Lab 5 - Advanced Sails

HTTP File upload & download

File upload

In traditional, we could upload file using HTML form element by specifying the enctype attribute to multipart/form-data together with method="post". However, Sails requires all <input type="file" /> elements placing at the end of a form. Meaning that, we could not mix <input type="file" /> with other input elements within a form.

Thus, here are two different examples. One is the traditional approach, another one uses HTML 5 File API reading the file content and put it in a hidden input element.

Example 1: Upload file using `<form enctype="multipart/form-data">`

In this example, we would store file path of the uploaded file in User model avatarPath attribute.

Let’s use your comp3047-ls05-2018 Sails project as example. Please clone the project and open it up in Visual Studio Code.

git clone https://github.com/HKBU-COMP3047/comp3047-ls05-2018-your_github_username

Adding `avatarPath` attribute in `api/models/User.js`.

We add an avatarPath attribute to User model in order to store the uploaded file path.

avatarPath: {
	type: 'string'
},
// you may also need this id attribute for new version Sails in all of the models
id: {
	type: 'number',
	autoIncrement: true
},

Add a `upload` action in `api/controllers/UserController.js`

Then we add a controller action to handle the file upload.

//...
upload: async function(req, res) {

	if (req.method == 'GET')
		return res.view('user/upload');

	req.file('avatarfile').upload({maxBytes: 10000000}, async function whenDone(err, uploadedFiles) {
		if (err) { return res.serverError(err); }
		if (uploadedFiles.length === 0){ return res.badRequest('No file was uploaded'); }

		console.log('req.body.agree = ' + req.body.agree);

		await User.update({username: req.session.username}, {
			avatarPath: uploadedFiles[0].fd
		});

		return res.ok('File uploaded.');
	});
},
//...

The req.file('avatarfile').upload(options, callback) function would handle <input type="file" name="avatarfile" /> file upload control.

Config a route to the upload action

As usual, we need to define a route to the controller action in order to reach it. Please add the following line to config/routes.js.

'/user/upload': 'UserController.upload',

Adding ejs file

In a form element like <form action="..." method="post">, application/x-www-form-urlencoded is the default enctype when not specified. Thus we need to specify it to multipart/form-data by adding the enctype attribute.

Please create views/user/upload.ejs file with the following content.

<form action="/user/upload" method="post" enctype="multipart/form-data">
	<input type="file" accept="image/*" name="avatarfile" />
	<input type="checkbox" name="agree" value="true" />
	<input type="submit" />
</form>

Example 2: Embed file data in a hidden input

In this example, we are going to store the file content in User model avatar attribute.

Adding `avatar` attribute in `api/models/User.js`.

avatar: {
	type: 'string'
},

Adding `upload2` Controller action

//...
upload2: async function(req, res) {

	if (req.method == 'GET')
		return res.view('user/upload2');

	console.log('req.body.agree = ' + req.body.agree);

	await User.update({username: req.session.username}, {
		avatar: req.body.User.avatar
	});
	
	return res.ok('File uploaded.');
},
//...

Config a route to the `upload2` action

As usual, we need to define a route to the controller action in order to reach it. Please add the following line to config/routes.js.

'/user/upload2': 'UserController.upload2',

Adding ejs file

In this example, we need not to specify the enctype attribute of the form element. Instead, we need to add a implement a function, handling onchange event of the file input control with HTML 5 File API.

Please create views/user/upload2.ejs file with the following content.

<form action="/user/upload2" method="post">
	<input type="file" accept="image/*" onchange="handleFile(this.files)" />
	<div id="preview"></div>
	<input type="checkbox" name="agree" value="true" />
	<input type="submit" />
</div>
<script>
function handleFile(files) {

  const file = files[0];
  if (!file.type.startsWith('image/')) return;

  var preview = document.getElementById('preview');
  
  var reader = new FileReader();
  reader.onload = function(e) {
	preview.innerHTML = "";
  
	var img = document.createElement('img');
    img.src = e.target.result;
    preview.appendChild(img);

	var input = document.createElement('input');
	input.type = 'hidden';
	input.name = 'User[avatar]';
	input.value = e.target.result;
	preview.appendChild(input);
  }
  reader.readAsDataURL(file);

}
</script>

File download

Example 1: Stream file content directly in controller action

Adding `avatar` Controller action

Please add the following avatar action to api/controllers/UserController.js.

avatar: async function(req, res) {
	var user = await User.findOne({username: req.params.username});
	if (!user || !user.avatarPath)
		return res.notFound();
	
	var SkipperDisk = require('skipper-disk');
	var fileAdapter = SkipperDisk();
	// set the filename
	// res.set("Content-disposition", "attachment; filename='avatar.jpg'");
	res.set('Content-type', 'image/jpeg');
	// Stream the file down
	fileAdapter.read(user.avatarPath).on('error', function (err){
		return res.serverError(err);
	}).pipe(res);
},

Adding route to controller action

As usual, we need to define a route to the controller action in order to reach it. Please add the following line to config/routes.js.

'GET /user/:username/avatar.jpg': 'UserController.avatar',

After that, you could view the image by http://localhost:1337/user/admin/avatar.jpg, if you uploaded avatar using admin user.

Usage

<img src="/user/admin/avatar.jpg" />

Example 2: Embed file Data URI in HTML element

Adding a controller action

Please add the following profile action to api/controllers/UserController.js.

profile: async function(req, res) {
	var user = await User.findOne({username: req.session.username});
	if (!user) return res.notFound();

	return res.view('user/profile',{model: user});
},

Adding the route

'GET /user/profile': 'UserController.profile',

Adding view ejs file

Please create view/user/profile.ejs with the following code.

<h1><%=model.username%></h1>
<% if(model.avatar) { %>
<img src="<%=model.avatar%>" />
<a download="avatar.jpg" href="<%= model.avatar%>" target="_blank">Download avatar</a>
<% } %>

For the file we stored its Data URI in mode, we could simply use it in src attribute of an image element. If you would like to make it a download link, you could use <a href> element with download="filename" attribute like <a href="..." download="filename">.

Example 3: Convert file to Data URI in Sails

We may like to convert a local file to Data URI for embedding in HTML. To do so, we could use datauri module.

https://www.npmjs.com/package/datauri

Install `datauri` module

As usual, we use npm install ... --save command to add this module to our project.

npm install datauri --save

Changing `upload` action to store Data URI in `UserController`

Please locate following lines in UserController, which storing the file path in avatarPath attribute.

await User.update({username: req.session.username}, {
	avatarPath: uploadedFiles[0].fd
});

We modify it to store Data URI in avatar attribute by reading the file as Data URI.

const  DataURI  =  require('datauri').sync;
await User.update({username: req.session.username}, {
	avatar: DataURI(uploadedFiles[0].fd)
});

Handling CSV file

CSV stands for comma-separated values. It is a simple file format for storing excel like data table.

`fast-csv` module

fast-csv is a module for parsing or formatting CSV file written in Javascript. You could found the complete documentation and examples at the following URL.

https://www.npmjs.com/package/fast-csv

Install `fast-csv` module in project

As usual, we use npm install ... --save command to add this module to our project.

npm isntall fast-csv --save

Export CSV

Example: Stream csv file of `Person` models

In this example, we construct a controller action for us to download all Person models in CSV format.

Adding a controller action

Let’s add csv controller action with following codes into api/controllers/PersonController.js.

csv: async function(req, res) {
	var csv = require("fast-csv");
	var models = await Person.find();
	if (!models)
		return res.notFound();
	res.set('Content-type', 'text/csv');
	csv.write(models , {headers:  true}).pipe(res);
},

Setting up the route

As usual, we need to define a route to the controller action in order to reach it. Please add the following line to config/routes.js.

'GET /person/export.csv': 'PersonController.csv',

After that let’s checkout the result by navigating to http://localhost:1337/person/export.csv

Import CSV

Example: Upload and import CSV

Let’s combine it with the upload function that we have just learned. In this example, we would construct a upload function for importing the CSV file that we have just downloaded.

Adding controller action

Here are codes for implementing the controller action. Please add it in api/controllers/PersonController.js. The code block for uploading file is similar to previous multipart/form-data example.

//...
upload: async function(req, res) {

	if (req.method == 'GET')
		return res.view('person/upload');

	req.file('csv').upload({maxBytes: 10000000}, async function whenDone(err, uploadedFiles) {
		if (err) { return res.serverError(err); }
		if (uploadedFiles.length === 0){ return res.badRequest('No file was uploaded'); }

		var csv = require("fast-csv");
		csv.fromPath(uploadedFiles[0].fd, {headers :  true}).on("data", async function(data){

			await Person.create(data);
	
		}).on("end",  function(){
			return res.ok('csv file imported.');
		});

	});
},
//...

After we have validate the uploaded file, we construct a fast-csv object using require('fast-csv'). Then we use its .fromPath(path, options) to parse the CSV file that have just uploaded. .on('data') event callback would give us actual data of each row in the CSV file. The .on('end') event callback tells us the parsing have been completed.

Adding ejs file of upload form

Please create views/person/upload.ejs file with the following content.

<form action="/person/upload" method="post" enctype="multipart/form-data">
	<input type="file" accept="text/csv" name="csv" />
	<input type="submit" />
</form>

Adding route to controller action

As usual, we need to define a route to the controller action in order to reach it. Please add the following line to config/routes.js.

'/person/upload': 'PersonController.upload',

After that let’s checkout the result by navigating to http://localhost:1337/person/upload

Sending Email

Setting up Mailgun, a free email service

Mailgun is a powerful transactional Email APIs that enable you to send, receive, and track emails.

Register a Mailgun account

Please register a Mailgun account at https://www.mailgun.com/.

enter image description here

Optional: Add a Authorized Recipients

Without providing credit card information, we could only send email to authorized recipients. Please add an authorized recipient email address for testing.

enter image description here

Using Mailgun in Sails

Mailgun comes with a Javascript module. we may use it directly in our Sails app.

Install Mailgun module

As usual, we use npm install ... --save command to add this module to our project.

npm install mailgun-js --save

Add Mailgun account information to `config/custom.js`

Please add your Mailgun account information according to your dashboard. mailgunDomain is your Mailgun domain, for example sandboxef5f56337bca4a36a491e6a91e8cd296.mailgun.org. mailgunSecret is the API Key showing on the domain dashboard. mailgunFrom is the email address showing in From of an email.

mailgunDomain: 'yourmailgundomain',
mailgunSecret: 'yourmailgunkey',
mailgunFrom:'yourmailgunfrommail',

Generate a helper function

We would like to construct a helper function, which would enable us to use the function globally in Sails. Please use the following command to create a blank helper function with name send-single-email.

sails generate helper send-single-email

Adding code to helper

Please replace the codes in api/helpers/send-single-email.js as follows.

var mailgun = require('mailgun-js')({apiKey: sails.config.custom.mailgunSecret, domain: sails.config.custom.mailgunDomain});

module.exports = {
	friendlyName: 'Send single email',
	description: '',
	inputs: {
		options:{
			type:'json'
		}
	},
	exits: {
	},
	fn: async function (inputs, exits) {
		mailgun.messages().send(inputs.options, function (error, body) {
		
			if(error) return exits.error(error)
			return exits.success(body);
		});
	}
};

Using the helper function

We could add the following to config/bootstrap.js for testing. Each time we sails lift, there is an email sending to destination@email.com. Please modify destination@email.com to the authorized email you have just added.

await sails.helpers.sendSingleEmail({
	to: 'destination@email.com',
	from: sails.config.custom.mailgunFrom,
	subject: 'Subject',
	text: 'Your message'
});

After that, just run sails lift to see result.

Example: Render ejs file as email content

We could use ejs rendering result as email content. sails.renderView(pathToView, templateData) function could render the ejs file content with dynamic data, and return the result as string. We could use option layout: false to exclude layout.ejs. Please add the following code to api/controllers/UserController.js.

userlistemail: async function(req, res) {
	var models = await User.find();
	var html = await sails.reanderView('user/email_userlist', {
		users: models,
		layout: false
	});
	await sails.helpers.sendSingleEmail({
		to: 'change_to_your@email.com',
		from: sails.config.custom.mailgunFrom,
		subject: 'Subject',
		html: html
	});
	return res.ok('users list sent!');
},

Please create views/user/email_userlist.ejs with the following content.

<% users.forEach(function(user) { %>
<div><%= user.username %></div>
<% }); %>

Lab 5 - Advanced Sails

HTTP File upload & download

File upload

Example 1: Upload file using <form enctype="multipart/form-data">

Adding avatarPath attribute in api/models/User.js.

Add a upload action in api/controllers/UserController.js

Config a route to the upload action

Adding ejs file

Example 2: Embed file data in a hidden input

Adding avatar attribute in api/models/User.js.

Adding upload2 Controller action

Config a route to the upload2 action

Adding ejs file

File download

Example 1: Stream file content directly in controller action

Adding avatar Controller action

Adding route to controller action

Usage

Example 2: Embed file Data URI in HTML element

Adding a controller action

Adding the route

Adding view ejs file

Example 3: Convert file to Data URI in Sails

Install datauri module

Changing upload action to store Data URI in UserController

Handling CSV file

fast-csv module

Install fast-csv module in project

Export CSV

Example: Stream csv file of Person models

Adding a controller action

Setting up the route

Import CSV

Example: Upload and import CSV

Adding controller action

Adding ejs file of upload form

Adding route to controller action

Sending Email

Setting up Mailgun, a free email service

Register a Mailgun account

Optional: Add a Authorized Recipients

Using Mailgun in Sails

Install Mailgun module

Add Mailgun account information to config/custom.js

Generate a helper function

Adding code to helper

Using the helper function

Example: Render ejs file as email content

Example 1: Upload file using `<form enctype="multipart/form-data">`

Adding `avatarPath` attribute in `api/models/User.js`.

Add a `upload` action in `api/controllers/UserController.js`

Adding `avatar` attribute in `api/models/User.js`.

Adding `upload2` Controller action

Config a route to the `upload2` action

Adding `avatar` Controller action

Install `datauri` module

Changing `upload` action to store Data URI in `UserController`

`fast-csv` module

Install `fast-csv` module in project

Example: Stream csv file of `Person` models

Add Mailgun account information to `config/custom.js`