How to build a custom Slack slash command using Express and Node.js

How to build a custom Slack slash command using Express and Node.js

In this tutorial we are going to learn how to build and deploy a custom Slack slash command using Node.js and the Express web framework.

In this tutorial we are going to learn how to build and deploy a custom Slack slash command using Node.js and the Express web framework.

Table of Contents

  • Slack "slash commands"... Wait, what?
  • Building a URL shortener slash command
  • Create a new Slack application
  • The integration flow
  • Application architecture
  • Preparing the project
  • The command parser
  • Validation module
  • The Rebrandly API Client module
  • The slash command module
  • Web server with Express
  • Local testing with Ngrok
  • Publishing the integration on Heroku
  • Wrapping up

Slack "slash commands"... Wait, what?

Slash commands are special messages that begin with a slash (/) and behave differently from regular chat messages. For example, you can use the /feed command to subscribe the current channel to an RSS feed and receive notifications directly into Slack everytime a new article is published into that feed.

There are many slash commands available by default, and you can create your custom ones to trigger special actions or to retrieve information from external sources without leaving Slack.

Building a URL shortener slash command

In this tutorial we are going to build a "URL shortener" slash command, which will allow us to generate personalised short urls with a versatile syntax. For example, we want the following command to generate the shorturl [http://loige.link/rome17](http://loige.link/rome17):

/urlshortener create a short url for the link http://loige.co/my-universal-javascript-web-applications-talk-at-codemotion-rome-2017/ using the domain @loige.link and the custom slashtag ~rome17


We are going to use Rebrandly as Short URL service. If you don't know this service I totally recommend you, essentially for 3 reasons:

  1. It offers a very extensive FREE plan.
  2. Has an easy to use and well documented API for creating short URLs programmatically.
  3. Supports custom domains (I personally use it for my blog with loige.link and for FullStack Bulletin with fstack.link).

So, before starting the tutorial, be sure to have a Rebrandly account and an API Key, which you can generate from the API settings page once you are logged in.

Create a new Slack application

In order to create a new custom slash command for a given Slack organisation you have to create an app in the Slack developer platform.

There are a number of easy steps you will need to follow to get started:

1 - Select the option to create a new slash command

2 - Specify some simple options for the slash command

Notice that, for now, we are passing a sample request bin URL as Request URL so that we can inspect what's the payload that gets sent by Slack before implementing our custom logic.

3 - Install the new app in your test organisation

Now the command is linked and can be already used in your slack organisation, as you can see in the following image:

When you submit this command, Slack servers will send a POST request to the Request URL with all the details necessary to implement your custom logic and provide a response to the user invoking the command:

The integration flow

Before moving on, let's understand how the data flows between the different components that make the Slack slash command work. Let's start with a picture:

In brief:

  1. A user types a slash command followed by some text (the arguments of the command) into a Slack chat window.
  2. The Slack server receives the command and forwards it with an HTTP POST request to the Request URL associated with the command (hosted by the slash command developer on a separate server). The POST request contains many details about the command that has been invoked, so that the server receiving it can react accordingly. Some of the fields passed by Slack to the application server are:
  • token: a unique value generated by Slack for this integration, it should be kept secret and can be used to verify that the slash command request is really coming from Slack and not from another external source. Take not of your because you will need it later on.
  • text: the full text passed as argument to the slash command
  • team_id: the Slack id of the team where the slash cammand has been installed
  • channel_name: the name of the channel where the command was invoked
  • user_name: the user name of the user who invoked the command
  • response_url: a special URL that can be used by the server to provide an asynchronous response to slack (useful for managing long lived tasks that might take more than 3 seconds to complete).
  1. The application server responds to the HTTP request with 200 OK and a message containing the output of the command that should be displayed to the user.
Application architecture

So it should be clear now that our goal is to implement a little web server app that receives url shortening commands, calls the rebrandly APIs to do so and returns the shortened URLs back to the Slack server.

We can break down our app into some well-defined components:

  1. The web server: deals with all the HTTP nuances, receives and decodes requests from the Slack server and forwards it to the underlying components. Collects the result from them and returns it as an HTTP response.
  2. The command parser: parses the text (arguments) of the slash commands and extract URLS, slashtags and domains.
  3. The url shortener: uses the result of the command parser to generate the short URLs by invoking the Rebrandly APIs.
Preparing the project

For this project we will Node.js 6.0 or higher, so, before moving on, be use you have this version in your machine.

Let's get ready to write some come, but first create a new folder and run npm init in it. In this tutorial we are going to use some external dependencies that we need to fetch from npm:

npm install \ 
  [email protected]^1.17.2 \ 
  [email protected]^4.15.3 \ 
  [email protected]^1.0.4 \ 
  [email protected]^0.0.8 \ 
  [email protected]^4.0.0

Now let's create a folder called src and inside of it we can create all the files that we will need to write:

mkdir src
touch \ 
  src/commandParser.js \ 
  src/createShortUrls.js \
  src/server.js \
  src/slashCommand.js \
  src/validateCommandInput.js

Quite some files, uh? Let's see what we need them for:

  • server.js: is our web server app. It spins up an HTTP server using Express that can be called by the Slack server. It servers as an entry point for the whole app, but the file itself will deal only with the HTTP nuances of the app (routing, request parsing, response formatting, etc.) while the actual business logic will be spread in the other files.
  • slashCommand.js: implements the high level business logic needed for the slash command to work. It reiceves the content of the HTTP request coming from the Slack server and will use other submodules to process it and validate it. It will also invoke the module that deals with the Rebrandly APIs and manage the response, properly formatting it into JSON objects that are recognized by Slack. It will delegate some of the business logic to other modules: commandParser, validateCommandInput and createShortUrls.
  • commandParser: this is probably the core module of our project. It has the goal to take an arbitrary string of text and extract some informations like URLs, domains and slashtags.
  • validateCommandInput: implements some simple validation rule to check if the result of the command parser is something that can be used with the Rebrandly APIs to create one or more short URLs.
  • createShortUrls: implements the business logic that invokes the Rebrandly APIs to create one or more custom short URLs.

This should give you a top-down view of the architecture of the app we are going to implement in a moment. If you are a visual person (like me), you might love to have a chart to visualize how those modules are interconnected, here you go, lady/sir:

The command parser

We said that the command parser is the core of our application, so it makes sense to start to code it first. Let's jump straight into the source code:

// src/commandParser.js
const tokenizer = require('string-tokenizer')
const createUrlRegex = require('url-regex')

const arrayOrUndefined = (data) => {
  if (typeof data === 'undefined' || Array.isArray(data)) {
    return data
  }

  return [data]
}

const commandParser = (commandText) => {
  const tokens = tokenizer()
    .input(commandText)
    .token('url', createUrlRegex())
    .token('domain', /(?:@)((?:(?:[a-z\\u00a1-\\uffff0-9]-*)*[a-z\\u00a1-\\uffff0-9]+)*\.[a-z\\u00a1-\\uffff]{2,})/, match => match[2])
    .token('slashtag', /(?:~)(\w{2,})/, match => match[2])
    .resolve()

  return {
    urls: arrayOrUndefined(tokens.url),
    domain: tokens.domain,
    slashtags: arrayOrUndefined(tokens.slashtag)
  }
}

module.exports = commandParser

This module exports the function commandParser. This function accepts a string called commandText as the only argument. This string will be the text coming from the slash command.

The goal of the function is to be able to extrapolate all the meaningful information for our task from a free format string. In particular we want to extrapolate URLs, domains and slashtags.

In order to do this we use the module [string-tokenizer](https://www.npmjs.com/package/string-tokenizer) and some regular expressions:

  • The module [url-regex](https://www.npmjs.com/package/url-regex) is used to recognize all valid formats of URLs.
  • Then we define our own regex to extract domains, assuming that they will be prefixed by the @ character. We also specify an inline function to normalize all the matches and get rid of the @ prefix in the resulting output.
  • Similarly we define a regular expression to extract slashtags, which needs to have the ~ character as prefix. Here as well we cleanup the resulting matches to get rid of the ~ prefix.

With this configuration, the string-tokenizer module will return an object with all the matching components organised by key: all the URLs will be stored in an array under the key url and the same will happen with domain and slashtag for domains and slashtags respectively.

The caveat is that, for every given token, string-tokenizer returns undefined if no match is found, a simple string if only one match is found and an array if there are several substring matching the token regex.

Since we want to have potentially many urls and many associated slashtags but only one URL at the time, we want to return an object with a very specific format that satisfies those expectations:

  • urls: an array of urls (or undefined if none is found)
  • domain: the domain as a string (or undefined if none is specified)
  • slashtags: an array of slashtags (or undefined if none is found)

We process the output obtained with the string-tokenizer module (also using the simple helper function arrayOrUndefined) and return the resulting object.

That's all for this module.

Validation module

The goal of the commandParser module was very clear: extract and normalize some information from a text in order to construct an object that describes all the short URLs that needs to be created and their options.

The issue is that the resulting command object might be inconsistent in respect to some business rules that we need to enforce to interact with the Rebrandly APIs:

  • There must be at least one URL.
  • There must be at most one domain per command (if none is specified a default one will be used).
  • The number of slashtags cannot exceed the number of URLs (slashtags will be mapped to URLs in order, if there are more URLs than slashtags, the remaining URLs will get a randomly generated slashtag).
  • A command cannot contain more than 5 URLs (Rebrandly standard APIs are limited to 10 requests per second, so with this rule we should reasonably avoid to reach the limit).

The module validateCommandInput is here to help use ensure that all those rules are respected. Let's see its code:

// src/validateCommandInput.js
const validateCommandInput = (urls, domain, slashtags) => {
  if (!urls) {
    return new Error('No url found in the message')
  }

  if (Array.isArray(domain)) {
    return new Error('Multiple domains found. You can specify at most one domain')
  }

  if (Array.isArray(slashtags) && slashtags.length > urls.length) {
    return new Error('Urls/Slashtags mismatch: you specified more slashtags than urls')
  }

  if (urls.length > 5) {
    return new Error('You cannot shorten more than 5 URLs at the time')
  }
}

module.exports = validateCommandInput

The code is very simple and pretty much self-descriptive. The only important thing to underline is that the validateCommandInput function will return undefined in case all the validation rules are respected or an Error object as soon as one validation rule catches an issue with the input data. We will see soon how this design decision will make our validation logic very concise in the next modules.

The Rebrandly API Client module

Ok, at this stage we start to see things coming together: we have a module to parse a free text and generate a command object, another module to validate this command, so now we need a module that uses the data in the command to actually interact with our short URL service of choice through rest APIs. The createShortUrls is here to address this need.

// src/createShortUrls.js
const request = require('request-promise-native')

const createErrorDescription = (code, err) => {
  switch (code) {
    case 400:
      return 'Bad Request'
    case 401:
      return 'Unauthorized: Be sure you configured the integration to use a valid API key'
    case 403:
      return `Invalid request: ${err.source} ${err.message}`
    case 404:
      return `Not found: ${err.source} ${err.message}`
    case 503:
      return `Short URL service currently under maintenance. Retry later`
    default:
      return `Unexpected error connecting to Rebrandly APIs`
  }
}

const createError = (sourceUrl, err) => {
  const errorDescription = createErrorDescription(err.statusCode, JSON.parse(err.body))
  return new Error(`Cannot create short URL for "${sourceUrl}": ${errorDescription}`)
}

const createShortUrlFactory = (apikey) => (options) => new Promise((resolve, reject) => {
  const body = {
    destination: options.url,
    domain: options.domain ? { fullName: options.domain } : undefined,
    slashtag: options.slashtag ? options.slashtag : undefined
  }

  const req = request({
    url: 'https://api.rebrandly.com/v1/links',
    method: 'POST',
    headers: {
      apikey,
      'Content-Type': 'application/json'
    },
    body: JSON.stringify(body, null, 2),
    resolveWithFullResponse: true
  })

  req
    .then((response) => {
      const result = JSON.parse(response.body)
      resolve(result)
    })
    .catch((err) => {
      resolve(createError(options.url, err.response))
    })
})

const createShortUrlsFactory = (apikey) => (urls, domain, slashtags) => {
  const structuredUrls = urls.map(url => ({url, domain, slashtag: undefined}))
  if (Array.isArray(slashtags)) {
    slashtags.forEach((slashtag, i) => (structuredUrls[i].slashtag = slashtag))
  }

  const requestsPromise = structuredUrls.map(createShortUrlFactory(apikey))
  return Promise.all(requestsPromise)
}

module.exports = createShortUrlsFactory

This module is probably the longest and the most complex of our application, so let's spend 5 minutes together to understand all it's parts.

The Rebrandly API allows to create one short URL at the time, but this module exposes an interface with which is possible to create multiple short URLs with a single function call. For this reason inside the module we have two abstraction:

  • createShortUrlFactory: that allows to create a single URL and remains private inside the module (it's not exported).
  • createShortUrlsFactory: (notice Url vs Urls) that uses the previous function multiple times. This is the publicly exported function from the module.

Another important details is that both functions here are implementing the factory function design pattern. Both functions are used to create two new functions were that contains the Rebrandly apikey in their scope, this way you don't need to pass the API key around everytime you want to create a short url and you can reuse and share the generated functions.

With all these details in mind, undestanding the rest of the code should be fairly easy, because we are only building some levels of abstraction over a REST request to the Rebrandly API (using [request-promise-native](https://www.npmjs.com/package/request-promise-native)).

The slash command module

Ok, now that we have the three main modules we can combine them together into our slashCommand module.

Before jumping into the code, remember that the goal of this module is to grab the request received from Slack, process it a generate a valid response using Slack application message formatting rules and Slack message attachments:

// src/slashCommand.js
const commandParser = require('./commandParser')
const validateCommandInput = require('./validateCommandInput')

const createErrorAttachment = (error) => ({
  color: 'danger',
  text: `*Error*:\n${error.message}`,
  mrkdwn_in: ['text']
})

const createSuccessAttachment = (link) => ({
  color: 'good',
  text: `** ():\n${link.destination}`,
  mrkdwn_in: ['text']
})

const createAttachment = (result) => {
  if (result.constructor === Error) {
    return createErrorAttachment(result)
  }

  return createSuccessAttachment(result)
}

const slashCommandFactory = (createShortUrls, slackToken) => (body) => new Promise((resolve, reject) => {
  if (!body) {
    return resolve({
      text: '',
      attachments: [createErrorAttachment(new Error('Invalid body'))]
    })
  }

  if (slackToken !== body.token) {
    return resolve({
      text: '',
      attachments: [createErrorAttachment(new Error('Invalid token'))]
    })
  }

  const { urls, domain, slashtags } = commandParser(body.text)

  let error
  if ((error = validateCommandInput(urls, domain, slashtags))) {
    return resolve({
      text: '',
      attachments: [createErrorAttachment(error)]
    })
  }

  createShortUrls(urls, domain, slashtags)
    .then((result) => {
      return resolve({
        text: `${result.length} link(s) processed`,
        attachments: result.map(createAttachment)
      })
    })
})

module.exports = slashCommandFactory

So, the main function here is slashCommandFactory, which is the function exported by the module. Again we are using the factory pattern. At this stage you might have noticed, how I tend to prefer this more functional approach as opposed to creating classes and constructors to keep track of initialization values.

In this module the factory generates a new function that has createShortUrls and slackToken in the function scope. The createShortUrls argument is a function that needs to be created with the createShortUrlsFactory that we saw in the previous module. We are using another important design pattern here, the Dependency injection pattern, that allows us to combine different modules in a very versatile way. This patterns offers many advantages, like:

  • Keep modules decoupled.
  • Allow to switch implementation of the dependency without changing the code of the dependant modules, for example we could switch to another short URL service without the need of changing a single line of code in this module.
  • Simplified testability.

Enough with design patterns and back to our slashCommandFactory function... The function it generates contains the real business logic of this module which, more or leass, reads like this:

  1. Verify if the current message body is present (otherwise stop and return error message).
  2. Verify that the request token is the one we were expecting from Slack (the request is very unlikely to have been forged by a third party). If the token is not valid stop and return error message.
  3. Use the commandParser to extrapolate information about the meaning of the current received command.
  4. Validate the command details using validateCommandInput (if the validation fails, stop and return an error message).
  5. Use the injected createShortUrls function to generate all the requested short URLs.
  6. Creates a response object for Slack containing details about every generated short URL (or errors that happened during the generation of one or more of them). For this last step we also use the internal utility function createAttachment.

Also notice that, since the operation performed by this module is asynchronous, we are returning a Promise, and that we resolve the promise also in case of errors. We didn't use a reject because we are managing those errors and we want to propagate them up to Slack as valid responses to the Slack server so that the user can visualize a meaningful error message.

Web server with Express

We are almost there, the last bit missing is the web server. With Express on our side and all the other business logic modules already written this should be an easy task:

// src/server.js
const Express = require('express')
const bodyParser = require('body-parser')
const createShortUrlsFactory = require('./createShortUrls')
const slashCommandFactory = require('./slashCommand')

const app = new Express()
app.use(bodyParser.urlencoded({extended: true}))

const {SLACK_TOKEN: slackToken, REBRANDLY_APIKEY: apiKey, PORT} = process.env

if (!slackToken || !apiKey) {
  console.error('missing environment variables SLACK_TOKEN and/or REBRANDLY_APIKEY')
  process.exit(1)
}

const port = PORT || 80

const rebrandlyClient = createShortUrlsFactory(apiKey)
const slashCommand = slashCommandFactory(rebrandlyClient, slackToken)

app.post('/', (req, res) => {
  slashCommand(req.body)
    .then((result) => {
      return res.json(result)
    })
    .catch(console.error)
})

app.listen(port, () => {
  console.log(`Server started at localhost:${port}`)
})

I believe the code above is quite self descriptive, but let's recap what's going on in there:

  1. We initialize a new Express app and activate the body parser extension (which allows us to parse urlencoded messages from Slack).
  2. We verify if the app has been initialized with all the necessary environment variables (SLACK_TOKEN for the Slack slash command token and REBRANDLY_APIKEY for the Rebrandly API key), otherwise we shutdown the application with an error. We can optionally specify also the environment variable PORT to use a different HTTP port for the server (by default 80).
  3. We use our factory functions to generate the rebrandlyClient and initialize the slashCommand.
  4. At this stage we are ready to register a POST route for the slash command which will just hook the parse the incoming HTTP requests and pass them to the slashCommand function we created before. When the slashCommand completes we just return its response as JSON to the Slack server using res.json.
  5. Finally, we can start the app with app.listen.

That's all, hooray! Let's move into running and test this Slack integration!

Local testing with Ngrok

Our app is complete and you can start it by running:

export SLACK_TOKEN="your slack token"
export REBRANDLY_APIKEY="your rebrandly API key"
export PORT=8080 #optional
node src/server

At this stage our app will be listening at localhost on port 8080 (or whatever other port you specified during the initialization). In order for Slack to reach it you will need a publicly available URL.

For now we don't need a permanent publicly available server, we just need a public URL to test the app. We can easily get a temporary one using ngrok.

After installing ngrok, we have to run:

ngrok http 8080


This command will print a public https URL. You can copy this into your Slack slash command Request URL.

Finally we are ready to go into our Slack app and invoke our custom slash command:

If we did everything correctly, at this stage, you should see a response from our app directly in Slack:

Publishing the integration on Heroku

If you are happy with the current status of the app and you want to have permanently available for your Slack team it's time to move it online. Generally for those kind of cases Heroku can be a quick and easy option.

If you want to host this app on Heroku, be sure to have an account and the Heroku CLI already installed, then initialize a new Heroku app in the current project folder with:

heroku create awesome-slack-shorturl-integration


Beware that you might need to replace awesome-slack-shorturl-integration with a unique name for an Heroku app (somebody else reading this tutorial might have taken this one).

Let's configure the app:

heroku config:set --app awesome-slack-shorturl-integration SLACK_TOKEN=<YOUR_SLACK_TOKEN> REBRANDLY_APIKEY=<YOUR_REBRANDLY_APIKEY>


Be sure to replace and with your actual configuration values and then you are ready to deploy the app with:

git push heroku master


This will produce a long output. At the end of it you should see the URL of the app on Heroku. Copy it and paste it as Request URL in the slash command config on your Slack app.

Now your server should be up and running on Heroku.

Enjoy it and keep shortening your URLs wisely!

Wrapping up

So, we are at the end of this tutorial, I really hope you had fun and that I inspired you to create some new cool Slack integration! Well, if you are out of ideas I can give you few:

  • /willIGoOutThisWeekend: to get the weather forecast in your area for the coming weekend.
  • /howManyHolidaysLeft: to tell you how many days of holiday you have left in this year.
  • /atlunch and /backfromlunch: to strategically change your availability status when you are going to lunch and when you are back.
  • /randomEmoji: in case you need help in finding new emojis to throw at your team members.

... OK ok, I am going to stop here, at this stage you will probably have better ideas :)

I hope you will share your creatins with me in the comments here, I might want to add some new integration in my Slack team!

Secure Node.js, Express.js and PostgreSQL API using Passport.js

Secure Node.js, Express.js and PostgreSQL API using Passport.js

The comprehensive step by step tutorial on building secure Node.js, Express.js, Passport.js, and PostgreSQL Restful Web Service

The comprehensive step by step tutorial on building secure Node.js, Express.js, Passport.js, and PostgreSQL Restful Web Service. Previously, we have shown you a combination of Node.js, Express.js, and PostgreSQL tutorial. Now, we just add a security for that RESTful Web Service endpoints. Of course, we will start this tutorial from scratch or from zero application. We will use JWT for this Node.js, Express.js, Passport.js, and PostgreSQL tutorial.

Table of Contents:

The following tools, frameworks, and modules are required for this tutorial:

We assume that you have installed PostgreSQL server in your machine or can use your own remote server (we are using PostgreSQL 9.5.13). Also, you have installed Node.js in your machine and can run node, npm or yarn command in your terminal or command line. Next, check their version by type this commands in your terminal or command line.

node -v
v8.12.0
npm -v
6.4.1
yarn -v
1.10.1

That the versions that we are uses. Let’s continue with the main steps.

1. Create Express.js Project and Install Required Modules

Open your terminal or node command line the go to your projects folder. First, install express generator using this command.

sudo npm install express-generator -g

Next, create an Express.js app using this command.

express secure-node --view=ejs

This will create Express.js project with the EJS view instead of Jade view template because using ‘–view=ejs’ parameter. Next, go to the newly created project folder then install node modules.

cd secure-node && npm install

You should see the folder structure like this.

There’s no view yet using the latest Express generator. We don’t need it because we will create a RESTful API.

2. Add and Configure Sequelize.js Module and Dependencies

Before installing the modules for this project, first, install Sequelize-CLI by type this command.

sudo npm install -g sequelize-cli

To install Sequelize.js module, type this command.

npm install --save sequelize

Then install the module for PostgreSQL.

npm install --save pg pg-hstore

Next, create a new file at the root of the project folder.

touch .sequelizerc

Open and edit that file then add this lines of codes.

const path = require('path');

module.exports = {
&nbsp; "config": path.resolve('./config', 'config.json'),
&nbsp; "models-path": path.resolve('./models'),
&nbsp; "seeders-path": path.resolve('./seeders'),
&nbsp; "migrations-path": path.resolve('./migrations')
};

That files will tell Sequelize initialization to generate config, models, seeders and migrations files to specific directories. Next, type this command to initialize the Sequelize.

sequelize init

That command will create config/config.json, models/index.js, migrations and seeders directories and files. Next, open and edit config/config.json then make it like this.

{
&nbsp; "development": {
&nbsp; &nbsp; "username": "djamware",
&nbsp; &nbsp; "password": "[email&nbsp;protected]@r3",
&nbsp; &nbsp; "database": "secure_node",
&nbsp; &nbsp; "host": "127.0.0.1",
&nbsp; &nbsp; "dialect": "postgres"
&nbsp; },
&nbsp; "test": {
&nbsp; &nbsp; "username": "root",
&nbsp; &nbsp; "password": "[email&nbsp;protected]@r3",
&nbsp; &nbsp; "database": "secure_node",
&nbsp; &nbsp; "host": "127.0.0.1",
&nbsp; &nbsp; "dialect": "postgres"
&nbsp; },
&nbsp; "production": {
&nbsp; &nbsp; "username": "root",
&nbsp; &nbsp; "password": "[email&nbsp;protected]@r3",
&nbsp; &nbsp; "database": "secure_node",
&nbsp; &nbsp; "host": "127.0.0.1",
&nbsp; &nbsp; "dialect": "postgres"
&nbsp; }
}

We use the same configuration for all the environment because we are using the same machine, server, and database for this tutorial.

Before run and test connection, make sure you have created a database as described in the above configuration. You can use the psql command to create a user and database.

psql postgres --u postgres

Next, type this command for creating a new user with password then give access for creating the database.

postgres-# CREATE ROLE djamware WITH LOGIN PASSWORD '[email&nbsp;protected]@r3';
postgres-# ALTER ROLE djamware CREATEDB;

Quit psql then log in again using the new user that previously created.

postgres-# \q
psql postgres -U djamware

Enter the password, then you will enter this psql console.

psql (9.5.13)
Type "help" for help.

postgres=>

Type this command to creating a new database.

postgres=> CREATE DATABASE secure_node;

Then give that new user privileges to the new database then quit the psql.

postgres=> GRANT ALL PRIVILEGES ON DATABASE secure_node TO djamware;
postgres=> \q

3. Create or Generate Models and Migrations

We will use Sequelize-CLI to generating a new model. Type this command to create a model for Products and User model for authentication.

sequelize model:create --name Product --attributes prod_name:string,prod_desc:string,prod_price:float
sequelize model:create --name User --attributes username:string,password:string

That command creates a model file to the model’s folder and a migration file to folder migrations. Next, modify models/user.js and then import this module.

var bcrypt = require('bcrypt-nodejs');

Add the new methods to the User model, so the user.js class will be like this.

module.exports = (sequelize, DataTypes) => {
&nbsp; const User = sequelize.define('User', {
&nbsp; &nbsp; username: DataTypes.STRING,
&nbsp; &nbsp; password: DataTypes.STRING
&nbsp; }, {});
&nbsp; User.beforeSave((user, options) => {
&nbsp; &nbsp; if (user.changed('password')) {
&nbsp; &nbsp; &nbsp; user.password = bcrypt.hashSync(user.password, bcrypt.genSaltSync(10), null);
&nbsp; &nbsp; }
&nbsp; });
&nbsp; User.prototype.comparePassword = function (passw, cb) {
&nbsp; &nbsp; bcrypt.compare(passw, this.password, function (err, isMatch) {
&nbsp; &nbsp; &nbsp; &nbsp; if (err) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return cb(err);
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; cb(null, isMatch);
&nbsp; &nbsp; });
&nbsp; };
&nbsp; User.associate = function(models) {
&nbsp; &nbsp; // associations can be defined here
&nbsp; };
&nbsp; return User;
};

For the models/product.js there’s no action needed, leave it as default generated the model class.

4. Create Routers for RESTful Web Service and Authentication

To authenticating users and secure the resources or endpoint create this file as a router.

touch routes/api.js

Open and edit routes/api.js then declares all require variables.

const express = require('express');
const jwt = require('jsonwebtoken');
const passport = require('passport');
const router = express.Router();
require('../config/passport')(passport);
const Product = require('../models').Product;
const User = require('../models').User;

Create a router for signup or register the new user.

router.post('/signup', function(req, res) {
&nbsp; console.log(req.body);
&nbsp; if (!req.body.username || !req.body.password) {
&nbsp; &nbsp; res.status(400).send({msg: 'Please pass username and password.'})
&nbsp; } else {
&nbsp; &nbsp; User
&nbsp; &nbsp; &nbsp; .create({
&nbsp; &nbsp; &nbsp; &nbsp; username: req.body.username,
&nbsp; &nbsp; &nbsp; &nbsp; password: req.body.password
&nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; .then((user) => res.status(201).send(user))
&nbsp; &nbsp; &nbsp; .catch((error) => {
&nbsp; &nbsp; &nbsp; &nbsp; console.log(error);
&nbsp; &nbsp; &nbsp; &nbsp; res.status(400).send(error);
&nbsp; &nbsp; &nbsp; });
&nbsp; }
});

Create a router for sign in or login with username and password.

router.post('/signin', function(req, res) {
&nbsp; User
&nbsp; &nbsp; &nbsp; .find({
&nbsp; &nbsp; &nbsp; &nbsp; where: {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; username: req.body.username
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; .then((user) => {
&nbsp; &nbsp; &nbsp; &nbsp; if (!user) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; return res.status(401).send({
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; message: 'Authentication failed. User not found.',
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; });
&nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; user.comparePassword(req.body.password, (err, isMatch) => {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; if(isMatch && !err) {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; var token = jwt.sign(JSON.parse(JSON.stringify(user)), 'nodeauthsecret', {expiresIn: 86400 * 30});
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; jwt.verify(token, 'nodeauthsecret', function(err, data){
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; console.log(err, data);
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; res.json({success: true, token: 'JWT ' + token});
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; } else {
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; res.status(401).send({success: false, msg: 'Authentication failed. Wrong password.'});
&nbsp; &nbsp; &nbsp; &nbsp; &nbsp; }
&nbsp; &nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; .catch((error) => res.status(400).send(error));
});

Create a secure router to get and post product data.

router.get('/product', passport.authenticate('jwt', { session: false}), function(req, res) {
&nbsp; var token = getToken(req.headers);
&nbsp; if (token) {
&nbsp; &nbsp; Product
&nbsp; &nbsp; &nbsp; .findAll()
&nbsp; &nbsp; &nbsp; .then((products) => res.status(200).send(products))
&nbsp; &nbsp; &nbsp; .catch((error) => { res.status(400).send(error); });
&nbsp; } else {
&nbsp; &nbsp; return res.status(403).send({success: false, msg: 'Unauthorized.'});
&nbsp; }
});

router.post('/product', passport.authenticate('jwt', { session: false}), function(req, res) {
&nbsp; var token = getToken(req.headers);
&nbsp; if (token) {
&nbsp; &nbsp; Product
&nbsp; &nbsp; &nbsp; .create({
&nbsp; &nbsp; &nbsp; &nbsp; prod_name: req.body.prod_name,
&nbsp; &nbsp; &nbsp; &nbsp; prod_desc: req.body.prod_desc,
&nbsp; &nbsp; &nbsp; &nbsp; prod_price: req.body.prod_price
&nbsp; &nbsp; &nbsp; })
&nbsp; &nbsp; &nbsp; .then((product) => res.status(201).send(product))
&nbsp; &nbsp; &nbsp; .catch((error) => res.status(400).send(error));
&nbsp; } else {
&nbsp; &nbsp; return res.status(403).send({success: false, msg: 'Unauthorized.'});
&nbsp; }
});

Create a function for extract the token.

getToken = function (headers) {
&nbsp; if (headers && headers.authorization) {
&nbsp; &nbsp; var parted = headers.authorization.split(' ');
&nbsp; &nbsp; if (parted.length === 2) {
&nbsp; &nbsp; &nbsp; return parted[1];
&nbsp; &nbsp; } else {
&nbsp; &nbsp; &nbsp; return null;
&nbsp; &nbsp; }
&nbsp; } else {
&nbsp; &nbsp; return null;
&nbsp; }
};

Finally, export the router as a module.

module.exports = router;

5. Run and Test Secure Node.js, Express.js, Passport.js, and PostgreSQL Web Service

To run and test this secure Node.js, Express.js, Passport.js, and PostgreSQL Web Service, run the PostgreSQL instance first then run this command from the Terminal.

nodemon

or

npm start

To test the secure Product endpoint, open the Postman then type fill all required fields like this image.

You should get the response message Unauthorized and status code 401. Next, test signup using the Postman by changing the method to POST, add the address localhost:3000/api/signup, add the header Content-type with value application/json and the body of request raw text like this.

{ "username":"[email&nbsp;protected]", "password":"qqqq1111" }

You should get this response when executing successfully.

Next, test to log in with the above signed/registered username and password by changing the URL to localhost:3000/api/signin. You should get this response when executes successfully.

Now, you can back using the previous GET method with additional header using the token get from the sign in/log in response. You should see the Product data like below.

That it’s, the secure Node.js, Express.js, Passport.js, and PostgreSQL Web Service. You can get the working source code from our GitHub.

Learn More

The Complete Node.js Developer Course (2nd Edition)

Learn and Understand NodeJS

Node JS: Advanced Concepts

GraphQL: Learning GraphQL with Node.Js

Angular (Angular 2+) & NodeJS - The MEAN Stack Guide

The Complete Python & PostgreSQL Developer Course

SQL & Database Design A-Z™: Learn MS SQL Server + PostgreSQL

The Complete SQL Bootcamp

The Complete Oracle SQL Certification Course

Creating a RESTful Web API with Node.js and Express.js from scratch

Creating a RESTful Web API with Node.js and Express.js from scratch

In this article, I’ll show you step by step how to create a RESTful Web API with Node.js and Express.js by building a simple and useful Todo API. This article assumes you have basic javascript knowledge and terminal using capabilities.

In this article, I’ll show you step by step how to create a RESTful Web API with Node.js and Express.js by building a simple and useful Todo API. This article assumes you have basic javascript knowledge and terminal using capabilities.

You can also build a Web API in Node.js by using another framework except Express.js but Express.js is one of the most popular web framework for Node.js.

You can found the final source code of this Web API in this github repository.

Let’s start to create our mentioned Web API.

Before start

If you have never used Node.js or npm package manager you should install them.

To check whether the Node.js is already installed on your computer, open your terminal and run node -v command. If you see your Node.js version it's installed. Otherwise go to below link.

Click here to download and install Node.js (You can choose LTS version)

And if you don’t have any IDE or text editor for writing javascript I advice you Visual Studio Code.

Click here to download VS Code (Optional)

About express-generator

In fact we could use <a href="https://expressjs.com/en/starter/generator.html" target="_blank">express-generator</a> tool which designed to creating an Express Web API quickly but I want to create this API from scratch because of that tool puts some extra files and folder structures that we don't need them now. But you can use this useful tool next time on creating new Web API. I won't use it now due to keep article simple.

Creating Project

Go to your workspace root folder and create a new folder there named "todo-api".

Then create "package.json" and "server.js" files into "todo-api" folder like below.

package.json

{
    "name": "todo-api",
    "version": "1.0.0",
    "scripts": {
        "start": "node server.js"
    },
    "dependencies": {
        "express": "^4.16.4"
    }
}

server.js

const http = require('http');
const express = require('express');
const app = express();
app.use(express.json());
app.use('/', function(req, res) {
    res.send('todo api works');
});
const server = http.createServer(app);
const port = 3000;
server.listen(port);
console.debug('Server listening on port ' + port);

After creating above files open your terminal in the "todo-api" folder and run npm installcommand.

This command will be install your project dependencies which pointed at the "package.json" file.

After finished package download process, downloaded dependency files will be installed into"node_modules" folder at the root of the "todo-api" folder.

After finished package installing then run npm start to start our Web API.

Now our Web API listening. To see result open your web browser then write localhost:3000 to address bar and press enter.

As result you’ll see our request handler response in your browser: “todo api works”.

This is a dead simple Express.js Web API. And it needs the some development. For example we need to an api endpoint to get todo items. So let’s add a new API endpoint for this.

Create a new folder named "routes" in the root of the "todo-api" folder.

Then create a "items.js" file inside of "routes" folder and put following codes inside it.

Your final folder structure should be like below;

/todo-api
/node_modules
/routes
    items.js
package.json
server.js

items.js

const express = require('express');
const router = express.Router();
const data = [
    {id: 1, title: 'Finalize project', order: 1, completed: false, createdOn: new Date()},
    {id: 2, title: 'Book ticket to London', order: 2, completed: false, createdOn: new Date()},
    {id: 3, title: 'Finish last article', order: 3, completed: false, createdOn: new Date()},
    {id: 4, title: 'Get a new t-shirt', order: 4, completed: false, createdOn: new Date()},
    {id: 5, title: 'Create dinner reservation', order: 5, completed: false, createdOn: new Date()},
];
router.get('/', function (req, res) {
    res.status(200).json(data);
});
router.get('/:id', function (req, res) {
    let found = data.find(function (item) {
        return item.id === parseInt(req.params.id);
    });
    if (found) {
        res.status(200).json(found);
    } else {
        res.sendStatus(404);
    }
});
module.exports = router;

Initial code of "items.js" file contains two endpoints. First one gets all todo items and second one gets one item which matches given id parameter.

Before testing items routes we should register it in the "server.js" file.

Modify "server.js" file like below to register new item routes.

server.js

const http = require('http');
const express = require('express');
const itemsRouter = require('./routes/items');
const app = express();
app.use(express.json());
app.use('/items', itemsRouter);
app.use('/', function(req, res) {
    res.send('todo api works');
});
const server = http.createServer(app);
const port = 3000;
server.listen(port);
console.debug('Server listening on port ' + port);

Now run npm start to start our Web API.

Then open your web browser and write localhost:3000/items to address bar and press enter.

You’ll see todo items json array in the response body.

And write localhost:3000/items/3 to address bar and press enter.

You’ll see the todo item which has id 3 in the response body.

But not finished up yet.

CRUD Operations and HTTP methods

I think we’ll need CRUD operations to Create, Read, Update and Delete todo items.

We have already two endpoints for getting items. So we need Create, Update and Delete endpoints.

Let’s add also these endpoints into the items.js file.

Our final "items.js" file and endpoints should be like below.

const express = require('express');
const router = express.Router();

const data = [
  {id: 1, title: 'Finalize project',          order: 1, completed: false, createdOn: new Date()},
  {id: 2, title: 'Book ticket to London',     order: 2, completed: false, createdOn: new Date()},
  {id: 3, title: 'Finish last article',       order: 3, completed: false, createdOn: new Date()},
  {id: 4, title: 'Get a new t-shirt',         order: 4, completed: false, createdOn: new Date()},
  {id: 5, title: 'Create dinner reservation', order: 5, completed: false, createdOn: new Date()},
];

router.get('/', function (req, res) {
  res.status(200).json(data);
});

router.get('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    res.status(200).json(found);
  } else {
    res.sendStatus(404);
  }
});

router.post('/', function (req, res) {
  let itemIds = data.map(item => item.id);
  let orderNums = data.map(item => item.order);

  let newId = itemIds.length > 0 ? Math.max.apply(Math, itemIds) + 1 : 1;
  let newOrderNum = orderNums.length > 0 ? Math.max.apply(Math, orderNums) + 1 : 1;

  let newItem = {
    id: newId,
    title: req.body.title,
    order: newOrderNum,
    completed: false,
    createdOn: new Date()
  };

  data.push(newItem);

  res.status(201).json(newItem);
});

router.put('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    let updated = {
      id: found.id,
      title: req.body.title,
      order: req.body.order,
      completed: req.body.completed
    };

    let targetIndex = data.indexOf(found);

    data.splice(targetIndex, 1, updated);

    res.sendStatus(204);
  } else {
    res.sendStatus(404);
  }
});

router.delete('/:id', function (req, res) {
  let found = data.find(function (item) {
    return item.id === parseInt(req.params.id);
  });

  if (found) {
    let targetIndex = data.indexOf(found);

    data.splice(targetIndex, 1);
  }

  res.sendStatus(204);
});

module.exports = router;

Short Explanation

I wanna explain shortly some points of our last codes.

First of all you must have noticed that our api works on a static data and keeps it on memory. All of our GET, POST, PUT and DELETE http methods just manipulate a json array. The purpose of this is to keep article simple and draw attention to the Web API structure.

Due to this situation our POST method has some extra logic such as calculating next item ids and order numbers.

So you can modify logic and data structures in these http methods to use a database or whatever you want.

Testing API with Postman

We have tested the GET methods of our Web API in our web browser and seen responses. But we can’t test directly POST, PUT and DELETE http methods in web browser.

If you want to test also other http methods you should use Postman or another http utility.

Now I’ll show you how to test the Web API with Postman

Before we start click here and install Postman.

When you first launch Postman after installing you’ll see start window. Close this start window by clicking close button on top right corner. Then you must see following screen.

An empty Postman request

Sending GET Request

Before sending a request to API we should start it by running npm startcommand as we do before.

After start the Web API and seeing “Server listening on…” message write localhost:3000/itemsto address bar as seen below and click Send button. You'll see todo items array as API response like below.

Sending a GET request with Postman

You can try similarly by giving an item id in request url like this localhost:3000/items/3

Sending POST Request

To sending a POST request and create a new todo item write localhost:3000/items to address bar and change HTTP verb to POST by clicking arrow at front of the address bar as seen below.

Sending a POST request with Postman

Before sending the POST request you should add request data to body of the request by clicking body tab and selecting raw and JSON as seen below.

Attaching a JSON body to POST request in Postman

Now click Send button to send POST request to the Web API. Then you must get “201 Created” http response code and seeing created item in the response body.

To see the last status of todo items send a get request to localhost:3000/itemsaddress. You must see newly created item at the end of the list.

Sending PUT Request

Sending PUT request is very similar to sending POST request.

The most obvious difference is request url should be pointed specific item like this localhost:3000/items/3

And you should choose PUT as http verb instead of POST and send all of the required data in the request body unlike POST.

For example you could send a JSON body in the PUT request as below.

An example JSON body for PUT request

{
    "title": "New title of todo item",
    "order": 3,
    "completed": false
}

When you click Send button you must get “204 No Content” http response code. You can check item you updated by sending a get request.

Sending DELETE Request

To send a DELETE request, change the request url to address a specific item id like this localhost:3000/items/3

And select DELETE as http verb and click Send button.

You must get “204 No Content” http response code as result of the DELETE operation.

Send a get request and see the last status of list.

About the DELETE Http Request

I want to say a few words about DELETE http request. You must have noticed something in our delete code. DELETE request returns “204 No Content” every situation.

Http DELETE requests are idempotent. So what that mean? If you delete a resource on server by sending DELETE request, it’s removed from the collection. And every next DELETE request on the same resource won’t change outcome. So you won’t get “404 Not Found” in the second request. Each request returns same response whether succeed or not. That’s mean idempotent operation.

Conclusion

Finally we’ve tested all http methods of our Web API.

As you can see, it works just fine.

Thanks for reading ❤

If you liked this post, share it with all of your programming buddies!

Top 7 Most Popular Node.js Frameworks You Should Know

Top 7 Most Popular Node.js Frameworks You Should Know

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser. In this post, you'll see top 7 of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Node.js is an open-source, cross-platform, runtime environment that allows developers to run JavaScript outside of a browser.

One of the main advantages of Node is that it enables developers to use JavaScript on both the front-end and the back-end of an application. This not only makes the source code of any app cleaner and more consistent, but it significantly speeds up app development too, as developers only need to use one language.

Node is fast, scalable, and easy to get started with. Its default package manager is npm, which means it also sports the largest ecosystem of open-source libraries. Node is used by companies such as NASA, Uber, Netflix, and Walmart.

But Node doesn't come alone. It comes with a plethora of frameworks. A Node framework can be pictured as the external scaffolding that you can build your app in. These frameworks are built on top of Node and extend the technology's functionality, mostly by making apps easier to prototype and develop, while also making them faster and more scalable.

Below are 7of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).

Express

With over 43,000 GitHub stars, Express is the most popular Node framework. It brands itself as a fast, unopinionated, and minimalist framework. Express acts as middleware: it helps set up and configure routes to send and receive requests between the front-end and the database of an app.

Express provides lightweight, powerful tools for HTTP servers. It's a great framework for single-page apps, websites, hybrids, or public HTTP APIs. It supports over fourteen different template engines, so developers aren't forced into any specific ORM.

Meteor

Meteor is a full-stack JavaScript platform. It allows developers to build real-time web apps, i.e. apps where code changes are pushed to all browsers and devices in real-time. Additionally, servers send data over the wire, instead of HTML. The client renders the data.

The project has over 41,000 GitHub stars and is built to power large projects. Meteor is used by companies such as Mazda, Honeywell, Qualcomm, and IKEA. It has excellent documentation and a strong community behind it.

Koa

Koa is built by the same team that built Express. It uses ES6 methods that allow developers to work without callbacks. Developers also have more control over error-handling. Koa has no middleware within its core, which means that developers have more control over configuration, but which means that traditional Node middleware (e.g. req, res, next) won't work with Koa.

Koa already has over 26,000 GitHub stars. The Express developers built Koa because they wanted a lighter framework that was more expressive and more robust than Express. You can find out more about the differences between Koa and Express here.

Sails

Sails is a real-time, MVC framework for Node that's built on Express. It supports auto-generated REST APIs and comes with an easy WebSocket integration.

The project has over 20,000 stars on GitHub and is compatible with almost all databases (MySQL, MongoDB, PostgreSQL, Redis). It's also compatible with most front-end technologies (Angular, iOS, Android, React, and even Windows Phone).

Nest

Nest has over 15,000 GitHub stars. It uses progressive JavaScript and is built with TypeScript, which means it comes with strong typing. It combines elements of object-oriented programming, functional programming, and functional reactive programming.

Nest is packaged in such a way it serves as a complete development kit for writing enterprise-level apps. The framework uses Express, but is compatible with a wide range of other libraries.

LoopBack

LoopBack is a framework that allows developers to quickly create REST APIs. It has an easy-to-use CLI wizard and allows developers to create models either on their schema or dynamically. It also has a built-in API explorer.

LoopBack has over 12,000 GitHub stars and is used by companies such as GoDaddy, Symantec, and the Bank of America. It's compatible with many REST services and a wide variety of databases (MongoDB, Oracle, MySQL, PostgreSQL).

Hapi

Similar to Express, hapi serves data by intermediating between server-side and client-side. As such, it's can serve as a substitute for Express. Hapi allows developers to focus on writing reusable app logic in a modular and prescriptive fashion.

The project has over 11,000 GitHub stars. It has built-in support for input validation, caching, authentication, and more. Hapi was originally developed to handle all of Walmart's mobile traffic during Black Friday.