PostgreSQL with Nodejs

Most of the applications need at some point to persist data. It can be through files, local storage, cloud services or often databases

If you do not have particular requirement relational database systems are usually a good default choice, and particularly PostgreSQL, a very powerful open source SQL server. Moreover, if you develop for the Nodejs platform you will not run short of tools and solutions.

So what would be the ideal library or tool for your next application?

If you are lucky enough to work for a company which has resources to have its own dedicated database team, you will probably have only to use the abstractions they provide in your language of choice (Javascript in our case) to call the stored procedures they will have carefully crafted. In that case, you will eventually see no SQL (Structured Query Language, the language used in many relational database systems) at all and won’t go out of your comfort zone. In practice, this case may not occur very often and the truth is that many Nodejs developers or even so-called full stack developers know very little about SQL and will seek for various abstractions in their favorite packages repository.

Lately, I came across a very interesting article written by Thomas Hunter which were describing the different levels of abstraction a database related library may have. The idea was to dissuade you to use frameworks called ORM (Object Relational Mapping) and to convince you to learn SQL. Of course, as often in software development, each kind of abstraction has its pros and cons and let’s review quickly what I learned in Thomas’ article.

If you want more details you should probably read Thomas’ article and skip the sections below.

Drivers

Drivers provide almost no abstraction at all. They usually only handle connections and the communication protocol with the database. They may also parse the responses into adapted data structures (most of the time in Javascript it will be JSON objects and arrays). The *de facto *PostgreSQL driver for Nodejs is pg.

Drivers are normally very fast and the most flexible solution as they will let you manually write SQL queries yourself. On the other hand, it might quickly become tedious and dangerous to write dynamic queries by hand as it would involve a lot of string manipulation.

Query builders

With query builders, instead of manipulating strings, you will be using functions whose API is usually close to the SQL grammar. In Javascript, functions compose very well with patterns such as partial application, decoration, composition, recursion, higher-order function, etc.

They will often let you create your own abstractions and solve many problems in an elegant way. They also provide a framework with more explicit constraints than strings can do (especially if you use some sort of types on top of the normal signatures) which usually makes you avoid making mistakes and fit better in your everyday tools (IDE and so).

A very popular query builders library is knex.

ORM

ORM is the highest level of abstraction and such frameworks come with usually a lot of features. The promise is to create a data model domain with idioms you know such classes, objects, etc; and then the framework will translate it into a database structure and convenient ways to query it through the models you will have defined. The obvious advantage is that you can handle everything in your language of choice and with concepts familiar to you, the underlying database entities and queries will stay very far from your application code. However, the way we think the application data model sometimes does not quite fit the way we would build a database for that model: it is known as the object-relational impedance mismatch. Such libraries have been creating passionate debates between their fans and the others. Of course, they definitely have their usage and the problem is not black or white, but there are recurrent criticisms:

  • When you learn an ORM (and this is already many hours of work) you learn syntax and concepts you probably will not be used outside of this particular ORM.
  • The SQL queries generated by an ORM may not be very efficient in the sense that ORM will have to tackle specialized queries in a more generalist way. Moreover, performances can also degrade as the ORM will often not return raw data but some model instances of it (in order to provide some features like validation, lazy loading, etc). You often trade performances for features.
  • ORM cannot generate some (basic sometimes) queries or have it wrong when used through their models. This problem is however easily worked around as they provide a low-level query interface to write the queries you want directly with the underlying driver.
  • You may not design the database, the database schema can be generated for you which may sometimes lead to very poor database design.

The most popular ORM for Nodejs is Sequelize.

I think we can do better…

Introducing Ship-Hold

Note all the examples will use the database generated by the script in annexes at the end of this article
In the 1970s (yes almost 50 years ago!). Edgar F. Codd created a Relational Algebra. SQL became the most common implementation of the algebra and has evolved a lot to become one of the most powerful query language and one of the most battle-tested. It includes now procedure capabilities and extensions (especially with PostgreSQL) which go beyond the scope of the algebra. Clearly, SQL is often neglected by application developers whereas its capabilities are enormous.

Through products (joins) you can cross data from different tables and organize your result data in infinite ways whereas with SQL functions you can aggregate data to format it the way you want. A single query although quite verbose is probably way more flexible than what you could do with Javascript even by composing small functions together.

You want to fetch all the posts including a field *author *with the first name and the last name of the author:

SELECT 
    “posts”.*, 
    concat_ws(‘ ‘,”users”.”first_name”,”users”.”last_name”) as “author” 
FROM “posts” JOIN “users” USING(“user_id”);

Actually, you prefer to include all author’s data as a JSON object:

SELECT 
    “posts”.*, 
    to_json(“users”.*) as “author” 
FROM “posts” JOIN “users” USING(“user_id”);

Or you want to know who are the five users who have published the most comments:

WITH 
    “commenters” AS (SELECT COUNT(*),”user_id” FROM “comments” GROUP BY “user_id” ORDER BY “count” DESC LIMIT 5)
SELECT “users”.* FROM “commenters” JOIN “users” USING (“user_id”);

Doing the same things in Javascript considering your table as arrays would require a great deal of code.

You are convinced and decide to learn SQL: great! What if you could write in Javascript with function calls, SQL queries the way you would write them in pure SQL? Good news, with ship-hold you can.

Let’s say you want to find all the posts about SQL published after 2010 including their author’s data ordered by publication date:

const posts = await sh
    .select('posts.*',{
        value: toJson('"users".*'),
        as:'author'
    })
    .from('posts')
    .where('title','ilike','%sql%')
    .and('published_at','>',new Date(2010,1,1))
    .join('users')
    .on('posts.user_id','"users"."user_id"')
    .orderBy('published_at','desc')
    .run();

And ship hold query builders are regular functions so you can add all the syntactic sugar you want. Even better you can compose builders together as you would do with SQL subqueries. For example, you could get the same result for the third example with the following code:

// A higher order function
const fromTable = table => (...args) => sh.select(...args).from(table);

const fromPosts = fromTable('posts');

const fromUsers = fromTable('users');
// A sub query
const getTopFiveCommenters = {
    value: fromPosts(count('*'), 'user_id')
        .groupBy('user_id')
        .orderBy('count', 'desc')
        .limit(5),
    as: 'talkative'
};

const users = await fromUsers()
    .where('user_id', 'IN', sh.select('user_id').from(getTopFiveCommenters))
    .run();

It basically has the power of SQL with the expressiveness of Javascript functions!

Eager loading: a not so easy problem.

SQL is very powerful but at some point, it becomes quite complicated for a human being to write a query to get the expected result. It is particularly true when you want to fetch related data from different tables and start to apply pagination to your result set or to a sub-result set. That’s one of the reasons ORM exist: they eventually help you to generate such queries.

Let’s say you want to get users with the posts they have written in case they have written some:

SELECT * FROM “users” LEFT JOIN “posts” USING (“user_id”)

Ok, now you want the users whose last name is Renard, and you want only the five first Renard ordered by alphanumeric order on their first name. Of course you still want the articles they eventually have written too.

SELECT * FROM “users” LEFT JOIN “posts” USING (“user_id”) WHERE “last_name” = ‘Renard’ ORDER BY “first_name” LIMIT 5

Is it correct ? Actually no: The limit will be applied to the join and as various rows may correspond to the same user (one for each written article) with this query you’ll be missing data.

SELECT * FROM (SELECT * FROM “users” WHERE “last_name”=’Renard’ ORDER BY “first_name” LIMIT 5) as “users” LEFT JOIN “posts” USING (“user_id”) ORDER BY “first_name”

This query would do the job, although the data parsed by the driver would then need to be aggregated: again various rows may correspond to the same user!

Now you also want to paginate the posts included: instead of getting all the articles written by a given user you want the three last published about Javascript. Ah, and you also want to include within these articles the five last published comment with their author’s data…

Ok, you get the point, it becomes quite complicated to write a query which will get you the right result.

Thankfully ORM and such will help you to get the right result. How they do may vary from one library to another. Some will make multiple queries then buffer the result and aggregate it programmatically. Others will inline the join queries, etc. A different strategy can get you the result but more or less efficiently.

Ship hold is not an ORM but it gives you the ability to define services related to a particular table. By using a service, it will simplify the syntax to make a query against a particular table, but it also gives you the ability to define relations between services. It will greatly help you then to write queries with inclusions. You basically only have to compose builders as you would construct your sentence in English.

const Users = sh.service({
    table: 'users',
    primaryKey: 'user_id'
});

const Posts = sh.service({
    table: 'posts',
    primaryKey: 'post_id'
});

const Comments = sh.service({
    table: 'comments',
    primaryKey: 'comment_id'
});
Users.hasMany(Posts);
Posts.belongsTo(Users, 'user_id', 'author');

Users.hasMany(Comments);
Posts.hasMany(Comments);
Comments.belongsTo(Users, 'user_id', 'author');
Comments.belongsTo(Posts, 'post_id', 'article');

Then get the result for our very complicated request aforementioned:

// Five last comments with their authors' data
const fiveLastPublishedComments = Comments
    .select()
    .orderBy('published_at', 'desc')
    .limit(5)
    .include(Users);
// Three last Javascript posts
const threeLastPostAboutJavascript = Posts
    .select()
    .where('title', 'ilike', '%javascript%')
    .orderBy('published_at', 'desc')
    .limit(3)
    .include(fiveLastPublishedComments);

// finally
const renards = await Users
    .select()
    .where('last_name', 'Renard')
    .orderBy('first_name')
    .limit(5)
    .include(threeLastPostAboutJavascript)
    .run();

And done!

I dare you to do it so easily with an other tool :)

About performances

You might wonder how good the queries generated will perform. It is very difficult to compare data access libraries together as the result might change quite a lot depending on the database itself: the indexes created, the amount of data, etc.

However, ship-hold tries to leverage as much as possible the database features where a lot of ORM will duplicate the effort in their Javascript code base. A good example is the aggregation. Where many ORM will perform several queries buffer the result (often with redundant data) then aggregate the result programmatically, ship-hold will ask the database to do the aggregation with its built-in mechanisms. It is also one of the reasons ship-hold is small compared to other libraries. The weight of the installed bundled is 771kbincluding pg driver and others dependencies.

Database makers strive to improve performances, and write usually faster code in C++, with better algorithms that I personally can do on the JS side.

Any performance improvement on the database engine would, therefore, improve ship-hold. Moreover, join queries and subqueries carry a part of redundant data. If you do the aggregation at the database level, I suspect that less data would need to go throw the wires which might improve the performances compared to a solution where data is buffered on the JS side and cleaned/aggregated. It could be especially true if your database is on a remote server.

Nevertheless, you can have a look at the numbers of a sample applicationimplemented with various popular libraries.

If you think ship-hold worth it, please share the project link and star the repository or clap this article. If you have any recommendations to improve it don’t hesitate to contribute. Thanks for reading.

Annexes

Database script

DROP TABLE IF EXISTS posts, users, comments, tags, posts_tags CASCADE;
CREATE TABLE users (
    user_id serial PRIMARY KEY,
    email VARCHAR(100) UNIQUE NOT NULL,
    biography TEXT,
    first_name VARCHAR(100) NOT NULL, 
    last_name VARCHAR(100) NOT NULL 
);

CREATE TABLE posts(
    post_id serial PRIMARY KEY,
    title VARCHAR NOT NULL,
    content TEXT,
    published_at TIMESTAMP,
    user_id integer REFERENCES users (user_id)
);

CREATE TABLE comments(
    comment_id serial PRIMARY KEY,
    content TEXT,
    published_at TIMESTAMP,
    user_id integer REFERENCES users (user_id),
    post_id integer REFERENCES posts (post_id)
);

CREATE TABLE tags(
    tag VARCHAR PRIMARY KEY,
    description TEXT
);

CREATE TABLE posts_tags(
    post_id INTEGER REFERENCES posts,
    tag VARCHAR REFERENCES tags,
    PRIMARY KEY (post_id, tag)
);

By : Laurent Renard

The Features you need to know when using Javascript in 2020

The Features you need to know when using Javascript in 2020

A peek into the future of the JavaScript language, take a look at the exciting stage and features you need to know when using Javascript in 2020

Javascript reach has increased from a web browser to all places of programming.

  1. Node.js — Used for CLI and Server.

  2. Electron — Used for cross-platform desktop Apps.

  3. React native — Used for cross-platform mobile Apps.

  4. IoT — Low-cost IoT devices, now support javascript.

Recent updates to v8 engine have increased the performance a lot. The javascript parsing is 2x faster, promise on an avg is 11x faster on node version ≥11 from node version 8. The memory consumption as decreased -20%. So there is an improvement in performance and usability.

In this article, we will see some of the presents and purposed features that you can test in chrome browser (version ≥76) or Node.js(version ≥11) CLI.

Private Class field👇

Until ES6, we were not able to declare private properties directly. Yes there were ways like underscore convention (_propertyName), closures, symbols, or WeakMaps.

But now private class fields use a hash # prefix. Let’s learn it by an example.

class Test {
  a = 1;          // .a is public
  #b = 2;         // .#b is private
  static #c = 3;  // .#c is private and static
  incB() {
    this.#b++;
  }
}
const testInstance = new Test();
// runs OK
testInstance.incB();
// error - private property cannot be modified outside class
testInstance.#b = 0;

*Note: There’s no way to define the private function as of now, although a TC39 stage 3: draft proposal suggests using a hash # prefix on names.

You may also like: Keeping your JavaScript code clean forever and scalable.

String.matchAll()👇

If I have a string, with a global regular expression which has many capturing groups, I often want to iterate through all groups. Currently, my options are the following:

  1. String.prototype.match() with /g — If we use .match() with a regular expression whose flag /g is set, you get all full matches for it in an Array.

  2. String.prototype.split() — If we use a split string and regular expression to specify the separator and if it contains at least one capture group then .split() returns an Array in which the substrings are interleaved.

The issues with the above approach are that they only work if /g is set on regular expression and the property .lastIndex of a regular expression is changed each time a match happens. This makes using the same regular expression at multiple locations risky.

The matchAll() help resolve all above. Let’s check out the definition and usage

Given a string and a regular expression, .matchAll() returns all results matching a string against a regular expression, including capturing groups.

let regexp = /t(e)(st(\d?))/g;
let str = 'test1test2';
let array = [...str.matchAll(regexp)];
console.log(array[0]);
// expected output: Array ["test1", "e", "st1", "1"]

Note: .matchAll() returns an iterator, not a true restartable iterable. That is, once the result is exhausted, you need to call the method again and create a new iterator.

Numeric Separators👇

If you have struggled to read a long sequence of number, this is where your search end.

Numeric Separators allow the human eye to parse quickly, especially when there are lots of repeating digits:

1000000000000 -> 1_000_000_000_000
1019436871.42 -> 1_019_436_871.42

Now it’s easier to tell that the first number is a trillion, and the second number is in the order of 1 billion.

It also works on other bases, for example:

const fileSystemPermission = 0b111_111_000;
const bytes = 0b1111_10101011_11110000_00001101;
const words = 0xFAB_F00D;

You can also use the separator in the fractions and exponents:

const massOfElectronInKg = 9.109_383_56e-31;
const trillionInShortScale = 1e1_2;

Note: Parsing the _ separated integer can be tricky as Number('123_456') gives NAN whereas parseInt('123_456') gives 123

BigInt’s👇

BigInts are a new numeric primitive in JavaScript that can represent integers with precision larger than 2⁵³–1. With BigInts, you can safely store and operate on large integers even beyond the safe integer limit for Numbers .

BigInts correctly perform integer arithmetic without overflowing. Let’s understand by an example:-

const max = Number.MAX_SAFE_INTEGER;
// 9007199254740991
max+1;
// 9007199254740992
max+2;
// 9007199254740991

We can see that max + 1 produces the same result as max + 2 .

Any calculation on integers outside the safe integer range (i.e. from Number.MIN_SAFE_INTEGER to Number.MAX_SAFE_INTEGER) potentially loses precision. For this reason, we can only rely on numeric integer values within the safe range.

Therefore, BigInts came to existence, BigInts can be created by adding the n suffix to any integer literal. For example, 123 becomes 123n or the global BigInt(number) function can be used to convert a Number into a BigInts.

Let’s revisit the above example with BigInts

BigInt(Number.MAX_SAFE_INTEGER) + 2n;
// 9007199254740993n
typeof 123n
// "bigint2"

Note: Numeric separators are especially helpful with BigInts, for eg: const massOfEarthInKg = 6_000_000_000_000_000_000_000_000n;

BigInts support the most common operators. Binary +, -, *, and ** all work as expected. / and % work, and round towards zero as needed.

(7 + 6 - 5) * 4 ** 3 / 2 % 3;
// → 1
(7n + 6n - 5n) * 4n ** 3n / 2n % 3n;
// → 1n

Note: One gotcha is that it’s not allowed to mix operations between BigInts and Numbers

Locale String with BigInt👇

The toLocaleString() method returns a string with a language-sensitive representation of the BigInt.

let bigint = 123456789123456789n;

// German uses period for thousands
console.log(bigint.toLocaleString('de-DE'));
// → 123.456.789.123.456.789

// Arabic in most Arabic speaking countries uses Eastern Arabic digits
console.log(bigint.toLocaleString('ar-EG'));
// → ١٢٣٬٤٥٦٬٧٨٩٬١٢٣٬٤٥٦٬٧٨٩

// India uses thousands/lakh/crore separators
console.log(bigint.toLocaleString('en-IN'));
// → 1,23,45,67,89,12,34,56,789

// the nu extension key requests a numbering system, e.g. Chinese decimal
console.log(bigint.toLocaleString('zh-Hans-CN-u-nu-hanidec'));
// → 一二三,四五六,七八九,一二三,四五六,七八九

// when requesting a language that may not be supported, such as
// Balinese, include a fallback language, in this case Indonesian
console.log(bigint.toLocaleString(['ban', 'id']));
// → 123.456.789.123.456.789
GlobalThis Keyword👇

JavaScript’s variable scopes are nested and form a tree whose root is the global scope and the value of the this keyword is a reference to the object that “owns” the currently executing code or the function where its looked at.

Usually to figure out the global this we use a function like

const getGlobalThis = () => {

  // in webworker or service worker
  if (typeof self !== 'undefined') return self;

  // in browser 
  if (typeof window !== 'undefined') return window;

  // in Node.js
  if (typeof global !== 'undefined') return global;

  // Standalone javascript shell
  if (typeof this !== 'undefined') return this;

  throw new Error('Unable to locate global object');
};
const theGlobalThis = getGlobalThis();

The above function does not cover all cases when we need global this value.

  1. In the case of use strict the value of this is undefined.

  2. When we form a bundle in the javascript it usually wraps under some code that might differ the global this.

  3. In Standalone javascript engine shell environment, the above code will not work.

To solve the above problem globalThis keyword is introduced which returns global this object in any environment at any time.

Note: The global object is now considered a mistake that JavaScript can’t get rid of, due to backward compatibility. It affects performance negatively and is generally confusing.

Promise.allSettled()👇

If you are wondering what’s promise is in javascript then check out this — JavaScript Promises: an Introduction.

A little gist, a promise is JavaScript’s way of promising you that work will be done (or might fail if the work could not be completed).

The new method returns a promise that resolves after all of the given promises have settled i.e either resolved or rejected, with an array of objects that each describe the outcome of each promise.

const promise1 = Promise.resolve(3);
const promise2 = new Promise((resolve, reject) => setTimeout(reject, 100, 'foo'));
const promises = [promise1, promise2];
Promise.allSettled(promises).
  then((results) => results.forEach((result) => console.log(result.status)));
// expected output:
// "fulfilled"
// "rejected"

This is different from the Promise.all as it rejects as soon as a promise within the iterable object rejected.

Below is the comparison of current supported promises method

![](https://i.imgur.com/NpAaXb2.png)

Promise.all, Promise.allSettled, Promise.race

Dynamic Import 👇
![](https://i.imgur.com/vovyPRX.png)

Static vs Dynamic Import

This one is crazy before we dive into it, let’s first see what static import is.

Static import only accepts a string literal as the module specifier and introduces bindings into the local scope via a pre-runtime “linking” process.

The static import syntax can only be used at the top-level of the file.

import * as module from './utils.mjs';

Static import enables important use cases such as static analysis, bundling tools, and tree-shaking.

But what about

  • import a module on-demand (or conditionally)

  • compute the module specifier at runtime

  • import a module from within a regular script (as opposed to a module)

This was not possible until dynamic importsimport(moduleSpecifier) returns a promise for the module namespace object of the requested module, which is created after fetching, instantiating, and evaluating all of the module’s dependencies, as well as the module itself.

<script type="module">
  (async () => {
    const moduleSpecifier = './utils.mjs';
    const module = await import(moduleSpecifier)
    module.default();
    // → logs 'Hi from the default export!'
    module.doStuff();
    // → logs 'Doing stuff…'
  })();
</script>

Note: Use static imports for initial paint dependencies, especially for above-the-fold content. In other cases, consider loading dependencies on-demand with dynamic import().

You may also like: Top 8 Trends and Tools Front-End JavaScript for 2020.

Stable Sorting — (Consistent and reliable result now)👇

By Stable in the algorithmic sense means: does it preserve the order or otherwise “equal” items?

Let’s understand by an example

const people = [
  {name: 'Gary', age: 20},
  {name: 'Ann', age: 20},
  {name: 'Bob', age: 17},
  {name: 'Sue', age: 21},
  {name: 'Sam', age: 17},
];

// Sort people by name
people.sort( (p1, p2) => {
  if (p1.name < p2.name) return -1;
  if (p1.name > p2.name) return 1;
  return 0;
});

console.log(people.map(p => p.name));
// ['Ann', 'Bob', 'Gary', 'Sam', 'Sue']
// Re-sort people by age

people.sort( (p1, p2) => {
  if (p1.age < p2.age) return -1;
  if (p1.age > p2.age) return 1;
  return 0;
});

console.log(people.map(p => p.name));
// We're expecting people sorted by age, then by name within age group:
// ['Bob', 'Sam', 'Ann', 'Gary', 'Sue']
// But we might get any of these instead, depending on the browser:
// ['Sam', 'Bob', 'Ann', 'Gary', 'Sue']
// ['Bob', 'Sam', 'Gary', 'Ann', 'Sue']
// ['Sam', 'Bob', 'Gary', 'Ann', 'Sue']

If you’re getting one of the last three results, then you’re probably using Google Chrome, or maybe one of an assortment of browsers that do not implement Array.sort() as a “stable” algorithm.

This is because different JS engines (across different browsers) taking different routes to implement sort, Also, some javascript engine use stable sort for short array but for long array uses unstable sort.

This lead to inconsistent in sort stability behavior and a lot of confusion. This is why in the development environment everything related to sort seems to work but in the production environment, we start to see something else due to the varying size of array the sort was tested on.

Note: There are 3rd party libraries, I heartily recommend Lodash, which has stable sort

But this has been resolved now, we have a stable sort on most of the browsers. The syntax remains the same.

Since this article has a lot to digest and try-test features, we will continue with the more new feature in the next one.

Thank you for reading!

How to Use a SQL Like and Regex Search in MongoDB and Node.JS

How to Use a SQL Like and Regex Search in MongoDB and Node.JS

In this article we will know how to use Regex to search in MongoDB like the SQL Like Statement

In this article we will know how to use Regex to search in MongoDB like the SQL Like Statement

To select the documents from a collection, you can use the db.collection.find() method. To select all documents in the collection, pass an empty document as the query filter document to the method.

In the shell, copy and paste the following to return all documents in the members collection.

db.members.find({})

To format the results, append the .pretty() to the find operation:

db.members.find({}).pretty()

Searching for Word Using Regex

Now that we are using .find() to query our collection, we can actually modify our syntax ever so slightly and begin searching for matches based on a word or phrase that may be a partial match within a given field, similar to the LIKE operator for SQL engines.

The trick is to utilize regular expressions (or regex for short), which is basically a text string that defines a search pattern. There are a number of regex engines that are written in slightly different syntax, but the fundamentals are all basically the same, and in this case, MongoDB uses the Perl Regex (PCRE) engine.

At the most basic level, a regex expression is a string (series of characters) enclosed on both sides by a single slash (/).

For example, if we want to use regex to perform the same query as above and find out how many members serve Neha, we can replace our string "Neha" with /Neha/ instead:

db.members.find( { name: /Neha/ } )

But imagine we want to find the number of restaurants where borough starts with the first three characters "Neha". We’d modify our regex very slightly, like so:

db.members.find( { name: /^Neha/ } )

The caret character (^) specifies the location in our string which should be the beginning, so if we had a document where those three letters were in the middle of the field, we wouldn’t get a match.

This informs the regex engine that we want to the search to be case insensitive, matching regardless of upper or lowercase. We also added the special i flag following our regex closing slash (/):

db.members.find( { name: /Neha/i } )

Using variable regex with MongoDB in Node.JS

var search ='Neha';

db.members.find(name: new RegExp(search)) //For substring search, case sensitive. 

db.members.find(name: new RegExp('^' + search + '$')) //For exact search, case sensitive

db.members.find(name: new RegExp(search, ‘i')) //For substring search, case insensitive

db.members.find(name: new RegExp('^' +search + '$', 'i')); //For exact search, case insensitive

Happy Coding!

Please like and share so I have the motivation to continue sharing!

What is the difference between PM2 Cluster vs Fork Mode in Node.JS

What is the difference between PM2 Cluster vs Fork Mode in Node.JS

97 The main difference between fork_mode and cluster_mode is that it orders pm2 to use either the child_process.fork api or the cluster api.

The main difference between fork_mode and cluster_mode is that it orders pm2 to use either the child_process.fork api or the cluster api.

Fork mode

Take the fork mode as a basic process spawning. This allows to change the exec_interpreter, so that you can run a php or a python server with pm2. Yes, the exec_interpreter is the "command" used to start the child process. By default, pm2 will use node so that pm2 start server.js will do something like:

require('child_process').spawn('node', ['server.js'])

This mode is very useful because it enables a lot of possibilities. For example, you could launch multiple servers on pre-established ports which will then be load-balanced by HAProxy or Nginx.

Cluster mode

The cluster will only work with node as it's exec_interpreter because it will access to the nodejs cluster module (eg: isMaster, fork methods etc.). This is great for zero-configuration process management because the process will automatically be forked in multiple instances.

For example pm2 start -i 4 server.js will launch 4 instances of server.js and let the cluster module handle load balancing.

How to Create Secure (JWT) Token Based Authentication API with Node.js

How to Create Secure (JWT) Token Based Authentication API with Node.js

In this tutorial, we are going to learn how to build a secure token-based user authentication REST APIs using JWT (JSON web token), bcrypt, Node, Express, and MongoDB

Creating authentication REST API with Node Js is merely effortless. We will be taking the help of Express js to create the authentication endpoints and also make the MongoDB connection to store the user’s data in it.

Authentication Workflow with JSON Web Tokens

Let’s understand from the below diagram how does the secure authentication system work with JSON web token.

Token Based Authentication API

  • A client makes the API call and sends the user information such as username and password to the webserver.
  • On successful authentication a webserver generates a string-based token and returns to the client system.
  • A client can store this token in the browser’s local storage or in a session.
  • Client sets this token in a header something like “Bearer xxx.xxx.xxx”.
  • On next API call JWT token communicateS with the server, and after the successful verification, the server returns the response to the client.
Initiate Node Token-Based Authentication Project

Create a project folder to build secure user authentication REST API, run the following command.

mkdir server

Get inside the project folder.

cd server

Let’s start the project by first creating the package.json file by running the following command.

npm init

Install NPM Packages to Create Secure Auth API

Next, install the NPM dependencies for the authentication API by running the given below command.

npm install express jsonwebtoken bcryptjs body-parser 
cors mongoose-unique-validator mongoose --save

This is image title

Next, install the nodemon NPM module, it helps in starting the node server when any change occurs in the server files.

npm install nodemon --save-dev
Define Mongoose Schema

Next, we are going to define user schema using mongoose ODM. It allows us to retrieve the data from the database.

Create a folder and name it models inside the project directory, create a file User.js in it.

To prevent storing the duplicate email id in MongoDB database install mongoose-unique-validator package. Below we will learn how to use in mongoose schema to validate duplicate email id from MongoDB database.

npm i mongoose-unique-validator --save

Next, add the following code in models/User.js file:

// models/User.js

const mongoose = require('mongoose');
const Schema = mongoose.Schema;
const uniqueValidator = require('mongoose-unique-validator');

let userSchema = new Schema({
    name: {
        type: String
    },
    email: {
        type: String,
        unique: true
    },
    password: {
        type: String
    }
}, {
    collection: 'users'
})

userSchema.plugin(uniqueValidator, { message: 'Email already in use.' });
module.exports = mongoose.model('User', userSchema)
  • The userSchema.plugin(uniqueValidator) method won’t let duplicate email id to be stored in the database.
  • The unique: true property in email schema does the internal optimization to enhance the performance.
Implement MongoDB Database in Node App

Create database folder in the project folder and create a new file database/db.js in it.

module.exports = {
    db: 'mongodb://localhost:27017/meanauthdb'
}
Create Secure Token-based Authentication REST API in Node

To build secure user authentication endpoints in node, create routes folder, and auth.routes.js file in it.

Here, we will define CRUD Restful APIs using the npm packages for log-in, sign-up, update-user, and delete-user.

// routes/auth.routes.js

const express = require("express");
const jwt = require("jsonwebtoken");
const bcrypt = require("bcrypt");
const router = express.Router();
const userSchema = require("../models/User");

// Sign-up
router.post("/register-user", (req, res, next) => {
    bcrypt.hash(req.body.password, 10).then((hash) => {
        const user = new userSchema({
            name: req.body.name,
            email: req.body.email,
            password: hash
        });
        user.save().then((response) => {
            res.status(201).json({
                message: "User successfully created!",
                result: response
            });
        }).catch(error => {
            res.status(500).json({
                error: error
            });
        });
    });
});

// Sign-in
router.post("/signin", (req, res, next) => {
    let getUser;
    userSchema.findOne({
        email: req.body.email
    }).then(user => {
        if (!user) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        getUser = user;
        return bcrypt.compare(req.body.password, user.password);
    }).then(response => {
        if (!response) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        let jwtToken = jwt.sign({
            email: getUser.email,
            userId: getUser._id
        }, "longer-secret-is-better", {
            expiresIn: "1h"
        });
        res.status(200).json({
            token: jwtToken,
            expiresIn: 3600,
            msg: getUser
        });
    }).catch(err => {
        return res.status(401).json({
            message: "Authentication failed"
        });
    });
});

// Get Users
router.route('/').get((req, res) => {
    userSchema.find((error, response) => {
        if (error) {
            return next(error)
        } else {
            res.status(200).json(response)
        }
    })
})

// Get Single User
router.route('/user-profile/:id').get((req, res, next) => {
    userSchema.findById(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

// Update User
router.route('/update-user/:id').put((req, res, next) => {
    userSchema.findByIdAndUpdate(req.params.id, {
        $set: req.body
    }, (error, data) => {
        if (error) {
            return next(error);
            console.log(error)
        } else {
            res.json(data)
            console.log('User successfully updated!')
        }
    })
})

// Delete User
router.route('/delete-user/:id').delete((req, res, next) => {
    userSchema.findByIdAndRemove(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

module.exports = router;
  • To secure the password, we are using the bcryptjs, It stores the hashed password in the database.
  • In the signin API, we are checking whether the assigned and retrieved passwords are the same or not using the bcrypt.compare() method.
  • In the signin API, we set the JWT token expiration time. Token will be expired within the defined duration.
Verify Node Authentication REST API

Next, we will verify the auth API using the JWT token. Create a middlewares folder and create a auth.js file inside of it, then include the following code in it.

Note: In the real world app the secret should not be kept in the code as declared below. The best practice is to store as an environment variable and it should be complex combination of numbers and strings.

// middlewares/auth.js

const jwt = require("jsonwebtoken");

module.exports = (req, res, next) => {
    try {
        const token = req.headers.authorization.split(" ")[1];
        jwt.verify(token, "longer-secret-is-better");
        next();
    } catch (error) {
        res.status(401).json({ message: "Authentication failed!" });
    }
};

Now, we will learn to implement JWT verification in the /user-profile endpoint. Import the following auth.js file from middlewares folder.

// Get User Profile
router.route('/user-profile/:id').get(authorize, (req, res, next) => {
    userSchema.findById(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

We added the authorize variable inside the user-profile API. It won’t render the data unless it has the valid JWT token. As you can see in the below screenshot, we have not defined the JWT token in get request, so we are getting the “No token provided” error.

JWT Verification Error in Express API

Adding Input Validation in Express RESTful API

Next, we will learn to implement validation in Express auth API using POST body request. Install express-validator npm library to validate name, email and password.

The express-validator is an express.js middleware for validating POST body requests.

Run the below command to install the express-validator package.

npm install express-validator --save

Add the following code in the middlewares/auth.routes.js file.

// routes/auth.routes.js

const express = require("express");
const jwt = require("jsonwebtoken");
const bcrypt = require("bcrypt");
const router = express.Router();
const userSchema = require("../models/User");
const authorize = require("../middlewares/auth");
const { check, validationResult } = require('express-validator');

// Sign-up
router.post("/register-user",
    [
        check('name')
            .not()
            .isEmpty()
            .isLength({ min: 3 })
            .withMessage('Name must be atleast 3 characters long'),
        check('email', 'Email is required')
            .not()
            .isEmpty(),
        check('password', 'Password should be between 5 to 8 characters long')
            .not()
            .isEmpty()
            .isLength({ min: 5, max: 8 })
    ],
    (req, res, next) => {
        const errors = validationResult(req);
        console.log(req.body);

        if (!errors.isEmpty()) {
            return res.status(422).jsonp(errors.array());
        }
        else {
            bcrypt.hash(req.body.password, 10).then((hash) => {
                const user = new userSchema({
                    name: req.body.name,
                    email: req.body.email,
                    password: hash
                });
                user.save().then((response) => {
                    res.status(201).json({
                        message: "User successfully created!",
                        result: response
                    });
                }).catch(error => {
                    res.status(500).json({
                        error: error
                    });
                });
            });
        }
    });

// Sign-in
router.post("/signin", (req, res, next) => {
    let getUser;
    userSchema.findOne({
        email: req.body.email
    }).then(user => {
        if (!user) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        getUser = user;
        return bcrypt.compare(req.body.password, user.password);
    }).then(response => {
        if (!response) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        let jwtToken = jwt.sign({
            email: getUser.email,
            userId: getUser._id
        }, "longer-secret-is-better", {
            expiresIn: "1h"
        });
        res.status(200).json({
            token: jwtToken,
            expiresIn: 3600,
            _id: getUser._id
        });
    }).catch(err => {
        return res.status(401).json({
            message: "Authentication failed"
        });
    });
});

// Get Users
router.route('/').get((req, res) => {
    userSchema.find((error, response) => {
        if (error) {
            return next(error)
        } else {
            res.status(200).json(response)
        }
    })
})

// Get Single User
router.route('/user-profile/:id').get(authorize, (req, res, next) => {
    userSchema.findById(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

// Update User
router.route('/update-user/:id').put((req, res, next) => {
    userSchema.findByIdAndUpdate(req.params.id, {
        $set: req.body
    }, (error, data) => {
        if (error) {
            return next(error);
            console.log(error)
        } else {
            res.json(data)
            console.log('User successfully updated!')
        }
    })
})

// Delete User
router.route('/delete-user/:id').delete((req, res, next) => {
    userSchema.findByIdAndRemove(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

module.exports = router;

We Passed the validation array with the check() method inside the post() method as a second argument. Next, we called the validationResult() method to validate errors, and it returns the errors if found any.

Following validation we implemented in ("/register-user") api.

  • Check if the value is required.
  • Check min and max character’s length.

Express input validation

Final auth.routes.js

// routes/auth.routes.js

const express = require("express");
const jwt = require("jsonwebtoken");
const bcrypt = require("bcrypt");
const router = express.Router();
const userSchema = require("../models/User");
const authorize = require("../middlewares/auth");
const { check, validationResult } = require('express-validator');

// Sign-up
router.post("/register-user",
    [
        check('name')
            .not()
            .isEmpty()
            .isLength({ min: 3 })
            .withMessage('Name must be atleast 3 characters long'),
        check('email', 'Email is required')
            .not()
            .isEmpty(),
        check('password', 'Password should be between 5 to 8 characters long')
            .not()
            .isEmpty()
            .isLength({ min: 5, max: 8 })
    ],
    (req, res, next) => {
        const errors = validationResult(req);
        console.log(req.body);

        if (!errors.isEmpty()) {
            return res.status(422).jsonp(errors.array());
        }
        else {
            bcrypt.hash(req.body.password, 10).then((hash) => {
                const user = new userSchema({
                    name: req.body.name,
                    email: req.body.email,
                    password: hash
                });
                user.save().then((response) => {
                    res.status(201).json({
                        message: "User successfully created!",
                        result: response
                    });
                }).catch(error => {
                    res.status(500).json({
                        error: error
                    });
                });
            });
        }
    });

// Sign-in
router.post("/signin", (req, res, next) => {
    let getUser;
    userSchema.findOne({
        email: req.body.email
    }).then(user => {
        if (!user) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        getUser = user;
        return bcrypt.compare(req.body.password, user.password);
    }).then(response => {
        if (!response) {
            return res.status(401).json({
                message: "Authentication failed"
            });
        }
        let jwtToken = jwt.sign({
            email: getUser.email,
            userId: getUser._id
        }, "longer-secret-is-better", {
            expiresIn: "1h"
        });
        res.status(200).json({
            token: jwtToken,
            expiresIn: 3600,
            msg: getUser
        });
    }).catch(err => {
        return res.status(401).json({
            message: "Authentication failed"
        });
    });
});

// Get Users
router.route('/').get(authorize, (req, res) => {
    userSchema.find((error, response) => {
        if (error) {
            return next(error)
        } else {
            res.status(200).json(response)
        }
    })
})

// Get Single User
router.route('/user-profile/:id').get((req, res, next) => {
    userSchema.findById(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

// Update User
router.route('/update-user/:id').put((req, res, next) => {
    userSchema.findByIdAndUpdate(req.params.id, {
        $set: req.body
    }, (error, data) => {
        if (error) {
            return next(error);
            console.log(error)
        } else {
            res.json(data)
            console.log('User successfully updated!')
        }
    })
})

// Delete User
router.route('/delete-user/:id').delete((req, res, next) => {
    userSchema.findByIdAndRemove(req.params.id, (error, data) => {
        if (error) {
            return next(error);
        } else {
            res.status(200).json({
                msg: data
            })
        }
    })
})

module.exports = router;
Node Server Configuration

Create a server.js file in the token-based authentication project’s folder and paste the following code in it.

const express = require('express');
const mongoose = require('mongoose');
const cors = require('cors');
const bodyParser = require('body-parser');
const dbConfig = require('./database/db');

// Express APIs
const api = require('./routes/auth.routes');

// MongoDB conection
mongoose.Promise = global.Promise;
mongoose.connect(dbConfig.db, {
    useNewUrlParser: true,
    useUnifiedTopology: true
}).then(() => {
    console.log('Database connected')
},
    error => {
        console.log("Database can't be connected: " + error)
    }
)

// Remvoe MongoDB warning error
mongoose.set('useCreateIndex', true);

// Express settings
const app = express();
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({
    extended: false
}));
app.use(cors());

// Serve static resources
app.use('/public', express.static('public'));

app.use('/api', api)

// Define PORT
const port = process.env.PORT || 4000;
const server = app.listen(port, () => {
    console.log('Connected to port ' + port)
})

// Express error handling
app.use((req, res, next) => {
    setImmediate(() => {
        next(new Error('Something went wrong'));
    });
});

app.use(function (err, req, res, next) {
    console.error(err.message);
    if (!err.statusCode) err.statusCode = 500;
    res.status(err.statusCode).send(err.message);
});

In this file we defined mongoDB database, express routes, PORT and errors.

Start Node Server

Now, we have placed everything at its place, and now it’s time to start the Node server. Open the terminal and run the given below commands to start the Node server and mongoDB:

Start the MongoDB database:

mongod

Start the nodemon server:

nodemon

You can test Node server on the following URL:
http://localhost:4000/api

Here, are the user authentication CRUD REST APIs built with Node.js.

API Methods API URL
GET (Users List) /api
POST (Sign in) /api/signin
POST (Sign up) /api/register-user
GET (User Profile) /api/user-profile/id
PUT (Update User) /api/update-user/id
DELETE (Delete User) /api/delete-user/id

Conclusion

Finally, we have completed secure Token-Based Authentication REST API with Node.js tutorial. So far, In this tutorial we have learned how to securely store the password in the database using the hash method with bcryptjs, how to create JWT token to communicate with the client and a server using jsonwebtoken. We also implemented the Express input validation using the express-validator plugin.

I hope you liked this tutorial, please share it with others, thanks for reading!

Security Best Node.js - Managing Sessions in Express.js

Security Best Node.js - Managing Sessions in Express.js

Every user interaction with your application is an isolated and individual request and response. The need to persist information between requests is vital for maintaining the ultimate experience for the user and the same applies for Node.js web...

Every user interaction with your application is an isolated and individual request and response. The need to persist information between requests is vital for maintaining the ultimate experience for the user and the same applies for Node.js web application such as the popular Express.js framework.

A common scenario that most developers can identify with is the need to maintain that a user has authenticated with an application.  In addition, it’s quite common to retain various personalized user information that is associated with a session as well.

Similarly, we are going to look at how we can securely set up sessions in our application to mitigate risks such as session hijacking. We’re going to look at how we can obfuscate session ID’s, enforce a time-to-live in our sessions, set up secure cookies for transporting sessions, and finally the importance and role of Transport Layer Security (TLS) when it comes to using sessions.

Setting of Sessions in Express.js

We’re going to use the NPM module express-sessions, a very popular session module that has been highly vetted by the community and constantly improved.

We’ll pass our express app object to a function to wire up the express-session module:

"use strict";
// provides a promise to a mongodb connection
import connectionProvider            from "../data_access/connectionProvider";
// provides application details such as MongoDB URL and DB name
import {serverSettings}              from "../settings";
import session                       from "express-session";
import mongoStoreFactory             from "connect-mongo";

export default function sessionManagementConfig(app) {

    // persistence store of our session
    const MongoStore = mongoStoreFactory(session);

    app.use(session({
        store: new MongoStore({
            dbPromise: connectionProvider(serverSettings.serverUrl, serverSettings.database)
        }),
        secret: serverSettings.session.password,
        saveUninitialized: true,
        resave: false,
        cookie: {
            path: "/",
        }
    }));

    session.Session.prototype.login = function(user, cb){
        this.userInfo = user;
        cb();
    };
}
What’s Going On Here

We're importing the session function from the express-session NPM module and passing the session function a configuration object to set properties such as:

  • **Store. **I’m using MongoDB as my backend, and I want to persist the application sessions in my database, so I am using the connect-mongo NPM module and setting the session store value to an instance of this module. However, you might be using a different backend, so your store option could be different. The default for express-session is an in-memory storage.

  • Secret. This is a value used in the signing of the session ID that is stored in the cookie.

  • Cookie. This determines the behavior of the HTTP cookie that stores the session ID.

We will come back to some of the elements I didn’t mention here shortly.  For now, let’s look at the first change we need to make with securely managing user sessions in our application.

Session Hijacking

The most prevalent risk that user sessions face is session hijacking. Sessions are much like a driver’s license or passport and provide identification for our users. If an attacker can steal the session of another user, they have essentially become that other person and can exploit the user or perform malicious activity on behalf of that user. The risk is even greater when the identity is someone with escalated privilege such as a site admin.

Session Identification

The first step that any attacker will perform is reconnaissance to determine where vulnerabilities lie in your application. Part of that reconnaissance is observing tell-tale signs of the underlying framework, third-party modules and any other software that in itself might contain vulnerabilities that can be exploited.

In the case of our sessions, a tell-tale sign is the name of the session cookie connect.sid, which can help an attacker identify the session mechanism being used and look for specific vulnerabilities.

Tip: Of course we don’t want to use vulnerable software. However, secure software today could be vulnerable tomorrow with a faulty update. So, keeping details about our application obfuscated can help make it that much more difficult for an attacker to exploit.

Therefore, the first thing we want to do is make that as hard to determine what session mechanism is used as possible, let’s update our session configuration object with a name property:

 app.use(session({
        store: new MongoStore({
            dbPromise: connectionProvider(serverSettings.serverUrl, serverSettings.database)
        }),
        secret: serverSettings.session.password,
        saveUninitialized: true,
        resave: false,
        cookie: {
            path: "/",
        }

       name: "id"
    }));

What Did We Do?

By providing a name property with a value of ID, it will be that much more difficult for any attacker to determine the underlying mechanisms used by our application.

Now that we have provided a level of obfuscation to our sessions, let’s look at how we can reduce the window of opportunity for session hijacking.

Session Time-to-Live

Unfortunately, sometimes the best-laid plans can be undermined. A perfect example is a user who didn’t log off a public computer and an attacker who was able to physically obtain access and operate as the previous user.

Therefore, we can reduce the window that the session is alive and directly impact the chances that an attacker can exploit a user or the system from a hijacked session by limiting the life of a session.

As I mentioned before, the express-sessions NPM module provides a store property where you can set a separate storage mechanism for storing your sessions (the default is in-memory). Therefore, the following change is tied to your backend storage of sessions. In my case, I am storing my sessions in a MongoDB database, and using the NPM module connect-mongo for easily storing sessions in the database.

In this case, I can provide the ttl property a value in seconds in the configuration object provided to the MongoStore, the default is 14 days (142460*60):

app.use(session({
   store: new MongoStore({
       dbPromise: connectionProvider(serverSettings.serverUrl, serverSettings.database),
       ttl: (1 * 60 * 60)
   }),
   //remaining - removed for brevity
}));

Cookie Time-to-Live

Not nearly as important as the session’s TTL, we can also set the expiration of the cookie, which is used for transporting the session ID, in the session configuration object.
We can provide a maxAge property and value in milliseconds in the cookie object.

app.use(session({
       //..previous removed for brevity

        cookie: {
            path: "/“,
       maxAge:  1800000  //30 mins
        }
}));

However, security and user experience is always a balancing act and in this case, reducing the time-to-live of the session will directly affect the user experience of when the user will be required to re-authenticate.

Tip: The most important thing is the life of the session, so whether you set a cookie’s age, you should never rely on it by itself and should always regulate the session’s time-to-live. One way to counterbalance the user’s experience is to require re-authenticating at the time that user’s attempt to access key access area’s of your site.

Regenerating New Sessions

It’s common practice to associate a session with an anonymous user, one who hasn’t authenticated with your application. However, when a user does successfully authenticate with your application, it is absolutely paramount that the authenticated user doesn’t continue to use the same session ID.

There are a number of creative ways attackers can obtain an authenticated user’s session, especially when sites make it easy by transporting the session ID in the URL.  In the case that an unauthenticated session had been hijacked by an attacker, when the legitimate user authenticates with the site, and the site has allowed the user to continue using the same session ID, the malicious user will also find that the session they had hijacked is now an authenticated session, allowing them to operate as the user.

You’ll notice in the original function where we wired up our Sessions, we’re adding a Login method to the Session’s prototype so it’s available to all instances of a Session object.

session.Session.prototype.login = function(user){
        this.userInfo = user;
};

This is simply a convenience method that I can access to associate user information with a user session, such as in the case of an authentication route API. For example, like we saw back in Node.js and Password Storage with Bcrypt, following a successful login, we have access to the session object off the request.

authenticationRouter.route("/api/user/login")
    .post(async function (req, res) {
        try {

          //removed for brevity….

          req.session.login(userInfo, function(err) {
            if (err) {
               return res.status(500).send("There was an error logging in. Please try again later.");
            }
          });
        //removed for brevity….
     });

However, it’s absolutely paramount not to continue using the same session ID after a user has successfully authenticated.  The express-sessions module provides a convenient regenerate method for regenerating the session ID. Our loginprototype method off of the Session object is a convenient place for regenerating the Session ID, so let's update:

session.Session.prototype.login = function (user, cb) {
   const req = this.req;
   req.session.regenerate(function(err){
      if (err){
         cb(err);
      }
   });

   req.session.userInfo = user;
   cb();
};

Now, when we provide information to be associated with a user’s session following the user’s successful authentication with our application, we can also ensure that a new session has been generated for this user.

Since cookies are what we use to transport our sessions, it’s important to implement secure cookies.  Let’s look at how we can do that next.

Cookies

We use sessions to maintain state between user requests and we use cookies to transport the session ID between those requests. You probably don’t drive through shady neighborhoods without locking your doors, nor should you throw your sessions out in the wild without any protection.

There are primarily 3 ways to protect the cookie, 2 that we will look at here, and a third we’ll examine in the next section when we look at serving our application content over HTTPS.

HTTPOnly Flag

Since the session ID is of no use to the client, there is absolutely no reason that the front-end application should ever have access to the session ID in the cookie.  However, by default, any script running on the front-end application will have access to a cookie’s contents.

We can limit access to our session cookie’s content by issuing the HTTP Set-cookie header and specifying the HTTPOnly flag for our session cookie.

HTTP header:

Set-cookie: mycookie=value; path=/; HttpOnly

We can easily provide this header by simply setting the httpOnly property on our cookie object to “true:”

app.use(session({
       //..previous removed for brevity

        cookie: {
            path: “/“,
            httpOnly: true,
       maxAge:  1800000
        }
}));

Now, only the agent (i.e., browser) will have access to the cookie in order to resubmit it on the next request to the same domain. This will directly help mitigate cross-site scripting threats that could have otherwise accessed the contents of our session cookie.

Secure Flag

With the HTTPOnly flag set, we limited application scripts from accessing our cookies at the front-end application, but what we haven’t stopped is prying eyes.

Man-in-the-middle (MitM) attacks are common and can easily be carried out by anyone that has access to the network traffic.  This goes along with any local coffee house WIFI that users might typically use.  If your session information is sent over the network without being encrypted, that information is available to anyone listening to the network traffic.

In addition to the HTTPOnly flag we specified, we can also set the Secure flag on our Set-CookieHTTP Header. This will notify the agent (i.e., browser) that we don’t want to send our cookie on any HTTP requests unless it is a secure connection.

HTTP header:

Set-cookie: mycookie=value; path=/; HttpOnly secure

Again, we can easily do so by setting the secure property on our cookie object to “true:”

app.use(session({
       //..previous removed for brevity

        cookie: {
            path: “/“,
            httpOnly: true,
            secure: true,
            maxAge:  1800000
        }
}));

In addition, a common scenario that the secure flag helps prevent is the risk of mixed content. It’s quite common to easily have a page or site setup to use secure HTTP (HTTPS), but an individual resource on that page is being requested insecurely over HTTP.

Without the Secure flag, that HTTP request would have leaked our session cookie on that insecure request for that resources.

Tip: Content Security Policies can help report when a page consist of mixed content. In a future post, we’ll look at how content security policies (CSP) can help to address mixed-content issues.

Unfortunately, with the Secure flag set, unless we are serving our page or site over HTTPS, we have eliminated the ability to send our session cookie on each request and that’s definitely not something we want. So, in the final section, let’s look at the importance of using Transport Layer Security (TLS) when it comes to securely managing our user sessions.

Transport Layer Security

We mentioned the risk of man-in-the-middle attacks earlier and how anyone with the means can easily listen to all network traffic. However, because the traffic is insecure and not encrypted, they can also eavesdrop on the information in that traffic.

Insecure Traffic for Man-in-the-middle Attacks

I have already covered the underlying details about TLS and since this isn’t strictly an article on TLS, I’m not going to rehash those details here. However, no topic on session management would be complete without mentioning the role and importance of TLS when it comes to securing user sessions.

If we only ever want to send our highly sensitive session cookies over a secure connection, then we must serve those requests and pages over an HTTP connection using Transport Layer Security (HTTPS).

TLS provides a number of benefits such as integrity of the information being exchanged and validity of the server you’re communicating with. In addition, transport layer security provides confidentiality of the information being exchanged through the means of encryption. Serving your site's content over TLS can ensure that your sensitive session cookie is encrypted and not viewable by the prying eyes of a man-in-the-middle attack.

In a future post, we’ll look at the details of serving our Node.js application over HTTPS.

Conclusion

Sessions are still a highly prominent tool for maintaining user state in a very stateless environment. However, sessions are usually tightly tied to sensitive data such as authentication and identification information regarding the user. Therefore, it is absolutely paramount to implement the necessary mitigations to protect against risks such as session hijacking as well as related threats that can lead to sessions hijacking such as man-in-the-middle and cross-site scripting attacks.

Thank you for reading !