Building a Multi-tenant Application on Node.js

Introduction

Multi-tenancy is a software architecture where a single instance of software runs on a server and serves multiple tenants. A good example would be Github where each user or organization has their separate work area. This concept is used while developing software that runs for different organizations.
The following image shows the two architecture for separating data. Both strategies that can be used to design your software and comes with their unique set of nuances.

Multi-tenancy is a key concept in building and delivering software-as-a-service (SaaS)solutions.

Why Multitenancy?

• Allows segregation of data across the clients.

• Manage and customize the app according to the client’s needs and feasibility without actually altering the source code. A simple example would be providing separate branding for each tenant or using feature flags to provide different features for each tenant.

• We can maximize efficiency while reducing the cost needed for centralized updates and maintenance. Maintenance for a multitenant application is easier as the updates applied to the system affects all the tenants and the upgrades and maintenance are usually handled by the SaaS company and not the individual customers.

Methods of Tenancy
When designing a multi-tenant SaaS application, you must carefully choose the tenancy design that best fits the needs of your application. A tenancy design determines the approach in managing, accessing, and separating tenant data.

I’d like to share two basic models that are commonly used when partitioning tenant data in a SaaS environment.

1. Logical Separation of Data: In this approach, there is only one database for all tenants. Their data is separated by using some unique identifier for each client. The codebase is responsible for storing and retrieving data using this unique identifier. This blog explains more about how we can implement a logical separation of data.

2. Physical Separation of Data: This approach separates the data by provisioning different database for different tenants/clients. This helps us to scale our application as the number of clients grows and also scale the database as per the clients need.

In this guide, we’ll be talking about multitenant architecture with physical separation of data.

Getting Started

Let’s dive into the implementation of the multi-tenant system. We will create an app using NodeJS and ExpressJS that will identify the tenant from each request and provide the data for that particular tenant database.

Let start with a few definitions:

• Tenant: A group of users with the specific privilege to their data.

• Connection Resolver: An utility that identifies a tenant and resolves a database connection pool for that particular tenant.

• Common Connection: An object which holds the information about the connection to a common database.

• Tenant Connection: An object which holds the information about the connection to tenant database.

• Common DB: Database which holds the information about all the tenant databases. It also stores the configuration for each tenant databases which can be used to enable/disable features for each tenant.

• Tenant DB: Separate database for every tenant that holds data as per tenant needs.

Prerequisites

• Make sure you have NodeJS and NPM setup in your system.

• Setup the backend application and install ExpressJS

• Make sure you have PostgreSQL. Reference:

Installing Required Packages

First, let’s add packages which we require for setting up the application. Go to your project root and initialize a new project using the command yarn init or npm init. Then run the following scripts:x

yarn add dotenv body-parser knex pg # npm install dotenv body-parser knex pg --save

We are using knex as a query builder and pg as the database library as we are connecting to PostgreSQL Server Database. You can use any database and its library for your purpose.

Modernizing the Script Using Babel

We all want to modernize our code and to do so we need to add and configure Babel for the project.

1. Add the required babel packages as the dev dependencies

yarn add @babel/cli @babel/core @babel/node @babel/preset-env — dev

2. Create a file called .babelrc and keep it on your project root directory. Add the following snippet to this file to enable transforms for ES2015+

{
  "presets": ["@babel/preset-env"]
}

3. Add the following start script in package.json

...
"scripts": {
  "start": "nodemon --watch src --exec babel-node src"
}
...

Booting up the App

Now let’s create a file index.js inside the src folder. We will initialize the express application and create a basic route.

import express from 'express';
import bodyParser from 'body-parser';

const PORT = 8080;

const app = express();

app.set('port', PORT);
app.use(bodyParser.json());

// API Route
app.get('/', (req, res, next) => {
  res.json({ body: 'Hello multi-tenant application.' });
});

app.listen(PORT, () => {
  console.log(`Express server started at port: ${PORT}`);
});

Run the application using the command yarn start.

Database Configuration

Metadata of the tenant database
Now we need to create the databases: common_db, tenant1_db and tenant2_db.

CREATE DATABASE common_db;
CREATE DATABASE tenant1_db;
CREATE DATABASE tenant2_db;

Make sure you create different users and credentials for these databases.
Let’s create a table in common_db which will hold connection information for the tenant databases.

CREATE TABLE tenants (
  id SERIAL PRIMARY KEY,
  slug VARCHAR(255) UNIQUE NOT NULL,
  db_name VARCHAR(100) UNIQUE NOT NULL,
  db_host VARCHAR(255),
  db_username VARCHAR(100),
  db_password TEXT,
  db_port INTEGER NOT NULL DEFAULT 5432,
  created_at TIMESTAMP DEFAULT NOW(),
  updated_at TIMESTAMP DEFAULT NOW()
);

INSERT INTO tenants (slug, db_name, db_host, db_username, db_password, db_port) VALUES
  ('tenant1', 'tenant1_db', 'localhost', 'tenant1_user', '***encrypted_password_for_tenant_1***', 5432),
  ('tenant2', 'tenant2_db', 'localhost', 'tenant2_user', '***encrypted_password_for_tenant_2***', 5432);

Connection Configuration for Common Database

We will now create an instance of knex and provide connection configuration of the common database. Make sure you have created a .env file in your project root.

# Database Connection
DB_CLIENT=pg
DB_USER=common_db_user
DB_PORT=1433
DB_HOST=localhost
DB_DATABASE=common_db
DB_PASSWORD=SOME_STRONG_PASSWORD

Create a file env.js inside src and add the following snippet.

import dotenv from 'dotenv';
dotenv.config();

Add the following snippet in a file commonDBConnection.js.

import knex from 'knex';

const knexConfig = {
  client: process.env.DB_CLIENT,
  connection: {
    user: process.env.DB_USER,
    host: process.env.DB_HOST,
    port: process.env.DB_PORT,
    database: process.env.DB_DATABASE,
    password: process.env.DB_PASSWORD
  },
  pool: { min: 2, max: 20 }
};

export default knex(knexConfig);

Adding a Special Package

We need to connect to the correct tenant database based on which tenant has requested. One way of doing this is to pass the information about the tenant (that is received from the request) to every underlying service. This approach is not clean and also introduces redundancy.

Hence we will use a context that will persist across the async task. Let’s use a package called continuation-local-storage.

This package works like thread-local storage in threaded programming but is based on chains of Node-style callbacks instead of threads.

yarn add continuation-local-storage

Adding a Service to Manage Connections

Let’s create util called connectionManager.js that will help us connect to our tenant databases and resolve the connection.

First, let’s add functions to connect to all the tenant databases.

import knex from 'knex';

import commonDBConnection from './commonDBConnection';

let connectionMap;

/**
 *  Create knex instance for all the tenants defined in common database and store in a map.
**/
export async function connectAllDb() {
  let tenants;

  try {
    tenants = await commonDBConnection.select('*').from('tenants');
  } catch (e) {
    console.log('error', e);

    return;
  }

  connectionMap =
    tenants
      .map(tenant => {
        return {
          [tenant.slug]: knex(createConnectionConfig(tenant))
        }
      })
      .reduce((prev, next) => {
        return Object.assign({}, prev, next);
      }, {});
}

/**
 *  Create configuration object for the given tenant.
**/
function createConnectionConfig(tenant) {
  return {
    client: process.env.DB_CLIENT,
    connection: {
      host: tenant.db_host,
      port: tenant.db_port,
      user: tenant.db_username,
      database: tenant.db_name,
      password: tenant.db_password
    },
    pool: { min: 2, max: 20 }
  };
}

Now we need to add functions to this file that will return a connection from the connectionMap we created earlier.

...
...
import { getNamespace } from 'continuation-local-storage';

/**
 *  Get the connection information (knex instance) for the given tenant's slug.
**/
export function getConnectionBySlug(slug) {
  if (connectionMap) {
    return connectionMap[slug];
  }
}

/**
 *  Get the connection information (knex instance) for current context.
**/
export function getConnection() {
  const nameSpace = getNamespace('unique context');
  const conn = nameSpace.get('connection');

  if (!conn) {
    throw 'Connection is not set for any tenant database.';
  }

  return conn;
}

We have now created a very sophisticated connection manager. To understand how we were able to use nameSpace.get('connection'), let’s see how we set it.

Creating Connection Resolver

Create a middleware to resolve the connection called connectionResolver.js inside the directory src/middlewares. This would be our middleware that figures out which connection to use throughout the request.

import { createNamespace } from 'continuation-local-storage';

import { getConnectionBySlug } from '../connectionManager';

// Create a namespace for the application.
let nameSpace = createNamespace('unique context');

/**
 * Get the connection instance for the given tenant's slug and set it to the current context.
**/
export function resolve(req, res, next) {
  const slug = req.query.slug;

  if (!slug) {
    res.json({ message: `Please provide tenant's slug to connect.` });
    return;
  }

  // Run the application in the defined namespace. It will contextualize every underlying function calls.
  nameSpace.run(() => {
    nameSpace.set('connection', getConnectionBySlug(slug)); // This will set the knex instance to the 'connection'
    next();
  });
}

Adding Data for Tenants

Let’s add some data to the tenants. In this example, we will do this writing a custom query. We will create a table called users and add some data to it. You can do this by adding migrations to the application and running the migration to the tenant databases.

-- Select the tenant1 database.
USE tenant1_db;

CREATE TABLE users(
  id INT IDENTITY(1, 1) PRIMARY KEY,
  first_name VARCHAR(255) NOT NULL,
  last_name VARCHAR(255) NOT NULL,
  email VARCHAR(255) NOT NULL,
  address VARCHAR(400)
);

INSERT INTO users(first_name, last_name, email, address) VALUES
 ('John', 'Doe', 'johndoe@email.com', 'USA');

-- Select the tenant2 database.
USE tenant2_db;

CREATE TABLE users(
  id SERIAL PRIMARY KEY,
  first_name VARCHAR(255) NOT NULL,
  last_name VARCHAR(255) NOT NULL,
  email VARCHAR(255) NOT NULL,
  address VARCHAR(400)
);
INSERT INTO users(first_name, last_name, email) VALUES
 ('Jane', 'Doe', 'janedoe@email.com');

Create a Service to Get User Information

We can now query from these databases. Let’s make use of the connectionResolver.js and connectionManager.js we created before.

1. Create a service users.js inside src/services

import { getConnection } from '../connectionManager';

/**
* Get all the users.
**/
export async function getAll(req, res, next) {
  res.json({ body: await getConnection().select('*').from('users') });
}

As shown in the above code, we now don’t need to let the services know what connection we are using. Now, this is the kind of abstraction we love in code. You can further abstract it by creating your own models and refactoring the connection manager.

2. Add the following code to index.js

import './env';

...
...

import * as userService from './services/users';
import { connectAllDb } from './connectionManager';
import * as connectionResolver from './middlewares/connectionResolver';

...
...

app.use(bodyParser.json());

connectAllDb();

app.use(connectionResolver.resolve);

...
...
// API Route for getting users
app.get('/users', userService.getAll);

...
...

All Done
We are now at the end. We can start our application by using the command yarn start. Let’s perform some API calls in the following routes:

• localhost:8080/users?slug=tenant1

• localhost:8080/users?slug=tenant2

This should now respond the right users for the given tenant.

Here is the complete code for this implementation → Multitenant Application Example!

Conclusion

We hope this post has helped you understand one of the architectures used in multi-tenant applications. This implementation will help you get started with an application which you can scale in the future. However, this post doesn’t cover the techniques of securing your API requests or how we can implement such APIs in the frontend.

Thanks for reading

#node-js #express