How to cluster Node.js app in multiple machines.Node.js has an amazing capacity of handling high loads, but it wouldn’t be wise to stop a request until the server finishes its process.
Scalability is a hot topic in tech, and every programming language or framework provides its own way of handling high loads of traffic.
Today, we’re going to see an easy and straightforward example about Node.js clustering. This is a programming technique which will help you parallelize your code and speed up performance.
“A single instance of Node.js runs in a single thread. To take advantage of multi-core systems, the user will sometimes want to launch a cluster of Node.js processes to handle the load.”
The complete example is available in this Github repository.
We’ll build a simple web server which will act as follows:
POSTrequest, we’ll pretend that user is sending us a picture.
Of course, this is not a real world application, but is pretty close to one. We just want to measure the benefits of using clustering.
I’m gonna use
yarn to install my dependencies and initialize my project:
Since Node.js is single threaded, if our web server crashes, it will remain down until some other process will restarts it. So we’re gonna install forever, a simple daemon which will restart our web server if it ever crashes.
We’ll also install Jimp, Koa and Koa Router.
This is the folder structure we need to create:
We’ll have an
The first one will be the file where we’ll experiment with the
cluster module. The second is a simple Koa server which will work without any clustering.
module directory, we’re gonna create two files:
job.js will perform the image manipulation work.
log.js will log every event that occurs during that process.
Log module will be a simple function which will take an argument and will write it to the
stdout (similar to
It will also append the current timestamp at the beginning of the log. This will allow us to check when a process started and to measure its performance.
I’ll be honest, this is not a beautiful and super-optimized script. It’s just an easy job which will allow us to stress our machine.
We’re gonna create a very simple webserver. It will respond on two routes with two different HTTP methods.
We’ll be able to perform a GET request on
[http://localhost:3000/](http://localhost:3000/.). Koa will respond with a simple text which will show us the current PID (process id).
The second route will only accept POST requests on the
/flip path, and will perform the job that we just created.
We’ll also create a simple middleware which will set an
X-Response-Time header. This will allow us to measure the performance.
Great! We can now start our server typing
node ./src/standard.js and test our routes.
The image I am currently manipulating (via Unsplash)
Let’s use my machine as a server:
If I make a POST request, the script above will send me a response in ~3800 milliseconds. Not so bad, given that the image I am currently working on is about 6.7MB.
I can try making more requests, but the response time won’t decrease too much. This is because the requests will be performed sequentially.
So, what would happen if I tried to make 10, 100, 1000 concurrent requests?
I made a simple Elixir script which performs multiple concurrent HTTP requests:
I chose Elixir because it’s really easy to create parallel processes, but you can use whatever you prefer!
As you can see, we spawn 10 concurrent processes from our iex (an Elixir REPL).
The Node.js server will immediately copy our image and start to flip it.
The first response will be logged after 16 seconds and the last one after 40 seconds.
Such a dramatic performance decrease! With just 10 concurrent requests,we decreased the webserver performance by 950%!
All credits to Pexels
Remember what I mentioned at the beginning of the article?
To take advantage of multi-core systems, the user will sometimes want to launch a cluster of Node.js processes to handle the load.
Depending on which server we’re gonna run our Koa application, we could have a different number of cores.
Every core will be responsible for handling the load individually. Basically, each HTTP request will be satisfied by a single core.
So for example — my machine, which has eight cores, will handle eight concurrent requests.
We can now count how many CPUs we have thanks to the
cpus() method will return an array of objects that describe our CPUs. We can bind its length to a constant which will be called
numWorkers, ’cause that’s the number of workers that we’re gonna use.
We’re now ready to require the
We now need a way of splitting our main process into
N distinct processes.
We’ll call our main process
master and the other processes
cluster module offers a method called
isMaster. It will return a boolean value that will tell us if the current process is directed by a worker or master:
Great. The golden rule here is that we don’t want to serve our Koa application under the master process.
We want to create a Koa application for each worker, so when a request comes in, the first free worker will take care of it.
cluster.fork() method will fit our purpose:
Ok, at first that may be a little tricky.
As you can see in the script above, if our script has been executed by the master process, we’re gonna declare a constant called
workers. This will create a worker for each core of our CPU, and will store all the information about them.
If you feel unsure about the adopted syntax, using
[…Array(x)].map() is just the same as:
I just prefer to use immutable values while developing a high-concurrency app.
All credit to Pexels
As we said before, we don’t want to serve our Koa application under the master process.
Let’s copy our Koa app structure into the
else statement, so we will be sure that it will be served by a worker:
As you can see, we also added a couple of event listeners in the
The first one will tell us that a new worker has been spawned. The second one will create a new worker when one other worker crashes.
That way, the master process will only be responsible for creating new workers and orchestrating them. Every worker will serve an instance of Koa which will be accessible on the
As you can see, we got our first response after about 10 seconds, and the last one after about 14 seconds. It’s an amazing improvement over the previous 40 second response time!
We made ten concurrent requests, and the Koa server took eight of them immediately. When the first worker has sent its response to the client, it took one of the remaining requests and processed it!
Node.js has an amazing capacity of handling high loads, but it wouldn’t be wise to stop a request until the server finishes its process.
In fact, Node.js webservers can handle thousands of concurrent requests only if you immediately send a response to the client.
A best practice would be to add a pub/sub messaging interface using Redis or any other amazing tool. When the client sends a request, the server starts a realtime communication with other services. This takes charge of expensive jobs.
Load balancers would also help a lot splitting out high traffic loads.
Once again, technology is giving us endless possibilities, and we’re sure to find the right solution to scale our application to infinity and beyond!
Node.js for Beginners - Learn Node.js from Scratch (Step by Step) - Learn the basics of Node.js. This Node.js tutorial will guide you step by step so that you will learn basics and theory of every part. Learn to use Node.js like a professional. You’ll learn: Basic Of Node, Modules, NPM In Node, Event, Email, Uploading File, Advance Of Node.Node.js for Beginners
Welcome to my course "Node.js for Beginners - Learn Node.js from Scratch". This course will guide you step by step so that you will learn basics and theory of every part. This course contain hands on example so that you can understand coding in Node.js better. If you have no previous knowledge or experience in Node.js, you will like that the course begins with Node.js basics. otherwise if you have few experience in programming in Node.js, this course can help you learn some new information . This course contain hands on practical examples without neglecting theory and basics. Learn to use Node.js like a professional. This comprehensive course will allow to work on the real world as an expert!
What you’ll learn:
Node is fast, scalable, and easy to get started with. Its default package manager is npm, which means it also sports the largest ecosystem of open-source libraries. Node is used by companies such as NASA, Uber, Netflix, and Walmart.
But Node doesn't come alone. It comes with a plethora of frameworks. A Node framework can be pictured as the external scaffolding that you can build your app in. These frameworks are built on top of Node and extend the technology's functionality, mostly by making apps easier to prototype and develop, while also making them faster and more scalable.
Below are 7of the most popular Node frameworks at this point in time (ranked from high to low by GitHub stars).Express
With over 43,000 GitHub stars, Express is the most popular Node framework. It brands itself as a fast, unopinionated, and minimalist framework. Express acts as middleware: it helps set up and configure routes to send and receive requests between the front-end and the database of an app.
Express provides lightweight, powerful tools for HTTP servers. It's a great framework for single-page apps, websites, hybrids, or public HTTP APIs. It supports over fourteen different template engines, so developers aren't forced into any specific ORM.Meteor
The project has over 41,000 GitHub stars and is built to power large projects. Meteor is used by companies such as Mazda, Honeywell, Qualcomm, and IKEA. It has excellent documentation and a strong community behind it.Koa
Koa is built by the same team that built Express. It uses ES6 methods that allow developers to work without callbacks. Developers also have more control over error-handling. Koa has no middleware within its core, which means that developers have more control over configuration, but which means that traditional Node middleware (e.g. req, res, next) won't work with Koa.
Koa already has over 26,000 GitHub stars. The Express developers built Koa because they wanted a lighter framework that was more expressive and more robust than Express. You can find out more about the differences between Koa and Express here.Sails
Sails is a real-time, MVC framework for Node that's built on Express. It supports auto-generated REST APIs and comes with an easy WebSocket integration.
The project has over 20,000 stars on GitHub and is compatible with almost all databases (MySQL, MongoDB, PostgreSQL, Redis). It's also compatible with most front-end technologies (Angular, iOS, Android, React, and even Windows Phone).Nest
Nest is packaged in such a way it serves as a complete development kit for writing enterprise-level apps. The framework uses Express, but is compatible with a wide range of other libraries.LoopBack
LoopBack is a framework that allows developers to quickly create REST APIs. It has an easy-to-use CLI wizard and allows developers to create models either on their schema or dynamically. It also has a built-in API explorer.
LoopBack has over 12,000 GitHub stars and is used by companies such as GoDaddy, Symantec, and the Bank of America. It's compatible with many REST services and a wide variety of databases (MongoDB, Oracle, MySQL, PostgreSQL).Hapi
Similar to Express, hapi serves data by intermediating between server-side and client-side. As such, it's can serve as a substitute for Express. Hapi allows developers to focus on writing reusable app logic in a modular and prescriptive fashion.
The project has over 11,000 GitHub stars. It has built-in support for input validation, caching, authentication, and more. Hapi was originally developed to handle all of Walmart's mobile traffic during Black Friday.
2:32 What is Node.js?
3:22 Client-Server Architecture
4:12 Multi-Threaded Model
6:13 Single-Threaded Model
7:43 Multi-Threaded vs Event-Driven
9:45 Uber Old Architecture
11:10 Uber New Architecture
12:30 What is Node.js?
13:05 Sucess Stories
14:20 Node.js Trend
14:40 Node.js Features
16:25 Node.js Installation
16:50 Node.js First Example
17:30 Blocking vs Non-blocking
23:50 Node.js Modules
25:10 Global Objects
26:55 File System
34:50 Hands On
1:09:45 Node.js Tutorial
1:10:45 What is Node.js?
1:12:10 Features of Node.js
1:13:00 Node.js Architecture
1:14:55 NPM(Node Package Manager)
1:16:20 Node.js Modules
1:16:30 Node.js Modules Types
1:16:35 Core Modules
1:16:55 Local Modules
1:17:10 3rd Party Modules
1:18:35 JSON File
1:23:30 Data Types
1:29:55 File Systems
1:34:20 HTTP Module
1:44:37 HTTP Module
1:45:27 Creating a Web Server using Node.js
1:58:37 Node.js NPM Tutorial
1:59:37 What is NPM?
2:03:12 Main Functions of NPM
2:04:27 Need For NPM
2:08:07 NPM Packages
2:17:42 NPM Installation
2:18:12 JSON File
2:31:32 Node.js Express Tutorial
2:32:02 Introduction to Express.js
2:32:32 Features of Express.js
2:35:27 Getting Started with Express.js
2:39:42 Routing Methods
2:48:12 Building RESTful API with Node.js
2:48:27 What is REST API?
2:49:42 Features of REST API
2:51:12 Principles of REST API
2:56:37 Methods of REST API
2:59:52 Building REST API with Node.js
3:24:07 Node.js MySQL Tutorial
3:24:32 What is MySQL?
3:25:13 Advantages of Using MySQL with Node.js
3:27:38 MySQL Installation
3:44:23 Node.js MongoDB Tutorial
3:44:58 What is NoSQL?
3:47:53 NoSQL Databases
3:48:38 Introduction to MongoDB
3:52:48 Features of MongoDB
3:53:03 MongoDB Installation
4:36:08 Node.js Docker Tutorial
4:36:38 What is Docker?
4:39:13 Docker Working
4:41:43 Docker Basics
4:42:03 Docker Images
4:42:23 Docker Container
4:44:38 Why use Node.js with Docker?
4:45:18 Demo: Node.js with Docker
4:58:38 MEAN Stack Application Tutorial
4:59:18 What is MEAN Application?
5:02:17 RESTful API
5:03:02 Contact List MEAN App
6:17:57 Node.js Interview Questions
Got a question on the topic? Please share it in the comment section below and our experts will answer it for you.