Docker Best Practices for Node Developers

Docker Best Practices for Node Developers

Welcome to the "Docker Best Practices for Node Developers"! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

Welcome to the best course on the planet for using Docker with Node.js! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

My talk on all the best of Docker for Node.js developers and DevOps dealing with Node apps. From DockerCon 2019. Get the full 9-hour training course with my coupon at http://bit.ly/365ogba

Get the source code for this talk at https://github.com/BretFisher/dockercon19

Some of the many cool things you'll do in this course
  • Build Node.js Images that auto-scan for security vulnerabilities
  • Use Docker's cutting-edge BuildKit with SSH Agents and NPM Caches for better image building
  • Use docker-compose with Visual Studio Code for full Node.js debug support
  • Use BuildKit and Multi-stage Builds to create minimal and flexible Dockerfiles
  • Build custom Node.js images using distro's like CentOS and Alpine
  • Test Docker init, tini, and Node.js as a PID 1 process in containers
  • Create Node.js apps that properly startup and respond to healthchecks
  • Develop ARM based Node.js apps with Docker Desktop, and deploy to AWS A1 Servers
  • Build graceful shutdown code into your apps for zero-downtime deploys
  • Dig into HTTP connections with orchestration, and how Proxies can help
  • Study examples of Docker Swarm and Kubernetes deployments for Node.js
  • Spend time Migrating traditional (legacy) Node.js apps into containers
  • Simplify your microservice solutions with advanced Docker Compose features
What you will learn in this course

You'll start with a quick review about getting set up with Docker, as well as Docker Compose basics. That way we're on the same page for the basics.

Then you'll jump into Node.js Dockerfile basics, that way you'll have a good Dockerfile foundation for new features we'll add throughout the course.

You'll be building on all the different things you learn from each Lecture in the course. Once you have the basics down of Compose, Dockerfile, and Docker Image, then you'll focus on nuances like how Docker and Linux control the Node process and how Docker changes that to make sure you know what options there are for starting up and shutting down Node.js and the right way to do it in different scenarios.

We'll cover advanced, newer features around making the Dockerfile the most efficient and flexible as possible using things like BuildKit and Multi-stage.

Then we'll talk about distributed computing and cloud design to ensure your Node.js apps have 12-factor design in your containers, as well as learning how to migrate old apps into this new way of doing things.

Next we cover Compose and its awesome features to get really efficient local development and test set-up using the Docker Compose command line and Docker Compose YAML file.

With all this knowledge, you'll progress to production concerns and making images production-ready.

Then we'll jump into deploying those containers and running them in production. Whether you use Docker Engine or orchestration with Kubernetes or Swarm, I've got you covered. In addition, we'll cover HTTP connections and reverse proxies for connection handling and routing with multi-container systems.

Lastly, you'll get a final, big assignment where you'll be building and deploying a large, complex solution, including multiple Node.js containers that are doing different things. You'll build Docker images, Dockerfiles, and compose files, and deploy them to a server to test. You'll need to check whether connections failover properly. You'll basically take everything you've learned and apply it in one big project!

How To Build a Node.js Application in Docker

How To Build a Node.js Application in Docker

In this article, you'll learn how to build a Node.js Application in Docker

In this tutorial, we’ll set up the socket.io chat example with Docker, from scratch to production-ready. In particular, we’ll see how to:

  • Get started bootstrapping a node application with Docker.
  • Not run everything as root (bad!).
  • Use binds to keep your test-edit-reload cycle short in development.
  • Manage node_modules in a container (there’s a trick to this).
  • Ensure repeatable builds with package-lock.json.
  • Share a Dockerfile between development and production using multi-stage builds.

This tutorial assumes you already have some familiarity with Docker and node. If you’d like a gentle intro to Docker first, I’d recommend running through Docker’s official introduction.

Getting Started

We’re going to set things up from scratch. The final code is available on github here, and there are tags for each step along the way. Here’s the code for the first step, in case you’d like to follow along.

Without Docker, we’d start by installing node and any other dependencies on the host and running npm init to create a new package. There’s nothing stopping us from doing that here, but we’ll learn more if we use Docker from the start. (And of course the whole point of using Docker is that you don’t have to install things on the host.) We’ll start by creating a “bootstrapping container” that has node installed, and we’ll use it to set up the npm package for the application.

The Bootstrapping Container and Service

We’ll need to write two files, a Dockerfile and a docker-compose.yml, to which we’ll add more later on. Let’s start with the bootstrapping Dockerfile:

FROM node:10.16.3

USER node

WORKDIR /srv/chat

It’s a short file, but there already some important points:

  1. It starts from the official Docker image for the latest long term support (LTS) node release, at time of writing. I prefer to name a specific version, rather than one of the ‘floating’ tags like node:lts or node:latest, so that if you or someone else builds this image on a different machine, they will get the same version, rather than risking an accidental upgrade and attendant head-scratching.

  2. The USER step tells Docker to run any subsequent build steps, and later the process in the container, as the node user, which is an unprivileged user that comes built into all of the official node images from Docker. Without this line, they would run as root, which is against security best practices and in particular the principle of least privilege. Many Docker tutorials skip this step for simplicity, and we will have to do some extra work to avoid running as root, but I think it’s very important.

  3. The WORKDIR step sets the working directory for any subsequent build steps, and later for containers created from the image, to /srv/chat, which is where we’ll put our application files. The /srv folder should be available on any system that follows the Filesystem Hierarchy Standard, which says that it is for “site-specific data which is served by this system”, which sounds like a good fit for a node app 1.

Now let’s move on to the bootstrapping compose file, docker-compose.yml:

version: '3.7'

services:
  chat:
    build: .
    command: echo 'ready'
    volumes:
      - .:/srv/chat

Again there is quite a bit to unpack:

  1. The version line tells Docker Compose which version of its file format we are using. Version 3.7 is the latest at the time of writing, so I’ve gone with that, but older 3.x and 2.x versions would also work fine here; in fact, the 2.x series might even be a better fit, depending on your use case 2.

  2. The file defines a single service called chat, built from the Dockerfile in the current directory, denoted .. All the service does for now is to echo ready and exit.

  3. The volume line, .:/srv/chat, tells Docker to bind mount the current directory . on the host at /srv/chat in the container, which is the WORKDIR we set up in the Dockerfile above. This means that changes we’ll make to source files on the host will be automatically reflected inside the container, and vice versa. This is very important for keeping your test-edit-reload cycles as short as possible in development. It will, however, create some issues with how npm installs dependencies, which we’ll come back to shortly.

Now we’re ready to build and test our bootstrapping container. When we run docker-compose build, Docker will create an image with node set up as specified in the Dockerfile. Then docker-compose up will start a container with that image and run the echo command, which shows that everything is set up OK.

$ docker-compose build
Building chat
Step 1/3 : FROM node:10.16.3
# ... more build output ...
Successfully built d22d841c07da
Successfully tagged docker-chat-demo_chat:latest

$ docker-compose up
Creating docker-chat-demo_chat_1 ... done
Attaching to docker-chat-demo_chat_1
chat_1  | ready
docker-chat-demo_chat_1 exited with code 0

This output indicates that the container ran, echoed ready and exited successfully. 🎉

Initializing an npm package

⚠️ Aside for Linux users: For this next step to work smoothly, the node user in the container should have the same uid (user identifier) as your user on the host. This is because the user in the container needs to have permissions to read and write files on the host via the bind mount, and vice versa. I’ve included an appendix with advice on how to deal with this issue. Docker for Mac users don’t have to worry about it because of some uid remapping magic behind the scenes, but Docker for Linux get much better performance, so I’d call it a draw.

Now we have a node environment set up in Docker, we’re ready to set up the initial npm package files. To do this, we’ll run an interactive shell in the container for the chat service and use it to set up the initial package files:

$ docker-compose run --rm chat bash
[email protected]:/srv/chat$ npm init --yes
# ... writes package.json ...
[email protected]:/srv/chat$ npm install
# ... writes package-lock.json ...
[email protected]:/srv/chat$ exit

And then the files appear on the host, ready for us to commit to version control:

$ tree
.
├── Dockerfile
├── docker-compose.yml
├── package-lock.json
└── package.json

Here’s the resulting code on github.

Installing Dependencies

Next up on our list is to install the app’s dependencies. We want these dependencies to be installed inside the container via the Dockerfile, so the container will contain everything needed to run the application. This means we need to get the package.json and package-lock.json files into the image and run npm install in the Dockerfile. Here’s what that change looks like:

diff --git a/Dockerfile b/Dockerfile
index b18769e..d48e026 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,5 +1,14 @@
 FROM node:10.16.3

+RUN mkdir /srv/chat && chown node:node /srv/chat
+
 USER node

 WORKDIR /srv/chat
+
+COPY --chown=node:node package.json package-lock.json ./
+
+RUN npm install --quiet
+
+# TODO: Can remove once we have some dependencies in package.json.
+RUN mkdir -p node_modules

And here’s the explanation:

  1. The RUN step with mkdir and chown commands, which are the only commands we need to run as root, creates the working directory and makes sure that it’s owned by the node user.

  2. It’s worth noting that there are two shell commands chained together in that single RUN step. Compared to splitting out the commands over multiple RUN steps, chaining them reduces the number of layers in the resulting image. In this example, it really doesn’t matter very much, but it is a good habit not to use more layers than you need. It can save a lot of disk space and download time if you e.g. download a package, unzip it, build it, install it, and then clean up in one step, rather than saving layers with all of the intermediate files for each step.

  3. The COPY to ./ copies the npm packaging files to the WORKDIR that we set up above. The trailing / tells Docker that the destination is a folder. The reason for copying in only the packaging files, rather than the whole application folder, is that Docker will cache the results of the npm install step below and rerun it only if the packaging files change. If we copied in all our source files, changing any one would bust the cache even though the required packages had not changed, leading to unnecessary npm installs in subsequent builds.

  4. The --chown=node:node flag for COPY ensures that the files are owned by the unprivileged node user rather than root, which is the default 3.

  5. The npm install step will run as the node user in the working directory to install the dependencies in /srv/chat/node_modules inside the container.

This last step is what we want, but it causes a problem in development when we bind mount the application folder on the host over /srv/chat. Unfortunately, the node_modules folder doesn’t exist on the host, so the bind effectively hides the node modules that we installed in the image. The final mkdir -p node_modules step and the next section are related to how we deal with this.

The node_modules Volume Trick

There are several ways around the node modules hiding problem, but I think the most elegant is to use a volume within the bind to contain node_modules. To do this, we have to add a few lines to our docker compose file:

diff --git a/docker-compose.yml b/docker-compose.yml
index c9a2543..799e1f6 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -6,3 +6,7 @@ services:
     command: echo 'ready'
     volumes:
       - .:/srv/chat
+      - chat_node_modules:/srv/chat/node_modules
+
+volumes:
+  chat_node_modules:

The chat_node_modules:/srv/chat/node_modules volume line sets up a named volume 4 called chat_node_modules that contains the directory /srv/chat/node_modules in the container. The top level volumes: section at the end must declare all named volumes, so we add chat_node_modules there, too.

So, it’s simple to do, but there is quite a bit going on behind the scenes to make it work:

  1. During the build, npm install installs the dependencies (which we’ll add in the next section) into /srv/chat/node_modules within the image. We’ll color the files from the image blue:
/srv/chat$ tree # in image
.
├── node_modules
│   ├── accepts
...
│   └── yeast
├── package-lock.json
└── package.json
  1. When we later start a container from that image using our compose file, Docker first binds the application folder from the host inside the container under /srv/chat. We’ll color the files from the host red:
/srv/chat$ tree # in container without node_modules volume
.
├── Dockerfile
├── docker-compose.yml
├── node_modules
├── package-lock.json
└── package.json

The bad news is that the node_modules in the image are hidden by the bind; inside the container, we instead see only an empty node_modules folder on the host.

  1. However, we’re not done yet. Docker next creates a volume that contains a copy of /srv/chat/node_modules in the image, and it mounts it in the container. This, in turn, hides the node_modules from the bind on the host:
/srv/chat$ tree # in container with node_modules volume
.
├── Dockerfile
├── docker-compose.yml
├── node_modules
│   ├── accepts
...
│   └── yeast
├── package-lock.json
└── package.json

This gives us what we want: our source files on the host are bound inside the container, which allows for fast changes, and the dependencies are also available inside of the container, so we can use them to run the app.

We can also now explain the final mkdir -p node_modules step in the bootstrapping Dockerfile above: we have not actually installed any packages yet, so npm install doesn’t create the node_modules folder during the build. When Docker creates the /srv/chat/node_modules volume, it will automatically create the folder for us, but it will be owned by root, which means the node user won’t be able to write to it. We can preempt that by creating node_modules as the node user during the build. Once we have some packages installed, we no longer need this line.

Package Installation

So, let’s rebuild the image, and we’ll be ready to install packages.

$ docker-compose build
... builds and runs npm install (with no packages yet)...

The chat app requires express, so let’s get a shell in the container and npm install it with --save to save the dependency to our package.json and update package-lock.json accordingly:

$ docker-compose run --rm chat bash
Creating volume "docker-chat-demo_chat_node_modules" with default driver
[email protected]:/srv/chat$ npm install --save express
# ...
[email protected]:/srv/chat$ exit

The package-lock.json file, which has for most purposes replaced the older npm-shrinkwrap.json file, is important for ensuring that Docker image builds are repeatable. It records the versions of all direct and indirect dependencies and ensures that npm installs in Docker builds on different machines will all get the same dependency tree.

Finally, it’s worth noting that the node_modules we installed are not present on the host. There may be an empty node_modules folder on the host, which is a side effect of the binds and volumes we created, but the actual files live in the chat_node_modules volume. If we run another shell in the chat container, we’ll find them there:

$ ls node_modules
# nothing on the host
$ docker-compose run --rm chat bash
[email protected]:/srv/chat$ ls -l node_modules/
total 196
drwxr-xr-x 2 node node 4096 Aug 25 20:07 accepts
# ... many node modules in the container
drwxr-xr-x 2 node node 4096 Aug 25 20:07 vary

The next time we run a docker-compose build, the modules will be installed into the image.

Here’s the resulting code on github.

Running the App

We are finally ready to install the app, so we’ll copy in the remaining source files, namely index.js and index.html.

Then we’ll install the socket.io package. At the time of writing, the chat example is only compatible with socket.io version 1, so we need to request version 1:

$ docker-compose run --rm chat npm install --save [email protected]
# ...

In our docker compose file, we then remove our dummy echo ready command and instead run the chat example server. Finally, we tell Docker Compose to export 3000 in the container on the host, so we can access it in a browser:

diff --git a/docker-compose.yml b/docker-compose.yml
index 799e1f6..ff92767 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -3,7 +3,9 @@ version: '3.7'
 services:
   chat:
     build: .
-    command: echo 'ready'
+    command: node index.js
+    ports:
+      - '3000:3000'
     volumes:
       - .:/srv/chat
       - chat_node_modules:/srv/chat/node_modules

Then we are ready to run with docker-compose up 5:

$ docker-compose up
Recreating dockerchatdemo_chat_1
Attaching to dockerchatdemo_chat_1
chat_1 | listening on *:3000

Then you can see it running on http://localhost:3000.

Here’s the resulting code on github.

Docker for Dev and Prod

We now have our app running in development under docker compose, which is pretty cool! Before we can use this container in production, we have a few problems to solve and possible improvements to make:

  • Most importantly, the container as we’re building it at the moment does not actually contain the source code for the application — it just contains the npm packaging files and dependencies. The main idea of a container is that it should contain everything needed to run the application, so clearly we will want to change this.

  • The /srv/chat application folder in the image is currently owned and writeable by the node user. Most applications don’t need to rewrite their source files at runtime, so again applying the principle of least privilege, we shouldn’t let them.

  • The image is fairly large, weighing in at 909MB according to the handy dive image inspection tool. It’s not worth obsessing over image size, but we don’t want to be needlessly wasteful either. Most of the image’s heft comes from the default node base image, which includes a full compiler tool chain that lets us build node modules that use native code (as opposed to pure JavaScript). We won’t need that compiler tool chain at runtime, so from both a security and performance point of view, it would be better not to ship it to production.

Fortunately, Docker provide a powerful tool that helps with all of the above: multi-stage builds. The main idea is that we can have multiple FROM commands in the Dockerfile, one per stage, and each stage can copy files from previous stages. Let’s see how to set that up:

diff --git a/Dockerfile b/Dockerfile
index d48e026..6c8965d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -1,4 +1,4 @@
-FROM node:10.16.3
+FROM node:10.16.3 AS development

 RUN mkdir /srv/chat && chown node:node /srv/chat

@@ -10,5 +10,14 @@ COPY --chown=node:node package.json package-lock.json ./

 RUN npm install --quiet

-# TODO: Can remove once we have some dependencies in package.json.
-RUN mkdir -p node_modules
+FROM node:10.16.3-slim AS production
+
+USER node
+
+WORKDIR /srv/chat
+
+COPY --from=development --chown=root:root /srv/chat/node_modules ./node_modules
+
+COPY . .
+
+CMD ["node", "index.js"]

  1. Our existing Dockerfile steps will form the first stage, which we’ll now give the name development by adding AS development to the FROM line at the start. I’ve now removed the temporary mkdir -p node_modules step needed during bootstrapping, since we now have some packages installed.

  2. The new second stage starts with the second FROM step, which pulls in the slim node base image for the same node version and calls the stage production for clarity. This slim image is also an official node image from Docker. As its name suggests, it is smaller than the default node image, mainly because it doesn’t include the compiler toolchain; it includes only the system dependencies needed to run a node application, which are far fewer than what may be required to build one.

    This multi-stage Dockerfile runs npm install in the first stage, which has the full node image at its disposal for the build. Then it copies the resulting node_modules folder to the second stage image, which uses the slim base image. This technique reduces the size of the production image from 909MB to 152MB, which is about a factor of 6 saving for relatively little effort 6.

  3. Again the USER node command tells Docker to run the build and the application as the unprivileged node user rather than as root. We also have to repeat the WORKDIR, because it doesn’t persist into the second stage automatically.

  4. The COPY --from=development --chown=root:root ... line copies the dependencies installed in the preceding development stage into the production stage and makes them owned by root, so the node user can read but not write them.

  5. The COPY . . line then copies the rest of the application files from the host to the working directory in the container, namely /srv/chat.

  6. Finally, the CMD step specifies the command to run. In the development stage, the application files came from bind mounts set up with docker-compose, so it made sense to specify the command in the docker-compose.yml file instead of the Dockerfile. Here it makes more sense to specify the command in the Dockerfile, which builds it into the container.

Now that we have our multi-stage Dockerfile set up, we need to tell Docker Compose to use only the development stage rather than going through the full Dockerfile, which we can do with the target option:

diff --git a/docker-compose.yml b/docker-compose.yml
index ff92767..2ee0d9b 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -2,7 +2,9 @@ version: '3.7'

 services:
   chat:
-    build: .
+    build:
+      context: .
+      target: development
     command: node index.js
     ports:
       - '3000:3000'

This will preserve the old behavior we had before we added multistage builds, in development.

Finally, to make the COPY . . step in our new Dockerfile safe, we should add a .dockerignore file. Without it, the COPY . . may pick up other things we don’t need or want in our production image, such as our .git folder, any node_modules that are installed on the host outside of Docker, and indeed all the Docker-related files that go into building the image. Ignoring these leads to smaller images and also faster builds, because the Docker daemon does not have to work as hard to create its copy of the files for its build context. Here’s the .dockerignore file:

.dockerignore
.git
docker-compose*.yml
Dockerfile
node_modules

With all of that set up, we can run a production build to simulate how a CI system might build the final image, and then run it like an orchestrator might:

$ docker build . -t chat:latest
# ... build output ...
$ docker run --rm --detach --publish 3000:3000 chat:latest
dd1cf2bf9496edee58e1f5122756796999942fa4437e289de4bd67b595e95745

and again access it in the browser on http://localhost:3000. When finished, we can stop it using the container ID from the command above 7.

$ docker stop dd1cf2bf9496edee58e1f5122756796999942fa4437e289de4bd67b595e95745

Setting up nodemon in Development

Now that we have distinct development and production images, let’s see how to make the development image a bit more developer-friendly by running the application under nodemon for automatic reloads within the container when we change a source file. After running

$ docker-compose run --rm chat npm install --save-dev nodemon

to install nodemon, we can update the compose file to run it:

diff --git a/docker-compose.yml b/docker-compose.yml
index 2ee0d9b..173a297 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -5,7 +5,7 @@ services:
     build:
       context: .
       target: development
-    command: node index.js
+    command: npx nodemon index.js
     ports:
       - '3000:3000'
     volumes:

Here we use npx to run nodemon through npm 8. When we bring up the service, we should see the familiar nodemon output 9:

docker-compose up
Recreating docker-chat-demo_chat_1 ... done
Attaching to docker-chat-demo_chat_1
chat_1  | [nodemon] 1.19.2
chat_1  | [nodemon] to restart at any time, enter `rs`
chat_1  | [nodemon] watching dir(s): *.*
chat_1  | [nodemon] starting `node index.js`
chat_1  | listening on *:3000

Finally, it’s worth noting that with the Dockerfile above the dev dependencies will be included in the production image. It is possible to break out another stage to avoid this, but I would argue it is not necessarily a bad thing to include them. Nodemon is unlikely to be wanted in production, it is true, but dev dependencies often include testing utilities, and including those means we can run the tests in our production container as part of CI. It also generally improves dev-prod parity, and as some wise people once said, ‘test as you fly, fly as you test.’ Speaking of which, we don’t have any tests, but it’s easy enough to run them when we do:

$ docker-compose run --rm chat npm test

> [email protected] test /srv/chat
> echo "Error: no test specified" && exit 1

Error: no test specified
npm ERR! Test failed.  See above for more details.

Here’s the final code on github.

Conclusion

We’ve taken an app and got it running in development and production entirely within Docker. Great job!

We jumped through some hopefully edifying hoops to bootstrap a node environment without installing anything on the host. We also jumped through some hoops to avoid running builds and processes as root, instead running them as an unprivileged user for better security.

Node / npm’s habit of putting dependencies in the node_modules subfolder makes our lives a little bit more complicated than other solutions, such as ruby’s bundler, that install your dependencies outside the application folder, but we were able to work around that fairly easily with the nested node modules volume trick.

Finally, we used Docker’s multi-stage build feature to produce a Dockerfile suitable for both development and production. This simple but powerful feature is useful in a wide variety of situations, and we’ll see it again in some future articles.

How to Dockerize a Node.js App from scratch

How to Dockerize a Node.js App from scratch

In this post, I will be giving you a complete walkthrough of how to Dockerize a Node.js application from scratch.

Originally published at https://www.edureka.co

Every Node.js developer out there always puts in utmost efforts to make his application free any type of environment dependencies. But despite their measures, surprises occur all the time leading to the failure of the application. Well, this is where Docker comes to the rescue. In this Node.js Docker Tutorial, I will be giving you a complete walkthrough of how to Dockerize a Node.js application from scratch.

Below are the topics I will be covering in this Node.js Docker article:

  • What is Docker?
  • Docker Basics: Dockerfile, Images & Containers
  • Why use Node.js with Docker?
  • Demo: Node.js Docker Tutorial

Let’s get started with this Node.js Docker Tutorial.

What is Docker?

Docker is a containerization platform which is used for packaging an application and its dependencies together within a Docker container. This ensures the effortless and smooth functioning of our application irrespective of the changes in the environment.

Thus, you can think of Docker as a tool that is designed to make the creation, deployment, and execution of applications using the containers easier and efficient.

Talking about Docker Container it is nothing but a standardized unit that is used to deploy a particular application or environment and can be built dynamically. You can have any container such as Ubuntu, CentOS, etc. based on your requirement with respect to Operating Systems. Moreover, these containers aren’t limited to just OS, you can have application-oriented OS as well. A few examples of such containers are CakePHP container, Tomcat-Ubuntu container, etc.

To understand this better refer to the below diagram:

Now in this diagram, you can see that each and every application is running on a separate container along with its own set of dependencies & libraries. This ensures that each application is independent of others, enabling developers to build applications independently without any interference from other applications. Thus, being a developer you can simply build a container with different applications installed in it and hand it over to the QA team. The QA team then just needs to execute the container for replicating the developer’s environment.

Now, let me now throw some light on the nitty-gritty of Docker which will aid you in understanding the process of Dockerizing your Node.js application better.

Docker Basics: Dockerfile, Images & Containers

The three most important aspects that you must know before you get your feet wet with Docker are:

  1. Dockerfile
  2. Docker Images
  3. Docker Containers

In the above diagram you can see that when a Dockerfile is built, it gives you a Docker Image. Furthermore, when you execute the Docker Image then it finally gives you a Docker Container.

Let’s now understand each of these in detail.

Dockerfile

A Dockerfile is basically a text document that contains the list of commands which can be invoked by a user using the command line in order to assemble an image. Thus, by reading instructions from this Dockerfile, Docker automatically builds images. 

For executing multiple command line instructions successively you can create an automated build using the following command:

docker build

Docker Image

A Docker Image can be considered something similar to a template which is typically used to build Docker Containers. In other words, these read-only templates are nothing but the building blocks of a Docker Container. In order to execute an image and build a container you need to use the following command:

docker run

The Docker Images that you create using this command are stored within the Docker Registry. It can be either a user’s local repository or a public repository like a Docker Hub which allows multiple users to collaborate in building an application.

Docker Container

A Docker Container is the running instance of a Docker Image. These containers hold the complete package that is required to execute the application. So, these are basically ready to use applications which are created from Docker Images that is the ultimate utility of Docker.

Why use Docker with Node.js?

Below I have listed down a few of the most intriguing reasons to use Docker with your Node.js application:

  • Docker helps in speeding up the application deployment process
  • It makes the application portability across other machines easier
  • It makes the version control easier and promotes component reuse
  • Using Docker, one can easily share the Docker images and Dockerfiles
  • Docker has a very lightweight footprint and thus puts minimal overhead on the applications
  • It’s easy and simple to use and maintain

I hope this gives you enough reasons to start using Docker right away. So, lets now dive deeper into this Node.js Docker Tutorial and see how exactly Docker can be used with Node.js applications.

Demo: Node.js Docker Tutorial

Before you start using Docker with Node.js, you need to make sure that Docker is already installed in your system and you have the right set of permissions to use it. If not then you can refer to the following articles:

  1. Installing Docker on Ubuntu/CentOS
  2. Installing Docker on Windows

Now that installation process is out of the way, let’s now concentrate on Dockerizing a Node.js Application. I am assuming you already have Node.js installed in your system.

In order to Dockerize a Node.js application, you need to go through the following steps:

  1. Create Node.js Application
  2. Create a Docker file
  3. Build Docker Image
  4. Execute

Create Node.js Application

In order to Dockerize a Node.js Application, the very initial thing you need is the Node.js Application. You can refer to my article on Building REST API with Node.js.

Once you are done developing the application, you need to make sure that the application is executing properly on the assigned port. In my case, I am using port 8080. If the application is working as expected, you can proceed to the next step.

Create a Dockerfile

In this step, we will be creating the Dockerfile which will enable us to recreate and scale our Node.js application as per our requirement. To complete this step, you need to create a new file in the root directory of the project and name it Dockerfile.

Here, I am using a lightweight alpine based image to build our Docker image on it. While creating a Dockerfile our main aim should be to keep the Docker image as small as possible in size all while availing everything that is required to run our application successfully.

Below I have written down the code that needs to add in your Dockerfile:

Dockerfile

FROM node:9-slim
 
# WORKDIR specifies the application directory
WORKDIR /app
 
# Copying package.json file to the app directory
COPY package.json /app
 
# Installing npm for DOCKER
RUN npm install
 
# Copying rest of the application to app directory
COPY . /app
 
# Starting the application using npm start
CMD ["npm","start"]

As you can see in the above code that I have used two distinct COPY commands to reduce the application rebuild time. As Docker can implicitly cache the result of each individual command, you don’t need to execute all the commands from the beginning each time you try to create a Docker image.

Now that you have successfully defined your Dockerfile, the next step is to Build a Docker Image. In the next section of this article, I will demonstrate how you can build your Docker image with ease.

Build Docker Image

Building a Docker image is rather easy and can be done using a simple command. Below I have written down the command that you need to type in your terminal and execute it:

docker build -t <docker-image-name> <file path>

Once you execute this command, you will see a 6 step output in your terminal. I have attached a screenshot to my output.

If you are getting an output something similar to the above screenshot, then it means that your application is working fine and the docker image has been successfully created. In the next section of this Node.js Docker article, I will show you how to execute this Docker Image.

Executing the Docker Image

Since you have successfully created your Docker image, now you can run one or more Docker containers on this image using the below-given command:

docker run it -d -p <HOST PORT>:<DOCKER PORT> <docker-image-name>

This command will start your docker container based on your Docker image and expose it on the specified port in your machine. In the above command -d flag indicates that you want to execute your Docker container in a detached mode. In other words, this will enable your Docker container to run in the background of the host machine. While the -p flag specifies which host port will be connected to the docker port.

To check whether your application has been successfully Dockerized or not, you can try launching it on the port you have specified for the host in the above command.

If you want to see the list of images currently running in your system, you can use the below command:

docker ps

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow us on Facebook | Twitter

Further reading

Best 50 Nodejs interview questions from Beginners to Advanced in 2019

Tutorial Laravel 6 with Docker and Docker-Compose

Docker for Absolute Beginners

Docker Tutorial From Beginner to Advanced



Crafting multi-stage builds with Docker in Node.js

Crafting multi-stage builds with Docker in Node.js

Learn how you can use a multi-stage Docker build for your Node.js application. Docker multi-stage builds enable us to create more complex build pipelines without having to resort to magic tricks.

Everyone knows about Docker. It’s the ubiquitous tool for packaging and distribution of applications that seemed to come from nowhere and take over our industry! If you are reading this, it means you already understand the basics of Docker and are now looking to create a more complex build pipeline.

In the past, optimizing our Docker images has been a challenging experience. All sorts of magic tricks were employed to reduce the size of our applications before they went to production. Things are different now because support for multi-stage builds has been added to Docker.

In this post, we explore how you can use a multi-stage build for your Node.js application. For an example, we’ll use a TypeScript build process, but the same kind of thing will work for any build pipeline. So even if you’d prefer to use Babel, or maybe you need to build a React client, then a Docker multi-stage build can work for you as well.

A basic, single-stage Dockerfile for Node.js

Let’s start by looking at a basic Dockerfile for Node.js. We can visualize the normal Docker build process as shown in Figure 1 below.

Figure 1: Normal Docker build process.

We use the docker build command to turn our Dockerfile into a Docker image. We then use the docker run command to instantiate our image to a Docker container.

The Dockerfile in Listing 1 below is just a standard, run-of-the-mill Dockerfile for Node.js. You have probably seen this kind of thing before. All we are doing here is copying the package.json, installing production dependencies, copying the source code, and finally starting the application.

This Dockerfile is for regular JavaScript applications, so we don’t need a build process yet. I’m only showing you this simple Dockerfile so you can compare it to the multi-stage Dockerfile I’ll be showing you soon.

Listing 1: A run-of-the-mill Dockerfile for Node.js

FROM node:10.15.2

WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install --only=production
COPY ./src ./src
EXPOSE 3000
CMD npm start

Listing 1 is a quite ordinary-looking Docker file. In fact, all Docker files looked pretty much like this before multi-stage builds were introduced. Now that Docker supports multi-stage builds, we can visualize our simple Dockerfile as the single-stage build process illustrated in Figure 2.


Figure 2: A single-stage build pipeline.

The need for multiple stages

We can already run whatever commands we want in the Dockerfile when building our image, so why do we even need a multi-stage build?

To find out why, let’s upgrade our simple Dockerfile to include a TypeScript build process. Listing 2 shows the upgraded Dockerfile. I’ve bolded the updated lines so you can easily pick them out.

Listing 2: We have upgraded our simple Dockerfile to include a TypeScript build process

FROM node:10.15.2

WORKDIR /usr/src/app
COPY package*.json ./
COPY tsconfig.json ./
RUN npm install
COPY ./src ./src
RUN npm run build
EXPOSE 80
CMD npm start

We can easily and directly see the problem this causes. To see it for yourself, you should instantiate a container from this image and then shell into it and inspect its file system.

I did this and used the Linux tree command to list all the directories and files in the container. You can see the result in Figure 3.

Notice that we have unwittingly included in our production image all the debris of development and the build process. This includes our original TypeScript source code (which we don’t use in production), the TypeScript compiler itself (which, again, we don’t use in production), plus any other dev dependencies we might have installed into our Node.js project.


FIgure 3: The debris from development and the build process is bloating our production Docker image.
Bear in mind this is only a trivial project, so we aren’t actually seeing too much cruft left in our production image. But you can imagine how bad this would be for a real application with many sources files, many dev dependencies, and a more complex build process that generates temporary files!

We don’t want this extra bloat in production. The extra size makes our containers bigger. When our containers are bigger than needed, it means we aren’t making efficient use of our resources. The increased surface area of the container can also be a problem for security, where we generally prefer to minimize the attackable surface area of our application.

Wouldn’t it be nice if we could throw away the files we don’t want and just keep the ones we do want? This is exactly what a Docker multi-stage build can do for us.

Crafting a Dockerfile with a multi-stage build

We are going to split out Dockerfile into two stages. Figure 4 shows what our build pipeline looks like after the split.


Figure 4: A multi-stage Docker build pipeline to build TypeScript.

Our new multi-stage build pipeline has two stages: Build stage 1 is what builds our TypeScript code; Build stage 2 is what creates our production Docker image. The final Docker image produced at the end of this pipeline contains only what it needs and omits the cruft we don’t want.

To create our two-stage build pipeline, we are basically just going to create two Docker files in one. Listing 3 shows our Dockerfile with multiple stages added. The first FROM command initiates the first stage, and the second FROM command initiates the second stage.

Compare this to a regular single-stage Dockerfile, and you can see that it actually looks like two Dockerfiles squished together in one.

Listing 3: A multi-stage Dockerfile for building TypeScript code

# 
# Build stage 1.
# This state builds our TypeScript and produces an intermediate Docker image containing the compiled JavaScript code.
#
FROM node:10.15.2

WORKDIR /usr/src/app
COPY package*.json ./
COPY tsconfig.json ./
RUN npm install
COPY ./src ./src
RUN npm run build

#
# Build stage 2.
# This stage pulls the compiled JavaScript code from the stage 1 intermediate image.
# This stage builds the final Docker image that we'll use in production.
#
FROM node:10.15.2

WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install --only=production
COPY --from=0 /usr/src/app/build ./build
EXPOSE 80
CMD npm start

To create this multi-stage Dockerfile, I simply took Listing 2 and divided it up into separate Dockerfiles. The first stage contains only what is need to build the TypeScript code. The second stage contains only what is needed to produce the final production Docker image. I then merged the two Dockerfiles into a single file.

The most important thing to note is the use of --from in the second stage. I’ve bolded this line in Listing 3 so you can easily pick it out. This is the syntax we use to pull the built files from our first stage, which we refer to here as stage 0. We are pulling the compiled JavaScript files from the first stage into the second stage.

We can easily check to make sure we got the desired result. After creating the new image and instantiating a container, I shelled in to check the contents of the file system. You can see in Figure 5 that we have successfully removed the debris from our production image.


Figure 5: We have removed the debris of development from our Docker image.

We now have fewer files in our image, it’s smaller, and it has less surface area. Yay! Mission accomplished.

But what, specifically, does this mean?

The effect of the multi-stage build

What exactly is the effect of the new build pipeline on our production image?

I measured the results before and after. Our single-stage image produced by Listing 2 weighs in at 955MB. After converting to the multi-stage build in Listing 3, the image now comes to 902MB. That’s a reasonable reduction — we removed 53MB from our image!

While 53MB seems like a lot, we have actually only shaved off just more than 5 percent of the size. I know what you’re going to say now: But Ash, our image is still monstrously huge! There’s still way too much bloat in that image.

Well, to make our image even smaller, we now need to use the alpine, or slimmed-down, Node.js base image. We can do this by changing our second build stage from node:10.15.2 to node:10.15.2-alpine.

This reduces our production image down to 73MB — that’s a huge win! Now the savings we get from discarding our debris is more like a whopping 60 percent. Alright, we are really getting somewhere now!

This highlights another benefit of multi-stage builds: we can use separate Docker base images for each of our build stages. This means you can customize each build stage by using a different base image.

Say you have one stage that relies on some tools that are in a different image, or you have created a special Docker image that is custom for your build process. This gives us a lot of flexibility when constructing our build pipelines.

How does it work?

You probably already guessed this: each stage or build process produces its own separate Docker image. You can see how this works in Figure 6.

The Docker image produced by a stage can be used by the following stages. Once the final image is produced, all the intermediate images are discarded; we take what we want for the final image, and the rest gets thrown away.


Figure 6: Each stage of a multi-stage Docker build produces an image.

Adding more stages

There’s no need to stop at two stages, although that’s often all that’s needed; we can add as many stages as we need. A specific example is illustrated in Figure 7.

Here we are building TypeScript code in stage 1 and our React client in stage 2. In addition, there’s a third stage that produces the final image from the results of the first two stages.


Figure 7: Using a Docker multi-stage build, we can create more complex build pipelines.

Pro tips

Now time to leave you with a few advanced tips to explore on your own:

  1. You can name your build stages! You don’t have to leave them as the default 0, 1, etc. Naming your build stages will make your Dockerfile more readable.
  2. Understand the options you have for base images. Using the right base image can relieve a lot of confusion when constructing your build pipeline.
  3. Build a custom base image if the complexity of your build process is getting out of hand.
  4. You can pull from external images! Just like you pull files from earlier stages, you can also pull files from images that are published to a Docker repository. This gives you an option to pre-bake an early build stage if it’s expensive and doesn’t change very often.
Conclusion and resources

Docker multi-stage builds enable us to create more complex build pipelines without having to resort to magic tricks. They help us slim down our production Docker images and remove the bloat. They also allow us to structure and modularize our build process, which makes it easier to test parts of our build process in isolation.

So please have some fun with Docker multi-stage builds, and don’t forget to have a look at the example code on GitHub.

Here’s the Docker documentation on multi-stage builds, too.