Welcome to the "Docker Best Practices for Node Developers"! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.
Welcome to the best course on the planet for using Docker with Node.js! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.
My talk on all the best of Docker for Node.js developers and DevOps dealing with Node apps. From DockerCon 2019. Get the full 9-hour training course with my coupon at http://bit.ly/365ogba
Get the source code for this talk at https://github.com/BretFisher/dockercon19Some of the many cool things you'll do in this course
You'll start with a quick review about getting set up with Docker, as well as Docker Compose basics. That way we're on the same page for the basics.
Then you'll jump into Node.js Dockerfile basics, that way you'll have a good Dockerfile foundation for new features we'll add throughout the course.
You'll be building on all the different things you learn from each Lecture in the course. Once you have the basics down of Compose, Dockerfile, and Docker Image, then you'll focus on nuances like how Docker and Linux control the Node process and how Docker changes that to make sure you know what options there are for starting up and shutting down Node.js and the right way to do it in different scenarios.
We'll cover advanced, newer features around making the Dockerfile the most efficient and flexible as possible using things like BuildKit and Multi-stage.
Then we'll talk about distributed computing and cloud design to ensure your Node.js apps have 12-factor design in your containers, as well as learning how to migrate old apps into this new way of doing things.
Next we cover Compose and its awesome features to get really efficient local development and test set-up using the Docker Compose command line and Docker Compose YAML file.
With all this knowledge, you'll progress to production concerns and making images production-ready.
Then we'll jump into deploying those containers and running them in production. Whether you use Docker Engine or orchestration with Kubernetes or Swarm, I've got you covered. In addition, we'll cover HTTP connections and reverse proxies for connection handling and routing with multi-container systems.
Lastly, you'll get a final, big assignment where you'll be building and deploying a large, complex solution, including multiple Node.js containers that are doing different things. You'll build Docker images, Dockerfiles, and compose files, and deploy them to a server to test. You'll need to check whether connections failover properly. You'll basically take everything you've learned and apply it in one big project!
In this article, you'll learn how to build a Node.js Application in Docker
In this tutorial, we’ll set up the socket.io chat example with Docker, from scratch to production-ready. In particular, we’ll see how to:
node_modulesin a container (there’s a trick to this).
Dockerfilebetween development and production using multi-stage builds.
This tutorial assumes you already have some familiarity with Docker and node. If you’d like a gentle intro to Docker first, I’d recommend running through Docker’s official introduction.Getting Started
We’re going to set things up from scratch. The final code is available on github here, and there are tags for each step along the way. Here’s the code for the first step, in case you’d like to follow along.
Without Docker, we’d start by installing node and any other dependencies on the host and running
npm init to create a new package. There’s nothing stopping us from doing that here, but we’ll learn more if we use Docker from the start. (And of course the whole point of using Docker is that you don’t have to install things on the host.) We’ll start by creating a “bootstrapping container” that has node installed, and we’ll use it to set up the npm package for the application.
We’ll need to write two files, a
Dockerfile and a
docker-compose.yml, to which we’ll add more later on. Let’s start with the bootstrapping
FROM node:10.16.3 USER node WORKDIR /srv/chat
It’s a short file, but there already some important points:
It starts from the official Docker image for the latest long term support (LTS) node release, at time of writing. I prefer to name a specific version, rather than one of the ‘floating’ tags like
node:latest, so that if you or someone else builds this image on a different machine, they will get the same version, rather than risking an accidental upgrade and attendant head-scratching.
USER step tells Docker to run any subsequent build steps, and later the process in the container, as the
node user, which is an unprivileged user that comes built into all of the official node images from Docker. Without this line, they would run as root, which is against security best practices and in particular the principle of least privilege. Many Docker tutorials skip this step for simplicity, and we will have to do some extra work to avoid running as root, but I think it’s very important.
WORKDIR step sets the working directory for any subsequent build steps, and later for containers created from the image, to
/srv/chat, which is where we’ll put our application files. The
/srv folder should be available on any system that follows the Filesystem Hierarchy Standard, which says that it is for “site-specific data which is served by this system”, which sounds like a good fit for a node app 1.
Now let’s move on to the bootstrapping compose file,
version: '3.7' services: chat: build: . command: echo 'ready' volumes: - .:/srv/chat
Again there is quite a bit to unpack:
version line tells Docker Compose which version of its file format we are using. Version 3.7 is the latest at the time of writing, so I’ve gone with that, but older 3.x and 2.x versions would also work fine here; in fact, the 2.x series might even be a better fit, depending on your use case 2.
The file defines a single service called
chat, built from the
Dockerfile in the current directory, denoted
.. All the service does for now is to echo
ready and exit.
The volume line,
.:/srv/chat, tells Docker to bind mount the current directory
. on the host at
/srv/chat in the container, which is the
WORKDIR we set up in the
Dockerfile above. This means that changes we’ll make to source files on the host will be automatically reflected inside the container, and vice versa. This is very important for keeping your test-edit-reload cycles as short as possible in development. It will, however, create some issues with how npm installs dependencies, which we’ll come back to shortly.
Now we’re ready to build and test our bootstrapping container. When we run
docker-compose build, Docker will create an image with node set up as specified in the
docker-compose up will start a container with that image and run the echo command, which shows that everything is set up OK.
$ docker-compose build Building chat Step 1/3 : FROM node:10.16.3 # ... more build output ... Successfully built d22d841c07da Successfully tagged docker-chat-demo_chat:latest $ docker-compose up Creating docker-chat-demo_chat_1 ... done Attaching to docker-chat-demo_chat_1 chat_1 | ready docker-chat-demo_chat_1 exited with code 0
This output indicates that the container ran, echoed
ready and exited successfully. 🎉
⚠️ Aside for Linux users: For this next step to work smoothly, the
nodeuser in the container should have the same
uid(user identifier) as your user on the host. This is because the user in the container needs to have permissions to read and write files on the host via the bind mount, and vice versa. I’ve included an appendix with advice on how to deal with this issue. Docker for Mac users don’t have to worry about it because of some uid remapping magic behind the scenes, but Docker for Linux get much better performance, so I’d call it a draw.
Now we have a node environment set up in Docker, we’re ready to set up the initial npm package files. To do this, we’ll run an interactive shell in the container for the
chat service and use it to set up the initial package files:
$ docker-compose run --rm chat bash [email protected]:/srv/chat$ npm init --yes # ... writes package.json ... [email protected]:/srv/chat$ npm install # ... writes package-lock.json ... [email protected]:/srv/chat$ exit
And then the files appear on the host, ready for us to commit to version control:
$ tree . ├── Dockerfile ├── docker-compose.yml ├── package-lock.json └── package.json
Here’s the resulting code on github.Installing Dependencies
Next up on our list is to install the app’s dependencies. We want these dependencies to be installed inside the container via the
Dockerfile, so the container will contain everything needed to run the application. This means we need to get the
package-lock.json files into the image and run
npm install in the
Dockerfile. Here’s what that change looks like:
diff --git a/Dockerfile b/Dockerfile index b18769e..d48e026 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,5 +1,14 @@ FROM node:10.16.3 +RUN mkdir /srv/chat && chown node:node /srv/chat + USER node WORKDIR /srv/chat + +COPY --chown=node:node package.json package-lock.json ./ + +RUN npm install --quiet + +# TODO: Can remove once we have some dependencies in package.json. +RUN mkdir -p node_modules
And here’s the explanation:
RUN step with
chown commands, which are the only commands we need to run as root, creates the working directory and makes sure that it’s owned by the node user.
It’s worth noting that there are two shell commands chained together in that single
RUN step. Compared to splitting out the commands over multiple
RUN steps, chaining them reduces the number of layers in the resulting image. In this example, it really doesn’t matter very much, but it is a good habit not to use more layers than you need. It can save a lot of disk space and download time if you e.g. download a package, unzip it, build it, install it, and then clean up in one step, rather than saving layers with all of the intermediate files for each step.
./ copies the npm packaging files to the
WORKDIR that we set up above. The trailing
/ tells Docker that the destination is a folder. The reason for copying in only the packaging files, rather than the whole application folder, is that Docker will cache the results of the
npm install step below and rerun it only if the packaging files change. If we copied in all our source files, changing any one would bust the cache even though the required packages had not changed, leading to unnecessary
npm installs in subsequent builds.
--chown=node:node flag for
COPY ensures that the files are owned by the unprivileged
node user rather than root, which is the default 3.
npm install step will run as the
node user in the working directory to install the dependencies in
/srv/chat/node_modules inside the container.
This last step is what we want, but it causes a problem in development when we bind mount the application folder on the host over
/srv/chat. Unfortunately, the
node_modules folder doesn’t exist on the host, so the bind effectively hides the node modules that we installed in the image. The final
mkdir -p node_modules step and the next section are related to how we deal with this.
There are several ways around the node modules hiding problem, but I think the most elegant is to use a volume within the bind to contain
node_modules. To do this, we have to add a few lines to our docker compose file:
diff --git a/docker-compose.yml b/docker-compose.yml index c9a2543..799e1f6 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -6,3 +6,7 @@ services: command: echo 'ready' volumes: - .:/srv/chat + - chat_node_modules:/srv/chat/node_modules + +volumes: + chat_node_modules:
chat_node_modules:/srv/chat/node_modules volume line sets up a named volume 4 called
chat_node_modules that contains the directory
/srv/chat/node_modules in the container. The top level
volumes: section at the end must declare all named volumes, so we add
chat_node_modules there, too.
So, it’s simple to do, but there is quite a bit going on behind the scenes to make it work:
npm installinstalls the dependencies (which we’ll add in the next section) into
/srv/chat/node_moduleswithin the image. We’ll color the files from the image blue:
/srv/chat$ tree # in image . ├── node_modules │ ├── accepts ... │ └── yeast ├── package-lock.json └── package.json
/srv/chat. We’ll color the files from the host red:
/srv/chat$ tree # in container without node_modules volume . ├── Dockerfile ├── docker-compose.yml ├── node_modules ├── package-lock.json └── package.json
The bad news is that the
node_modules in the image are hidden by the bind; inside the container, we instead see only an empty
node_modules folder on the host.
/srv/chat/node_modulesin the image, and it mounts it in the container. This, in turn, hides the
node_modulesfrom the bind on the host:
/srv/chat$ tree # in container with node_modules volume . ├── Dockerfile ├── docker-compose.yml ├── node_modules │ ├── accepts ... │ └── yeast ├── package-lock.json └── package.json
This gives us what we want: our source files on the host are bound inside the container, which allows for fast changes, and the dependencies are also available inside of the container, so we can use them to run the app.
We can also now explain the final
mkdir -p node_modules step in the bootstrapping
Dockerfile above: we have not actually installed any packages yet, so
npm install doesn’t create the
node_modules folder during the build. When Docker creates the
/srv/chat/node_modules volume, it will automatically create the folder for us, but it will be owned by root, which means the node user won’t be able to write to it. We can preempt that by creating
node_modules as the node user during the build. Once we have some packages installed, we no longer need this line.
So, let’s rebuild the image, and we’ll be ready to install packages.
$ docker-compose build ... builds and runs npm install (with no packages yet)...
The chat app requires express, so let’s get a shell in the container and
npm install it with
--save to save the dependency to our
package.json and update
$ docker-compose run --rm chat bash Creating volume "docker-chat-demo_chat_node_modules" with default driver [email protected]:/srv/chat$ npm install --save express # ... [email protected]:/srv/chat$ exit
package-lock.json file, which has for most purposes replaced the older
npm-shrinkwrap.json file, is important for ensuring that Docker image builds are repeatable. It records the versions of all direct and indirect dependencies and ensures that
npm installs in Docker builds on different machines will all get the same dependency tree.
Finally, it’s worth noting that the
node_modules we installed are not present on the host. There may be an empty
node_modules folder on the host, which is a side effect of the binds and volumes we created, but the actual files live in the
chat_node_modules volume. If we run another shell in the
chat container, we’ll find them there:
$ ls node_modules # nothing on the host $ docker-compose run --rm chat bash [email protected]:/srv/chat$ ls -l node_modules/ total 196 drwxr-xr-x 2 node node 4096 Aug 25 20:07 accepts # ... many node modules in the container drwxr-xr-x 2 node node 4096 Aug 25 20:07 vary
The next time we run a
docker-compose build, the modules will be installed into the image.
Here’s the resulting code on github.Running the App
We are finally ready to install the app, so we’ll copy in the remaining source files, namely
Then we’ll install the
socket.io package. At the time of writing, the chat example is only compatible with socket.io version 1, so we need to request version 1:
$ docker-compose run --rm chat npm install --save [email protected] # ...
In our docker compose file, we then remove our dummy
echo ready command and instead run the chat example server. Finally, we tell Docker Compose to export 3000 in the container on the host, so we can access it in a browser:
diff --git a/docker-compose.yml b/docker-compose.yml index 799e1f6..ff92767 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -3,7 +3,9 @@ version: '3.7' services: chat: build: . - command: echo 'ready' + command: node index.js + ports: + - '3000:3000' volumes: - .:/srv/chat - chat_node_modules:/srv/chat/node_modules
Then we are ready to run with
docker-compose up 5:
$ docker-compose up Recreating dockerchatdemo_chat_1 Attaching to dockerchatdemo_chat_1 chat_1 | listening on *:3000
Then you can see it running on
Here’s the resulting code on github.Docker for Dev and Prod
We now have our app running in development under docker compose, which is pretty cool! Before we can use this container in production, we have a few problems to solve and possible improvements to make:
Most importantly, the container as we’re building it at the moment does not actually contain the source code for the application — it just contains the npm packaging files and dependencies. The main idea of a container is that it should contain everything needed to run the application, so clearly we will want to change this.
/srv/chat application folder in the image is currently owned and writeable by the
node user. Most applications don’t need to rewrite their source files at runtime, so again applying the principle of least privilege, we shouldn’t let them.
The image is fairly large, weighing in at 909MB according to the handy dive image inspection tool. It’s not worth obsessing over image size, but we don’t want to be needlessly wasteful either. Most of the image’s heft comes from the default
Fortunately, Docker provide a powerful tool that helps with all of the above: multi-stage builds. The main idea is that we can have multiple
FROM commands in the
Dockerfile, one per stage, and each stage can copy files from previous stages. Let’s see how to set that up:
diff --git a/Dockerfile b/Dockerfile index d48e026..6c8965d 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM node:10.16.3 +FROM node:10.16.3 AS development RUN mkdir /srv/chat && chown node:node /srv/chat @@ -10,5 +10,14 @@ COPY --chown=node:node package.json package-lock.json ./ RUN npm install --quiet -# TODO: Can remove once we have some dependencies in package.json. -RUN mkdir -p node_modules +FROM node:10.16.3-slim AS production + +USER node + +WORKDIR /srv/chat + +COPY --from=development --chown=root:root /srv/chat/node_modules ./node_modules + +COPY . . + +CMD ["node", "index.js"]
Dockerfile steps will form the first stage, which we’ll now give the name
development by adding
AS development to the
FROM line at the start. I’ve now removed the temporary
mkdir -p node_modules step needed during bootstrapping, since we now have some packages installed.
The new second stage starts with the second
FROM step, which pulls in the
slim node base image for the same node version and calls the stage
production for clarity. This
slim image is also an official node image from Docker. As its name suggests, it is smaller than the default
node image, mainly because it doesn’t include the compiler toolchain; it includes only the system dependencies needed to run a node application, which are far fewer than what may be required to build one.
npm install in the first stage, which has the full node image at its disposal for the build. Then it copies the resulting
node_modules folder to the second stage image, which uses the
slim base image. This technique reduces the size of the production image from 909MB to 152MB, which is about a factor of 6 saving for relatively little effort 6.
USER node command tells Docker to run the build and the application as the unprivileged
node user rather than as root. We also have to repeat the
WORKDIR, because it doesn’t persist into the second stage automatically.
COPY --from=development --chown=root:root ... line copies the dependencies installed in the preceding
development stage into the production stage and makes them owned by root, so the node user can read but not write them.
COPY . . line then copies the rest of the application files from the host to the working directory in the container, namely
CMD step specifies the command to run. In the development stage, the application files came from bind mounts set up with docker-compose, so it made sense to specify the command in the
docker-compose.yml file instead of the
Dockerfile. Here it makes more sense to specify the command in the
Dockerfile, which builds it into the container.
Now that we have our multi-stage
Dockerfile set up, we need to tell Docker Compose to use only the
development stage rather than going through the full
Dockerfile, which we can do with the
diff --git a/docker-compose.yml b/docker-compose.yml index ff92767..2ee0d9b 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -2,7 +2,9 @@ version: '3.7' services: chat: - build: . + build: + context: . + target: development command: node index.js ports: - '3000:3000'
This will preserve the old behavior we had before we added multistage builds, in development.
Finally, to make the
COPY . . step in our new
Dockerfile safe, we should add a
.dockerignore file. Without it, the
COPY . . may pick up other things we don’t need or want in our production image, such as our
.git folder, any
node_modules that are installed on the host outside of Docker, and indeed all the Docker-related files that go into building the image. Ignoring these leads to smaller images and also faster builds, because the Docker daemon does not have to work as hard to create its copy of the files for its build context. Here’s the
.dockerignore .git docker-compose*.yml Dockerfile node_modules
With all of that set up, we can run a production build to simulate how a CI system might build the final image, and then run it like an orchestrator might:
$ docker build . -t chat:latest # ... build output ... $ docker run --rm --detach --publish 3000:3000 chat:latest dd1cf2bf9496edee58e1f5122756796999942fa4437e289de4bd67b595e95745
and again access it in the browser on
http://localhost:3000. When finished, we can stop it using the container ID from the command above 7.
$ docker stop dd1cf2bf9496edee58e1f5122756796999942fa4437e289de4bd67b595e95745
Now that we have distinct development and production images, let’s see how to make the development image a bit more developer-friendly by running the application under nodemon for automatic reloads within the container when we change a source file. After running
$ docker-compose run --rm chat npm install --save-dev nodemon
to install nodemon, we can update the compose file to run it:
diff --git a/docker-compose.yml b/docker-compose.yml index 2ee0d9b..173a297 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -5,7 +5,7 @@ services: build: context: . target: development - command: node index.js + command: npx nodemon index.js ports: - '3000:3000' volumes:
docker-compose up Recreating docker-chat-demo_chat_1 ... done Attaching to docker-chat-demo_chat_1 chat_1 | [nodemon] 1.19.2 chat_1 | [nodemon] to restart at any time, enter `rs` chat_1 | [nodemon] watching dir(s): *.* chat_1 | [nodemon] starting `node index.js` chat_1 | listening on *:3000
Finally, it’s worth noting that with the
Dockerfile above the dev dependencies will be included in the production image. It is possible to break out another stage to avoid this, but I would argue it is not necessarily a bad thing to include them. Nodemon is unlikely to be wanted in production, it is true, but dev dependencies often include testing utilities, and including those means we can run the tests in our production container as part of CI. It also generally improves dev-prod parity, and as some wise people once said, ‘test as you fly, fly as you test.’ Speaking of which, we don’t have any tests, but it’s easy enough to run them when we do:
$ docker-compose run --rm chat npm test > [email protected] test /srv/chat > echo "Error: no test specified" && exit 1 Error: no test specified npm ERR! Test failed. See above for more details.
Here’s the final code on github.Conclusion
We’ve taken an app and got it running in development and production entirely within Docker. Great job!
We jumped through some hopefully edifying hoops to bootstrap a node environment without installing anything on the host. We also jumped through some hoops to avoid running builds and processes as root, instead running them as an unprivileged user for better security.
Node / npm’s habit of putting dependencies in the
node_modules subfolder makes our lives a little bit more complicated than other solutions, such as ruby’s bundler, that install your dependencies outside the application folder, but we were able to work around that fairly easily with the nested node modules volume trick.
Finally, we used Docker’s multi-stage build feature to produce a
Dockerfile suitable for both development and production. This simple but powerful feature is useful in a wide variety of situations, and we’ll see it again in some future articles.
In this post, I will be giving you a complete walkthrough of how to Dockerize a Node.js application from scratch.
Originally published at https://www.edureka.co
Every Node.js developer out there always puts in utmost efforts to make his application free any type of environment dependencies. But despite their measures, surprises occur all the time leading to the failure of the application. Well, this is where Docker comes to the rescue. In this Node.js Docker Tutorial, I will be giving you a complete walkthrough of how to Dockerize a Node.js application from scratch.
Below are the topics I will be covering in this Node.js Docker article:
Let’s get started with this Node.js Docker Tutorial.What is Docker?
Docker is a containerization platform which is used for packaging an application and its dependencies together within a Docker container. This ensures the effortless and smooth functioning of our application irrespective of the changes in the environment.
Thus, you can think of Docker as a tool that is designed to make the creation, deployment, and execution of applications using the containers easier and efficient.
Talking about Docker Container it is nothing but a standardized unit that is used to deploy a particular application or environment and can be built dynamically. You can have any container such as Ubuntu, CentOS, etc. based on your requirement with respect to Operating Systems. Moreover, these containers aren’t limited to just OS, you can have application-oriented OS as well. A few examples of such containers are CakePHP container, Tomcat-Ubuntu container, etc.
To understand this better refer to the below diagram:
Now in this diagram, you can see that each and every application is running on a separate container along with its own set of dependencies & libraries. This ensures that each application is independent of others, enabling developers to build applications independently without any interference from other applications. Thus, being a developer you can simply build a container with different applications installed in it and hand it over to the QA team. The QA team then just needs to execute the container for replicating the developer’s environment.
The three most important aspects that you must know before you get your feet wet with Docker are:
In the above diagram you can see that when a Dockerfile is built, it gives you a Docker Image. Furthermore, when you execute the Docker Image then it finally gives you a Docker Container.
Let’s now understand each of these in detail.
A Dockerfile is basically a text document that contains the list of commands which can be invoked by a user using the command line in order to assemble an image. Thus, by reading instructions from this Dockerfile, Docker automatically builds images.
For executing multiple command line instructions successively you can create an automated build using the following command:
A Docker Image can be considered something similar to a template which is typically used to build Docker Containers. In other words, these read-only templates are nothing but the building blocks of a Docker Container. In order to execute an image and build a container you need to use the following command:
The Docker Images that you create using this command are stored within the Docker Registry. It can be either a user’s local repository or a public repository like a Docker Hub which allows multiple users to collaborate in building an application.
A Docker Container is the running instance of a Docker Image. These containers hold the complete package that is required to execute the application. So, these are basically ready to use applications which are created from Docker Images that is the ultimate utility of Docker.Why use Docker with Node.js?
Below I have listed down a few of the most intriguing reasons to use Docker with your Node.js application:
I hope this gives you enough reasons to start using Docker right away. So, lets now dive deeper into this Node.js Docker Tutorial and see how exactly Docker can be used with Node.js applications.Demo: Node.js Docker Tutorial
Before you start using Docker with Node.js, you need to make sure that Docker is already installed in your system and you have the right set of permissions to use it. If not then you can refer to the following articles:
Now that installation process is out of the way, let’s now concentrate on Dockerizing a Node.js Application. I am assuming you already have Node.js installed in your system.
In order to Dockerize a Node.js application, you need to go through the following steps:
Create Node.js Application
In order to Dockerize a Node.js Application, the very initial thing you need is the Node.js Application. You can refer to my article on Building REST API with Node.js.
Once you are done developing the application, you need to make sure that the application is executing properly on the assigned port. In my case, I am using port 8080. If the application is working as expected, you can proceed to the next step.
Create a Dockerfile
In this step, we will be creating the Dockerfile which will enable us to recreate and scale our Node.js application as per our requirement. To complete this step, you need to create a new file in the root directory of the project and name it Dockerfile.
Here, I am using a lightweight alpine based image to build our Docker image on it. While creating a Dockerfile our main aim should be to keep the Docker image as small as possible in size all while availing everything that is required to run our application successfully.
Below I have written down the code that needs to add in your Dockerfile:
FROM node:9-slim # WORKDIR specifies the application directory WORKDIR /app # Copying package.json file to the app directory COPY package.json /app # Installing npm for DOCKER RUN npm install # Copying rest of the application to app directory COPY . /app # Starting the application using npm start CMD ["npm","start"]
As you can see in the above code that I have used two distinct COPY commands to reduce the application rebuild time. As Docker can implicitly cache the result of each individual command, you don’t need to execute all the commands from the beginning each time you try to create a Docker image.
Now that you have successfully defined your Dockerfile, the next step is to Build a Docker Image. In the next section of this article, I will demonstrate how you can build your Docker image with ease.
Build Docker Image
Building a Docker image is rather easy and can be done using a simple command. Below I have written down the command that you need to type in your terminal and execute it:
docker build -t <docker-image-name> <file path>
Once you execute this command, you will see a 6 step output in your terminal. I have attached a screenshot to my output.
If you are getting an output something similar to the above screenshot, then it means that your application is working fine and the docker image has been successfully created. In the next section of this Node.js Docker article, I will show you how to execute this Docker Image.
Executing the Docker Image
Since you have successfully created your Docker image, now you can run one or more Docker containers on this image using the below-given command:
docker run it -d -p <HOST PORT>:<DOCKER PORT> <docker-image-name>
This command will start your docker container based on your Docker image and expose it on the specified port in your machine. In the above command -d flag indicates that you want to execute your Docker container in a detached mode. In other words, this will enable your Docker container to run in the background of the host machine. While the -p flag specifies which host port will be connected to the docker port.
To check whether your application has been successfully Dockerized or not, you can try launching it on the port you have specified for the host in the above command.
If you want to see the list of images currently running in your system, you can use the below command:
Thanks for reading ❤
If you liked this post, share it with all of your programming buddies!
Learn how you can use a multi-stage Docker build for your Node.js application. Docker multi-stage builds enable us to create more complex build pipelines without having to resort to magic tricks.
Everyone knows about Docker. It’s the ubiquitous tool for packaging and distribution of applications that seemed to come from nowhere and take over our industry! If you are reading this, it means you already understand the basics of Docker and are now looking to create a more complex build pipeline.
In the past, optimizing our Docker images has been a challenging experience. All sorts of magic tricks were employed to reduce the size of our applications before they went to production. Things are different now because support for multi-stage builds has been added to Docker.
In this post, we explore how you can use a multi-stage build for your Node.js application. For an example, we’ll use a TypeScript build process, but the same kind of thing will work for any build pipeline. So even if you’d prefer to use Babel, or maybe you need to build a React client, then a Docker multi-stage build can work for you as well.A basic, single-stage Dockerfile for Node.js
Let’s start by looking at a basic Dockerfile for Node.js. We can visualize the normal Docker build process as shown in Figure 1 below.
Figure 1: Normal Docker build process.
We use the
docker build command to turn our Dockerfile into a Docker image. We then use the
docker run command to instantiate our image to a Docker container.
The Dockerfile in Listing 1 below is just a standard, run-of-the-mill Dockerfile for Node.js. You have probably seen this kind of thing before. All we are doing here is copying the
package.json, installing production dependencies, copying the source code, and finally starting the application.
FROM node:10.15.2 WORKDIR /usr/src/app COPY package*.json ./ RUN npm install --only=production COPY ./src ./src EXPOSE 3000 CMD npm start
Listing 1 is a quite ordinary-looking Docker file. In fact, all Docker files looked pretty much like this before multi-stage builds were introduced. Now that Docker supports multi-stage builds, we can visualize our simple Dockerfile as the single-stage build process illustrated in Figure 2.
The need for multiple stages
Figure 2: A single-stage build pipeline.
We can already run whatever commands we want in the Dockerfile when building our image, so why do we even need a multi-stage build?
To find out why, let’s upgrade our simple Dockerfile to include a TypeScript build process. Listing 2 shows the upgraded Dockerfile. I’ve bolded the updated lines so you can easily pick them out.
FROM node:10.15.2 WORKDIR /usr/src/app COPY package*.json ./ COPY tsconfig.json ./ RUN npm install COPY ./src ./src RUN npm run build EXPOSE 80 CMD npm start
We can easily and directly see the problem this causes. To see it for yourself, you should instantiate a container from this image and then shell into it and inspect its file system.
I did this and used the Linux tree command to list all the directories and files in the container. You can see the result in Figure 3.
Notice that we have unwittingly included in our production image all the debris of development and the build process. This includes our original TypeScript source code (which we don’t use in production), the TypeScript compiler itself (which, again, we don’t use in production), plus any other dev dependencies we might have installed into our Node.js project.
FIgure 3: The debris from development and the build process is bloating our production Docker image.
Bear in mind this is only a trivial project, so we aren’t actually seeing too much cruft left in our production image. But you can imagine how bad this would be for a real application with many sources files, many dev dependencies, and a more complex build process that generates temporary files!
We don’t want this extra bloat in production. The extra size makes our containers bigger. When our containers are bigger than needed, it means we aren’t making efficient use of our resources. The increased surface area of the container can also be a problem for security, where we generally prefer to minimize the attackable surface area of our application.
Wouldn’t it be nice if we could throw away the files we don’t want and just keep the ones we do want? This is exactly what a Docker multi-stage build can do for us.Crafting a Dockerfile with a multi-stage build
We are going to split out Dockerfile into two stages. Figure 4 shows what our build pipeline looks like after the split.
Figure 4: A multi-stage Docker build pipeline to build TypeScript.
Our new multi-stage build pipeline has two stages: Build stage 1 is what builds our TypeScript code; Build stage 2 is what creates our production Docker image. The final Docker image produced at the end of this pipeline contains only what it needs and omits the cruft we don’t want.
To create our two-stage build pipeline, we are basically just going to create two Docker files in one. Listing 3 shows our Dockerfile with multiple stages added. The first
FROM command initiates the first stage, and the second
FROM command initiates the second stage.
Compare this to a regular single-stage Dockerfile, and you can see that it actually looks like two Dockerfiles squished together in one.
To create this multi-stage Dockerfile, I simply took Listing 2 and divided it up into separate Dockerfiles. The first stage contains only what is need to build the TypeScript code. The second stage contains only what is needed to produce the final production Docker image. I then merged the two Dockerfiles into a single file.
The most important thing to note is the use of
We can easily check to make sure we got the desired result. After creating the new image and instantiating a container, I shelled in to check the contents of the file system. You can see in Figure 5 that we have successfully removed the debris from our production image.
Figure 5: We have removed the debris of development from our Docker image.
We now have fewer files in our image, it’s smaller, and it has less surface area. Yay! Mission accomplished.
But what, specifically, does this mean?The effect of the multi-stage build
What exactly is the effect of the new build pipeline on our production image?
I measured the results before and after. Our single-stage image produced by Listing 2 weighs in at 955MB. After converting to the multi-stage build in Listing 3, the image now comes to 902MB. That’s a reasonable reduction — we removed 53MB from our image!
While 53MB seems like a lot, we have actually only shaved off just more than 5 percent of the size. I know what you’re going to say now: But Ash, our image is still monstrously huge! There’s still way too much bloat in that image.
Well, to make our image even smaller, we now need to use the
alpine, or slimmed-down, Node.js base image. We can do this by changing our second build stage from
This reduces our production image down to 73MB — that’s a huge win! Now the savings we get from discarding our debris is more like a whopping 60 percent. Alright, we are really getting somewhere now!
This highlights another benefit of multi-stage builds: we can use separate Docker base images for each of our build stages. This means you can customize each build stage by using a different base image.
Say you have one stage that relies on some tools that are in a different image, or you have created a special Docker image that is custom for your build process. This gives us a lot of flexibility when constructing our build pipelines.How does it work?
You probably already guessed this: each stage or build process produces its own separate Docker image. You can see how this works in Figure 6.
The Docker image produced by a stage can be used by the following stages. Once the final image is produced, all the intermediate images are discarded; we take what we want for the final image, and the rest gets thrown away.
Adding more stages
Figure 6: Each stage of a multi-stage Docker build produces an image.
There’s no need to stop at two stages, although that’s often all that’s needed; we can add as many stages as we need. A specific example is illustrated in Figure 7.
Here we are building TypeScript code in stage 1 and our React client in stage 2. In addition, there’s a third stage that produces the final image from the results of the first two stages.
Figure 7: Using a Docker multi-stage build, we can create more complex build pipelines.
Now time to leave you with a few advanced tips to explore on your own:
Docker multi-stage builds enable us to create more complex build pipelines without having to resort to magic tricks. They help us slim down our production Docker images and remove the bloat. They also allow us to structure and modularize our build process, which makes it easier to test parts of our build process in isolation.
So please have some fun with Docker multi-stage builds, and don’t forget to have a look at the example code on GitHub.
Here’s the Docker documentation on multi-stage builds, too.