A Deep Dive Into Docker

A Deep Dive Into Docker

Docker: Enterprise Container Platform. Docker is a set of coupled software-as-a-service and platform-as-a-service products that use operating-system-level virtualization to develop and deliver software in packages called containers.

Docker: Enterprise Container Platform. Docker is a set of coupled software-as-a-service and platform-as-a-service products that use operating-system-level virtualization to develop and deliver software in packages called containers.

Introduction

Since its open source launch in 2013, Docker became one of the most popular pieces of technology out there. A lot of companies are contributing, and a huge amount of people are using and adopting it. But why is it so popular? What does it offer that was not there before? In this blog post we want to dive deeper into the internals of Docker to understand how it works.

The first part of this post will give a quick overview about the basic architectural concepts. In the second part we will introduce four main functionalities that form the foundation for isolation in Docker containers: 1) cgroups, 2) namespaces, 3) stackable image-layers and copy-on-write, and 4) virtual network bridges. In the third section there will be a discussion about opportunities and challenges when using containers and Docker. We conclude by answering some frequently asked questions about Docker.

Basic Architecture

"Docker is an open-source project that automates the deployment of applications inside software containers." - Wikipedia
People usually refer to containers when talking about operating system level virtualization. Operating system level virtualization is a method in which the kernel of an operating system allows the existence of multiple isolated application instances. There are many implementations of containers available, one of which is Docker.

**Docker **launches containers based off of images. An image is like a blueprint, defining what should be inside the container when it is being created. The usual way to define an image is through a Dockerfile. A Dockerfile contains instructions on how to build your image step by step (don't worry you will understand more about what is going on internally later on). The following Dockerfile, for example, will start from an image containing OpenJDK, install Python 3 there, copy the requirements.txt inside the image and then install all Python packages from the requirements file.

FROM openjdk:8u212-jdk-slim RUN apt-get update \ && apt-get install -y --no-install-recommends \ Python3=3.5.3-1 \ Python3-pip=9.0.1-2+deb9u1 \ && rm -rf /var/lib/apt/lists/* COPY requirements.txt requirements.txt RUN pip3 install --upgrade -r requirements.txt 

Images are usually stored in image repositories called Docker registries. Dockerhub is a public Docker registry. In order to download images and start containers you need to have a Docker host. The Docker host is a Linux machine which runs the Docker daemon (a daemon is a background process that is always running, waiting for work to be done).

In order to launch a container, you can use the Docker client, which submits the necessary instructions to the Docker daemon. The Docker daemon is also talking to the Docker registry if it cannot find the requested image locally. The following picture illustrates the basic architecture of Docker:

What is important to note already is that Docker itself does not provide the actual containerization but merely uses what is available in Linux. Let's dive into the technical details.

Container Isolation

Docker achieves isolation of different containers through the combination of four main concepts: 1) cgroups, 2) namespaces, 3) stackable image-layers and copy-on-write, and 4) virtual network bridges. In the following sub sections we are going to explain these concepts in detail.

Control Groups (cgroups)

The Linux operating system manages the available hardware resources (memory, CPU, disk I/O, network I/O, ...) and provides a convenient way for processes to access and utilize them. The CPU scheduler of Linux, for example, takes care that every thread will eventually get some time on a CPU core so that no applications are stuck waiting for CPU time.

Control groups (cgroups) are a way to assign a subset of resources to a specific group of processes. This can be used to, e.g., ensure that even if your CPU is super busy with Python scripts, your PostgreSQL database still gets dedicated CPU and RAM. The following picture illustrates this in an example scenario with 4 CPU cores and 16 GB RAM.

All Zeppelin notebooks started in the zeppelin-grp will utilize only core 1 and 2, while the PostgreSQL processes share core 3 and 4. Same applies to the memory. Cgroups are one important building block in container isolation as they allow hardware resource isolation.

Namespaces

While cgroups isolate hardware resources, namespaces isolate and virtualize system resources. Examples of system resources that can be virtualized include process IDs, hostnames, user IDs, network access, interprocess communication, and filesystems. Let's first dive into an example of process ID (PID) namespaces to make this more clear and then briefly discuss other namespaces as well.

PID Namespaces

The Linux operating system organizes processes in a so called process tree. The tree root is the first process that gets started after the operating system is booted and it has the PID 1. As only one process tree can exist and all other processes (e.g. Firefox, terminal emulators, SSH servers) need to be (directly or indirectly) started by this process. Due to the fact that this process initializes all other processes it is often referred to as the init process.

The following figure illustrates parts of a typical process tree where the init process started a logging service (syslogd), a scheduler (cron) and a login shell (bash):

1 /sbin/init +-- 196 /usr/sbin/syslogd -s +-- 354 /usr/sbin/cron -s +-- 391 login +-- 400 bash +-- 701 /usr/local/bin/pstree 

Inside this tree, every process can see every other process and send signals (e.g. to request the process to stop) if they wish. Using PID namespaces virtualizes the PIDs for a specific process and all its sub processes, making it think that it has PID 1. It will then also not being able to see any other processes except its own children. The following figure illustrates how different PID namespaces isolate the process sub trees of two Zeppelin processes.

1 /sbin/init | + ... | +-- 506 /usr/local/zeppelin 1 /usr/local/zeppelin +-- 2 interpreter.sh +-- 3 interpreter.sh +-- 511 /usr/local/zeppelin 1 /usr/local/zeppelin +-- 2 java 

Filesystem Namespaces

Another use case for namespaces is the Linux filesystem. Similar to PID namespaces, filesystem namespaces virtualize and isolate parts of a tree - in this case the filesystem tree. The Linux filesystem is organized as a tree and it has a root, typically referred to as /.

In order to achieve isolation on a filesystem level, the namespace will map a node in the filesystem tree to a virtual root inside that namespace. Browsing the filesystem inside that namespace, Linux does not allow you to go beyond your virtualized root. The following drawing shows part of a filesystem that contains multiple "virtual" filesystem roots inside the /drives/xx folders, each containing different data.

Other Namespaces

Besides the PID and the filesystem namespaces there are also other kinds of namespaces. Docker allows you to utilize them in order to achieve the amount of isolation you require. The user namespace, e.g., allows you to map a user inside a container to a different user outside. This can be used to map the root user inside the container to a non-root user outside, so the process inside the container acts like an admin inside but outside it has no special privileges.

Stackable Image Layers and Copy-On-Write

Now that we have a more detailed understanding of how hardware and system resource isolation helps us to build containers, we are going to take a look into the way that Docker stores images. As we saw earlier, a Docker image is like a blueprint for a container. It comes with all dependencies required to start the application that it contains. But how are these dependencies stored?

Docker persists images in stackable layers. A layer contains the changes to the previous layer. If you, for example, install first Python and then copy a Python script, your image will have two additional layers: One containing the Python executables and another one containing the script. The following picture shows a Zeppelin, an Spring and a PHP image, all based on Ubuntu.

In order not to store Ubuntu three times, layers are immutable and shared. Docker uses copy-on-write to only make a copy of a file if there are changes.

When starting a container-based on an image, the Docker daemon will provide you with all the layers contained in that image and put it in an isolated filesystem namespace for this container. The combination of stackable layers, copy-on-write, and filesystem namespaces enable you to run a container completely independent of the things "installed" on the Docker host without wasting a lot of space. This is one of the reasons why containers are more lightweight compared to virtual machines.

Virtual Network Bridge

Now we know ways to isolate hardware resources (cgroups) and system resources (namespaces) and how to provide each container with a predefined set of dependencies to be independent from the host system (image layers). The last building block, the virtual network bridge, helps us in isolating the network stack inside a container.

A network bridge is a computer networking device that creates a single aggregate network from multiple communication networks or network segments. Let's look at a typical setup of a physical network bridge connecting two network segments (LAN 1 and LAN 2):

Usually we only have a limited amount of network interfaces (e.g. physical network cards) on the Docker host and all processes somehow need to share access to it. In order to isolate the networking of containers, Docker allows you to create a virtual network interface for each container. It then connects all the virtual network interfaces to the host network adapter, as shown in the following picture:

The two containers in this example have their own eth0 network interface inside their network namespace. It is mapped to a corresponding virtual network interfaces veth0 and veth1 on the Docker host. The virtual network bridge docker0 connects the host network interface eth0 to all container network interfaces.

Docker gives you a lot of freedom in configuring the bridge, so that you can expose only specific ports to the outside world or directly wire two containers together (e.g. a database container and an application which needs access to it) without exposing anything to the outside.

Connecting the Dots

Taking the techniques and features described in the previous sub sections, we are now able to "containerize" our applications. While it is possible to manually create containers using cgroups, namespaces, virtual network adapters, etc., Docker is a tool that makes it convenient and with almost no overhead. It handles all the manual, configuration intensive tasks, making containers accessible to software developers and not only Linux specialists.

In fact there is a nice talk available from one of the Docker engineers where he demonstrates how to manually create a container, also explaining the details we covered in this sub section.

Opportunities and Challenges of Docker

By now, many people are using Docker on a daily basis. What benefits do containers add? What does Docker offer that was not there before? In the end everything you need for containerizing your applications was already available in Linux for a long time, wasn't it?

Let's look at some opportunities (not an exhaustive list of course) that you have when moving to a container-based setup. Of course there are not only opportunities, but also challenges that might give you a hard time when adopting Docker. We are also going to name a few in this section.

Opportunities

Docker enables DevOps. The DevOps philosophy tries to connect development and operations activities, empowering developers to deploy their applications themselves. You build it, you run it. Having a Docker based deployment, developers can ship their artifacts together with the required dependencies directly without having to worry about dependency conflicts. Also it allows developers to write more sophisticated tests and execute them faster, e.g., creating a real database in another container and linking it to their application on their laptop in a few seconds (see Testcontainers).

Containers increase the predictability of your deployment. No more "runs on my machine". No more failing application deployments because one machine has a different version of Java installed. You build the image once and you can run it anywhere (given there is a Linux Kernel and Docker installed).

High adoption rate and good integration with many prominent cluster managers. One big part about using Docker is the software ecosystem around it. If you are planning to operate at scale, you won't get around using one or the other cluster manager. It doesn't matter if you decide to let someone else manage your deployment (e.g. Google Cloud, Docker Cloud, Heroku, AWS, ...) or want to maintain your own cluster manager (e.g. Kubernetes, Nomad, Mesos), there are plenty of solutions out there.

Lightweight containers enable fast failure recovery or auto-scaling. Imagine running an online shop. During Christmas time, people will start hitting your web servers and your current setup might not be sufficient in terms of capacity. Given that you have enough free hardware resources, starting a few more containers hosting your web application will take only a few seconds. Also failing machines can be recovered by just migrating the containers to a new machine.

Challenges

Containers give a false sense of security. There are many pitfalls when it comes to securing your applications. It is wrong to assume that one way to secure them is to put them inside containers. Containers do not secure anything, per se. If someone hacks your containerized web application he might be locked into the namespaces but there are several ways to escape this depending on the setup. Be aware of this and put as much effort into security as you would without Docker.

Docker makes it easy for people to deploy half baked solutions. Pick your favorite piece of software and enter its name it to the Google search bar, adding "Docker". You will probably find at least one if not dozens of already publicly available images containing your software at Dockerhub. So why not just execute it and give it a shot? What can go wrong? Many things can go wrong. Things happen to look shiny and awesome when put into containers and people stop paying attention to the actual software and configuration inside.

The fat container anti-pattern results in large, hard-to-manage deployment artifacts. I have seen Docker images which require you to expose more than 20 ports for different applications inside when a the container. The philosophy of Docker is that one container should do one job and you should rather compose them instead of making them heavier. If you end up putting all your tools together in one container you lose all the advantages, might have different versions of Java or Python inside and end up with a 20 GB, unmanageable image.

Deep Linux knowledge might still be required to debug certain situations. You might have heard your colleague saying that XXX does not work with Docker. There are multiple reasons why this could happen. Some applications have issues running inside a bridged network namespace if they do not distinguish properly between the network interface they bind to and the one they advertise. Another issue can be related to cgroups and namespaces where default settings in terms of shared memory are not the same as on your favorite Linux distribution, leading to OOM errors when running inside containers. However, most of the issues are not actually related to Docker but to the application not being designed properly and they are not that frequent. Still they require some deeper understanding of how Linux and Docker works which not every Docker user has.

Frequently Asked Questions

Q: What's the difference between a container and a virtual machine?

Without diving too much into details about the architecture of virtual machines (VMs), let us look at the main difference between the two on a conceptual level. Containers run inside an operating system, using kernel features to isolate applications. VMs on the other hand require a hypervisor which runs inside an operating system. The hypervisor then creates virtual hardware which can be accessed by another set of operating systems. The following illustration compares a virtual machine based application setup and a container-based setup.

As you can see, the container-based setup has less overhead as it does not require an additional operating system for each application. This is possible because the container manager (e.g. Docker) uses operating system functionality directly to isolate applications in a more lightweight fashion.

Does that mean that containers are superior to virtual machines? It depends. Both technologies have their use cases and it sometimes even make sense to combine them, running a container manager inside a VM. There are many blog posts out there discussing the pros and cons of both solutions so we're not going to go into detail right now. It is important to understand the difference and to not see containers as some kind of "lightweight VM", because internally they are different.

Q: Do containers contain?

Looking at the definition of containers and what we've learned so far, we can safely say that it is possible to use Docker to deploy isolated applications. By combining control groups and namespaces with stackable image layers and virtual network interfaces plus a virtual network bridge, we have all the tools required to completely isolate an application, possibly also locking the process in the container. The reality shows that it's not that easy though. First, it needs to be configured correctly and secondly, you will notice that completely isolated containers don't make a lot of sense most of the time.

In the end your application somehow needs to have some side effect (persisting data to disk, sending packets over the network, ...). So you will end up breaking the isolation by forwarding network traffic or mounting host volumes into your filesystem namespace. Also it is not required to use all available namespace features. While the network, PID and filesystem namespace features are enabled by default, using the user ID namespace requires you to add extra configuration options.

So it is false to assume that just by putting something inside a container makes it secure. AWS, e.g., uses a lightweight VM engine called Firecracker for secure and multi-tenant execution of short-lived workloads.

Q: Do containers make my production environment more stable?

Some people argue that containers increase stability because they isolate errors. While this is true to the extent that properly configured namespaces and cgroups will limit side effects of one process going rogue, in practice there are some things to keep in mind.

As mentioned earlier, containers do only contain if configured properly and most of the time you want them to interact with other parts of your system. It is therefore possible to say that containers can help to increase stability in your deployment but you should always keep in mind that it does not protect your applications from failing.

Conclusion

Docker is a great piece of technology to independently deploy applications in a more or less reproducible and isolated way. As always, there is no one-size-fits-all solution and you should understand your requirements in terms of security, performance, deployability, observability, and so on, before choosing Docker as the tool of your choice.

Kubernetes Fundamentals - Learn Kubernetes from the Beginning

Kubernetes Fundamentals - Learn Kubernetes from the Beginning

Kubernetes is about orchestrating containerized apps. Docker is great for your first few containers. As soon as you need to run on multiple machines and need to scale/up down and distribute the load and so on, you need an orchestrator - you need Kubernetes

Kubernetes is about orchestrating containerized apps. Docker is great for your first few containers. As soon as you need to run on multiple machines and need to scale/up down and distribute the load and so on, you need an orchestrator - you need Kubernetes

This is the first part of a series of articles on Kubernetes, cause this topic is BIG!.

  • Part I - From the beginning, Part I, Basics, Deployment and Minikube: In this part, we cover why Kubernetes, some history and some basic concepts like deploying, Nodes, Pods.
  • Part II - Introducing Services and Labeling: In this part, we deepen our knowledge of Pods and Nodes. We also introduce Services and labeling using labels to query our artifacts.
  • Part III - Scaling: Here we cover how to scale our app
  • Part IV - Auto scaling: In this part we look at how to set up auto-scaling so we can handle sudden large increases of incoming requests
Resources Kubernetes

So what do we know about Kubernetes?

It's an open-source system for automating deployment, scaling, and management of containerized applications

Let'start with the name. It's Greek for Helmsman, the person who steers the ship. Which is why the logo looks like this, a steering wheel on a boat:

It's Also called K8s so K ubernete s, 8 characters in the middle are removed. Now you can impress your friends that you know why it's referred to as K8.

Here is some more Jeopardy knowledge on its origin. Kubernetes was born out of systems called Borg and Omega. It was donated to CNCF, Cloud Native Computing Foundation in 2014. It's written in Go/Golang.

If we see past all this trivia knowledge, it was built by Google as a response to their own experience handling a ton of containers. It's also Open Source and battle-tested to handle really large systems, like planet-scale large systems.

So the sales pitch is:

Run billions of containers a week, Kubernetes can scale without increasing your ops team

Sounds amazing right, billions of containers cause we are all Google size. No? :) Well even if you have something like 10-100 containers, it's for you.

In this part I hope to cover the following:

  • Why Kubernetes and Orchestration in General
  • Hello world: Minikube basics, talking through Minikube, simple deploy example
  • Cluster and basic commands, Nodes,
  • Deployments, what it is and deploying an app
  • Pods and Nodes, explain concepts and troubleshooting
Part I - From the beginning, Part I, Basics, Deployment and Minikube Why Orchestration

Well, it all started with containers. Containers gave us the ability to create repeatable environments so dev, staging, and prod all looked and functioned the same way. We got predictability and they were also light-weight as they drew resources from the host operating system. Such a great breakthrough for Developers and Ops but the Container API is really only good for managing a few containers at a time. Larger systems might consist of 100s or 1000+ containers and needs to be managed as well so we can do things like scheduling, load balancing, distribution and more.

At this point, we need orchestration the ability for a system to handle all these container instances. This is where Kubernetes comes in.

Getting started

Ok ok, let's say I buy into all of this, how do I get started?

Impatient ey, sure let's start to do something practical with Minikube

Ok, sounds good I'm a coder, I like practical stuff. What is Minikube?

Minikube is a tool that lets us run Kubernetes locally

Oh, sweet, millions of containers on my little machine?

Well, no, let's start with a few and learn Kubernetes basics while at it.

Installation

To install Minikube lets go to this installation page

It's just a few short steps that means we install

  • a Hypervisor
  • Kubectl (Kube control tool)
  • Minikube

Run

Get that thing up and running by typing:

minikube start

It should look something like this:

You can also ensure that kubectl have been correctly installed and running:

kubectl version

Should give you something like this in response:

Ok, now we are ready to learn Kubernetes.

Learning kubectl and basic concepts

In learning Kubernetes lets do so by learning more about kubectl a command line program that lets us interact with our Cluster and lets us deploy and manage applications on said Cluster.

The word Cluster just means a group of similar things but in the context of Kubernetes, it means a Master and multiple worker machines called Nodes. Nodes were historically called Minions

, but not so anymore.

The master decides what will run on the Nodes, which includes things like scheduled workloads or containerized apps. Which brings us to our next command:

kubectl get nodes

This should give us a result like this:

What this tells us what Nodes we have available to do work.

Next up let's try to run our first app on Kubernetes with the run command like so:

kubectl run kubernetes-first-app --image=gcr.io/google-samples/kubernetes-bootcamp:v1 --port=8080

This should give us a response like so:

Next up lets check that everything is up and running with the command:

kubectl get deployments

This shows the following in the terminal:

In putting our app on the Kluster, by invoking the run command, Kubernetes performed a few things behind the scenes, it:

  • searched for a suitable node where an instance of the application could be run, there was only one node so it got chosen
  • scheduled the application to run on that Node
  • configured the cluster to reschedule the instance on a new Node when needed

Next up we are going to introduce the concept Pod, so what is a Pod?

A Pod is the smallest deployable unit and consists of one or many containers, for example, Docker containers. That's all we are going to say about Pods at the moment but if you really really want to know more have a read here

The reason for mentioning Pods at this point is that our container and app is placed inside of a Pod. Furthermore, Pods runs in a private isolated network that, although visible from other Pods and services, it cannot be accessed outside the network. Which means we can't reach our app with say a curl command.

We can change that though. There is more than one way to expose our application to the outside world for now however we will use a proxy.

Now open up a 2nd terminal window and type:

kubectl proxy

This will expose the kubectl as an API that we can query with HTTP request. The result should look like:

Instead of typing kubectl version we can now type curl http://localhost:8001/version and get the same results:

The API Server inside of Kubernetes have created an endpoint for each pod by its pod name. So the next step is to find out the pod name:

kubectl get pods

This will list all the pods you have, it should just be one pod at this point and look something like this:

Then you can just save that down to a variable like so:

Lastly, we can now do an HTTP call to learn more about our pod:

curl http://localhost:8001/api/v1/namespaces/default/pods/$POD_NAME

This will give us a long JSON response back (I trimmed it a bit but it goes on and on...)

Maybe that's not super interesting for us as app developers. We want to know how our app is doing. Best way to know that is looking at the logs. Let's do that with this command:

kubectl logs $POD_NAME

As you can see below we know get logs from our app:

Now that we know the Pods name we can do all sorts of things like checking its environment variables or even step inside the container and look at the content.

kubectl exec $POD_NAME env

This yields the following result:

Now lets step inside the container:

kubectl exec -ti $POD_NAME bash

We are inside! This means we can see what the source code looks like even:

cat server.js

Inside of our container, we can now reach the running app by typing:

curl http://localhost:8080

Summary Part I

This is where we will stop for now.
What did we actually learn?

  • Kubernetes, its origin what it is
  • Orchestration why you will soon need it
  • Concepts like Master, Nodes and Pods
  • Minikube, kubectl and how to deploy an image onto our Cluster
Part II - Introducing Services and Labeling

In this part we will cover the following:

  • Deepen our knowledge on Pods and Nodes
  • Introduce Services and labeling
  • Perform an exercise, involving setting labels on Pods and use labels to query our artifacts
Concepts revisited

When we create a Deployment on Kubernetes, that Deployment creates Pods with containers inside them. So Pods are tied to Nodes and will continue to exist until terminated or deleted. Let's try to educate ourselves a bit more on Pods, Nodes and let's also introduce a new topic Services.

Pods

Pods are the atomic unit on the Kubernetes platform, i.e smallest possible deployable unit

We've stated the above before but it's worth mentioning again.

What else is there to know?

A Pod is an abstraction that represents a group of one or more containers, for example, Docker or rkt, and some shared resources for those containers. Those resources include:

  • Shared storage, as Volumes
  • Networking, as a unique cluster IP address
  • Information about how to run each container, such as the container image version or specific ports to use

A Pod can have more than one container. If it does contain more than one container it is so the other containers can support the primary application.
Typical examples of helper applications are data pullers, data pushers, and proxies. You can read more on that use case here

  1. The containers in a Pod share an IP Address and port space and are:
  2. Always co-located
  3. Co-scheduled

Let me show you an image to make it easier to visualize:

As we can see above a Pod can have a lot of different artifacts in them that are able to communicate and support the app in some way.

Nodes

A Pod always runs on a Node

So Node is the Pods parent?

Yes.

A Node is a worker machine and may be either a virtual or a physical machine, depending on the cluster

Each Node is managed by the Master. A Node can have multiple pods.

So it's a one to many relationship

The Kubernetes master automatically handles scheduling the pods across the Nodes in the cluster

Every Kubernetes Node runs at least a:

  • Kubelet, is responsible for the pod spec and talks to the cri interface

  • Kube proxy, is the main interface for coms between nodes

  • A container runtime, (like Docker, rkt) responsible for pulling the container image from a registry, unpacking the container, and running the application.

Ok so a Node contains a Kubelet and container runtime and one to many Pods. I think I got it.

Let's show an image to make this info stick, cause it's quite important that we know what goes on, at least at a high level:

Services

Pods are mortal, they can die. Pods, in fact, have a lifecycle.

When a worker node dies, the Pods running on the Node are also lost.

What happens to our apps? :(

You might think them and their data are lost but not so. The whole point with Kubernetes is to not let that happen. We normally deploy something like a ReplicaSet.

A ReplicaSet, what do you mean?

A ReplicaSet is a high-level artifact that can drive the cluster back to desired state via the creation of new Pods to keep your application running.

Ok so if a Pod goes down the ReplicaSet just creates a new Pod/s in its place?

Yes, exactly that. If you focus on defining a desired state the rest is up to Kubernetes.

Phew sounds really great then.

This concept of desired state is a very important one. You need to specify how many containers you want of each kind, at all times.

Oh so 4 database containers, 3 services etc?

Yes exactly.

So you don't have to care about the details just tell Kubernetes what state you want and it does the rest. If something goes up, Kubernetes ensures it comes back up again to desired state.

Each Pod in a Kubernetes cluster has a unique IP address, even Pods on the same Node, so there needs to be a way of automatically reconciling changes among Pods so that your applications continue to function.

Ok?

Yea, think like this. If a Pod containing your app goes down and another Pod is created in its place, running your app. Users should still be able to use your app after that.

Ok I got it. Makes me think...

The motivation for a Service

You should never refer to a Pod by it's IP address, just think what happens when a Pod goes down and comes back up again but this time with a different IP. It is for that reason a Service exists.

A Service in Kubernetes is an abstraction which defines a logical set of Pods and a policy by which to access them.

Makes me think of a routers and subnets

Yea I guess you can say there is a resemblance in there somewhere.

Services enable a loose coupling between dependent Pods and are defined using YAML or JSON file, just like all Kubernetes objects.

That's handy, just JSON and YAML :)

Services and Labels

The set of Pods targeted by a Service is usually determined by a LabelSelector.

Although each Pod has a unique IP address, those IPs are not exposed outside the cluster without a Service. We can expose them through a proxy though as we showed in part I.

Wait, go back a second here, you said LabelSelector. I wasn't quite following?

Remember how we couldn't refer to Pods by IP, cause Pods might go down and a new Pod could come back in its place?

Yes

Well, labels are the answer to how Services and Pods are able to communicate. This is what we mean by loose coupling. By applying labels like for example frontend, backend, release and so on to Pods, we are able to refer to Pods by their logical name rather than their specifics, i.e IP number.

Oh I get it, so it's a high-level domain language

Mm, kind of.

Services and Traffic

Services allow your applications to receive traffic.

Services can be exposed in different ways by specifying a type in ServiceSpec, service specification.

  • ClusterIP (default) - Exposes the Service on an internal IP in the cluster. This type makes the Service only reachable from within the cluster.
  • NodePort - Exposes the Service on the same port of each selected Node in the cluster using NAT. Makes a Service accessible from outside the cluster using :. Superset of ClusterIP.
  • LoadBalancer - Creates an external load balancer in the current cloud (if supported) and assigns a fixed, external IP to the Service. Superset of NodePort.
  • ExternalName - Exposes the Service using an arbitrary name (specified by externalName in the spec) by returning a CNAME record with the name. No proxy is used. This type requires v1.7 or higher of kube-dns.

Ok I think I get it. Ensure I'm speaking externally to a Service instead of specific Pods. Depending on what I expose the Service as, that leads to different behavior?

Yea that's correct.

You said something about labels though, how do we create and apply them to Pods?

Yea lets talk about that next.

Labels

As we just mentioned, Services are the abstraction that allows pods to die and replicate in Kubernetes without impacting your application.

Now, Services match a set of Pods using labels and selectors, it allows us to operate on Pods like a group.

Labels are key/value pairs attached to objects and can be used in any number of ways:

  • Designate objects for development, test, and production
  • Embed version tags
  • Classify an object using tags

Labels can be attached to objects at creation time or later on. They can be modified at any time.

Lab - Fun with Labels and kubectl

It's a good idea to have read the first part of this series where we create a deployment. If you haven't you need to first create a deployment like so:

kubectl run kubernetes-first-app --image=gcr.io/google-samples/kubernetes-bootcamp:v1 --port=8080

Now we should be good to go.

Ok. I know you are probably all tired from all theory by now.

I bet you are just itching to learn more hands on Kubernetes with kubectl.

Well, the time for that has come :). We will do two things:

  1. Create a Service and learn how we can expose our app using said Service
  2. Learn about Labeling and how we can improve our querying game by having appropriate labels on our artifacts.

Let's create a new service.

We will get acquainted with the expose command.

Let's check for existing pods,

kubectl get pods

Next let's see what services we have:

kubectl get services

Next lets create a Service like so:

kubectl expose deployment/kubernetes-first-app --type="NodePort" --port 8080

As you can see above we are just targeting one of our deployments kubernetes-first-app and referring to it with [type]/[deployment name] and type being deployment.

We expose it as service of type NodePort and finally, we choose to expose it at port 8080.

Now run kubectl get services again and see the results:

As you can see we now have two services in use, our basic kubernetes service and our newly created kubernetes-first-app.

Next up we need to grab the port of our service and assign that to a variable:

export NODE_PORT=$(kubectl get services/kubernetes-first-app -o go-template='{{(index .spec.ports 0).nodePort}}')
echo NODE_PORT=$NODE_PORT

We now have a our port stored on environment variable NODE_PORT and we are ready to start communicating with our service like so:

curl $(minikube ip):$NODE_PORT

Which leads to the following output:

Creating and applying Labels

When we created our deployment and our Pod, it was automatically assigned with a label.

By typing

kubectl describe deployment

we can see the name of said label.

Next up we can query the pods by that same label

kubectl get pods -l run=kubernetes-first-app

Above we are using -l to query for a specific label and kubernetes-bootcamp as the name of the label. This gives us the following result:

You can do a similar query to your services:

kubectl get services -l run=kubernetes-first-app

That just shows that you can query on different levels, for specific Pods or Services that have Pods with that label.

Next up we will look at how to change the label

First let's get the name of the pod, like so:

POD_NAME=kubernetes-first-app-669789f4f8-6glpx

Above I'm just assigning what my Pod is called to a variable POD_NAME. Check with a kubectl getpods what your Pod is called.

Then we can add/apply the new label like so:

kubectl label pod $POD_NAME app=v1

Verify that the new label have been set, like so:

kubectl describe pod

or

kubectl describe pods $POD_NAME

As you can see from the result our new label app=v1 has been appended to existing labels.

Now we can query like so:

kubectl get pods -l app=v1

That's pretty much how labeling works, how to get available labels, apply them and use them in a query. Ensure to give them a descriptive name like an app version, a certain environment or a name like frontend or backend, something that makes sense to your situation.

Clean up

Ok, so we created a service. We should learn how to clean up after ourselves. Run the following command to remove our service:

kubectl delete service -l run=kubernetes-bootcamp

Verify the service is no longer there with:

kubectl get services

also, ensure our exposed IP and port can no longer be reached:

curl $(minikube ip):$NODE_PORT

Just because the service is gone doesn't mean the app is gone. The app should still be reachable on:

kubectl exec -ti $POD_NAME curl localhost:8080

Summary Part II

So what did we learn? We learned a bit more on Pods and Nodes. Furthermore, we learned that we shouldn't speak directly to Pods but rather use a high-level abstraction such as Services. Services use labels as a way to define a domain language and apply those to different Pods.

Ok, so we understand a bit more on Kubernetes and how different concepts relate. We mentioned something called desired state a number of times but we didn't go into detail on how to set such a state. That's our next part in this series where we will cover how to set the desired state and how Kubernetes maintains it, so stay tuned.

Part III - Scaling

This third part aims to show how you scale your application. We can easily set the number of Replicas we want of a certain application and let Kubernetes figure out how to do that. This is us defining a so-called desired state.

When traffic increases, we will need to scale the application to keep up with user demand. We've talked about deployments and services, now lets talk scaling.

What does scaling mean in the context of Kubernetes?

We get more Pods. More Pods that are scheduled to nodes.

Now it's time to talk about desired state again, that we mentioned in previous parts.

This is where we relinquish control to Kubernetes. All we need to do is tell Kubernetes how many Pods we want and Kubernetes does the rest.

So we tell Kubernetes about the number of Pods we want, what does that mean? What does Kubernetes do for us?

It means we get multiple instances of our application. It also means traffic is being distributed to all of our Pods, ie. load balancing.

Furthermore, Kubernetes, or more specifically, services within Kubernetes will monitor which Pods are available and send traffic to those Pods.

Scaling demo Lab

If you haven't followed the first two parts I do recommend you go back and have a read. What you need for the following to work is at least a deployment. So if you haven't created one, here is how:

kubectl run kubernetes-first-app --image=gcr.io/google-samples/kubernetes-bootcamp:v1 --port=8080

Let's have a look at our deployments:

kubectl get deployments

Let's look closer at the response we get:

We have three pieces of information that are important to us. First, we have the READY column in which we should read the value in the following way, CURRENT STATE/DESIRED STATE. Next up is the UP_TO_DATE column which shows the number of replicas that were updated to match the desired state.
Lastly, we have the AVAILABLE column that shows how many replicas we have available to do work.

Let's scale

Now, let's do some scaling. For that we will use the scale command like so:

kubectl scale deployments/kubernetes-first-app --replicas=4 

as we can see above the number of replicas was increased to 4 and kubernetes is thereby ready to load balance any incoming requests.

Let's have a look at our Pods next:

When we asked for 4 replicas we got 4 Pods.

We can see that this scaling operation took place by using the describe command, like so:

kubectl describe deployments/kubernetes-first-app

In the above image, we are given quite a lot of information on our Replicas for example, but there is some other information in there that we will explain later on.

Does it load balance?

The whole point with the scaling was so that we could balance the load on incoming requests. That means that not the same Pod would handle all the requests but that different Pods would be hit.
We can easily try this out, now that we have scaled our app to contain 4 replicas of itself.

So far we used the describe command to describe the deployment but we can use it to describe the IP and port of. Once we have the IP and port we can then hit it with different HTTP requests.

kubectl describe services/kubernetes-first-app

Especially look at the NodePort and the Endpoints. NodePort is the port value that we want to hit with an HTTP request.

Now we will actually invoke the cURL command and ensure that it hits a different port each time and thereby prove our load balancing is working. Let's do the following:

NODE_PORT=30450

Next up the cURL call:

curl $(minikube ip):$NODE_PORT

As you can see above we are doing the call 4 times. Judging by the output and the name of the instance we see that we are hitting a different Pod for each request. Thereby we see that the load balancing is working.

Scaling down

So far we have scaled up. We managed to go from one Pod to 4 Pods thanks to the scale command. We can use the same command to scale down, like so:

kubectl scale deployments/kubernetes-first-app --replicas=2 

Now if we are really fast adding the next command we can see how the Pods are being removed as Kubernetes is trying to adjust to desired state.

2 out of 4 Pods are saying Terminating as only 2 Pods are needed to maintain the new desired state.

Running our command again we see that only 2 Pods remain and thereby our new desired state have been reached:

We can also look at our deployment to see that our scale instruction has been parsed correctly:

Self-healing

Self-healing is Kubernetes way of ensuring that the desired state is maintained. Pods don't self heal cause Pods can die. What happens is that a new Pod appears in its place, thanks to Kubernetes.

So how do we test this?

Glad you asked, we can delete a Pod and see what happens. So how do we do that? We use the delete command. We need to know the name of our Pod though so we need to call get pods for that. So let's start with that:

kubectl get pods

Then lets pick one of our two Pods kubernetes-first-app-669789f4f8-6glpx and assign it to a variable:

POD_NAME=kubernetes-first-app-669789f4f8-6glpx

Now remove it:

kubectl delete pods $POD_NAME

Let's be quick about it and check our Pod status with get pods. It should say Terminating like so:

Wait some time and then echo out our variable $POD_NAME followed by get pods. That should give you a result similar to the below.

So what does the above image tell us? It tells us that the Pod we deleted is truly deleted but it also tells us that the desired state of two replicas has been achieved by spinning up a new Pod. What we are seeing is * self-healing* at work.

Different ways to scale

Ok, we looked at a way to scale by explicitly saying how many replicas we want of a certain deployment. Sometimes, however, we might want a different way to scale namely auto-scaling. Auto-scaling is about you not having to set the exact number of replicas you want but rather rely on Kubernetes to create the number of replicas it thinks it needs. So how would Kubernetes know that? Well, it can look at more than one thing but a common metric is CPU utilization. So let's say you have a booking site and suddenly someone releases Bruce Springsteen tickets you are likely to want to rely on auto-scaling, cause the next day when the tickets are all sold out you want the number of Pods to go back to normal and you wouldn't want to do this manually.

Auto-scaling is a topic I plan to cover more in detail in a future article so if you are really curious how that is done I recommend you have a look here

Summary Part III

Ok. So we did it. We managed to scale an app by creating replicas of it. It wasn't so hard to accomplish. We showed how we only needed to provide Kubernetes with a desired state and it would do its utmost to preserve said state, also called * self-healing*. Furthermore, we mentioned that there was another way to scale, namely auto-scaling but decided to leave that topic for another article. Hopefully, you are now more in awe of how amazing Kubernetes is and how easy it is to scale your app.

Part IV - Auto scaling

In this article, we will cover the following:

  • Why auto scaling, we will discuss different scenarios in which it makes sense to rely on auto scaling over defining it statically like we do with desired state
  • How, lets talk about Horizontal Auto Scaling the concept/feature that allows us to scale in an elastic way.
  • Lab - lets scale, we will look at how to actually set this up in kubectl and simulate a ton of incoming requests. We will then inspect the results and see that Kubernetes acts the way we think
Why

So in our last part, we talked about desired state. That's an OK strategy until something unforeseen happens and suddenly you got a great influx of traffic. This is likely to happen to businesses such as e-commerce around a big sale or a ticket vendor when you release tickets to a popular event.

Events like these are an anomaly which forces you to quickly scale up. The other side of the coin though is that at some point you need to scale down or you suddenly have overcapacity you might need to pay for. What you really want is for the scaling to act in an elastic way so it scaled up when you need it to and scales down when there is less traffic.

How

Horizontal auto-scaling, what does it mean?

It's a concept in Kubernetes that can scale the number of Pods we need. It can do so on a replication controller, deployment or replica set. It usually looks at CPU utilization but can be made to look at other things by using something called custom metrics support, so it's customizable.

It consists of two parts a resource and a controller. The controller checks utilization, or whatever metric you decided, to ensure that the number of replicas matches your specification. If need be it spins up more Pods or removes them. The default is checking every 15 seconds but you can change that by looking at a flag called --horizontal-pod-autoscaler-sync-period.

The underlying algorithm that decides the number of replicas looks like this:

desiredReplicas = ceil[currentReplicas * ( currentMetricValue / desiredMetricValue )]

Lab - lets scale

Ok, the first thing we need to do is to scale our deployment to use something other than desired state.

We have two things we need to specify when we do autoscaling:

  • min/max, we define a minimum and maximum in terms of how many Pods we want
  • CPU, in this version we set a certain CPU utilization percentage. When it goes above that it scales out as needed. Think of this one as an IF clause, if CPU value greater than the threshold, try to scale

Set up

Before we can attempt our scaling experiment we need to make sure we have the correct add-ons enabled. You can easily see what add-ons you have enabled by typing:

minikube addons list

If it looks like the above we are all good. Why am I saying that? Well, what we need, to be able to auto-scale, is that heapster and metrics-server add ons are enabled.

Heapster enables Container Cluster Monitoring and Performance Analysis.

Metrics server provide metrics via the resource metrics API. Horizontal Pod Autoscaler uses this API to collect metrics

We can easily enable them both with the following commands (we will need to for auto-scaling to show correct data):

minikube addons enable heapster

and

minikube addons enable metrics-server

We need to do one more thing, namely to enable Custom metrics, which we do by starting minikube with such a flag like so:

minikube start --extra-config kubelet.EnableCustomMetrics=true

Ok, now we are good to go.

Running the experiment

We need to do the following to run our experiment

  • Create a deployment
  • Apply autoscaling
  • Bombard the deployment with incoming requests
  • Watch the auto scaling how it changes

Create a deployment

kubectl run php-apache --image=k8s.gcr.io/hpa-example --requests=cpu=200m --expose --port=80

Above we are creating a deployment php-apache and expose it as a service on port 80. We can see that we are using the image k8s.gcr.io/hpa-example

It should tell us the following:

service/php-apache created
deployment.apps/php-apache created

Autoscaling

Next up we will use the command autoscale. We will use it like so:

kubectl autoscale deployment php-apache --cpu-percent=50 --min=1 --max=10

It should say something like:

horizontalpodautoscaler.autoscaling/php-apache autoscaled

Above we are applying the auto-scaling on the deployment php-apache and as you can see we are applying both min-max and cpu based auto scaling which means we give a rule for how the auto scaling should happen:

If CPU load is >= 50% create a new Pod, but only maximum 10 Pods. If the load is low go back gradually to one Pod

Bombard with requests

Next step is to send a ton of requests against our deployment and see our auto-scaling doing its work. So how do we do that?

First off let's check the current status of our horizontal pod auto-scaler or hpa for short by typing:

kubectl get hpa

This should give us something like this:

The above shows us two pieces of information. The first is the TARGETS column which shows our CPU utilization, actual usage/trigger value. The next bit of interest is the column REPLICAS that shows us the number of copies, which is 1 at the moment.

For our next trick open up a separate terminal tab. We need to do is to set things up so we can send a ton of requests.

Next up we create a container using this command:

kubectl run -i --tty load-generator --image=busybox /bin/sh

This should take us to a prompt within the container. This is followed by:

while true; do wget -q -O- http://php-apache.default.svc.cluster.local; done

The command above should result in something looking like this.

This will just go on and on until you hit CTRL+C, but leave it be for now.

This throws a ton on requests in while true loop.

I thought while true loops were bad?

They are but we are only going to run it for a minute so that the auto scaling can happen. Yes, the CPU will sound a lot but don't worry :)

Let this go on for a minute or so, then enter the following command into the first terminal tab (not the one running the requests), like so:

kubectl get hpa

It should now show something like this:

As you can see from the above the column TARGETS looks different and now says 339%/50% which means the current load on the CPU and REPLICAS is 7 which means it has gone from 1 to 7 replicas. So as you can see we have been bombarding it pretty hard.

Now go to the second terminal and hit CTRL+C or you will have a situation like this:

It will actually take a few minutes for Kubernetes to cool off and get the values back to normal. A first look at the Pods situation shows us the following:

kubectl get pods

As you can see we have 7 Pods up and running still but let's wait a minute or two and it should look like this:

Ok, now we are back to normal.

Summary Part IV

Now we did some great stuff in this article. We managed to set up auto-scaling, bombard it with requests and without frying our CPU, hopefully ;)

We also managed to learn some new Kubernetes commands while at it and got to see auto-scaling at work giving us new Pods based on our specification.

Docker Best Practices for Node Developers

Docker Best Practices for Node Developers

Welcome to the "Docker Best Practices for Node Developers"! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

Welcome to the best course on the planet for using Docker with Node.js! With your basic knowledge of Docker and Node.js in hand, Docker Mastery for Node.js is a course for anyone on the Node.js path. This course will help you master them together.

My talk on all the best of Docker for Node.js developers and DevOps dealing with Node apps. From DockerCon 2019. Get the full 9-hour training course with my coupon at http://bit.ly/365ogba

Get the source code for this talk at https://github.com/BretFisher/dockercon19

Some of the many cool things you'll do in this course
  • Build Node.js Images that auto-scan for security vulnerabilities
  • Use Docker's cutting-edge BuildKit with SSH Agents and NPM Caches for better image building
  • Use docker-compose with Visual Studio Code for full Node.js debug support
  • Use BuildKit and Multi-stage Builds to create minimal and flexible Dockerfiles
  • Build custom Node.js images using distro's like CentOS and Alpine
  • Test Docker init, tini, and Node.js as a PID 1 process in containers
  • Create Node.js apps that properly startup and respond to healthchecks
  • Develop ARM based Node.js apps with Docker Desktop, and deploy to AWS A1 Servers
  • Build graceful shutdown code into your apps for zero-downtime deploys
  • Dig into HTTP connections with orchestration, and how Proxies can help
  • Study examples of Docker Swarm and Kubernetes deployments for Node.js
  • Spend time Migrating traditional (legacy) Node.js apps into containers
  • Simplify your microservice solutions with advanced Docker Compose features
What you will learn in this course

You'll start with a quick review about getting set up with Docker, as well as Docker Compose basics. That way we're on the same page for the basics.

Then you'll jump into Node.js Dockerfile basics, that way you'll have a good Dockerfile foundation for new features we'll add throughout the course.

You'll be building on all the different things you learn from each Lecture in the course. Once you have the basics down of Compose, Dockerfile, and Docker Image, then you'll focus on nuances like how Docker and Linux control the Node process and how Docker changes that to make sure you know what options there are for starting up and shutting down Node.js and the right way to do it in different scenarios.

We'll cover advanced, newer features around making the Dockerfile the most efficient and flexible as possible using things like BuildKit and Multi-stage.

Then we'll talk about distributed computing and cloud design to ensure your Node.js apps have 12-factor design in your containers, as well as learning how to migrate old apps into this new way of doing things.

Next we cover Compose and its awesome features to get really efficient local development and test set-up using the Docker Compose command line and Docker Compose YAML file.

With all this knowledge, you'll progress to production concerns and making images production-ready.

Then we'll jump into deploying those containers and running them in production. Whether you use Docker Engine or orchestration with Kubernetes or Swarm, I've got you covered. In addition, we'll cover HTTP connections and reverse proxies for connection handling and routing with multi-container systems.

Lastly, you'll get a final, big assignment where you'll be building and deploying a large, complex solution, including multiple Node.js containers that are doing different things. You'll build Docker images, Dockerfiles, and compose files, and deploy them to a server to test. You'll need to check whether connections failover properly. You'll basically take everything you've learned and apply it in one big project!

A beginner’s guide to Kubernetes Architecture

A beginner’s guide to Kubernetes Architecture

Kubernetes is an open-source container orchestration engine for automating deployment, scaling, and management of containerized applications. In this post, we will go through the architecture of Kubernetes for beginners and what are the different options to get started for beginners.

Kubernetes is an open-source container orchestration engine for automating deployment, scaling, and management of containerized applications. In this post, we will go through the architecture of Kubernetes for beginners and what are the different options to get started for beginners.

  • Kubernetes Architecture
  • Different Options To Get Started
  • Minikube
  • Kubernetes Dashboard
  • How To Deploy In Cluster
  • Some Examples
  • Summary
  • Conclusion
Kubernetes Architecture

Kubernetes is the orchestration platform for the containerized applications. It follows the declarative configuration which defines the desired state of the applications and Kubernetes work hard to maintain that state throughout the lifecycle.

It’s very important for anyone to understand the architecture of the Kubernetes before getting started. Let’s dive into the architecture principles of this tool.
A beginner’s guide to Kubernetes Architecture

If you look at the above diagram, we have master nodes and worker nodes. Worker nodes are managed by the master. The worker node has the following services to be able to run the container applications in the pods.

  • Kubelet
  • Kube-proxy
  • container runtime

Kubelet

This is the most important service in the Kubernetes which is responsible for the execution of the container execution layer. Without kubelet, Kubernetes is just REST API backed by key-value store. Kubernetes executes isolated container application by default. containers are not only isolated from each other and they are also isolated from the underlying host system. This is critical to decoupling the management of individual applications from each other and from management of the underlying cluster physical/virtual infrastructure.

API admission control may reject pods or add additional scheduling constraints to them, but Kubelet is the final arbiter of what pods can and cannot run on a given node, not the schedulers or DaemonSets.

Kube-proxy

This provides an abstraction layer for the group of pods in the node under a common access policy, for example, load balancer. Every node implements Kube-proxy which provides the virtual IP address for the clients to access the dynamic set of pods. This provides a highly available load balancing solution with low-performance overhead.

container runtime

This is responsible for running the container such as Docker, rkt, containerd

The main components of the master nodes are as follows

  • API Server
  • Scheduler
  • Controller-manager
  • etcd

API Server

This is the basis for all the communication in the cluster. All the components in the cluster communicate through this. It exposes the kubernetes API.

Scheduler

The scheduler is responsible for assigning applications or kubernetes objects to the worker node. It is responsible for placing the pods on the nodes based on the resource requirements.

Controller-manager

This maintains the cluster such as node failures, maintaining the correct amount of pods, for example, if you deploy the application with 3 replicas, it makes sure that there are three pods running at any particular point of time.

etcd

This is the key value store that stores the cluster configuration. If you want to back up the cluster, all you need to save is this key value distributed store.

Different Options To Get Started

Once you understand the basic principles and architecture of kubernetes there are different options to get started or start practicing kubernetes. We can build a single node cluster and multi-node cluster. Here are these options.

Minikube

Minikube is a tool that runs a single-node Kubernetes cluster in a virtual machine on your personal computer. If you want to practice and deploy kubernetes objects on your laptop, this is a great tool. The only problem with this is that it has only one node which is master. You don’t get to experience the multi-node deployments of kubernetes. But, this is a great starting tool to get used to kubernetes. Here are the instructions on how to install it on your laptop.

katacoda

You can actually run kubernetes scenarios right out of the box in the interactive browser-based terminal. You can just sign in and start working on these scenarios.

[]Google Kubernetes Engine(https://cloud.google.com/kubernetes-engine/docs/concepts/kubernetes-engine-overview "")

Google Kubernetes Engine provides a managed environment for deploying, managing, and scaling your containerized applications using Google infrastructure. You can create a free tier account and you can start a multi-node cluster right away and start working on it.

Microsoft Azure Kubernetes Service

The fully managed Azure Kubernetes Service (AKS) makes deploying and managing containerized applications easy. It offers serverless Kubernetes, an integrated continuous integration and continuous delivery (CI/CD) experience, and enterprise-grade security and governance. You can create a free tier account and start with this right away.

Amazon Elastic Kubernetes Service

Amazon Elastic Kubernetes Service (Amazon EKS) makes it easy to deploy, manage, and scale containerized applications using Kubernetes on AWS. You can create a free tier account with AWS and start working.

Minikube

Minikube is a tool that runs a single-node Kubernetes cluster in a virtual machine on your personal computer. In this Section, we install Minikube, any other prerequisites and test some commands with kubectl.

These are all the required things to run Minikube on your laptop.

  • Set up Kubectl
  • Install VirtualBox
  • Install Minikube

The first thing we need is to install and set up kubectl. Here are the instructions from kubernetes official docs. Once you install that one based on your OS check the version with the YAML format with this commandkubectl version --client -o yaml. Since we didn’t install minikube yet we are checking only client version. If you are seeing this you are good to go.
A beginner’s guide to Kubernetes Architecture

VirtualBox is the next thing we need to install. When we start minikube, it actually runs in the VirtualBox. You can follow the Installing VirtualBox section of the below article or you can follow official docs here.
Now it’s time to install Minikube on your laptop. Follow these instructions.

Install Minikube

Once installed, test the version and start it with these commands minikube version and minikube start
A beginner’s guide to Kubernetes Architecture

You can check the version of kubectl to get the server version along with the client. Let’s test with some commands.

// get component status
kubectl get componentstatus// get nodes. you get only master node since minikube is single node // cluster
kubectl get nodeskubectl get pods
Kubernetes Dashboard

Kubernetes dashboard is a web-based UI that can be used to deploy containerized applications into kubernetes cluster, troubleshoot applications and manage resources in the cluster. You can also get the complete view of resources or objects deployed in the cluster. You can also create the deployment using the deployment wizard.

Kubernetes Dashboard UI is not installed by default. We have to use the following command to install it.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta1/aio/deploy/recommended.yaml

A beginner’s guide to Kubernetes Architecture

Once installed, we need to run this commandkubectl proxy so that the dashboard is available at this location. When you first hit this URL, you will see the below screen.

http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
A beginner’s guide to Kubernetes Architecture

There are two ways to access the dashboard one is to create a service account and another way is to use a kubeconfig file. Let’s use the token to sign into the dashboard.

You need to follow the documentation here to create a token. Once you create the token, you can sign in with the token. Here is the dashboard view. I have some objects already in the cluster. Here is the kubernetes dashboard to view all the objects in the cluster in the UI.
A beginner’s guide to Kubernetes Architecture

The token expires after a certain amount of time. When it does, generate the token again with the below command and sign in again.

kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep admin-user | awk '{print $1}')
How To Deploy In Cluster

You create objects in the cluster by feeding YAML or json file into the API server. The YAML file is converted to JSON format before submitted to the API server.

We use kubectl to create and update objects in the cluster. Kubectl create just creates the object in the cluster and kubectl apply creates and updates the objects in the cluster. This is the recommended way of managing Kubernetes applications on production.

Here is an example of kubernetes object pod creating and updating with the YAML file.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: null
  labels:
    run: nginx
  name: nginx
spec:
  containers:
  - image: nginx
    name: nginx
    resources: {}
  dnsPolicy: ClusterFirst
  restartPolicy: Never
status: {}
// raw file of the above
kubectl create -f https://gist.githubusercontent.com/bbachi/bcdcfaab157d6461b64cd64d50d9890a/raw/2062d7c17f6c07473309fa7508f8fba46d16944d/nginx-pod.yaml// verify the pod is created
kubectl get po

You can create the same nginx pod with the kubectl run command. This command is limited and you can’t pass all the specifications with this command.

kubectl run nginx --image=nginx --restart=Never

Here is the complete cheat sheet of the kubectl.

Some Examples

Let’s create some objects in the cluster.

Pods

In the above section, we have created a pod. If you notice the kubernetes dashboard, you can actually see this pod in the pod section of the workloads.
A beginner’s guide to Kubernetes Architecture

Deployments

Let’s create a deployment of nginx with 10 replicas. Here is the file

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx-web
  name: nginx-web
spec:
  replicas: 10
  selector:
    matchLabels:
      app: nginx-web
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx-web
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}
// create this object
kubectl apply -f https://gist.githubusercontent.com/bbachi/4eef188f8687e82fb8e570e6ae1ca405/raw/e1eae137150ff713c02ddd66db4acde7b5ffff69/nginx-web.yaml// get deploy, replicasets and pods
kubectl get deploy
kubectl get rs
kubectl get po

Services

Let’s create a service for this deployment. A Service is n abstract way to expose an application running on a set of pods as a network service.

kubectl expose deploy nginx-web --port=80// get service
kubectl get svc

CronJob

Cron jobs are useful for creating periodic and recurring tasks, like running backups or sending emails. Cron jobs can also schedule individual tasks for a specific time, such as if you want to schedule a job for a low activity period.

Let’ s create one from the official doc here.

kubectl create -f https://k8s.io/examples/application/job/cronjob.yamlkubectl get cj

Let’s look again at the kubernetes dashboard to see all these objects.
A beginner’s guide to Kubernetes Architecture

Let’s clean up the cluster by deleting all the objects in the cluster.

kubectl delete cj hello
kubectl delete deploy nginx-web
kubectl delete svc nginx-web
kubectl delete po nginx
Summary
  • Kubernetes is an open-source container orchestration engine for automating deployment, scaling, and management of containerized applications.
  • It’s very important to get to know the kubernetes architecture before diving into actual deployment with it
  • We have master nodes and worker nodes in the cluster.
  • Every master node contains these components: API server, Scheduler, controller-manager and etc.
  • Every worker node contains these components: Kube-proxy, kubelet, and container runtime
  • You can run multiple master nodes for the high available cluster
  • There are different options to get hands-on with kubernetes: katacoda. minikube, managed kubernetes services from the major cloud providers
  • Kubernetes dashboard is the web interface where you can view, edit and, delete objects in the cluster.
  • Kubernetes dashboard is not installed by default.
  • Kubectl apply is the recommended way to create/deploy objects in the production cluster
  • You create objects in the cluster by feeding YAML or json file into the API server. The YAML file is converted to JSON format before submitted to the API server.
  • Kubectl run is another way to create objects but it is limited because we can’t specify all the specs as options.
Conclusion

Kubernetes is the future of deployment. A lot of fortune 500 companies have already moved and some are transitioning now. So, it is here to stay. I would recommend start using this and get familiar as soon as possible.

Understanding Kubernetes Security on Docker Enterprise 3.0

Understanding Kubernetes Security on Docker Enterprise 3.0

Docker began including Kubernetes with Docker Enterprise 2.0 last year. The recent 3.0 release includes CNCF Certified Kubernetes 1.14, which has many additional security features. In this post, I will review Pod Security Policies and Admission Controllers.

Docker began including Kubernetes with Docker Enterprise 2.0 last year. The recent 3.0 release includes CNCF Certified Kubernetes 1.14, which has many additional security features. In this post, I will review Pod Security Policies and Admission Controllers.

What are Kubernetes Pod Security Policies?

Pod Security Policies are rules created in Kubernetes to control security in pods. A pod will only be scheduled on a Kubernetes cluster if it passes these rules. These rules are defined in the “PodSecurityPolicy” resource and allow us to manage host namespace and filesystem usage, as well as privileged pod features. We can use the PodSecurityPolicy resource to make fine-grained security configurations, including:

  • Privileged containers.
  • Host namespaces (IPC, PID, Network and Ports).
  • Host paths and their permissions and volume types.
  • User and group for containers process execution and setuid capabilities inside container.
  • Change default containers capabilities.
  • Behaviour of Linux security modules.
  • Allow host kernel configurations using sysctl.

The Docker Universal Control Plane (UCP) 3.2 provides two Pod Security Policies by default – which is helpful if you’re just getting started with Kubernetes.These default policies will allow or prevent execution of privileged containers inside pods. To manage Pod Security Policies, you need to have administrative privileges on the cluster.

Reviewing and Configuring Pod Security Policies

To review defined Pod Security Policies in a Docker Enterprise Kubernetes cluster, we connect using an administrator’s UCP Bundle:

$ kubectl get PodSecurityPolicies
NAME           PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES                                                
privileged     true    *      RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *
unprivileged   false          RunAsAny   RunAsAny    RunAsAny   RunAsAny   false            *

These default policies control the execution of privileged containers inside pods.

Let’s create a policy to disallow execution of containers using root for main process. If you are not familiar with Kubernetes, we can reuse the “unprivileged” Pod Security Policy content as a template:

$ kubectl get psp  privileged -o yaml --export > /tmp/mustrunasnonroot.yaml

We removed non-required values and will have the following Pod Security Policy file: /tmp/mustrunasnonroot.yaml

Change the runAsUser rule with “MustRunAsNonRoot” value:

apiVersion: extensions/v1beta1
kind: PodSecurityPolicy
metadata:  
  name: psp-mustrunasnonroot
spec:
  allowPrivilegeEscalation: false
  allowedHostPaths:
  - pathPrefix: /dev/null
    readOnly: true
  fsGroup:
    rule: RunAsAny
  hostPorts:
  - max: 65535
    min: 0
  runAsUser:
    rule: MustRunAsNonRoot
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - '*'

We create this new policy as an administrator user in the current namespace (if none was selected, the policy will be applied to the “default” namespace):

$ kubectl create -f mustrunasnonroot.yaml                      
podsecuritypolicy.extensions/psp-mustrunasnonroot created

Now we can review Pod Security Policies:

$ kubectl get PodSecurityPolicies --all-namespaces
NAME               PRIV    CAPS   SELINUX    RUNASUSER          FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
psp-mustrunasnonroot   true    *      RunAsAny   MustRunAsNonRoot   RunAsAny   RunAsAny   false            *
privileged         true    *      RunAsAny   RunAsAny           RunAsAny   RunAsAny   false            *
unprivileged       false          RunAsAny   RunAsAny           RunAsAny   RunAsAny   false            *

Next, we create a Cluster Role that will allow our test user to use the Pod Security Policy we just created, using role-mustrunasnonroot.yaml.

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: role-mustrunasnonroot
rules:
- apiGroups:
  - policy
  resourceNames:
  - psp-mustrunasnonroot
  resources:
  - podsecuritypolicies
  verbs:
  - use

Next, we add a Cluster Role Binding to associate a new non-admin role to our user (jramirez for this example). We created rb-mustrunasnonroot-jramirez.yaml with following content:

kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: rb-mustrunasnonroot-jramirez
  namespace: default
roleRef:
  kind: ClusterRole
  name: role-mustrunasnonroot
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: User
  name: jramirez
  namespace: default

We create both the Cluster Role and Cluster Role Binding to allow jramirez to use the defined Pod Security Policy:

$ kubectl create -f role-mustrunasnonroot.yaml
clusterrole.rbac.authorization.k8s.io/role-mustrunasnonroot created

$ kubectl create -f rb-mustrunasnonroot-jramirez.yaml
rolebinding.rbac.authorization.k8s.io/rb-mustrunasnonroot-jramirez created

Now that we’ve applied this policy, we should delete the default rules (privileged or unprivileged). In this case, the default “ucp:all:privileged-psp-role” was applied.

$ kubectl delete clusterrolebinding ucp:all:privileged-psp-role
clusterrolebinding.rbac.authorization.k8s.io "ucp:all:privileged-psp-role" deleted

We can review jramirez’s permissions to create new pods on the default namespace.

$ kubectl auth can-i create pod --as jramirez
yes

Now we can create a pod using the following manifest from nginx-as-root.yaml:

apiVersion: v1
kind: Pod
metadata:
 name: nginx-as-root
 labels:
   lab: nginx-as-root
spec:
 containers:
 - name: nginx-as-root
   image: nginx:alpine

We’ll now need to login as jramirez using ucp-bundle, our test non-admin user. We can then test deployment to see if it works:

$ kubectl create -f nginx-as-root.yaml
pod/nginx-as-root created

We will get a CreateContainerConfigError because the image doesn’t have any users defined, so the command will try to create a root container, which the policy blocks.

Events:
 Type     Reason     Age                    From               Message
 ----     ------     ----                   ----               -------
 Normal   Scheduled  6m9s                   default-scheduler  Successfully assigned default/nginx-as-root to vmee2-5
 Warning  Failed     4m12s (x12 over 6m5s)  kubelet, vmee2-5   Error: container has runAsNonRoot and image will run as root
 Normal   Pulled     54s (x27 over 6m5s)    kubelet, vmee2-5   Container image "nginx:alpine" already present on machine

What can we do to avoid this? As a best practice, we should not allow containers with root permissions. However, we can create an Nginx image without root permissions. Here’s a lab image that will work for our purposes (but it’s not production ready):

FROM alpine

RUN addgroup -S nginx \
&& adduser -D -S -h /var/cache/nginx -s /sbin/nologin -G nginx -u 10001 nginx \
&& apk add --update --no-cache nginx \
&& ln -sf /dev/stdout /var/log/nginx/access.log \
&& ln -sf /dev/stderr /var/log/nginx/error.log \
&& mkdir /html

COPY nginx.conf /etc/nginx/nginx.conf

COPY html /html

RUN chown -R nginx:nginx /html

EXPOSE 1080

USER 10001

CMD ["nginx", "-g", "pid /tmp/nginx.pid;daemon off;"]

We created a new user nginx to launch the nginx main process under this one (in fact, the nginx installation will provide a special user www-data or nginx, depending on base operating system). We added the user under a special UID because we will use that UID on Kubernetes to specify the user that will be used to launch all containers in our nginx-as-nonroot pod.

You can see that we are using a new nginx.conf. Since we are not using root to start Nginx, we can’t use ports below 1024. Consequently, we exposed port 1080 in the Dockerfile. This is the simplest Nginx config required.

worker_processes  1;

events {
   worker_connections  1024;
}


http {
   include       mime.types;
   default_type  application/octet-stream;
   sendfile        on;
   keepalive_timeout  65;
   server {
       listen       1080;
       server_name  localhost;

       location / {
           root   /html;
           index  index.html index.htm;
       }


       error_page   500 502 503 504  /50x.html;
       location = /50x.html {
           root   /html;
       }

   }

}

We added a simple index.html with just one line:

$ cat html/index.html  
It worked!!

And our pod definition has new security context settings:

apiVersion: v1
kind: Pod
metadata:
 name: nginx-as-nonroot
 labels:
   lab: nginx-as-root
spec:
 containers:
 - name: nginx-as-nonroot
   image: frjaraur/non-root-nginx:1.2
   imagePullPolicy: Always
 securityContext:
   runAsUser: 10001

We specified a UID for all containers in that pod. Therefore, the Nginx main process will run under 10001 UID, the same one specified in image.

If we don’t specify the same UID, we will get permission errors because the main process will use pod-defined settings with different users and Nginx will not be able to manage files:

nginx: [alert] could not open error log file: open() "/var/lib/nginx/logs/error.log" failed (13: Permission denied)
2019/10/17 07:36:10 [emerg] 1#1: mkdir() "/var/tmp/nginx/client_body" failed (13: Permission denied)

If we do not specify any security context, it will use the image-defined UID with user 10001. It will work correctly since the process doesn’t require root access.

We can go back to the previous situation by deleting the custom Cluster Role Binding we created earlier (rb-mustrunasnonroot-jramirez) and adding the UCP role again:

ucp:all:privileged-psp-role

Create rb-privileged-psp-role.yaml with following content:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: ucp:all:privileged-psp-role
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: privileged-psp-role
subjects:
- kind: Group
  name: system:authenticated
  apiGroup: rbac.authorization.k8s.io
- kind: Group
  name: system:serviceaccounts
  apiGroup: rbac.authorization.k8s.io

And create the ClusterRoleBinding object using **$ kubectl create -f rb-privileged-psp-role.yaml **as administrator.

Kubernetes Admission Controllers

Admission Controllers are a feature added to Kubernetes clusters to manage and enforce default resource values or properties and prevent potential risks or misconfigurations. They occur before workload execution, intercepting requests to validate or modify its content. The Admission Controllers gate user interaction with cluster API, applying policies to any actions on Kubernetes.

We can review which Admission Controllers are defined in Docker Enterprise by taking a look at the ucp-kube-apiserver command-line used to start this Kubernetes API Server container. On any of our managers, we can describe container configuration:

$ docker inspect ucp-kube-apiserver --format 'json {{ .Config.Cmd }}'  
json [--enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,PersistentVolumeLabel,DefaultStorageClass,DefaultTolerationSeconds,
NodeRestriction,ResourceQuota,PodNodeSelector,PodSecurityPolicy, UCPAuthorization,CheckImageSigning,UCPNodeSelector
…
…

These are the Admission Controllers deployed with Docker Enterprise Kubernetes:

  • NamespaceLifecycle will manage important namespace features. It will prevent users from removing the default, kube-system and kube-public namespaces, and it will provide the integrity for other namespaces deletion, removing all objects on it prior to deletion (for example). It will also prevent new object creation on a namespace that is in the process of being removed (it can take time because running objects must be removed).
  • LimitRanger will manage default resource requests to pods that don’t specify any. It also verifies that Namespace associated resources doesn’t pass its defined limit.
  • ServiceAccount will associate pods to a default ServiceAccount if they don’t provide one, and ensure that one exists if it is present on Pod definition. It will also manage API account accessibility.
  • PersistentVolumeLabel will add special labels for regions or zones to ensure that right volumes are mounted per region or zone.
  • DefaultStorageClass will add a default StorageClass when none was declared, and a PersistentVolumeClaim ask for storage.
  • DefaultTolerationSeconds will set default pod toleration values, evicting nodes not ready or unreachable for more than 300 seconds.
  • NodeRestriction will allow only kubelet modifications to its own Node or Pods.
  • ResourceQuota will manage resource quota limits not reached within namespaces.
  • PodNodeSelector provides default node selections within namespaces.
  • PodSecurityPolicy reviews Pod Security Policies to determine if a Pod can be executed or not.
  • UCPAuthorization provides UCP Roles to Kubernetes integration, preventing deletion of system-required cluster roles and bindings. It will also prevents using host paths volumes or privileged containers for non-admins (or non-privileged accounts), even if it is allowed in Pod Security Policies.
  • CheckImageSigning prevents execution of Pods based on unsigned images by authorized users.
  • UCPNodeSelector manages execution of non-system Kubernetes workloads only on non-mixed UCP hosts.

The last few are Docker designed and created to ensure UCP and Kubernetes integration and improved access and security. These Admission Controllers will be set up during installation. They can’t be disabled since doing so can compromise cluster security, or even break some unnoticeable but important functionalities.

As we learned, Docker Enterprise 3.0 now provides Kubernetes security features by default that will complement and improve users interaction with the cluster, maintaining the highest security environment out-of-box.

How to using Docker Compose for NodeJS Development

How to using Docker Compose for NodeJS Development

Docker is an amazing tool for developers. It allows us to build and replicate images on any host, removing the inconsistencies of dev environments and reducing onboarding timelines considerably.

Docker is an amazing tool for developers. It allows us to build and replicate images on any host, removing the inconsistencies of dev environments and reducing onboarding timelines considerably.

To provide an example of how you might move to containerized development, I built a simple todo API using NodeJS, Express, and PostgreSQL using Docker Compose for development, testing, and eventually in my CI/CD pipeline.

In a two-part series, I will cover the development and pipeline creation steps. In this post, I will cover the first part: developing and testing with Docker Compose.

Requirements for This Tutorial

This tutorial requires you to have a few items before you can get started.

The todo app here is essentially a stand-in, and you could replace it with your own application. Some of the setup here is specific for this application, and the needs of your application may not be covered, but it should be a good starting point for you to get the concepts needed to Dockerize your own applications.

Once you have everything set up, you can move on to the next section.

Creating the Dockerfile

At the foundation of any Dockerized application, you will find a Dockerfile. The Dockerfile contains all of the instructions used to build out the application image. You can set this up by installing NodeJS and all of its dependencies; however the Docker ecosystem has an image repository (the Docker Store) with a NodeJS image already created and ready to use.

In the root directory of the application, create a new Dockerfile

/> touch Dockerfile

Open the newly created Dockerfile in your favorite editor. The first instruction, FROM, will tell Docker to use the prebuilt NodeJS image. There are several choices, but this project uses the node:7.7.2-alpine image.

FROM node:7.7.2-alpine

If you run docker build ., you will see something similar to the following:

Sending build context to Docker daemon 249.3 kB
Step 1/1 : FROM node:7.7.2-alpine
7.7.2-alpine: Pulling from library/node
709515475419: Pull complete
1a7746e437f7: Pull complete
662ac7b95f9d: Pull complete
Digest: sha256:6dcd183eaf2852dd8c1079642c04cc2d1f777e4b34f2a534cc0ad328a98d7f73
Status: Downloaded newer image for node:7.7.2-alpine
 ---> 95b4a6de40c3
Successfully built 95b4a6de40c3

With only one instruction in the Dockerfile, this doesn’t do too much, but it does show you the build process without too much happening. At this point, you now have an image created, and running docker images will show you the images you have available:

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
node                7.7.2-alpine        95b4a6de40c3        6 weeks ago         59.2 MB

The Dockerfile needs more instructions to build out the application. Currently it’s only creating an image with NodeJS installed, but we still need our application code to run inside the container. Let’s add some more instructions to do this and build this image again.

This particular Docker file uses RUN, COPY, and WORKDIR. You can read more about those on Docker’s reference page to get a deeper understanding.

Let’s add the instructions to the Dockerfile now:

FROM node:7.7.2-alpine

WORKDIR /usr/app

COPY package.json .
RUN npm install --quiet

COPY . .

Here is what is happening:

  • Set the working directory to /usr/app
  • Copy the package.json file to /usr/app
  • Install node_modules
  • Copy all the files from the project’s root to /usr/app

You can now run docker build . again and see the results:

Sending build context to Docker daemon 249.3 kB
Step 1/5 : FROM node:7.7.2-alpine
  ---> 95b4a6de40c3
Step 2/5 : WORKDIR /usr/app
 ---> e215b737ca38
Removing intermediate container 3b0bb16a8721
Step 3/5 : COPY package.json .
 ---> 930082a35f18
Removing intermediate container ac3ab0693f61
Step 4/5 : RUN npm install --quiet
 ---> Running in 46a7dcbba114

 ### NPM MODULES INSTALLED ###

 ---> 525f662aeacf
 ---> dd46e9316b4d
Removing intermediate container 46a7dcbba114
Step 5/5 : COPY . .
 ---> 1493455bcf6b
Removing intermediate container 6d75df0498f9
Successfully built 1493455bcf6b

You have now successfully created the application image using Docker. Currently, however, our app won’t do much since we still need a database, and we want to connect everything together. This is where Docker Compose will help us out.

Docker Compose Services

Now that you know how to create an image with a Dockerfile, let’s create an application as a service and connect it to a database. Then we can run some setup commands and be on our way to creating that new todo list.

Create the file docker-compose.yml:

/> touch docker-compose.yml

The Docker Compose file will define and run the containers based on a configuration file. We are using compose file version 2https://docs.docker.com/compose/compose-file/compose-file-v2/ syntax, and you can read up on it on Docker’s site.

An important concept to understand is that Docker Compose spans “buildtime” and “runtime.” Up until now, we have been building images using docker build ., which is “buildtime.” This is when our containers are actually built. We can think of “runtime” as what happens once our containers are built and being used.

Compose triggers “buildtime” — instructing our images and containers to build — but it also populates data used at “runtime,” such as env vars and volumes. This is important to be clear on. For instance, when we add things like volumes and command, they will override the same things that may have been set up via the Dockerfile at “buildtime.”

Open your docker-compose.yml file in your editor and copy/paste the following lines:

version: '2'
services:
  web:
    build: .
    command: npm run dev
    volumes:
      - .:/usr/app/
      - /usr/app/node_modules
    ports:
      - "3000:3000"
    depends_on:
      - postgres
    environment:
      DATABASE_URL: postgres://[email protected]/todos
  postgres:
    image: postgres:9.6.2-alpine
    environment:
      POSTGRES_USER: todoapp
      POSTGRES_DB: todos

This will take a bit to unpack, but let’s break it down by service.

The web service

The first directive in the web service is to build the image based on our Dockerfile. This will recreate the image we used before, but it will now be named according to the project we are in, nodejsexpresstodoapp. After that, we are giving the service some specific instructions on how it should operate:

  • command: npm run dev – Once the image is built, and the container is running, the npm run dev command will start the application.

  • volumes: – This section will mount paths between the host and the container.

  • .:/usr/app/ – This will mount the root directory to our working directory in the container.

- /usr/app/node_modules – This will mount the node_modules directory to the host machine using the buildtime directory.

  • environment: – The application itself expects the environment variable DATABASE_URL to run. This is set in db.js.

  • ports: – This will publish the container’s port, in this case 3000, to the host as port 3000.

The DATABASE_URL is the connection string. postgres://[email protected]/todos connects using the todoapp user, on the host postgres, using the database todos.

The Postgres service

Like the NodeJS image we used, the Docker Store has a prebuilt image for PostgreSQL. Instead of using a build directive, we can use the name of the image, and Docker will grab that image for us and use it. In this case, we are using postgres:9.6.2-alpine. We could leave it like that, but it hasenvironmentvariables to let us customize it a bit.

environment: – This particular image accepts a couple environment variables so we can customize things to our needs. POSTGRES_USER: todoapp – This creates the user todoapp as the default user for PostgreSQL. POSTGRES_DB: todos – This will create the default database as todos.

Running The Application

Now that we have our services defined, we can build the application using docker-compose up. This will show the images being built and eventually starting. After the initial build, you will see the names of the containers being created:

Pulling postgres (postgres:9.6.2-alpine)...
9.6.2-alpine: Pulling from library/postgres
627beaf3eaaf: Pull complete
e351d01eba53: Pull complete
cbc11f1629f1: Pull complete
2931b310bc1e: Pull complete
2996796a1321: Pull complete
ebdf8bbd1a35: Pull complete
47255f8e1bca: Pull complete
4945582dcf7d: Pull complete
92139846ff88: Pull complete
Digest: sha256:7f3a59bc91a4c80c9a3ff0430ec012f7ce82f906ab0a2d7176fcbbf24ea9f893
Status: Downloaded newer image for postgres:9.6.2-alpine
Building web
...
Creating nodejsexpresstodoapp_postgres_1
Creating nodejsexpresstodoapp_web_1
...
web_1       | Your app is running on port 3000

At this point, the application is running, and you will see log output in the console. You can also run the services as a background process, using docker-compose up -d. During development, I prefer to run without -d and create a second terminal window to run other commands. If you want to run it as a background process and view the logs, you can run docker-compose logs.

At a new command prompt, you can run docker-compose ps to view your running containers. You should see something like the following:

            Name                            Command              State           Ports
------------------------------------------------------------------------------------------------
nodejsexpresstodoapp_postgres_1   docker-entrypoint.sh postgres   Up      5432/tcp
nodejsexpresstodoapp_web_1        npm run dev                     Up      0.0.0.0:3000->3000/tcp

This will tell you the name of the services, the command used to start it, its current state, and the ports. Notice nodejsexpresstodoapp_web_1 has listed the port as 0.0.0.0:3000->3000/tcp. This tells us that you can access the application using localhost:3000/todos on the host machine.

/> curl localhost:3000/todos

[]

The package.json file has a script to automatically build the code and migrate the schema to PostgreSQL. The schema and all of the data in the container will persist as long as the postgres:9.6.2-alpine image is not removed.

Eventually, however, it would be good to check how your app will build with a clean setup. You can run docker-compose down, which will clear things that are built and let you see what is happening with a fresh start.

Feel free to check out the source code, play around a bit, and see how things go for you.

Testing the Application

The application itself includes some integration tests built using jest. There are various ways to go about testing, including creating something like Dockerfile.test and docker-compose.test.ymlfiles specific for the test environment. That’s a bit beyond the current scope of this article, but I want to show you how to run the tests using the current setup.

The current containers are running using the project name nodejsexpresstodoapp. This is a default from the directory name. If we attempt to run commands, it will use the same project, and containers will restart. This is what we don’t want.

Instead, we will use a different project name to run the application, isolating the tests into their own environment. Since containers are ephemeral (short-lived), running your tests in a separate set of containers makes certain that your app is behaving exactly as it should in a clean environment.

In your terminal, run the following command:

/> docker-compose -p tests run -p 3000 --rm web npm run watch-tests

You should see jest run through integration tests and wait for changes.

The docker-compose command accepts several options, followed by a command. In this case, you are using -p tests to run the services under the tests project name. The command being used is run, which will execute a one-time command against a service.

Since the docker-compose.yml file specifies a port, we use -p 3000 to create a random port to prevent port collision. The--rmoption will remove the containers when we stop the containers. Finally, we are running in the web service npm run watch-tests.

Thanks for reading !