Top Docker Tools: 5 Docker Utilities You Should Know

Top Docker Tools: 5 Docker Utilities You Should Know

The Docker community has created many open source utilities, making it helpful with even more use cases than you might imagine.

You can find a lot of cool Docker utilities on the web. Most of these are open source and available on Github. Over the last two years, I have become quite active with Docker, using it for most of my development projects. As you start using Docker, you will find it is suitable for more use cases than you may have initially envisioned. You will want Docker to do a little more for you—and it will not disappoint!

The Docker community is very active, with a lot of useful utilities popping up daily. It is difficult to keep check of all the innovation happening in the community. To help you, I have collected some interesting and useful Docker utilities, which I use in my daily work. These utilities make me more productive, eliminating what otherwise would have been manual work.

Let’s go through each of the utilities I find useful in my journey to Dockerize stuff.

1. watchtower: Automatically Update Docker Containers

Watchtower monitors running containers and watches for changes to the images those containers were originally started from. When Watchtower detects that an image has changed, it automatically restarts the container using the new image. I use it in my local development where I would like to try out the latest built image.

Watchtower is itself packaged as a Docker image so you can run it just the way you would run any other container. To run Watchtower, you would run the following command:

In the command above, we started Watchtower container with a mounted file /var/run/docker.sock . This is required so that Watchtower can interact with Docker daemon API. We passed an option interval of 30 seconds. This option defines the Watchtower poll interval. Watchtower supports a few more options, which you can use as described in their documentation.

Let’s now start a container that Watchtower can monitor.

Now, Watchtower will start monitoring friendlyhello container. When I push the new image to Docker Hub, Watchtower, in its next run, will detect that a new image is available. It will gracefully stop the container and start the container using the new image. It will pass the options that we passed to the run command. In other words, the container will be started with 4000:80 publish ports option.

By default, Watchtower will poll the Docker Hub registry to look for updated images. You can configure Watchtower to poll private registry by passing the registry credentials in environment variables REPO_USER and REPO_PASS.

To learn more about Watchtower, I recommend reviewing the Watchtower documentation.

2. docker-gc: Garbage Collection of Container and Images

The docker-gc utility helps clean up your Docker host by removing containers and images that are not required. It removes all the containers that existed more than an hour ago. Also, it removes images that don’t belong to any remaining containers.

You can use docker-gc both as a script and container. We will run docker-gc as a container. Let’s use docker-gc to find all the containers and images that can be removed.

In the command shown above, we mounted Docker socket file so that docker-gc can interact with Docker API. We passed an environment variable DRY_RUN=1 to find which containers and images will be removed. If we don’t provide this option, docker-gc will remove all of them. It is good to first verify everything docker-gc will clean. The output of the above command appears below.

If you are fine with the docker-gc clean up plan, you can again run docker-gc without DRY_RUN flag to perform the clean up.

The output of the above command will tell you all the images and the containers that docker-gc removed.

There are few more options that docker-gc supports. I recommend you read the docker-gc documentation to learn more about it.

3. docker-slim: Magic Diet Pill for Your Containers

If you are worried about the size of your Docker images, you will be blown away by docker-slim.

The docker-slim utility uses static and dynamic analysis to create skinny image variants of your fat images. To use docker-slim, you have to download its binary from Github. Binaries are available for Linux and Mac. Once you download the binary, add it to your PATH.

I created a Docker image for example application friendlyhello used in the Docker official documentation. The image size, as you can see below, is 194 MB.

As you can see for a simple application, we have to download 194 MB of data. Let’s use docker-slim to see how much fat it can remove.

The docker-slim utility will carry out a series of steps–inspecting fat image, instrument fat image, finally creating a slim version of the image. Let’s look at the size of the slim variant.

As you can see above, the image size was reduced to 24.9 MB. You can start the container and it will behave in the same manner. The docker-slim utility works well with Java, Python, Ruby, and Node.js application.

Try it yourself and see how much you can gain. In my personal projects, I have found that it worked for most cases. You can learn more about docker-slim from its documentation.

4. rocker: Breaks the Limits of Dockerfile

Most of the developers using Docker use Dockerfile for building images. *Dockerfile *is a declarative way to define all the commands a user could call on the command line in order to assemble an image.

Rocker adds new instructions to the Dockerfile instruction set. Rocker was created by Grammarly to solve problems they faced with Dockerfile format. The Grammarly team wrote an in-depth blog explaining their reasons for creating it. I recommend you read it to better understand Rocker. The two problems they highlight in their post are:

  1. Size of Docker images.
  2. Slower builds.

The blog also mentions some of the new instructions added by Rocker. Refer to Rocker documentation to learn about all the instructions Rocker supports.

  1. MOUNT is used to share volumes between builds so they can be reused by dependency management tools.
  2. FROM instruction exists in the Dockerfile as well. Rocker makes it feasible to add more than one FROM instruction. This means you can create more than one image from a single Rockerfile. The first set of instructions will build the artifact using all the dependencies. The second set of instructions can use the build artifact. This reduces the image size drastically.
  3. TAG is used to tag the image at different stages of the build. This means you don’t have to tag images manually.
  4. PUSH is used to push images to a registry.
  5. ATTACH allows you to run an intermediate step interactively. This is useful for debugging.

To use Rocker, you must install it on your machine. For Mac users, it is as simple as running couple of brew commands:

Once installed, you can use Rocker to build an image by passing it to Rockerfile:

To build an image and push it to Docker Hub, you can run the following command:

Rocker has a good set of features. To learn more about it, refer to its documentation.

5. ctop: Top-like interface for containers

The utility I have started using lately is ctop, which provides a real-time metrics view of multiple containers. If you are a mac user, then you can install ctop using brew as shown below.

Once installed, you can start using ctop. It only needs the DOCKER_HOST environment variable configured.

To view the state of all the containers, you can run ctop command.

To view the running containers only, you can use ctop -a command.

The ctop is a simple utility and very useful to learn about containers running on your host. You can read more about it in the ctop documentation.

These are some of the Docker utilities I find useful. Do you use any Docker utilities in your daily work? If so, let us know in the comments section below.

WordPress in Docker. Part 1: Dockerization

WordPress in Docker. Part 1: Dockerization

This entry-level guide will tell you why and how to Dockerize your WordPress projects.

This entry-level guide will tell you why and how to Dockerize your WordPress projects.

What is Docker | Docker Tutorial for Beginners

What is Docker | Docker Tutorial for Beginners

This DevOps Docker Tutorial on what is docker will help you understand how to use Docker Hub, Docker Images, Docker Container & Docker Compose. This tutorial explains Docker's working Architecture and Docker Engine in detail.

This Docker tutorial also includes a Hands-On session around Docker by the end of which you will learn to pull a centos Docker Image and spin your own Docker Container. You will also see how to launch multiple docker containers using Docker Compose. Finally, it will also tell you the role Docker plays in the DevOps life-cycle.

The Hands-On session is performed on an Ubuntu-64bit machine in which Docker is installed.

Deploying Dockerized .NET Apps Without Being a DevOps Guru

Deploying Dockerized .NET Apps Without Being a DevOps Guru

This article will demonstrate first using the tooling to publish a simple ASP.NET Core API in an image to the Docker hub, and then creating a Linux virtual machine in Azure to host the API

Originally published by Julie Lerman at https://blog.docker.com

.NET Developers who use Visual Studio have access to a great extension to help them create Docker images for their apps. The Visual Studio Tools for Docker simplify the task of developing and debugging apps destined for Docker images. But what happens when you are ready to move from debugging in Visual Studio to deploying your image to a container in the cloud?

This blog post will demonstrate first using the tooling to publish a simple ASP.NET Core API in an image to the Docker hub, and then creating a Linux virtual machine in Azure to host the API. It will also engage Docker Compose and Microsoft SQL Server for Linux in a Docker container, along with a Docker Volume for persistence. The goal is to create a simple test environment and a low-stress path to getting your first experience with publishing an app in Docker.

Using the Docker Tools to aid in building and debugging the API is the focus of a series of articles that were published in the April, May and June 2019 issues of MSDN Magazine. So I’ll provide only a high level look at the solution.

Overview of the Sample App

The API allows me to track the names of Docker Captains. It’s not a real-world solution, but enough to give me something to work with. You can download the solution from github.com/julielerman/dockercaptains. I’ll provide a few highlights here.

   public class Captain
   {
       public int CaptainId { get; set; }
       public string Name { get; set; }
   }

The API leverages Entity Framework Core (EF Core) for its data persistence. This requires a class that inherits from the EF Core DbContext. My class, CaptainContext, specifies a DbSet to work from and defines a bit of seed data for the database.

Enabling a Dynamic Connection String

The startup.cs file uses ASP.NET Core’s dependency injection to configure a SQL Server provider for the CaptainContext. There is also code to read a connection string from an environment variable within the Docker container and update a password placeholder that’s less visible to prying eyes.

public void ConfigureServices(IServiceCollection services)
{
 services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_2);
 var conn = Configuration["ConnectionStrings:CaptainDB"];
 conn = connectionstring.Replace("ENVPW", Configuration["DB_PW"]);
 services.AddDbContext<CaptainContext>(options => options.UseSqlServer(conn));
}

The VS Tools generated a Dockerfile and I only made one change to the default — adding the CaptainDB connection string ENV variable with its ENVPW placeholder:

ENV ConnectionStrings:CaptainDB "Server=db;Database=CaptainDB;User=sa;Password=ENVPW;"

ASP.NET Core can discover Docker environment variables when running in a Docker container.

Orchestrating with a docker-compose file

Finally comes the docker-compose.yml file. This sets up a service for the API image, another for the database server image and a volume for persisting the data.

version: '3.4'

services:
 dataapidocker:
   image: ${DOCKER_REGISTRY-}dataapidocker
   build:
     context: .
     dockerfile: DataAPIDocker/Dockerfile
   environment:
     - DB_PW
   depends_on:
     - db
   ports:
     - 80:80
 db:
   image: mcr.microsoft.com/mssql/server
   volumes:
     - mssql-server-julie-data:/var/opt/mssql/data
   environment:
     SA_PASSWORD: "${DB_PW}"
     ACCEPT_EULA: "Y"
   ports:
     - "1433:1433"
volumes:
 mssql-server-julie-data: {}

Notice that I’m declaring the DB_PW environment variable in the API’s service definition and referencing it in the db’s service definition.

There’s also an .env file in the solution where the value of DB_PW is hidden.

[email protected]

Docker will read that file by default.

I got this solution set up and running from within Visual Studio on my development machine. And I love that even when the debugger publishes the app to a local container, I can still debug while it’s running in that container. That’s a super-power of the tools extension.

Using the Tools to Publish to Docker Hub

Once I was happy with my progress, I wanted to get this demo running in the cloud. Although I can easily use the CLI to push and pull, I love that the Docker Tools in VS can handle this part. The Dockerfile created by the tool has instructions for a multi-stage build. When you target Visual Studio to a release build, the tools will build the release image described in the Dockerfile. Publishing will rebuild that release image and publish it to your destination registry.

You can see my full solution in the screenshot below. My API project is called DataAPIDocker. Notice there is also a docker-compose project. This was created by the Docker Tools. But it is the DataAPIDocker project that will be published first into an image and then to a repository.

This will present a Publish page where you can choose to create a New Profile. A publish profile lets you define where to publish your app and also predefine any needed credentials. Creating a profile begins with selecting from a list of targets; for publishing a Docker image, select Container Registry. That option then gives you predefined registries to choose, such as Azure Container Registry, Docker Hub, or a custom registry – which could be an instance of Docker Trusted Registry. 

I’ll choose Docker Hub and click Publish. 

The last step is to provide your Docker Hub repository name. If you don’t already have docker.config set up with your credentials, then you also need to supply your password. 

After creating a profile, it gets stored in the Visual Studio project.

You’ll be returned to the Publish overview page with this profile selected, where you can edit the default “latest” tag name. Click the Publish button to trigger the Docker Tools to do their job. 

A window will open up showing the progress of the docker push command run by the tools.

After the push is complete you can open the repository to see your new repository which by default is public.

Setting up an Azure Linux VM to Host the Containers

Now that the image is hosted in the cloud, you can turn your sights to hosting a container instance for running the app. Since my Visual Studio Subscription includes credits on Azure, I’ll use those. I will create a Linux Virtual Machine on Azure with Docker and Docker Compose, then run an instance of my new image along with a SQL Server and a data volume.

I found two interesting paths for doing this at the command line. One was by using the Azure CLI at the command line in Windows, macOS or Linux. It is so much easier than doing it through the Azure Portal.

I found this doc to be really helpful as I was doing this for the first time. The article walks you through installing the Azure CLI, logging into Azure, creating a Linux VM with Docker already installed then installing Docker Compose. Keep in mind that this will create a default machine using “Standard DS1 v2 (1 vcpus, 3.5 GB memory)” setup. That VM size has an estimated cost of about $54 (USD) per month. 

Alternatively, you can use Docker Machine, a Docker tool for installing Docker on virtual hosts and managing the hosts. This path is a little more automated but it does require that you use bash and that you start by using the Azure CLI to log into your Azure account using the command az login.

Once that’s done, you can use parameters of docker-machine to tell it you’re creating this in Azure, specify your subscription, ssh username, port and size of the machine to create. The last uses standard Azure VM size names. 

I found it interesting to use the Azure CLI workflow which was educational and then consider the docker-machine workflow as a shortcut version.

Since I was still working on my Windows machine, and don’t have the Windows Subsystem for Linux installed there, I opened up Visual Studio Code and switched my terminal shell to use bash. That let me use docker-machine without issue.I also have the Azure Login extension in VS Code, so I was already logged in to Azure.

I first had to get the subscription ID of my Azure Account which I did using the CLI. Then I plugged the id into the docker-machine command:

docker-machine create -d azure 
   --azure-subscription-id [this is where I pasted my subscript id]
   --azure-ssh-user azureuser
   --azure-open-port 80
   --azure-size "Standard_DS1_v2"
   mylinuxvm

There are more settings you can apply, such as defining the resource and location. The output from this command will pause, providing you with details for how to allow docker-machine authorization to the VM by plugging a provided code into a browser window. Once that’s done the command will continue its work and the output will forge ahead.

When it’s finished, you’ll see the message “Docker is up and running!” (on the new VM), Followed by a very important message to configure a shell on the VM by running:

"C:\Program Files\Docker\Docker\Resources\bin\docker-machine.exe" env mylinuxvm

Recall that I’m doing these tasks on Windows, so docker-machine is ensuring that I know where to find the executable. After performing this task, I can see the machine up and running in the Azure Portal. This lets me inspect other default configuration choices made because I didn’t specify them in the docker-machine command.

By default, all of the needed ports are set up for access such as 80 for http and 22 for ssh.

Re-Creating Docker-Compose and .env on the VM

We only need two files on this machine: the docker-compose.yml and the .env file.

Docker-machine allows you to easily ssh into the VM in order for your command line commands to execute on that machine.

docker-machine ssh mylinuxvm

Then you can use a linux editor such as nano to re-create the two files.

nano docker-compose.yml

And you can paste the contents of your docker-compose file into there. This is the docker-compose file in my solution for the sample app. However, there are two edits you’ll need to make.

  1. The original file depends on a variable supplied by the VS Docker Tools for the registry location. Change the value of image to point to your Docker Hub image: image: julielerman/dataapidocker:formylinuxvm
  2. You’ll also need to change the version of docker-compose specified at the top of the file to 2.0 since you’re moving from hosting on Windows to hosting on Linux.

In nano, you can save the docker-compose file with ^O. Then exit nano and run it again to create the .env file using the command:

nano .env

Paste the key value pair environment variable from the app and save the .env file.

Running the Container

I still had to install docker-compose on the new machine. Docker is nice enough to feed you the command for that if you try to run docker-compose before installing it.

 sudo apt install docker-compose

Then I was able to run my containers with: 

 sudo docker-compose up

One important thing I learned: The VS Docker tooling doesn’t define port mapping for the API service in docker-compose. That’s hidden in a docker-compose.override.yml file used by the debugger. If you look at the docker-compose file listed earlier in this article, you’ll see that I added it myself. Without it, when you try to browse to the API, you will get a Connection refused error.

My ASP.NET Core API is now running and I can browse to it at public IP address specified for the VM. The HTTP Get of my Captains controller returns a list of the captains seeded in the database. 

DevOps are for Devs, Too

As a developer who is often first in line to claim “I don’t do DevOps”, I was surprised at how simple it turned out to be to deploy the app I had created. So often I have allowed my development machine to be a gate that defined the limitations of my expertise. I can build the apps and watch them work on my development machine but I’ve usually left deployment to someone else.

While I have ventured into the Azure Portal frequently, the fact that the Docker Tools and the Azure CLI made it so simple to create the assets I needed for deploying the app made me wonder why I’d waited so long to try that out. And in reality, I didn’t have to deploy the app, just an image and then a docker-compose file. That the Docker Machine made it even easier to create those cloud assets was something of a revelation. 

Part of this workflow leveraged the Docker Tools for Visual Studio on Windows. But because I spend a lot of time in Visual Studio Code on my MacBook, I now have the confidence to explore using the Docker CLI for publishing the image to Docker Hub. After that I can just repeat the Docker Machine path to create the Azure VM where I can run my containers. 

Thanks for reading

If you liked this post, share it with all of your programming buddies!

Follow me on Facebook | Twitter

Further reading

Docker and Kubernetes: The Complete Guide

Docker Mastery: The Complete Toolset From a Docker Captain

Docker for the Absolute Beginner - Hands On - DevOps

Docker for Absolute Beginners

How to debug Node.js in a Docker container?

Docker Containers for Beginners

Deploy Docker Containers With AWS CodePipeline

Build Docker Images and Host a Docker Image Repository with GitLab

How to create a full stack React/Express/MongoDB app using Docker