Introducing Docker BuildKit

Introducing Docker BuildKit

Docker Build is one of the most used features of the Docker Engine. Docker Build integrating BuildKit, users should see an improvement on performance, storage management, feature functionality, and security. Docker images created with BuildKit can be pushed to Docker Hub just like Docker images created with legacy build

I am working as a SRE and a lot of the work that I do is around docker containers. And a lot of the work around docker containers is on building them. Some Dockerfiles are really simple, like when just adding a Go binary to an alpine/scratch image, while some are more complicated especially when using multistage builds. In the later cases usually you also build your binaries with docker, besides creating the images. And sometimes in cases like these running those docker build commands might take more than a few minutes (for example Argo-CD takes around 12–15 min, though not just one image is created).

Image for post

Starting with Docker 18.09 there is a new way of building images called BuildKit. This is considered the V2 of docker build and it currently isn't the default way even for Docker 19.03, you need to enable it. The easiest way to make it active is to set an environment variable: DOCKER_BUILKIT=1. This V2 adds many interesting features and some of them are out of box. Also the resulting docker image are similar to the ones created with normal build, so there shouldn’t be any problems running that image or pushing it to a registry.

BuildKit performance improvements

In the normal docker build each layer is built sequentially, you need to wait for layer n to build n+1. This limitation wasn't very visible initially, but with the introduction of multi-stage builds it became a bigger issue because you can start more steps in parallel until to the point they depend on each other. So valuable time can be gained here by introducing some parallelism.

In order to find which steps can be built in parallel and which ones need to wait for each other BuildKit will build a graph of the dependencies and use it to increase the build efficiency. The output of a docker build command will be completely different to what you were used until now, with some steps running in parallel, similar to the below (where you can see #4 running in parallel with #10):

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.15kB done
#2 DONE 0.0s

#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#3 [internal] load metadata for docker.io/library/openjdk:8
#3 DONE 1.7s

#4 [1/8] FROM docker.io/library/openjdk:[email protected]:291ef47999c4ee7160cc1208ff...
#4 resolve docker.io/library/openjdk:[email protected]:291ef47999c4ee7160cc1208ff49244bf93a43b7eca1c31842615fc529efc24e done
#4 ...

#10 [internal] load build context
#10 transferring context: 10.60kB done
#10 DONE 0.0s

#4 [1/8] FROM docker.io/library/openjdk:[email protected]:291ef47999c4ee7160cc1208ff...

Performance is not the only improvement that BuildKit delivers, but we can have it out of the box without any changes to the Dockerfile. And this is what I would like to check next, how to enable BuildKit on multi-stage docker builds and if it is really a noticeable improvement.

docker devops

Bootstrap 5 Complete Course with Examples

Bootstrap 5 Tutorial - Bootstrap 5 Crash Course for Beginners

Nest.JS Tutorial for Beginners

Hello Vue 3: A First Look at Vue 3 and the Composition API

Building a simple Applications with Vue 3

Deno Crash Course: Explore Deno and Create a full REST API with Deno

How to Build a Real-time Chat App with Deno and WebSockets

Convert HTML to Markdown Online

HTML entity encoder decoder Online

Ever Wondered Why We Use Containers In DevOps?

At some point we've all said the words, "But it works on my machine." It usually happens during testing or when you're trying to get a new project set up. Sometimes it happens when you pull down changes from an updated branch.

Docker manifest - A peek into image's manifest.json files

The docker manifest command does not work independently to perform any action. In order to work with the docker manifest or manifest list, we use sub-commands along with it. This manifest sub-command can enable us to interact with the image manifests. Furthermore, it also gives information about the OS and the architecture, that a particular image was built for. The image manifest provides a configuration and a set of layers for a container image. This is an experimenta

Docker Explained: Docker Architecture | Docker Registries

Following the second video about Docker basics, in this video, I explain Docker architecture and explain the different building blocks of the docker engine; docker client, API, Docker Daemon. I also explain what a docker registry is and I finish the video with a demo explaining and illustrating how to use Docker hub.

How to Extend your DevOps Strategy For Success in the Cloud?

DevOps and Cloud computing are joined at the hip, now that fact is well appreciated by the organizations that engaged in SaaS cloud and developed applications in the Cloud. During the COVID crisis period, most of the organizations have started using cloud computing services and implementing a cloud-first strategy to establish their remote operations. Similarly, the extended DevOps strategy will make the development process more agile with automated test cases.

What Is DevOps and Is Enterprise DevOps Any Good?

What is DevOps? How are organizations transitioning to DevOps? Is it possible for organizations to shift to enterprise DevOps? Read more to find out!