Dfg - Generates Dockerfiles Based on Various input Channels

dfg - Dockerfile Generator

dfg is both a go library and an executable that produces valid Dockerfiles using various input channels.   

Overview

dfg is a Dockerfile generator that accepts input data from various sources, produces and redirects the generated Dockerfile to an output target such as a file or stdout.

It is especially useful for generating Dockerfile instructions conditionally since Dockerfile language has no control flow logic.

Installation

Installing as an Executable

• MacOS

curl -o dfg -L https://github.com/ozankasikci/dockerfile-generator/releases/download/v1.0.0/dfg_v1.0.0_darwin_amd64
chmod +x dfg && sudo mv dfg /usr/local/bin

• Linux

curl -o dfg -L https://github.com/ozankasikci/dockerfile-generator/releases/download/v1.0.0/dfg_v1.0.0_linux_amd64
chmod +x dfg && sudo mv dfg /usr/local/bin

• Windows

curl -o dfg.exe -L https://github.com/ozankasikci/dockerfile-generator/releases/download/v1.0.0/dfg_v1.0.0_windows_amd64.exe

Installing as a Library

go get -u github.com/ozankasikci/dockerfile-generator

Getting Started

Using dfg as an Executable

Available commands:

dfg generate --input path/to/yaml --out Dockerfile generates a file named Dockerfile

dfg generate --input path/to/yaml --target-field ".server.dockerfile" --out Dockerfile generates a file named Dockerfile reading the .server.dockerfile field of the YAML file.

dfg generate --help lists available flags

Using dfg as a Library

When using dfg as a go library, you need to pass a []dfg.Stage slice as data. This approach enables and encourages multi staged Dockerfiles. Dockerfile instructions will be generated in the same order as in the []dfg.Instruction slice.

Some Instructions accept a runForm field which specifies if the Instruction should be run in the shell form or the exec form. If the runForm is not specified, it will be chosen based on Dockerfile best practices.

For detailed usage example please see Library Usage Example

Examples

Single YAML File per Dockerfile Example (Expects a stages key on top level)

stages:
  final:
    - from:
        image: kstaken/apache2
    - run:
        runForm: shell
        params:
          - apt-get update &&
          - apt-get install -y
          - php5
          - libapache2-mod-php5 &&
          - apt-get clean &&
          - rm -rf /var/lib/apt/lists/*
    - cmd:
        params:
          - /usr/sbin/apache2
          - -D
          - FOREGROUND

Use dfg as binary:

dfg generate -i ./example-input-files/apache-php.yaml --stdout

Or as a library

data, err := dfg.NewDockerFileDataFromYamlFile("./example-input-files/apache-php.yaml")
tmpl := dfg.NewDockerfileTemplate(data)
err = tmpl.Render(output)

Output

FROM kstaken/apache2
RUN apt-get update && apt-get install -y php5 libapache2-mod-php5 && apt-get clean && rm -rf /var/lib/apt/lists/*
CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]

YAML File Example With Target Field (Allows using any field)

someConfig:
  key: value
serverConfig:
  dockerfile:
    stages:
      final:
        - from:
            image: kstaken/apache2
        - run:
            runForm: shell
            params:
              - apt-get update &&
              - apt-get clean &&
              - rm -rf /var/lib/apt/lists/*

Use dfg as binary:

dfg generate -i ./example-input-files/test-input-with-target-key-6.yaml --target-field ".serverConfig.dockerfile" --stdout

Or as a library

data, err := dfg.NewDockerFileDataFromYamlField("./example-input-files/test-input-with-target-key-6.yaml", ".serverConfig.dockerfile")
tmpl := dfg.NewDockerfileTemplate(data)
err = tmpl.Render(output)

Output

FROM kstaken/apache2
RUN apt-get update && apt-get clean && rm -rf /var/lib/apt/lists/*

Library Usage Example

package main

import dfg "github.com/ozankasikci/dockerfile-generator"

func main() {
    data := &dfg.DockerfileData{
        Stages: []dfg.Stage{
            // Stage 1 - Builder Image
            // An instruction is just an interface, so you can pass custom structs as well
            []dfg.Instruction{
                dfg.From{
                    Image: "golang:1.7.3", As: "builder",
                },
                dfg.User{
                    User: "ozan",
                },
                dfg.Workdir{
                    Dir: "/go/src/github.com/ozankasikci/dockerfile-generator/",
                },
                dfg.RunCommand{
                    Params: []string{"go", "get", "-d", "-v", "golang.org/x/net/html"},
                },
                dfg.CopyCommand{
                    Sources: []string{"app.go"}, Destination: ".",
                },
                dfg.RunCommand{
                    Params: []string{"CGO_ENABLED=0", "GOOS=linux", "go", "build", "-a", "-installsuffix", "cgo", "-o", "app", "."},
                },
            },
            // Stage 2 - Final Image
            []dfg.Instruction{
                dfg.From{
                    Image: "alpine:latest", As: "final",
                },
                dfg.RunCommand{
                    Params: []string{"apk", "--no-cache", "add", "ca-certificates"},
                },
                dfg.User{
                    User: "root", Group: "admin",
                },
                dfg.Workdir{
                    Dir: "/root/",
                },
                dfg.CopyCommand{
                    From: "builder", Sources: []string{"/go/src/github.com/ozankasikci/dockerfile-generator/app"}, Destination: ".",
                },
                dfg.Cmd{
                    Params: []string{"./app"},
                },
            },
        },
    }
    tmpl := dfg.NewDockerfileTemplate(data)
    
    // write to a file
    file, err := os.Create("Dockerfile")
    err = tmpl.Render(file)
    
    // or write to stdout
    err = tmpl.Render(os.Stdout)
}

Output

FROM golang:1.7.3 as builder
USER ozan
WORKDIR /go/src/github.com/ozankasikci/dockerfile-generator/
RUN go get -d -v golang.org/x/net/html
COPY app.go .
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o app .

FROM alpine:latest as final
RUN apk --no-cache add ca-certificates
USER root:admin
WORKDIR /root/
COPY --from=builder /go/src/github.com/ozankasikci/dockerfile-generator/app .
CMD ["./app"]

TODO

•  Add reading Dockerfile data from an existing yaml file support
•  Implement json file input channel
•  Implement stdin input channel
•  Implement toml file input channel

Author: Ozankasikci
Source Code: https://github.com/ozankasikci/dockerfile-generator 
License: GPL-3.0 license

#go #golang #dockerfiles 

Docker Image Pipelines and Patterns

Docker Image Pipelines and Patterns

Stephen Kuenzli has designed, built, deployed, and operated highly available, scalable software systems for nearly 20 years. In this live coding session, Stephen showcases how to build images automatically with Dockerfiles and patterns for tagging them.

📚📚📚
This video references the book | Docker in Action, Second Edition 
Get it here: http://mng.bz/aD0B
For 40% discount use the code: watchkuenzli40
📚📚📚

About the book:
Docker in Action, Second Edition" teaches you to create, deploy, and manage applications hosted in Docker containers running on Linux. Fully updated, with four new chapters and revised best practices and examples, this second edition begins with a clear explanation of the Docker model. With examples showing how Docker benefits the whole dev lifecycle, you’ll discover techniques for everything from dev-and-test machines to full-scale cloud deployments.

#docker #dockerfiles #devops

Docker Image for Novice

We will learn how to create docker image of Node.js app/service. There are lots of article already on this topic but they have lack of simplicity. When I went through them to create my first docker image. I realized they are complicated. But making a docker image of Node.js app is not as tough as it seems.

Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly.

I am assuming that you already have knowledge of working in Node.js. Here I will not go through creating a Node.js project. You can refer this repo for code.

When you start learning Docker, two main terms come up

1. Image
2. Container

In this article, we will learn about Images.

What is Docker Image?

A docker image is collection of files that bundle together all essentials (installations, application code and dependencies) required to configure a fully operational container environment. In simple words, it bundle all files necessary to run a container.

For creating docker image, docker needs a manual. This manual is called Dockerfile. Developer just needs to create this. Sounds cool!

Lets create a Dockerfile for our case.

#nodejs #docker #docker-beginner #docker-image #dockerfiles

Dockerfile and Its Components

Dockerfile is basically a text file. It contains some set of instructions. Automation of docker image creation.

Docker Components =>

FROM: for a base image the command must be on top of the docker

RUN: To execute Command, it will create a layer in the image.

MAINTAINER: Author/owner/description

COPY: Copy files from the local system (docker VM) we need to provide a source, destination(We cant download file from the internet and any remote directory)

ADD: Similar to copy but, it provides a feature to download files from the internet, also we extract files at the docker image side.

EXPOSE: To Expose ports such as port 8080 for tomcat, port 80 for Nginx, etc.

WORKDIR: To set a working directory for a container.

CMD: Execute commands but during container creation

ENTRYPOINT: Similar to CMD, but has higher priority over CMD, first commands will be executed by ENTRYPOINT only.

#devops #docker #dockerfiles #containerization

Hello World using Docker and Python

Introduction to docker basics with “Hello, world” in python

Dockerfile

Docker is an open-source platform that allows your code to run in an isolated environment from your infrastructure. It’s lightweight and takes care of all your dependencies. According to the Stack overflow developers survey of 2020, docker is the second most beloved platform after Linux and the most wanted platform for new developers.

In this article, we will be learning the basics of docker for beginners a step at a time. We are using python cause python is the most popular programming language and the distribution of python is the most horrible and docker helps to make it a tolerable process.

#dockerfiles #python3 #devops #docker

Exposing Docker Container to Outside World

Allowing docker container to share data or communicate between each other and outside application using few basic commands

docker file

Docker containers are like your babies. You made them, you take care of them, and you want to protect them from the outside world. Outside world is scary after all, all the bad things you can think of is in outside world. But as a mature parent you make a decision that you should be too restrictive. Docker allows you to explicitly expose little to the outside world as you need.

This article is in continuation to the below article. I assume that you know basics of creating docker file, starting and removing the images and containers. So kindly glance over the article below so we are on the same page.

#docker #devops #dockerfiles #container

Components of Docker & Basic Commands

Prerequisite: https://bansalkushagra.medium.com/docker-architecture-o-s-virtualization-in-docker-a32ba4042215

Components of Docker

A. Docker Daemon:

Ø Docker runs on host O.S

Ø It is responsible for the running container to manage docker services.

Ø Docker daemons can communicate with other daemons.

B. Docker Client

Ø Docker users can interact with docker through a client.

Ø Docker Client uses commands and REST APIS to communicate with the docker daemon.

Ø When a Client runs any server command on the docker client terminal, the client terminals send these docker commands to the docker daemon.

Ø It is possible for Docker client to communicate with more than one daemon.

C. Docker Host

Ø It is used to provide an environment to execute and run applications. It contains the docker daemon, images, containers, networks, and Storage.

D. Docker Hub/Registry

Ø It manages and Stores the docker Images.

Ø There are two types of registry in the docker.

o Public Registry

o Private Registry

E. Docker Images

Ø Docker images are the read-only binary templates used to create a container.

Ø Single files with all dependencies and configurations required to run a program.

#dockerfiles #devops-tool #devops #docker

Easy Way to Manage and Operate your Applications on Kubernetes

K8s is awesome, but at the same time, it’s complicated!

K8s is Awesome!

Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. So it’s a container orchestration tool, but it is just not limited to it as does storage orchestration, service discovery, load balancing, automated rollouts, brings in self-healing, provides secret and configuration management, provides horizontal scaling, and is a declarative way to define a cluster state.

Source: https://kubernetes.io/docs/concepts/overview/components/ (open source)

The biggest use-case for K8s is that it encourages micro-service based architecture. It does so by allowing micro-services to be independently scaled.

#docker #dockerfiles #kubernetes

Docker in a nutshell

Docker is has become a common topic in the industry. It is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries, and configuration files; they can communicate with each other through well-defined channels.This article will guide you through a quick overview of the Docker command.

Let’s start with running a Docker image. To run a docker image you use the command docker run ubuntu. When you run the command what happens is, it runs the instance of the Ubuntu image and exists immediately.

If you list down the docker containers, docker ps you will see there are no running containers. When you check all the docker containers docker ps -a you will see that the container has started and stopped.

#dockerfiles #docker #devops-practice #devops-training #devops

Understanding Docker image tags and publishing images to Docker Hub

In this lesson, we are going to discuss how to create and manage tags associated with the Docker images. Then we are going to publish a sample Docker image to Docker Hub and consume it on a separate machine.

In the previous lessons, we learned about the anatomy of Docker images, how Docker containers work, how to create a Docker image from a Dockerfile, and learned a few characteristics of Docker images. However, we did not focus on sharing a Docker image with the world. In this lesson, we are going to learn just that as well as Docker image tags.

For this lesson, we are going to create a simple Docker image that prints a version number to the console when a container is created from it. The source code for this application is located in this GitHub repository.

docker-version-app/ 
├── .dockerignore
├── .gitignore
├── Dockerfile
└── version.txt

Our Docker image will look really simple. We are going to have a Dockerfile and version.txt file that contains a version string. The Dockerfile has the CMD instruction that prints the content of the version.txt file. Since this operation doesn’t need a complicated setup, we will use the alpine:3.12.2 image as the parent image for our Docker image.

#programming #dockerhub #docker #dockerfiles #devops

Dockerizing spring app with Gradle

Docker is a tool designed to make it easier to create, deploy, and run applications by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.

Today we will cover bellow things

1. The importance of docker
2. Overview on docker
3. Dockerizing spring app with detailed explanations
4. Docker various operations eg: Image creation and deletion, container creation and deletion and so more
5. Monitor Docker images using command line and docker desktop apps
6. Advance part: Automotion of docker task eg: docker build, run and push to docker hub from Gradle command

**Prerequisites: **Before starting this article you must have knowledge of spring boot with Gradle dependencies. You can learn this from the below link

1. The importance of docker

• is a tool designed to make it easier to create, deploy, and run applications by using containers.
• Containers are great for continuous integration and continuous delivery (CI/CD) workflows.
• Docker’s portability and lightweight nature also make it easy to dynamically manage workloads, scaling up or tearing down applications and services as business needs dictate, in near real-time.
• Docker is lightweight and fast. It provides a viable, cost-effective alternative to hypervisor-based virtual machines, so you can use more of your compute capacity to achieve your business goals.
• Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package. So the developer can rest assured that the application will run on any other Linux machine regardless of any customized settings that the machine might have that could differ from the machine used for writing and testing the code.
• Docker is a bit like a virtual machine. But unlike a virtual machine, rather than creating a whole virtual operating system, Docker allows applications to use the same Linux kernel as the system that they’re running on and only requires applications to be shipped with things not already running on the host computer. This gives a significant performance boost and reduces the size of the application.
• importantly, Docker is open source. This means that anyone can contribute to Docker and extend it to meet their own needs if they need additional features that aren’t available out of the box.

#docker-image #docker #dockerfiles #spring-boot #gradle

Start with Docker for Data Science

After reading this article you will be able to build your entire project on Docker.

Docker is an open source platform for developing apps that enable separate your applications from your infrastructure.

The aim of this is quickly delivering software.

So the first step to dockerize your project begin with installing Docker.

I. Install Docker

On desktop

• For macOs
• Just download the dmg programme there and double click on it then drag the Docker icon to Applications folder.
• Requirements are :
• • Mac hardware must be a 2010 or a newer model with an Intel processor
• • macOS must be version 10.14 or newer
• • At least 4 GB of RAM
• For Windows
• Just download the dmg programme there and double click on it and follow instructions.
• Requirements are :
• • Windows 10 64-bit: Pro, Enterprise, or Education (Build 16299 or later).
• • 64 bit processor with Second Level Address Translation (SLAT)
• • 4GB system RAM
• For Ubuntu
• You can just run this command :
• sudo snap install docker
• or
• sudo apt install docker.io
• or go to here and install it from repository.
• To install Docker Engine, you need the 64-bit version of one of these Ubuntu versions:
• • Ubuntu Groovy 20.10
• • Ubuntu Focal 20.04 (LTS)
• • Ubuntu Bionic 18.04 (LTS)
• • Ubuntu Xenial 16.04 (LTS)

#data-science #machine-learning #docker-compose #dockerfiles #docker

Kubernetes for Beginners  (Kubernetes commands)

Welcome back to “Kubernetes for Beginners” tutorial series. Kubernetes is a trending topic in the IT world. I am writing this series is for the beginners who as a help for their CKAD examination. If you haven’t checked my previous tutorials, I recommend you to have a look at them. Let’s start Part 06 of the tutorial series, Kubernetes commands.

Before starting I would like to recommend you to have a look at Docker in a nutshell if you are new to Docker. If not, you can continue this article.

In the previous Docker article (Docker in a nutshell), I showed you how to use commands and arguments in Docker commands. Now let’s check how to put commands and arguments into a pod-definition.yaml file.

Look at this docker command.

docker run ubuntu sleep 5

Let’s see how to put the sleep 5 command and argument to the pod definition file.

#devops #dockerfiles #docker #kubernetes #cka-training

Lightweight and Performance Dockerfile for Node.js

Hello World! Today I’m about to share my notes and thoughts about Dockerfile for Node.js applications. I hope you will find it useful and create Dockerfile, which will suit all your needs.

Docker Introduction

If you are here it means, that you have faced Docker before, but to be sure that we are on the same page let’s remind what it is.

So, there are no secrets — Dockerfile is an instruction of how our image will be built. Step by step, layer by layer this file describes how our image will look like.

Okay, so another question — what is an image? We can treat it as a template. Once you build an image from your Dockerfile, it can be shared with other people. You can run a container from an image.

The last but not the least — container, it’s a running image, the actual process.

Why do we need this system? It solves the particular problem — environment. You can run a container where you need to, regardless of your OS. So, as you can see everything begins from Dockerfile.

#nodejs #javascript #programming #dockerfiles #docker

Dockerize Your Python Flask application and deploy it onto Heroku

In this blog, we can see how to dockerize a simple python flask app and deploy it onto Heroku. This blog assumes that you have already installed Docker and Heroku in your PC. If that’s not the case, you can easily install it from here and here.

At First, create a new directory my-dir and move inside the directory.

#python #docker #flask #dockerfiles