Samuel Tucker

Samuel Tucker


Docker Compose Tutorial for Beginners

Docker Compose is a tool that helps us overcome this problem and easily handle multiple containers at once. In this tutorial, we’ll have a look at its main features and powerful mechanisms.

1. Overview

When using Docker extensively, the management of several different containers quickly becomes cumbersome.

Docker Compose is a tool that helps us overcome this problem and easily handle multiple containers at once.

In this tutorial, we’ll have a look at its main features and powerful mechanisms.

2. The YAML Configuration Explained

In short, Docker Compose works by applying many rules declared within a single docker-compose.yml configuration file.

These YAML rules, both human-readable and machine-optimized, provide us an effective way to snapshot the whole project from ten-thousand feet in a few lines.

Almost every rule replaces a specific **Docker **command so that in the end we just need to run:

docker-compose up

We can get dozens of configurations applied by Compose under the hood. This will save us the hassle of scripting them with Bash or something else.

In this file, we need to specify the version of the Compose file format, at least one service, and optionally volumes and networks:

version: "3.7"

Let’s see what these elements actually are.

2.1. Services

First of all, services refer to containers’ configuration.

For example, let’s take a dockerized web application consisting of a front end, a back end, and a database: We’d likely split those components into three images and define them as three different services in the configuration:

    image: my-vue-app
    image: my-springboot-app
    image: postgres

There are multiple settings that we can apply to services, and we’ll explore them deeply later on.

2.2. Volumes & Networks

Volumes, on the other hand, are physical areas of disk space shared between the host and a container, or even between containers. In other words, a volume is a shared directory in the host, visible from some or all containers.

Similarly, networks define the communication rules between containers, and between a container and the host. Common network zones will make containers’ services discoverable by each other, while private zones will segregate them in virtual sandboxes.

Again, we’ll learn more about them in the next section.

3. Dissecting a Service

Let’s now begin to inspect the main settings of a service.

3.1. Pulling an Image

Sometimes, the image we need for our service has already been published (by us or by others) in Docker Hub, or another Docker Registry.

If that’s the case, then we refer to it with the image attribute, by specifying the image name and tag:

    image: ubuntu:latest

3.2. Building an Image

Instead, we might need to build an image from the source code by reading its Dockerfile.

This time, we’ll use the build keyword, passing the path to the Dockerfile as the value:

    build: /path/to/dockerfile/

We can also use a URL instead of a path:


Additionally, we can specify an image name in conjunction with the build attribute, which will name the image once created, making it available to be used by other services:

    image: my-project-image

3.3. Configuring the Networking

Docker containers** communicate between themselves in networks created, implicitly or through configuration, by Docker Compose**. A service can communicate with another service on the same network by simply referencing it by container name and port (for example network-example-service:80), provided that we’ve made the port accessible through the expose keyword:

    image: karthequian/helloworld:latest
      - "80"

In this case, by the way, it would also work without exposing it, because the expose directive is already in the image Dockerfile.

To reach a container from the host, the ports must be exposed declaratively through the ports keyword, which also allows us to choose if exposing the port differently in the host:

    image: karthequian/helloworld:latest
      - "80:80"
    image: myapp:latest
      - "8080:3000"
    image: myapp:latest
      - "8081:3000"

Port 80 will now be visible from the host, while port 3000 of the other two containers will be available on ports 8080 and 8081 in the host. This powerful mechanism allows us to run different containers exposing the same ports without collisions.

Finally, we can define additional virtual networks to segregate our containers:

    image: karthequian/helloworld:latest
      - my-shared-network
    image: alpine:latest
      - my-shared-network
    image: alpine:latest
      - my-private-network
  my-shared-network: {}
  my-private-network: {}

In this last example, we can see that another-service-in-the-same-network will be able to ping and to reach port 80 of network-example-service, while another-service-in-its-own-network won’t.

3.4. Setting Up the Volumes

There are three types of volumes: anonymous, named, and host ones.

Docker manages both anonymous and named volumes, automatically mounting them in self-generated directories in the host. While anonymous volumes were useful with older versions of Docker (pre 1.9), named ones are the suggested way to go nowadays. Host volumes also allow us to specify an existing folder in the host.

We can configure host volumes at the service level and named volumes in the outer level of the configuration, in order to make the latter visible to other containers and not only to the one they belong:

    image: alpine:latest
      - my-named-global-volume:/my-volumes/named-global-volume
      - /tmp:/my-volumes/host-volume
      - /home:/my-volumes/readonly-host-volume:ro
    image: alpine:latest
      - my-named-global-volume:/another-path/the-same-named-global-volume

Here, both containers will have read/write access to the my-named-global-volume shared folder, no matter the different paths they’ve mapped it to. The two host volumes, instead, will be available only to volumes-example-service.

The /tmp folder of the host’s file system is mapped to the /my-volumes/host-volume folder of the container.

This portion of the file system is writeable, which means that the container can not only read but also write (and delete) files in the host machine.

We can mount a volume in read-only mode by appending :ro to the rule, like for the /home folder (we don’t want a Docker container erasing our users by mistake).

3.5. Declaring the Dependencies

Often, we need to create a dependency chain between our services, so that some services get loaded before (and unloaded after) other ones. We can achieve this result through the depends_on keyword:

    image: wurstmeister/kafka:2.11-
      - zookeeper
    image: wurstmeister/zookeeper

We should be aware, however, that Compose will not wait for the zookeeper service to finish loading before starting the kafka service: it will simply wait for it to start. If we need a service to be fully loaded before starting another service, we need to get deeper control of startup and shutdown order in Compose.

4. Managing Environment Variables

Working with environment variables is easy in Compose. We can define static environment variables, and also define dynamic variables with the ${} notation:

    image: "postgres:${POSTGRES_VERSION}"
      DB: mydb
      USER: "${USER}"

There are different methods to provide those values to Compose.

For example, one is setting them in a .env file in the same directory, structured like a .properties file, key=value:


Otherwise, we can set them in the OS before calling the command:

export POSTGRES_VERSION=alpine
export USER=foo
docker-compose up

Finally, we might find handy using a simple one-liner in the shell:

POSTGRES_VERSION=alpine USER=foo docker-compose up

We can mix the approaches, but let’s keep in mind that Compose uses the following priority order, overwriting the less important with the higher ones:

  1. Compose file
  2. Shell environment variables
  3. Environment file
  4. Dockerfile
  5. Variable not defined

5. Scaling & Replicas

In older Compose versions, we were allowed to scale the instances of a container through the docker-compose scale command. Newer versions deprecated it and replaced it with the –__–scale option.

On the other side, we can exploit Docker Swarm – a cluster of Docker Engines – and autoscale our containers declaratively through the replicas attribute of the deploy section:

    image: dockersamples/examplevotingapp_worker
      - frontend
      - backend
      mode: replicated
      replicas: 6
          cpus: '0.50'
          memory: 50M
          cpus: '0.25'
          memory: 20M

Under deploy, we can also specify many other options, like the resources thresholds. Compose, however, considers the whole deploy section only when deploying to Swarm, and ignores it otherwise.

6. A Real-World Example: Spring Cloud Data Flow

While small experiments help us understanding the single gears, seeing the real-world code in action will definitely unveil the big picture.

Spring Cloud Data Flow is a complex project, but simple enough to be understandable. Let’s download its YAML file and run:


Compose will download, configure, and start every component, and then intersect the container’s logs into a single flow in the current terminal.

It’ll also apply unique colors to each one of them for a great user experience:

We might get the following error running a brand new Docker Compose installation:

lookup no such host

While there are different solutions to this common pitfall, using as DNS is probably the simplest.

7. Lifecycle Management

Let’s finally take a closer look at the syntax of Docker Compose:

docker-compose [-f ...] [options] [COMMAND] [ARGS...]

While there are many options and commands available, we need at least to know the ones to activate and deactivate the whole system correctly.

7.1. Startup

We’ve seen that we can create and start the containers, the networks, and the volumes defined in the configuration with up:

docker-compose up

After the first time, however, we can simply use start to start the services:

docker-compose start

In case our file has a different name than the default one (docker-compose.yml), we can exploit the -f and –__–file flags to specify an alternate file name:

docker-compose -f custom-compose-file.yml start

Compose can also run in the background as a daemon when launched with the -d option:

docker-compose up -d

7.2. Shutdown

To safely stop the active services, we can use stop, which will preserve containers, volumes, and networks, along with every modification made to them:

docker-compose stop

To reset the status of our project, instead, we simply run down, which will destroy everything with only the exception of external volumes:

docker-compose down

8. Conclusion

In this tutorial, we’ve learned about Docker Compose and how it works.

As usual, we can find the source docker-compose.yml file on GitHub, along with a helpful battery of tests immediately available in the following image:

#docker #devops

What is GEEK

Buddha Community

Docker Compose Tutorial for Beginners
Jeromy  Lowe

Jeromy Lowe


Data Visualization in R with ggplot2: A Beginner Tutorial

A famous general is thought to have said, “A good sketch is better than a long speech.” That advice may have come from the battlefield, but it’s applicable in lots of other areas — including data science. “Sketching” out our data by visualizing it using ggplot2 in R is more impactful than simply describing the trends we find.

This is why we visualize data. We visualize data because it’s easier to learn from something that we can see rather than read. And thankfully for data analysts and data scientists who use R, there’s a tidyverse package called ggplot2 that makes data visualization a snap!

In this blog post, we’ll learn how to take some data and produce a visualization using R. To work through it, it’s best if you already have an understanding of R programming syntax, but you don’t need to be an expert or have any prior experience working with ggplot2

#data science tutorials #beginner #ggplot2 #r #r tutorial #r tutorials #rstats #tutorial #tutorials

Iliana  Welch

Iliana Welch


Docker Tutorial for Beginners 8 - Build and Run C++ Applications in a Docker Container

Docker is an open platform that allows use package, develop, run, and ship software applications in different environments using containers.
In this course We will learn How to Write Dockerfiles, Working with the Docker Toolbox, How to Work with the Docker Machine, How to Use Docker Compose to fire up multiple containers, How to Work with Docker Kinematic, Push images to Docker Hub, Pull images from a Docker Registery, Push stacks of servers to Docker Hub.
How to install Docker on Mac.

#docker tutorial #c++ #docker container #docker #docker hub #devopstools

Willie  Beier

Willie Beier


Tutorial: Getting Started with R and RStudio

In this tutorial we’ll learn how to begin programming with R using RStudio. We’ll install R, and RStudio RStudio, an extremely popular development environment for R. We’ll learn the key RStudio features in order to start programming in R on our own.

If you already know how to use RStudio and want to learn some tips, tricks, and shortcuts, check out this Dataquest blog post.

Table of Contents

#data science tutorials #beginner #r tutorial #r tutorials #rstats #tutorial #tutorials

Tutorial: Loading and Cleaning Data with R and the tidyverse

1. Characteristics of Clean Data and Messy Data

What exactly is clean data? Clean data is accurate, complete, and in a format that is ready to analyze. Characteristics of clean data include data that are:

  • Free of duplicate rows/values
  • Error-free (e.g. free of misspellings)
  • Relevant (e.g. free of special characters)
  • The appropriate data type for analysis
  • Free of outliers (or only contain outliers have been identified/understood), and
  • Follows a “tidy data” structure

Common symptoms of messy data include data that contain:

  • Special characters (e.g. commas in numeric values)
  • Numeric values stored as text/character data types
  • Duplicate rows
  • Misspellings
  • Inaccuracies
  • White space
  • Missing data
  • Zeros instead of null values

2. Motivation

In this blog post, we will work with five property-sales datasets that are publicly available on the New York City Department of Finance Rolling Sales Data website. We encourage you to download the datasets and follow along! Each file contains one year of real estate sales data for one of New York City’s five boroughs. We will work with the following Microsoft Excel files:

  • rollingsales_bronx.xls
  • rollingsales_brooklyn.xls
  • rollingsales_manhattan.xls
  • rollingsales_queens.xls
  • rollingsales_statenisland.xls

As we work through this blog post, imagine that you are helping a friend launch their home-inspection business in New York City. You offer to help them by analyzing the data to better understand the real-estate market. But you realize that before you can analyze the data in R, you will need to diagnose and clean it first. And before you can diagnose the data, you will need to load it into R!

3. Load Data into R with readxl

Benefits of using tidyverse tools are often evident in the data-loading process. In many cases, the tidyverse package readxl will clean some data for you as Microsoft Excel data is loaded into R. If you are working with CSV data, the tidyverse readr package function read_csv() is the function to use (we’ll cover that later).

Let’s look at an example. Here’s how the Excel file for the Brooklyn borough looks:

The Brooklyn Excel file

Now let’s load the Brooklyn dataset into R from an Excel file. We’ll use the readxlpackage. We specify the function argument skip = 4 because the row that we want to use as the header (i.e. column names) is actually row 5. We can ignore the first four rows entirely and load the data into R beginning at row 5. Here’s the code:

library(readxl) # Load Excel files
brooklyn <- read_excel("rollingsales_brooklyn.xls", skip = 4)

Note we saved this dataset with the variable name brooklyn for future use.

4. View the Data with tidyr::glimpse()

The tidyverse offers a user-friendly way to view this data with the glimpse() function that is part of the tibble package. To use this package, we will need to load it for use in our current session. But rather than loading this package alone, we can load many of the tidyverse packages at one time. If you do not have the tidyverse collection of packages, install it on your machine using the following command in your R or R Studio session:


Once the package is installed, load it to memory:


Now that tidyverse is loaded into memory, take a “glimpse” of the Brooklyn dataset:

## Observations: 20,185
## Variables: 21
## $ BOROUGH <chr> "3", "3", "3", "3", "3", "3", "…
## $ `TAX CLASS AT PRESENT` <chr> "1", "1", "1", "1", "1", "1", "…
## $ BLOCK <dbl> 6359, 6360, 6364, 6367, 6371, 6…
## $ LOT <dbl> 70, 48, 74, 24, 19, 32, 65, 20,…
## $ `EASE-MENT` <lgl> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `BUILDING CLASS AT PRESENT` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ ADDRESS <chr> "8684 15TH AVENUE", "14 BAY 10T…
## $ `APARTMENT NUMBER` <chr> NA, NA, NA, NA, NA, NA, NA, NA,…
## $ `ZIP CODE` <dbl> 11228, 11228, 11214, 11214, 112…
## $ `RESIDENTIAL UNITS` <dbl> 1, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `COMMERCIAL UNITS` <dbl> 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0…
## $ `TOTAL UNITS` <dbl> 2, 1, 1, 1, 1, 1, 1, 1, 2, 1, 1…
## $ `LAND SQUARE FEET` <dbl> 1933, 2513, 2492, 1571, 2320, 3…
## $ `GROSS SQUARE FEET` <dbl> 4080, 1428, 972, 1456, 1566, 22…
## $ `YEAR BUILT` <dbl> 1930, 1930, 1950, 1935, 1930, 1…
## $ `TAX CLASS AT TIME OF SALE` <chr> "1", "1", "1", "1", "1", "1", "…
## $ `BUILDING CLASS AT TIME OF SALE` <chr> "S1", "A5", "A5", "A9", "A9", "…
## $ `SALE PRICE` <dbl> 1300000, 849000, 0, 830000, 0, …
## $ `SALE DATE` <dttm> 2020-04-28, 2020-03-18, 2019-0…

The glimpse() function provides a user-friendly way to view the column names and data types for all columns, or variables, in the data frame. With this function, we are also able to view the first few observations in the data frame. This data frame has 20,185 observations, or property sales records. And there are 21 variables, or columns.

#data science tutorials #beginner #r #r tutorial #r tutorials #rstats #tidyverse #tutorial #tutorials

Tutorial: How to Use Docker Volumes to Code Faster

If you are a developer who uses Docker, odds are you may have heard that you can use volumes to maintain persistent state for your containers in production. But what many developers don’t realize is that volumes can also be an excellent tool for speeding up your development workflow.
In this post, I’ll give you a brief overview of what is a Docker volume, how docker host volumes work, and show you a tutorial and example of how you can use volumes and nodemon to make coding with Docker easier and faster.

#microservices #tutorial #docker #developer-tutorials #docker-compose