How To Automate Jenkins Setup with Docker and Jenkins Configuration as Code

The author selected the Wikimedia Foundation to receive a donation as part of the Write for DOnations program.

Introduction

Jenkins is one of the most popular open-source automation servers, often used to orchestrate continuous integration (CI) and/or continuous deployment (CD) workflows.

Configuring Jenkins is typically done manually through a web-based setup wizard; this can be a slow, error-prone, and non-scalable process. You can see the steps involved by following Step 4 — Setting Up Jenkins of the How To Install Jenkins on Ubuntu 18.04 guide. Furthermore, configurations cannot be tracked in a version control system (VCS) like Git, nor be under the scrutiny of any code review process.

In this tutorial, you will automate the installation and configuration of Jenkins using Docker and the Jenkins Configuration as Code (JCasC) method.

Jenkins uses a pluggable architecture to provide most of its functionality. JCasC makes use of the Configuration as Code plugin, which allows you to define the desired state of your Jenkins configuration as one or more YAML file(s), eliminating the need for the setup wizard. On initialization, the Configuration as Code plugin would configure Jenkins according to the configuration file(s), greatly reducing the configuration time and eliminating human errors.

Docker is the de facto standard for creating and running containers, which is a virtualization technology that allows you to run isolated, self-contained applications consistently across different operation systems (OSes) and hardware architectures. You will run your Jenkins instance using Docker to take advantage of this consistency and cross-platform capability.

This tutorial starts by guiding you through setting up JCasC. You will then incrementally add to the JCasC configuration file to set up users, configuration authentication and authorization, and finally to secure your Jenkins instance. After you’ve completed this tutorial, you’ll have created a custom Docker image that is set up to use the Configuration as Code plugin on startup to automatically configure and secure your Jenkins instance.

Prerequisites

To complete this tutorial, you will need:

• Access to a server with at least 2GB of RAM and Docker installed. This can be your local development machine, a Droplet, or any kind of server. Follow Step 1 — Installing Docker from one of the tutorials in the How to Install and Use Docker collection to set up Docker.

Note:  This tutorial is tested on Ubuntu 18.04; however, because Docker images are self-contained, the steps outlined here would work for any OSes with Docker installed.

Step 1 — Disabling the Setup Wizard

Using JCasC eliminates the need to show the setup wizard; therefore, in this first step, you’ll create a modified version of the official [jenkins/jenkins](https://hub.docker.com/r/jenkins/jenkins/) image that has the setup wizard disabled. You will do this by creating a Dockerfile and building a custom Jenkins image from it.

The jenkins/jenkins image allows you to enable or disable the setup wizard by passing in a system property named jenkins.install.runSetupWizard via the JAVA_OPTS environment variable. Users of the image can pass in the JAVA_OPTS environment variable at runtime using the [--env](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file) flag to docker run. However, this approach would put the onus of disabling the setup wizard on the user of the image. Instead, you should disable the setup wizard at build time, so that the setup wizard is disabled by default.

You can achieve this by creating a Dockerfile and using the [ENV](https://docs.docker.com/engine/reference/builder/#env) instruction to set the JAVA_OPTS environment variable.

First, create a new directory inside your server to store the files you will be creating in this tutorial:

mkdir -p $HOME/playground/jcasc

Then, navigate inside that directory:

cd $HOME/playground/jcasc

Next, using your editor, create a new file named Dockerfile:

nano $HOME/playground/jcasc/Dockerfile

Then, copy the following content into the Dockerfile:

~/playground/jcasc/

FROM jenkins/jenkins:latest
ENV JAVA_OPTS -Djenkins.install.runSetupWizard=false

Here, you’re using the [FROM](https://docs.docker.com/engine/reference/builder/#from) instruction to specify jenkins/jenkins:latest as the base image, and the ENV instruction to set the JAVA_OPTS environment variable.

Save the file and exit the editor by pressing CTRL+X followed by Y.

#docker

What is GEEK

Buddha Community

Softagram - Making Code Reviews Humane

The story of Softagram is a long one and has many twists. Everything started in a small company long time ago, from the area of static analysis tools development. After many phases, Softagram is focusing on helping developers to get visual feedback on the code change: how is the software design evolving in the pull request under review.

Benefits of code change visualization and dependency checks

While it is trivial to write 20 KLOC apps without help of tooling, usually things start getting complicated when the system grows over 100 KLOC.

The risk of god class anti-pattern, and the risk of mixing up with the responsibilities are increasing exponentially while the software grows larger.

To help with that, software evolution can be tracked safely with explicit dependency change reports provided automatically to each pull request. Blocking bad PR becomes easy, and having visual reports also has a democratizing effect on code review.

Example visualization

Basic building blocks of Softagram

• Architectural analysis of the code, identifying how delta is impacting to the code base. Language specific analyzers are able to extract the essential internal/external dependency structures from each of the mainstream programming languages.

• Checking for rule violations or anomalies in the delta, e.g. finding out cyclical dependencies. Graph theory comes to big help when finding out unwanted or weird dependencies.

• Building visualization for humans. Complex structures such as software is not easy to represent without help of graph visualization. Here comes the vital role of change graph visualization technology developed within the last few years.

#automated-code-review #code-review-automation #code-reviews #devsecops #software-development #code-review #coding #good-company

Integrating SonarQube with Jenkins

Welcome back to the second article in our #BacktoBasics series. As many of us already know, SonarQube is an open-source tool for continuous inspection of code quality. It performs static analysis of code, thus detecting bugs, code smells and security vulnerabilities. In addition, it also can report on the duplicate code, unit tests, code coverage and code complexities for multiple programming languages. Hence, in order to achieve Continuous Integration with fully automated code analysis, it is important to integrate SonarQube with CI tools such as Jenkins. Here, we are going to discuss integrating SonarQube with Jenkins to perform code analysis.

Running Jenkins and SonarQube on Docker

Enough on the introductions. Let’s jump into the configurations, shall we? First of all, let’s spin up Jenkins and SonarQube using Docker containers. Note that, we are going to use docker compose as it is an easy method to handle multiple services. Below is the content of the docker-compose.yml file which we are going to use.

docker-compose.yml file

version: '3'
services:
  sonarqube: 
    ports: 
      - '9000:9000' 
    volumes: 
      - 'E:\work\sonar\conf\:/opt/sonarqube/conf' 
      - 'E:\work\sonar\data\:/opt/sonarqube/data' 
      - 'E:\work\sonar\logs\:/opt/sonarqube/logs' 
      - 'E:\work\sonar\extensions\:/opt/sonarqube/extensions' 
    image: sonarqube
  jenkins:
    image: 'ravindranathbarathy/jenkins'
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - 'E:\work\jenkins_home\:/var/jenkins_home'  
    ports:
      - '8080:8080'
      - '5000:50000'
  jenkins-slave:
    container_name: jenkins-slave
    restart: always
    environment:
            - 'JENKINS_URL=http://jenkins:8080'
    image: kaviyakulothungan/jenkins-slave-node:v2
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - 'E:\work\jenkins_slave\:/home/jenkins'
    depends_on:
      - jenkins

docker-compose up is the command to run the docker-compose.yml file.

docker-compose command to spin up Jenkins and Sonarqube

Shell

1

docker-compose up

Note: The _docker-compose_ command must be run from folder where the _docker-compose.yml_ file is placed

This file, when run, will automatically host the Jenkins listening on port 8080 along with a slave.

Jenkins hosted using Docker

The SonarQube will be hosted listening on port 9000.

SonarQube hosted using Docker

Configuring Jenkins for SonarQube Analysis

In order to run the SonarQube analysis in Jenkins, there are few things we have to take care before creating the Jenkins job. First of all, we need to install the**_ ‘_SonarQube Scanner” plugin. For this, let’s go to Jenkins -> Manage Jenkins -> Manage Plugins. There, navigate to “Available” view and look for the plugin “SonarQube Scanner”. Select the plugin and click on “Install without restart**” and wait for the plugin to be installed.

Installing SonarQube Scanner Plugin

Once the plugin is installed, we need to configure a few things in the Jenkins global configuration page.

For that, let’s click on Jenkins -> Manage Jenkins -> Configure System -> SonarQube Servers and fill in the required details.

SonarQube Server Configuration

Here,

• Name: Anything meaningful. Eg. sonarqube
• Server URL:
• Server Authentication Token: Refer below

To get the server authentication token, login to SonarQube and go to Administration -> Security -> Users and then click on Tokens. There, Enter a Token name and click on Generate and copy the token value and paste it in the Jenkins field and then click on “Done”.

Creating Authorization Token

Finally, save the Jenkins Global configurations by clicking on the “Save” icon.

There is one last configuration which has to be set up. In order to run SonarQube scan for our project, we need to install and configure the SonarQube scanner in our Jenkins. For that, let’s go to Manage Jenkins -> Global Tool Configuration -> SonarQube Scanner -> SonarQube Scanner installations. Enter any meaningful name under the Name field and select an appropriate method in which you want to install this tool in Jenkins. Here, we are going to select “Install automatically” option. Then, click on “Save”.

SonarQube Scanner Configuration in Jenkins

Creating and Configuring Jenkins Pipeline Job

Since we are all set with the global configurations, let’s now create a Jenkins Pipeline Job for a simple node.js application for which code analysis will be done by SonarQube.

For that, let’s click on “New Item” in Jenkins home page and enter the job name as “sonarqube_test_pipeline” and then select the “Pipeline” option and then click on “OK”.

Creating Jenkins Pipeline job

Now, inside the job configuration, let’s go to the Pipeline step and select Pipeline Script from SCM and then select Git and enter the Repository URL and then save the job.

##backtobasics #continuous integration #devops #blueocean #ci #code review #continous integration #docker #docker-compose #git #github #jenkins #jenkins pipeline #nodejs #sonarqube #sonarqube scanner #static code analysis

Static Code Analysis: What It Is? How to Use It?

Static code analysis refers to the technique of approximating the runtime behavior of a program. In other words, it is the process of predicting the output of a program without actually executing it.

Lately, however, the term “Static Code Analysis” is more commonly used to refer to one of the applications of this technique rather than the technique itself — program comprehension — understanding the program and detecting issues in it (anything from syntax errors to type mismatches, performance hogs likely bugs, security loopholes, etc.). This is the usage we’d be referring to throughout this post.

“The refinement of techniques for the prompt discovery of error serves as well as any other as a hallmark of what we mean by science.”

• J. Robert Oppenheimer

Outline

We cover a lot of ground in this post. The aim is to build an understanding of static code analysis and to equip you with the basic theory, and the right tools so that you can write analyzers on your own.

We start our journey with laying down the essential parts of the pipeline which a compiler follows to understand what a piece of code does. We learn where to tap points in this pipeline to plug in our analyzers and extract meaningful information. In the latter half, we get our feet wet, and write four such static analyzers, completely from scratch, in Python.

Note that although the ideas here are discussed in light of Python, static code analyzers across all programming languages are carved out along similar lines. We chose Python because of the availability of an easy to use ast module, and wide adoption of the language itself.

How does it all work?

Before a computer can finally “understand” and execute a piece of code, it goes through a series of complicated transformations:

As you can see in the diagram (go ahead, zoom it!), the static analyzers feed on the output of these stages. To be able to better understand the static analysis techniques, let’s look at each of these steps in some more detail:

Scanning

The first thing that a compiler does when trying to understand a piece of code is to break it down into smaller chunks, also known as tokens. Tokens are akin to what words are in a language.

A token might consist of either a single character, like (, or literals (like integers, strings, e.g., 7, Bob, etc.), or reserved keywords of that language (e.g, def in Python). Characters which do not contribute towards the semantics of a program, like trailing whitespace, comments, etc. are often discarded by the scanner.

Python provides the tokenize module in its standard library to let you play around with tokens:

Python

1

import io

2

import tokenize

3

4

code = b"color = input('Enter your favourite color: ')"

5

6

for token in tokenize.tokenize(io.BytesIO(code).readline):

7

    print(token)

Python

1

TokenInfo(type=62 (ENCODING),  string='utf-8')

2

TokenInfo(type=1  (NAME),      string='color')

3

TokenInfo(type=54 (OP),        string='=')

4

TokenInfo(type=1  (NAME),      string='input')

5

TokenInfo(type=54 (OP),        string='(')

6

TokenInfo(type=3  (STRING),    string="'Enter your favourite color: '")

7

TokenInfo(type=54 (OP),        string=')')

8

TokenInfo(type=4  (NEWLINE),   string='')

9

TokenInfo(type=0  (ENDMARKER), string='')

(Note that for the sake of readability, I’ve omitted a few columns from the result above — metadata like starting index, ending index, a copy of the line on which a token occurs, etc.)

#code quality #code review #static analysis #static code analysis #code analysis #static analysis tools #code review tips #static code analyzer #static code analysis tool #static analyzer

Docker Explained: Docker Architecture | Docker Registries

Following the second video about Docker basics, in this video, I explain Docker architecture and explain the different building blocks of the docker engine; docker client, API, Docker Daemon. I also explain what a docker registry is and I finish the video with a demo explaining and illustrating how to use Docker hub

In this video lesson you will learn:

• What is Docker Host
• What is Docker Engine
• Learn about Docker Architecture
• Learn about Docker client and Docker Daemon
• Docker Hub and Registries
• Simple demo to understand using images from registries

#docker #docker hub #docker host #docker engine #docker architecture #api

Jenkins Is Getting Old — It’s Time to Move On

By far, Jenkins is the most adopted tool for continuous integration, owning nearly 50% of the market share. As so many developers are using it, it has excellent community support, like no other Jenkins alternative. With that, it has more than 1,500 plugins available for continuous integration and delivery purposes.

We love and respect Jenkins. After all, it’s the first tool we encountered at the beginning of our automation careers. But as things are rapidly changing in the automation field, Jenkins is** left behind with his old approach**. Even though many developers and companies are using it, most of them aren’t happy with it. Having used it ourselves on previous projects, we quickly became frustrated by its lack of functionality, numerous maintenance issues, dependencies, and scaling problems.

We decided to investigate if other developers face the same problems and quickly saw the need to create a tool ourselves. We asked some developers at last year’s AWS Summit in Berlin about this. Most of them told us that they chose Jenkins because it’s free in the first place. However, many of them expressed interest in trying to use some other Jenkins alternative.

#devops #continuous integration #jenkins #devops adoption #jenkins ci #jenkins pipeline #devops continuous integration #jenkins automation #jenkins scripts #old technology