Using Spinnaker with Kubernetes for CI/CD

Kubernetes is now the de-facto standard for container orchestration. With more and more organizations adopting Kubernetes. This post will focus on pushing out new releases of the application to our Kubernetes cluster i.e. Continuous Delivery.

Kubernetes is now the de-facto standard for container orchestration. With more and more organizations adopting Kubernetes, it is essential that we get our fundamental ops-infra in place before any migration. This post will focus on pushing out new releases of the application to our Kubernetes cluster i.e. Continuous

Pre-requisites

1. Running Kubernetes Cluster (GKE is used for the purpose of this blog).
2. A Spinnaker set-up with Jenkins CI enabled.
3. Github webhook enabled for Jenkins jobs.

Strategy Overview

1. Github + Jenkins : CI System to build the docker image and push to registry.
2. Docker hub: Registry to store docker images.
3. Spinnaker : CD System to enable automatic deployments to Staging environment and supervised deployment to Production.

Continuous Delivery Pipeline for Kubernetes

Continuous Integration System

Although this post is about the CD system using Spinnaker. I want to briefly go over the CI pipeline so that the bigger picture is clear.

1. Whenever the master branch of a Git repo gets merged, a Jenkins job is triggered via Github webhook. The commit message for the master merge should include the updated version of the app, and whether it is Kubernetes Deploy action or Patch action.
2. The Jenkins job checks out the repo, builds code and the docker image according to the Dockerfile, and pushes it to Docker hub.
3. It then triggers a Spinnaker pipeline and sends a trigger.properties file as a build artifact. This properties file contains very crucial info that is consumed by Spinnaker and will be explained later in this post.

Properties File

TAG=1.7-SNAPSHOT
ACTION=DEPLOY

Continuous Delivery System

This is the crucial part. Spinnaker offers a ton of options for Kubernetes deployments. You can either consume manifests from GCS or S3 bucket or you can provide manifest as text within the pipeline.

Consuming manifests from GCS or S3 buckets includes more moving parts, and since this is an introductory blog, it is beyond this article's scope right now. However, with that being said, I extensively use that approach as it is best in scenarios where you need to deploy a large number of micro-services running in Kubernetes as such pipelines are highly templatized and re-usable.

Today, we will deploy a sample Nginx Service which reads the app version from a 

pom.xml file and renders it on the browser. Application code and Dockerfile can be found here. The part where index.html is updated can be seen below (This is what the Jenkins job does basically).

#!/bin/bash
#Author: Vaibhav Thakur

#Checking whether commit was in master of not.
if_master=`echo $payload | jq '.ref' | grep master`
if [ $? -eq 1 ]; then
    echo "Pipeline should not be triggered"
    exit 2
fi

#Getting tag from pom.xml
TAG=`grep SNAPSHOT pom.xml | sed 's|[<,>,/,version ]||g'`
echo $TAG

#Getting action from commit message
ACTION=$(echo $payload | jq -r '.commits[0].message' | cut -d',' -f2)

#Updating index.html
sed -i -- "s/VER/${TAG}/g" app/index.html

#Pushing to dockerhub
docker build -t vaibhavthakur/nginx-demo:$TAG .
docker push vaibhavthakur/nginx-demo:$TAG

echo TAG=${TAG} > trigger.properties
echo ACTION=${ACTION} >> trigger.properties

The manifest for the Nginx deployment and service is below:

apiVersion: v1
kind: Namespace
metadata:
  name: nginx
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  namespace: nginx
  name: nginx-deployment
spec:
  replicas: 1
  template:
    metadata:
      annotations:
        prometheus.io/path: "/status/format/prometheus"
        prometheus.io/scrape: "true"
        prometheus.io/port: "80"
      labels:
        app: nginx-server
    spec:
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
          - weight: 100
            podAffinityTerm:
              labelSelector:
                matchExpressions:
                - key: app
                  operator: In
                  values:
                  - nginx-server
              topologyKey: kubernetes.io/hostname
      containers:
      - name: nginx-demo
        image: vaibhavthakur/nginx-demo:1.0-SNAPSHOT
        imagePullPolicy: Always
        resources:
          limits:
            cpu: 2500m
          requests:
            cpu: 2000m
        ports:
        - containerPort: 80
          name: http
---
apiVersion: v1
kind: Service
metadata:
  namespace: nginx
  name: nginx-service
  annotations:
    cloud.google.com/load-balancer-type: Internal
spec:
  ports:
  - port: 80
    targetPort: 80
    name: http
  selector:
    app: nginx-server
  type: LoadBalancer

k8s kubernetes cicd continuous-integration continuous-delivery kubernetes-cluster pipeline jenkins

Our original Kubernetes tool list was so popular that we've curated another great list of tools to help you improve your functionality with the platform.

In this article, we’ll describe how to install Jenkins cluster with slave nodes auto-discovering and self-registering inside a master node. Jelastic PaaS implemented this solution in Jenkins DevOps Pack that can be installed from the Marketplace or through the environment setup wizard as a New Environment. In this tutorial, we’ll cover both. Also, you will find out how to build a simple Java project hosted on GitHub using the Jelastic Maven plugin.

After using Jenkins on several projects, we say it's time to move on. Jenkins is left behind with his old approach — found out more!

How to build and deploy Continuous Integration, Delivery with Jenkins for microservices application on Kubernetes and serverless

How to dockerize a Python application with Microservices architecture and Continuous Integration and jenkins pipeline with kubernetes