We have a project running in Kubernetes that needs to run SQL migrations during deployment.

To run migrations need to clone a Github repository and run actually migrations stored in it.

Currently, this is done with Kubernetes initContainers , and there are two of them — the first one with git clones the repository with migrations files into a Kubernetes Volume, and then another one with sql-migrate runs those migrations from this shared volume.

Still, there are a few issues with this approach:

1. each time when we are running a new pod it runs its initContainers and migrations
2. if a Deployment spin up a couple of pods — each will run migrations
3. if migrations will run for w while and will not respond to Kubernetes readiness — it could be killed without finishing migrations

To avoid all the above let’s reconfigure the process by using a Kubernetes Job to run only one pod and by adding Helm Hooks to trigger migrations during deployment.

Note: the kk here is an alias for the kubectl.

Preparation

Docker image

At first, let’s create our own Docker image with blackjack and git and https://github.com/rubenv/sql-migrate.

Create a Dockerfile:

FROM golang:alpine AS builder
RUN apk add --no-cache git gcc g++
RUN go get -v github.com/rubenv/sql-migrate/sql-migrate
RUN mv /go/bin/sql-migrate /bin/sql-migrate

Build and push it:

$ docker build -t projectname/sql-migrate-git .
$ docker push projectname/sql-migrate-git

Git authentification

The second thing is the Github authentification.

At this moment our git-container authenticates via an RSA key which is held in a Kubernetes Secrets, then it passes to a pod via an environment variable from where it’s taken by a bash script /opt/git/git.sh which is used to create a key-file /root/.ssh/id_rsa inside of the container, and this key finally is used to authenticate.

The initContainers in our Deployment currently looks like the next:

...
      initContainers:
      - name: git-clone
        image: projectname/git-cloner
        env:
        - name: SSH_PRIVATE_KEY
          valueFrom:
            secretKeyRef:
              name: git-ssh-key
              key: id_rsa
        - name: REPOSITORY_URL
          value: {{ .Values.git.repo }}
        - name: GIT_BRANCH
          value: {{ .Values.git.branch }}
        command: ['sh', '-c', '/opt/git/git.sh']
        volumeMounts:
          - name: git-volume
            mountPath: "/git"
            readOnly: false
      - name: init-migration
        image: fufuhu/sql-migrate:latest
        command: ['sh', '-c', 'while [ ! -d /git/db/migrations ]; do sleep 2; done && sleep 2; /bin/sql-migrate up -config=/config/config.yaml -env=main']
        volumeMounts:
          - name: migration-config
            mountPath: "/config/"
            readOnly: true
          - name: git-volume
            mountPath: "/git"
            readOnly: false
...

A lot of steps, a lot of objects, complicated process.

Instead, let’s use a login and a Github token which will be passed into our container from environment variables, and then we can clone the repository via HTTPS.

#helm #kubernetes #automation #sql #tutorial