TL;DR: you can leverage the pre-install and pre-upgrade Helm hooks to run database migrations before your application is installed or updated. Keep reading to understand why simpler solutions might not be good ideas and a couple of gotchas when using Helm hooks.

If your application uses a relational database, your schema will evolve over time. Before deploying a new version of you application, you will have to ensure that the database schema is up to date. This article is not about how to generate and manage schema migrations (there are multiple tools to handle that) but how to apply them as part of the application deployment process on Kubernetes.

Running migrations within the application

You could decide to run migrations automatically as part of your service startup. This might seem like a good idea: it guarantees that your service will not start before the migrations are applied and removes the risk of running an application on an outdated schema.

But when running on Kubernetes, this poses a couple of issues:

• every time a new pod is created or restarted, it will try to run the migrations again. If your migration scripts are written correctly, this should not be an issue, but it’s not a clean design.
• if the migration takes a while (think adding a column on a huge table), your pod might miss its readiness check and be killed before the migration is done. You could increase the initial delay for the pod’s readiness check, but then it makes it hard to understand if a pod is not starting because it’s applying database migrations or because of a different issue.
• depending on the disruption budget of your deployment (how many pods can be added/removed at any point of time during a deployment), you may have 2 identical migrations running at the same time, resulting in conflicts and errors.

#kubernetes #helm #devops #database