This post going to be short and it’s gonna address a specific issue for the Kubernetes development cluster you may also face.

Issue:

You have a cluster for development so it used by CI/CD chain. During working hours, the cluster will scale up/down depending on the build jobs. I assume you have already enabled auto-scaling for your cluster. In some cases, some pods which run as a runner for CI/CD chain, stay in a running state always. Since runner pods are technically healthy, they will not be killed or remove automatically by CI configuration unless your system owns a kind of feature.

In this case, these pods will run always until it manually removed from the cluster. This hinders the cluster to scale down out of working time.

#devops #kubernetes #cluster #terminator