Golang is a beautiful programming language that offers a plethora of feature sets to optimise and speed up your program execution.
Courtesy: https://github.com/dpaks/goworkers
With this power of Golang, let me show you how I reduced an API latency of our server from around 15 seconds to 300 milliseconds, i.e., a staggering 98% reduction in latency or 4900% increase in performance.
I work in the product team of DKube, an enterprise grade MLOps platform at my organisation. We have a lot of microservices and they sit atop Kubernetes v1.15.3. We use Golang v1.10, which will be upped to v1.14 soon.
We have an API to query the status of our microservices by utilising the metrics data from Kubernetes. The microservices are a combination of daemonsets, deployments and statefulsets. Therefore, as the size of cluster increases, the number of microservices increases due to increased replica size.
The Problem
I noticed that this particular API was taking a very long time, indeed 15 seconds even in a single node setup. I checked the code and found 2 issues:
Problem #1
We were using kubectl commands (Kubernetes CLI) to fetch the list of replicas under a microservice. We loop over this list to process data of a particular replica.
E.g: Suppose we have a service ‘foo’ and it has 10 replicas. In backend, we use 1 kubectl command to get the names of replicas. We use another 10 to fetch metrics data of each of the replicas. Hence, a total of 11 (i.e., number of replicas + 1) kubectl calls for a service with 10 replicas.
Problem #2
The API handler was running as a single go routine. We use Swagger http server. The list and fetch operations were sequential.
E.g: Suppose we have 100 microservices, each with 2 replicas. In backend, each of them are processed sequentially. A microservice with 2 replicas costs 3 (i.e., number of replicas + 1) kubectl calls as we saw in point 1. Therefore, a total of 300 sequential kubectl calls for querying the status of 100 microservices with 2 replicas each.
#kubernetes #performance #go #golang #api