When running Azure Kubernetes Service (AKS), it can be hard to understand and allocate costs in environments with multiple teams, projects, or even departments. With Kubecost, you gain full transparency into your Kubernetes usage and cost within minutes of installation. Officially launched in 2019 and built on open source, Kubecost now monitors over one billion dollars in Kubernetes spend, and enables startups and global enterprises alike to understand their spend and identify cost savings ranging from 30% to over 50%. Kubecost supports a wide range of self-managed and hosted Kubernetes environments, including Azure Kubernetes Service, which we’ll cover today in this article.

The Microsoft Azure Kubernetes Service (AKS) is a popular fully managed Kubernetes service that offers embedded continuous integration and continuous delivery as well as enterprise-grade security and governance— powerful tools for teams adopting Kubernetes. As with any complex infrastructure, AKS requires proper governance and financial transparency for successful organizational adoption. Kubecost, an open source tool that provides teams with visibility into Kubernetes spend and supports environments hosted in Azure, is a widely recommended solution for engineers and finance teams facing this problem.

The Cost Allocation ChallengePermalink

Kubernetes clusters are often shared across teams, microservices, projects, applications, departments, and cost centers to save on infrastructure and administrative overhead. The most popular methods for organizing this shared platform is by simply using labels (similar to tags) or by creating namespaces. A Kubernetes namespace is a logical separation inside a Kubernetes cluster which could be assigned to a particular team, application, or even a business unit.

Most organizations map a namespace to a specific workload type or purpose. For example, a typical cluster might have one namespace for monitoring and one namespace for logging for use by the DevOps teams who maintain the cluster. A customer-facing application hosted in that same cluster might have one namespace for its front-end components and a separate namespace for its back-end services.

Creating these logical divisions inside a cluster is administratively convenient, but adds complexity when trying to accurately measure resource usage and allocate costs to each tenant based on detailed billing data.

Sharing a Kubernetes Cluster in AzurePermalink

There are three models for sharing Azure Kubernetes Service (AKS) clusters across cost centers:

1. Dedicated AKS cluster: Each team or project gets its own dedicated AKS cluster to easily track its billing. This approach does not benefit from resource sharing.
2. Dedicated AKS node: Each team or project gets its pods assigned to a specific node in a shared AKS cluster using labels and a configuration constraint known as nodeSelector. Each team can then isolate its spending in the Azure bill according to the cost of its assigned nodes. Resource sharing is not efficient in this model.
3. Dedicated AKS Namespace: Each team or project gets its own dedicated namespace along with more flexibility and administrative freedom. This model, as we have mentioned, can be challenging for cost allocation. However, this is where Kubecost comes in.

The Measurement ChallengePermalink

Suppose you have chosen to support multiple projects and teams on your AKS cluster using a few dozen namespaces. In this example, each team has its own namespace and autonomous administrative control over provisioning pods and containers. After a couple of months, the cluster costs are higher than expected, and you would like to identify the cause of the spending jump. It’s a simple question, but complicated to answer, because you must measure resource usage by workloads in each namespace over multiple weeks to come to a conclusion.

This means that you must allocate the following by namespace:

1. The actual usage of CPU, GPU, memory, network, and persistent storage.
2. The excessive requests for CPU and memory leaving resources unused.
3. The usage of shared resources such as databases or blob storage.
4. The proportionate use of cloud instance reservations and support services.

If you take just network usage as one example, you are required to:

1. Measure the egress traffic generated by containers belonging to each namespace across regions, availability zones, and out to the internet.
2. Determine the data transfer rates that may be negotiated with your cloud provider.

Only then can you measure the networking costs associated with each namespace.

This process is daunting, especially when you consider the fact that it requires:

• Specialized telemetry for collecting relevant data in real-time
• Storage of time-series data over time
• Collection and analysis of cloud provider’s detailed billing logs
• Analysis of usage and cost data to calculate the accurate costs
• Presentation of the results in the form of reports or dashboards

Don’t Sidestep the ChallengePermalink

Leaving this challenge unresolved leads to organizational frustration in the long run as you struggle to identify the root cause of a spending jump or struggle to respond to a management request for what seems a simple report of cost allocation by application, team, or project. A sudden increase in spending due to misconfiguration would be a routine operational matter if noticed within hours or days, but when left lurking in the shadows for weeks or months, even a simple misconfiguration can fester into an end-of-quarter budgeting drama.

Although Kubernetes delivers on the promise of economies of scale and auto-scaling, it requires proper governance and financial transparency for sustained, successful organizational adoption.

#devops #azure #kubernetes #monitoring #aks