Observability is essential to running huge workloads in Kubernetes clusters. Prometheus is a monitoring system and time-series database that has proven to be adept at managing large-scale, dynamic Kubernetes environments. In fact, Prometheus is considered a foundational building block for running applications on Kubernetes and has become the de-facto open source standard for visibility and monitoring in Kubernetes environments.

Although open source, Prometheus does not come for free in terms of configuration that is required to properly monitor Kubernetes workloads. In this article, part one of a two-part piece on Prometheus, I highlight the most common challenges facing platform operators and site reliability engineers (SREs) for onboarding new workloads to Prometheus and configuring the tool ecosystem needed to manage Prometheus, along with potential solutions for overcoming each of these challenges.

Disclaimer: In this article, I don’t discuss the challenge of high-availability setups of Prometheus and multicluster setups. Instead, I focus on how to scale Prometheus to onboard more applications and to create dashboards for each application, so that more people can use it. If you are interested in the high-availability setups, you can refer to projects such as Thanos or VictoriaMetrics.

To start getting Prometheus ready in your organization, you can configure scraping to pull metrics from your services, build dashboards on top of your data using Grafana, and define alerts for important metrics breaching thresholds in your production environment (see figure below).

#kubernetes #microservices #monitoring #contributed #sponsored