Let’s deploy Grafana Loki to monitor logs in Grafana and create custom metric based on logs and configure alert accordingly.
In a production environment, a downtime of even a few microseconds is intolerable. Debugging such issues is time-critical. Proper logging and monitoring of infrastructure help in debugging such scenarios. It also helps in optimizing cost and other resources proactively, as well as helps to detect any impending issue which may arise in the near future. There are various logging and monitoring solutions available in the market. In this post, we will walk through the steps to deploy Grafana Loki in a Kubernetes environment. This is due to its seamless compatibility with Prometheus, a widely used software for collecting metrics. Grafana Loki consists of three components: Promtail, Loki, and Grafana (PLG), which we will see in brief before proceeding to the deployment. This article provides a better insight into the architectural differences of PLG and other primary logging and monitoring stack like Elasticsearch-FluentD-Kibana (EFK).
Logging, Monitoring, and Alerting With Grafana Loki
Before proceeding with the steps for deploying Grafana Loki, we will see each tool briefly.
Promtail
Promtail helps to monitor applications by shipping the container logs to Loki or Grafana cloud. This process primarily involves discovering targets, attaching labels to log streams from both log files and the systemd journal, and shipping them to Loki. Promtail’s service discovery is based on the Prometheus’ service discovery mechanism.
Loki
As quoted by creators of Loki, Loki is a horizontally scalable, highly available, multi-tenant log aggregation system inspired by Prometheus. Loki uses the same service discovery mechanism as that of Prometheus and adds labels to the log stream instead of indexing. Due to which, logs received from Promtail consist of the same set of labels as that of application metrics. Thus, it not only provides better context switching between logs and metrics but also avoids full index logging.
#open source #tutorial #microservices #grafana
