LOGS. When it comes to logging, developers have to go an extra mile by inserting those log lines in their application code and also make sure their infrastructure can handle huge log ingestion to maintain scalability. The benefits? Apart from debugging, logs tell us about the flow of transactions, monitor errors, generate alerts, and do Root Cause Analysis when things go haywire.

In this read, I tried to put forward certain ways on how you can set up a reliable and scalable logging architecture for your application deployed either in VMs or in Kubernetes.

Note: We should not log just for the sake of logging data. Take a proactive approach because it will eventually effect your storage and billing!

Centralized logging

We fetch. We process. We ship.

There are multiple benefits of storing logs in a central location. It reduces overhead on your application servers, you can effectively control and exploit log data and it eliminates the conundrum of tracking logs.

Logging architecture — the EFK stack

Elasticsearch — Real time, distributed, and scalable search engine. Can be used to index and search through volumes of log data.

Fluentd — Acts as a shipper or collector. As the shipper, it forwards log data to Elasticsearch and as a collector, it collects and forwards logs to shipper.

Kibana — Provides visualizations and dashboards for Elasticsearch. Also used to query the Elasticsearch.

#log-management-tool #kubernetes #log-management #logging #devops