In my last 2 articles about maintenance, we have talked about properly handling exceptions and how to log properly. Both subjects are simple enough, but done improperly will make our application unmaintainable.

In this article, I won’t explain how to configure these log aggregation and analysis tool. We rather are going to see the difference between the various tools I have tested. This way we can choose which tool we wish to have. There is plenty of tutorials on how to set up the tools.

Why aggregate logs

If we can’t find something it is as if we didn’t have it.

It is true for things we own, it’s also true for logs. For most small applications we can search and find logs using a simple grep command on the server. Most often this is enough. But when we start having multiple files and multiple servers, with thousands of users, finding the right log might start to become a problem. Log aggregation will help us find these logs.

Errors/Exceptions; are usually easy to find; because we often have one error repeating itself; or a single error log that’s easy to find. In this case, finding the log and fixing the issue shouldn’t be hard. But this means that either someone checks our error logs manually, or someone has noticed the error and is complaining. Log aggregation will allow us to see if there have been any errors and with some tools even alert us.

What aggregation also allows us to do, is statistics. As discussed in a previous article, this will allow us to understand the impact of a particular error or log. If our website registers thousands of orders per day and we have a single order in error; even if the error is important the issue is not critical.

#datadog #grafana #elasticsearch #programming #data-analysis