Tale a look at a number of tools that help you set up log forwarding, to monitor errors and exceptions. Here’s how they typically work.

As a developer, I love Lambda functions. They allow me to focus on the purpose of the functionality and save tons of time writing and deploying code. At the same time, one of the biggest challenges of using Lambda functions in production has been the troubleshooting of issues. This stems from a visibility gap between the code and how the user experiences the application and a lack of monitoring tools that specifically address this key problem in serverless environments.

Certainly, Amazon’s monitoring tool  CloudWatch provides a way to track function metrics and deep dive in the logs for debugging. However, combing through logs is not how I want to debug my issues — it takes _hours. _

I took a look at a number of tools that help you set up log forwarding, to monitor errors and exceptions. Here’s how they typically work:

1. Use the pre-configured CloudFormation stack to setup cloud resources and permissions in your environment.
2. Use CloudWatch APIs to stream (usually using Kinesis+Firehose) filtered logs into their own tool.
3. Apply formatting on ingested logs to present the errors and exceptions in a more consumable way.

This process works great. I was able to set up error monitoring in a few minutes without changing my code. In addition, I now had stack traces and interesting function details like:

1. Function memory usage
2. Function invocation time
3. Cost of executing my lambda functions

Challenges

Now, here are the challenges with the approach:

1. I don’t like that a tool has control over my AWS account because it is using assumeRole to access my account info.
2. The stack traces are still quite hard to read. Here’s an example:
3. A lot of run-time contexts are lost, for example, other threads and the ability to configure additional parameters.
4. I can’t see traces or transactions, to be able to debug issues across my application to correlate front-end and back-end behavior.

So, while log forwarding from CloudWatch is better than using the CloudWatch itself, it is not without flaws.

#monitoring #serverless #node #cloudwatch