Why the AWS Console Isn’t the Best for Serverless Debugging

We all know that debugging serverless is time-consuming and hard and that AWS Console doesn’t make it much easier. CloudWatch isn’t quite known for its ease of use. Why? Well to start with, it has suboptimal search features, logs scattered across multiple buckets and groups, little visualization capability, and no structure of Lambda function invocations. Across the AWS console, different types of monitoring data like logs, metrics, and traces are scattered in silos, adding tons of friction to debugging and troubleshooting efforts.

With CloudWatch, you have to look into multiple places to find out what executions happened when your latency metric spiked and that’s the case for debugging a Lambda function alone.

Things get even messier if your transaction spans multiple Lambda functions and a bunch of other managed services like DynamoDB or SQS. And then there are different regions and AWS accounts.

All this disconnect makes debugging serverless applications a real pain.

Here’s a typical case scenario of starting out with serverless. You start building your serverless application and you’re blown away by the development speed, you build many features in such a small amount of time, and naturally, you fall in love with serverless technology really quick. Then things go wrong because, well, that’s the nature of software development. Some misconfigured service, or some bug in a Lambda function you wrote, and now it’s time to debug. You’re forced to dig deep into CloudWatch, X-Ray, and whatnot, just to find out where your error is located.

Every error event that goes in that weakens your confidence in your stack a bit more and in the end, you get wary about moving fast and serverless becomes as slow and brittle as all other paradigms before it.

But what you want is to keep that speed of delivery and for this, you have to know what’s going on. Let’s face it, very often, CloudWatch just isn’t enough.

Here’s why you should stop digging around in AWS Console

In the last years, many serverless advocates sold you this new paradigm with function as a service (FaaS) solutions like AWS Lambda, and while it’s pretty awesome to be able to get a small part of code running in the cloud, without managing servers or containers, FaaS is just the catalyst of serverless.

You should think of serverless more like functionality as a service, using managed services like AppSync, S3, and Cognito, whenever possible and only fall back to Lambda when things simply won’t support your use-case. 

The problem is that most monitoring solutions go the same route as the serverless advocate went, and focus on Lambda. This leads to a dissonance; you can either get full insights in your system but you have to build most of it with custom Lambda functions or you get the full power of serverless with managed services, but debugging them will be a pain; which leads us back to the loss of trust from the beginning.

Dashbird gives you one source of truth for all your AWS related monitoring needs. You’ll find metrics, logs, and tracing in one place, which makes correlating cause and effect much simpler.

Dashbird tries to home in on managed AWS services so you don’t get lost in throwing buckets of hand-tailored code at your projects. SQS, DynamoDB, Step Functions, API Gateway, all these services that lower your time to market feel snug as a bug in the Dashbird monitoring platform.

Grouping Resources by Project Instead of Type Just Makes Sense

Dashbird groups all your resources by service, like you are used to from the AWS console, but also allows grouping by project. Projects group services that are related to each other in a logical sense, but not in a technical sense. For example, a project can consist of API Gateway, Lambda, and DynamoDB.

When you get an error it is usually related to a request that went through your pipeline of services.

To find an error you have to follow the request through all these services. With the AWS console, which groups services by technology, you would have to navigate multiple pages to find your error. In a project all this is consolidated, so you can find all the services the request hit in one place.

Find Errors Before they Happen

Naturally, you always want to keep the time from finding a bug to fixing it as low as possible. Dashbird alarms and insights, which are based on the AWS Well-Architected Framework and Dashbirds know-how of monitoring years of serverless production-ready systems, help you to do just that.

Dashbird’s insights immediately notify you if something fails, isn’t configured right or if metrics go in a dangerous range well before they can lead to an error. This way you can continuously improve your architecture and prevent errors from happening.

With a rather young technology like serverless, it’s always a pain to figure out best practices.

If it’s used right, you get an advantage over all competitors that do it the old way; if it’s used wrong, you might be worse off than them.

So getting this knowledge, not just from a generic article on the internet, but tailored to your specific system will save you valuable time and dollars off your Cloud bill.

Conclusion

If you encounter problems within your serverless systems, it’s crucial to find out what caused it in the shortest amount of time possible. Clicking around in umpteen different places around the AWS Console to correlate your errors with the requests that caused them isn’t your best way of action here. Sure, all data is there, but often there is much more data than you need and combing through it takes time too.

Third-party monitoring services like Dashbird help you cut through the noise and dramatically reduce the time to debug and troubleshoot. Dashbird consolidates all the log, metric, and tracing data into one place so you can query them as needed. With projects you can group serverless resources logically, which cuts down on log-lines again.

Finally, there are the Dashbird metrics based on the AWS Well-Architected Framework and Dashbird’s experience in monitoring serverless systems in production for years. These metrics can find errors for you before they even happen.

Many problematic configurations can be surfaced to you in the Dashbird console before the first customer even uses your system.

While a happy user is good, this feature can help especially with security related problems, which are a huge liability in the long run.

Also published at https://dashbird.io/blog/aws-console-serverless-debugging/