December 2020 — AWS released a new way to explore metrics across various dimensions. This is the Metrics Explorer.
I'm not one to blow my own horn (since I worked on this product), but as a customer of AWS — I love the product.
A bit of background: my side project uses a lot of Lambda functions to crawl the Latvian real estate market. Up till now, it was quite difficult to see any issues in my platform. CloudWatch alarms weren't quite useful as (1) using them adds cost to the project (which I'm trying to keep as frugal as it's possible) and (2) there's only a handful of crawlers which would make me worried if they went down. All the rest can go down for a week or even a month and I wouldn't be losing any data.
So my usual approach was to just take a peek at the CloudWatch metrics page once a month to see if there have been any recurring issues with the not-so-important Lambdas. This is all fine-and-dandy, but it's labor-intensive — to select each and every Lambda (there are hundreds).
But then the Metrics Explorer came along. With it — it's super easy to select a metric I'm interested in (e.g. errors), select a specific Lambda tag, and then split by another tag. This immediately gives a birds-eye view of all the crawlers (grouped by the site they are crawling) and the errors in each of the groups.
Not only that — it makes discovering other metrics super easy too. For example, one of my crawlers was using running a total of ~10m milliseconds per month. But this was not an important crawler, so I could immediately make the judgment call to change the Lambda to run less frequently. This in turn means I would spend less money on it. Just fantastic!
If you're also an AWS customer — I highly recommend checking out the new Metrics Explorer. See for yourself, how using it could help reduce costs, find issues, or just have a birds-eye view of the system you're running.
Disclaimer: I am an AWS employee and I worked on the Metrics Explorer product. This article contains only my personal opinion. In no way does it reflect the views of Amazon.