Alan Edwardes

Cheap Synthetic Monitoring with AWS Lambda

📅 24 January 2021 🔖 cloud-software ⏲️ 1 minute to read

⚠️ This post was last updated in 2021, meaning its contents may be outdated.

I have a few miscellaneous services which I host for various personal things; MediaWiki, Jenkins, Grafana, Rocket Chat, etc. These are usually hosted on VPS machines with Docker, which means a "one size fits all" monitoring solution is hard.

If I were operating as a business, I would just opt for an off-the-shelf synthetic monitoring solution with a support contract and call it done, but in a personal capacity I don't particularly want to pay.

I had a few criteria:

• Cheap/free
• Ability to write synthetic checks using HTTP requests (not a headless browser, that is a bit OTT)
• Synthetic checks preferably in C#
• Alerting via email for when the checks fail
• Hosted redundantly & managed (e.g. AWS, not on a VPS)
• Low/zero maintenance

Eventually I decided upon writing an AWS Lambda function, since you can schedule one to run every 5 minutes on a CloudWatch schedule. If that Lambda only runs for 30 seconds and is set to use 128MB of RAM, its usage will fall within the free tier so won't cost anything.

Ae.Synthetics

I created a small C# library to handle the heavy lifting of:

• Scheduling the synthetic checks in parallel
• Error handling / killing zombies
• Alerting via Amazon SES

In order to use it, you just need to implement your synthetic check as a C# class with the ISyntheticTest interface and register it with the runner.

Here's an example of a check:

internal sealed class EstrangedWikiTest : ISyntheticTest
{
    public async Task Run(ILogger logger, CancellationToken token)
    {
        logger.LogInformation($"Calling GET https://wiki.iamestranged.com/Main_Page");
        var response = await HttpClientStatics.SharedHttpClient.GetAsync("https://wiki.iamestranged.com/Main_Page", token);
        response.EnsureSuccessStatusCode();
    }
}

It could do more things, like submitting a username and password to the login form, but it depends on what's important for you to test. I typically split each endpoint into a separate synthetic test, since having multiple tests makes failures easier to track.

If there is a problem, you'll get an email like this:

Synthetic test EstrangedWikiTest failed with the following exception, in 2.2398347s:
System.Net.Http.HttpRequestException: Error while copying content to a stream. > System.IO.IOException: Unable to read data from the transport connection: Connection reset by peer. > System.Net.Sockets.SocketException: Connection reset by peer
    End of inner exception stack trace 
   at async ValueTask System.Net.Security.SslStreamInternal.FillBufferAsync(TReadAdapter adapter, int minSize)+InternalFillBufferAsync(?)
   at async ValueTask System.Net.Security.SslStreamInternal.ReadAsyncInternal(TReadAdapter adapter, Memory buffer)
...
Log entries before exception:
1.	[INFORMATION 2020-06-07T08:01:38.9730779+00:00] Calling GET https://wiki.iamestranged.com/Main_Page

... and you will continue to get the email every time the Lambda runs until the check passes again.

Full usage examples including the stack to deploy can be found in the README on the GitHub project.

🏷️ synthetic lambda monitoring aws c♯ email cheap vps free alerting schedule host mediawiki jenkins grafana