When we specify the memory size for a Lambda function, AWS will allocate CPU proportionally. For example, a 256 MB function will receive twice the processing power of a 128 MB function. That looks simple and straightforward, but…
I had this question: would there be an ideal memory size that minimizes the cost of running a given task on Lambda?
In order to answer that, I tested the same task running on multiple memory sizes to check whether such cost/memory trade-off sweet spot exists.
Benchmark Lambdas
I created two Lambda functions to run this test:
- Fibonacci: basic code that generates a sequence of… you guessed it, Fibonacci numbers! It’s just a low-memory, CPU-intensive task.
- Benchmarker: invokes the Fibonacci function (or any other function) multiple times, switching memory sizes; in the end, it averages out the results to determine which memory size optimizes speed and cost.
The code is open sourced, in case you’d like to test your own Lambdas. The results presented below will certainly vary according to the function you test, so I encourage you to download the Benchmarker Lambda and run it for yourself.
Photo by Stephen Dawson on Unsplash
Test Parameters
- AWS region: us-east-1 (N. Virginia)
- Memory sizes tested: 128, 256, 512, 768, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2752, 3008
- Fibonacci was invoked 20 times for each memory size
- Invocations ran in batches of 10 concurrent requests to speed up the process
- On each invocation, the Fibonacci function built a sequence of the first 30 Fibonacci numbers
- Cold starts were ignored to standardize duration results
Test Results
Average cost (USD) per million executions of the Fibonacci sequence builder (n=30)
The sweetest spots in terms of cost were:
- 768 MB: that’s the cheapest we can get for this task on Lambda; why 128 MB isn’t cheaper? It takes longer to process, long enough to make it more expensive in total!
- 2048 MB: although the price is ~3% higher than 768 MB, it runs 2.5x faster; in some cases, it might be worth spending the extra pennies to speed up the processing.
It’s counter-intuitive that a task running with 768 MB can cost less in comparison to 128 MB, for example. It means we can actually lower our AWS bill by increasing memory size in some cases. Of course, we need to know what is the minimum memory our function requires when considering changing our settings. There are monitoring services — like Dashbird — that make it easier to profile Lambda memory usage and identify thresholds for this kind of benchmarking analysis.
The sharp rise in the line slope (chart above) for higher memory sizes caught my attention. From that point on, it has been reported — although not officially — that Lambda provides two cores. My hypothesis is that the processing power is split among cores and, since my job was using only one core, the test was actually punishing the dual-core function setting. That’s something to look more closely in a future test, with a task that can take advantage of multiple cores.
Average duration (milliseconds) for running the Fibonacci sequence builder (n=30)
In terms of duration, the chart above seems to have no surprises, but I actually found something consistently weird in the results: 2048 MB always performs faster than 2304 and 2560 MB, which is unexpected. Zooming in to the highest memory sizes we can notice the difference.
Average duration (milliseconds) for running the Fibonacci sequence builder (n=30)
It might be negligible since it represents roughly 2% in extra execution time. Nonetheless, if we’re running this function millions of times or if latency is super important, those extra milliseconds can be relevant.
Understanding exactly which factors are playing a role in producing these unexpected results is hard. Lambda infrastructure is sort of a black box. Maybe there are differences in hardware serving each request, which would introduce some undesirable variability in our tests. Bottom line is: if you want to optimize your Lambda usage for either the fastest execution or lowest cost, you should definitely benchmark your functions.
I’ve released the benchmarking function so that you can deploy and test your own Lambda functions for yourself. Just follow the instructions in the repository README file.