This article highlights the steps we took to reduce the size of our NodeJs apps running on AWS Lambda but it still relates to any Node app running anywhere. In the end, you'll find out how we dropped our 50MB - 75MB (compressed) node apps to 8MB - 10MB. Awesome, right? But first, How did we get here? How did this set of software developers exhaust the 75GB AWS Lambda storage limit?
Oh yes, we did. 🙈 The Mistake Deploying microservices on AWS lambda could mean you have to work with other AWS services like SQS, SNS, API Gateway, etc., and using the , the default practice will be to define handlers for events coming from each of these services. Serverless Framework Under the hood, the Serverless framework creates new lambda functions for each of the handlers you define. Let's say you want to handle events from SNS, SQS, S3, and API Gateway, four Lambda functions will be created for each of those events using the same code base. This means that our 50MB app when deployed, becomes a huge 200MB app in total. Here's the interesting part, this was only on our staging environment.
Considering the number of microservices we had running, we were already at 50% usage, pushing our microservices to a new environment immediately doubled our storage usage and our deployments broke. Wahala 🙆🏽‍♂️ The Fix: How We Reduced Our AWS Lambda Size 1. Set AWS-SDK as Dev Dependency This is the mother of all. I won't even be bothered if you quit reading after this point. The package alone is over 60MB (uncompressed). This is huge! aws-sdk This was almost everything about our app size issue, our misfortune and also our miracle. The good news is that the comes pre-installed in your Lambda runtime, so you don't need to install it again. Only set it as a dev dependency. aws-sdk If we only we knew this. I'm in severe pain now 😭 2. Remove Unnecessary Packages There is a good chance you've done this already. However, for large projects mostly, some unused packages can go undetected easily. Try packages like . It helps you scan your app for unused and missing dependencies. depcheck npm install -g depcheck The flag is important, please. Let's not complicate things here. -g 3. Cross-Check for Dev Packages Just like the , there are other packages that could disguise as production dependencies such as and its plugins like . aws-sdk serverless serverless-offline These packages, in combination with the extra packages it installs for you, is over 100MB (uncompressed) in size, that is including . Hence, removing without removing these guys won't make any difference. It will be re-installed. aws-sdk aws-sdk Since these types of packages are used for deployment and need to be installed while deployment is ongoing, one can easily mistake them as production packages.
The best approach will be to set them as dev dependencies in your app and install them globally in your CI/CD Docker image. This approach also buys you more deployment time as you won't need to reinstall them every time your CI/CD pipeline runs. Created this Docker image for this purpose, feel free to reuse. It comes with a plugin and other necessary packages you need to deploy to lambda successfully. avonnadozie/serverless serverless-offline 4. Use Flag --production This should be a common step as well, however, It doesn't hurt to repeat it. Always remember to use the when running npm on production or in your CI/CD script to not install dev dependencies. --production npm install --production 5. Reuse Available Lambda Runtime Packages Just like , there are other packages such as and that are already available in the lambda runtime which you can reuse. aws-sdk uuid dotenv You can refer to this . list of node packages in lambda runtime 6. Inspect Deployment The crude way works well too. Download your live code and inspect it manually. To download, go to your Lambda console, click on the function and choose "Export function" from the "Actions" dropdown. Then click on "Download deployment package" afterwards. Once the download is complete, go to the folder, rank the folders by size and take a look. It reveals a lot. node_modules This was how we found out was still installed even after we set it as a dev dependency. aws-sdk 7. Move Out Large Non-js Files Consider hosting large files such as images, or JSON on a private CDN (most likely S3) and read it from there. This will cause a trade-off in speed but it could be worth it for you depending on the design of your app. 8. Merge Serverless Handlers This is completely up to you and your app architecture, but you can consider reusing lambda functions where necessary to prevent serverless from creating additional and unnecessary functions for you. For us, we found a way to merge SQS and SNS handlers given that their event input data and the way they work is similar and it saved us a whole lot of MBs. The lambda console provides you with sample event input data on its test tab you can work with. Choose from the list. Benefits to Reducing Your Application Size You might not be deploying multiple apps that will exhaust your AWS limit as we did, but there's still more you stand to gain by dropping your app size regardless. Faster time cold start Faster deployment with CI/CD as the zipping and upload process will be faster Less cost on storage and memory Also published . on Hashnode

Amazon

Avoiding ‘Too Many Connections’ in Lambda + RDS Workflows

How to Survive Testing on NodeJs and Jest 🤒

Portfolio

2021 - HackerNoon Contributor of the Year - NODEJS

2021 - HackerNoon Contributor of the Year - TESTING

2021 - HackerNoon Contributor of the Year - CODING

Nominated for 2022 - HackerNoon Contributor of the Year - Coding Skills

Nominated for 2022 - HackerNoon Contributor of the Year - Aws

Nominated for 2022 - HackerNoon Contributor of the Year - Amazon

Nominated for 2022 - No No No Nodejs

Too Long; Didn't Read

Using AWS Lambda to Reduce NodeJS App Size

Using AWS Lambda to Reduce NodeJS App Size

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Avoiding ‘Too Many Connections’ in Lambda + RDS Workflows

The Noonification: How Amazon Treats Warehouse Workers Who Contracted COVID (11/30/2022)

10 Free Ways to Promote Your Amazon Products

10 Failed Startup Product Examples by Google, Microsoft and Amazon

10 Best Infographics Of 2018

The Noonification: The Destroyer (12/29/2022)

Avoiding ‘Too Many Connections’ in Lambda + RDS Workflows

The Noonification: How Amazon Treats Warehouse Workers Who Contracted COVID (11/30/2022)

10 Free Ways to Promote Your Amazon Products

10 Failed Startup Product Examples by Google, Microsoft and Amazon

10 Best Infographics Of 2018

The Noonification: The Destroyer (12/29/2022)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps