I’ve recently settled on a clever use of npm pack to prep AWS Lambda Functions for deployment. I’d like to tell you how I’m using NPM and why it’s better than Webpack or Serverless Framework.
If you’re deploying JavaScript to AWS Lambda you’re likely aware there’s more than one way to do it. JavaScript and Serverless do not suffer from a lack of packaging tools. So far I’ve tried:
- Serverless
- Serverless + Webpack plugin + Babel
- SAM + Webpack
- SAM + NPM (lately)
Why I use SAM instead of Serverless
Serverless Framework is great if you’re new to the AWS ecosystem, especially CloudFormation. It also manages packaging and deployment of your JavaScript. After some time with Serverless I discovered a few things:
- I found myself guessing about the specific CloudFormation Serverless would generate
- I became concerned that I didn’t understand why it was generating what it was
- Many plugins jump outside of CloudFormation and use the AWS SDK. This can result in config that is not idempotent
- I found Serverless’ lifecycle hooks for plugins were not well documented
- I had to learn CloudFormation anyway so I could configure the resources I needed
- Over time I found most of my Serverless YAML was hand-written CloudFormation
- SAM abstracts only the most tedious bits of CloudFormation so I know what’s going on in there
In a nutshell, SAM offers me the right level of visibility and control.
Why I’m not using Webpack + Babel
This is simple: debugging. Yes, Webpack + Babel produce sourcemaps. Yes, Babel lets you use new language features. Yes, Webpack can produce lean packages. Here’s how things work for me in practice:
- Theres a whole class of bugs that cannot be understood in a reasonable amount of time, except by editing your code in the AWS console. If you’re redeploying between runs you’ll never get there
- The output of Webpack and Babel are not intended to be human readable. Good luck working in a tight feedback loop in the console
- Lambda supports Node 8.10 which has good enough coverage of the latest language features. The only “big” features 8.10 lacks are async generators and for-await-of. In my experience, a tiny fraction of Node developers understand how to use these features properly
Thus, I stick to the language features available in Node 8.10 and I don’t use Babel. For a list of Node 8.10's working features see Node 9.11.2 in Node.green .
Why I use NPM instead of Webpack
Why might I use Node Package Manager to package JavaScript to run on Node? The possibilities abound.
npm pack
NPM has a command called “pack” which builds a package of everything you want and none of what you don’t want. It’s much easier to use than Webpack.
By default npm pack grabs everything except:
- What’s in
.npmignore - What’s in
node_modules
To include dependencies from node_modules add them (explicitly) to the bundleDependencies key in package.json. No more Googling “webpack node externals aws-sdk”. NPM will trace the dependencies of the packages you deem necessary and add them to your package.
bundleDependencies in package.json
There’s also a handy flag -B . For example,npm install -B axios will add axios to bundleDependencies at install time.
Here’s what npm pack output looks like:
Output from 'npm pack'
TAR to ZIP
Here’s the wrinkle: npm pack produces a TAR and Lambda expects a ZIP. In addition, NPM places everything under package/. This is a simple fix in the NPM postpack script in package.json. The postpack script is called after pack. For more info on NPM pre/post scripts see this page.
"postpack": "tarball=$(npm list — depth 0 | sed ‘s/@/-/g; s/ .*/.tgz/g; 1q;’); tar -tf $tarball | sed ‘s/^package\\///’ | zip -[@r](http://twitter.com/r "Twitter profile for @r") package; rm $tarball"
Postpack script in package.json
Now we’ll get a package.zip each time we runnpm pack.
If you aren’t big on shell commands this one can look daunting. I promise it’s not that bad. Here’s an explanation of what it does.
First, we figure out the name of the TAR file, without running npm pack again (explain-shell). We can’t run npm pack in postpack (even though it gives us the file name) because it’ll cause a recursive loop.
tarball=$(npm list — depth 0 | sed ‘s/@/-/g; s/ .*/.tgz/g; 1q;’);
Next, we take everything in the tar file and put it in package.zip (explain-shell).
tar -tf $tarball | sed ‘s/^package\\///’ | zip -@r package;
Finally, we delete the tar file (explain-shell).
rm $tarball
You don’t have to leave the script as a long one-liner in package.json, feel free to pull it out into a bash file and make it more readable.
Deploy
Have a look at the sample repository I created for this article, particularly this commit.
git clone [https://github.com/ryanwmarsh/sam-with-npm](https://github.com/ryanwmarsh/sam-with-npm)
cd sam-with-npm/hello_world
npm install
npm pack
cd ..
aws s3 mb s3://sam-with-npm
sam package --template-file template.yaml --output-template-file packaged.yaml --s3-bucket sam-with-npm
sam deploy --template-file packaged.yaml --stack-name sam-with-npm --capabilities CAPABILITY_IAM
aws cloudformation describe-stacks --stack-name sam-with-npm --query 'Stacks[].Outputs'
Output of ‘describe-stacks’
Finally, check it with your endpoint
curl https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/Prod/hello/
Closing
I hope you found that this simplifies your tooling and workflow. If you have any questions about Serverless or DevOps I’m a freelance coach and I help teams reach maximum development output with minimal pain. You can always find me at http://thestack.io or [email protected]