Build a cache layer for secrets stored in AWS Secrets manager using AWS Lambda extension
A month back AWS announced a preview of Lambda Extensions, a new way to easily integrate Lambda with your favorite monitoring, observability, security, and governance tools. Extensions can be published as Lambda layers, there are two types are extension:
The whole idea of extensions was fascinating to me. So I thought, instead of just using the extensions for just implementing observability patterns, why can't we build a cache layer for secrets using extensions.
As always, I googled my idea to see anyone has already implemented this. A famous company named “Square” has many smarter engineers than me, and they have already published a Lambda extension with a similar idea in GitHub 😟 . To be clear, they did a fantastic job, but like always, there is room for improvement, isn’t it right?
The extension they published was storing the secret in “/tmp” directory of the Lambda, a BIG NO NO from a security perspective, and no cache refresh logic. So I decided to let me build an extension that can take care of all the below use cases:
Here is the high level view of all the components
This extension demonstrates how to host an HTTP server which cache secrets required for the lambda function. The extension does the following:
Note: This extension requires the Node.js 12 runtime to be present in the Lambda execution environment of your function.
Below sequence diagram explains the initialization of lambda extension and how lambda function reads cached secrets using HTTP server hosted inside the extension
Checkout the code from the GitHub repoNodeJS runtime should be installed in the local system, for more information click here
Note: This command assumes you’ve placed your secret, such as this example JSON text structure {“username”:”anika”,”password”:”aDM4N3*!8TT”}, in a file named mycreds.json.
aws secretsmanager create-secret --name secret_now --secret-string file://mycreds.json
Create a new NodeJS runtime Lambda function with the name “Secrets-Extension-Lambda-Test”
Note: Sample of the below code can be found in “example-function/index.js” under the code repo
Note: The code invokes the local HTTP server hosted inside lambda extension to read the secret’s value instead of directly going to AWS Secret’s manager
Note: Sample of the below yaml file can be found in “example-function/config.yaml” under the code repo. “secret_now” is the name of the secret we have created in the previous step, and we want to be cached by the extension. If you want more secrets to be cached you can keep adding them here
SecretManagers:
1. secrets:
Note: If the environment value not found, then cache gets refreshed every 10 minutes
> chmod +x deploy.sh
> ./deploy.sh
You can invoke the Lambda function using the following CLI command
aws lambda invoke \
--function-name "Secrets-Extension-Lambda-Test" \
--payload '{"payload": "hello"}' /tmp/invoke-result \
--cli-binary-format raw-in-base64-out \
--log-type Tail
The function should return "StatusCode": 200.
Browse to the Amazon CloudWatch Console. Navigate to Logs\Log Groups. Select the log group /aws/lambda/Secrets-Extension-Lambda-Test.
View the log stream to see the runtime log with pattern "Response from cache" followed by username and password stored in mycred.json
If you like the write-up and found it helpful, give a clap 👏 or leave a comment in the article. Stay home and stay safe 😷