The answer is pretty straightforward if you need a response. But what if you don't want to wait for a reply? Non-Waiting HTTP Consider the case of a Lambda function that after processing, accumulates some metrics and then emits those back to a REST aggregator service for processing. If the metrics are counters, it may not matter if the metrics request fails occasionally, as the next request will send the updated counter totals. In cases like these, it will be faster not to wait for the HTTP response. So how should we do this in NodeJS? Let's look first at a naive example request (without error handling) where we wait for the response. Just for this example, we'll create an async "request" node function that returns a promise that we can wait on. const https = require('https')
const URL = require('url')

async function request(url, data) {
    return new Promise((resolve, reject) => {
        let req = https.request(URL.parse(url), function (res) {
            let body = ''
            res.on('data', (chunk) => { body += chunk })
            res.on('end', () => { resolve(body) })
        })
        req.write(data)
        req.end()
    })
} Our Lambda would then look like: exports.handler = async (event, context) {
    ...
    let result = await request('https://metric-service.com', metrics)
    return result
} To invoke our HTTP request inside a Lambda, we use "await" which issues the request and then waits for the response. If the metric aggregator service takes 950 milliseconds to process the request and return a status code, we will be billed for an additional second on every invocation. During this wait time, our Lambda function is asleep, but AWS is still billing us for that time. With AWS Lambda, you are billed for elapsed time, not for utilized CPU time. While Lambda is extraordinarily cheap, with high transaction volumes, these short waits of 950 milliseconds can add up to a significant bill. Don't Wait So what happens if we simply do not call "await" and thus not wait on the response from our HTTP request? exports.handler = async (event, context) {
    /* nowait */ request('https://example.com', metrics)
    return 'done'
} Strange Things Happen Sometimes the request is sent and sometimes the request is not. Sometimes the request is received immediately by the metrics aggregator and sometimes the request is received after our Lambda next runs. What is happening? Freezing Lambda Containers Lambda functions run inside an AWS Firecracker container. When you return from your Lambda function, AWS immediately freezes the container and its global state. When the Lambda is next invoked, it will thaw the container to service the new invocation. If we send a HTTP request and that request is not fully sent over the network and we return from the Lambda function, AWS will immediately suspend our Lambda container AND the partially sent request will be suspended as well. The request will remain frozen until the Lambda is next invoked and the Node event loop will then resume processing and the request will be fully transmitted. How to Solve? We could try a short sleep to give time for the request to be sent? But how long should we wait. exports.handler = async (event, context) {
    /* nowait */ request('https://example.com', metrics)
    await sleep(100)
    return 'done'
} This hardly seems reliable. Wait for Request Transmission The correct solution is to use the Node API and wait until the request is fully sent, but not wait for the response to be received. req.end(,,callback) Here is a sample: const https = require('https')
const URL = require('url')

async function request(url, data) {
    return new Promise((resolve, reject) => {
        let req = https.request(URL.parse(url))
        req.write(data)
        req.end(null, null, () => {
            /* Request has been fully sent */
            resolve(req)
        })
    })
} Notice that the request is resolved via the end callback on the "req" object and not on the "res" object in the previous example. This modified request function should be invoked by our Lambda with "await". In this case, we are not waiting for the HTTP response, but rather for the request to be fully sent. This is much faster than the 750 milliseconds to receive our metrics response, typically under 20 milliseconds. exports.handler = async (event, context) {
    await request('https://example.com', metrics)
    return 'done'
} Alternative Design Patterns There are many other excellent ways to avoid waiting and blocking in Lambda functions such as using Step functions, SQS queues and directly invoking Lambda functions as events without waiting. Consider the best approach for your app, but if you must use HTTP and you don't need to wait for a response, consider the technique above to lower your wait time and AWS bill. SenseDeep Our SenseDeep Serverless Developer Studio uses this technique in a Watcher Lambda that monitors your Lambdas, runs alarms and ingests log data. We needed the Watcher to be exceptionally fast and not wait for any REST/HTTP API requests. So the Watcher uses this non-waiting technique when sending status back to the SenseDeep service. To learn more about the SenseDeep Serverless Developer Studio and try it for free, please go to: . https://www.sensedeep.com References Here are some other good reads about Lambda and AWS asynchronous programming. Node event loop freezing: https://levelup.gitconnected.com/avoiding-the-pitfalls-of-async-node-js-functions-in-aws-lambda-941220582e7a Lambda Billing: https://www.jeremydaly.com/serverless-tip-dont-overpay-when-waiting-on-remote-api-calls/ Async programming: https://read.acloud.guru/save-time-and-money-with-aws-lambda-using-asynchronous-programming-3548ea65f751 Perils of step functions: https://blog.scottlogic.com/2018/06/19/step-functions.html Invoke Lambda async: https://docs.aws.amazon.com/lambda/latest/dg/invocation-async.html Lambda invocation overview: https://docs.aws.amazon.com/lambda/latest/dg/lambda-invocation.html

Amazon

Get SenseDeep 

Nominated for 2022 - HackerNoon Contributor of the Year - Aws

Too Long; Didn't Read

Make resilience your competitive advantage

The Fastest Way to Invoke a HTTP/REST URL from an AWS Lambda

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

10 Things in Engineering We Don't Spend Enough Time On

254 Stories To Learn About Amazon

193 Stories To Learn About AWS Lambda

17 of the Best Amazon Web Services (AWS) for Web Developers to Learn

3 Min Hack for Locally building a Native Extension

10 Things in Engineering We Don't Spend Enough Time On

254 Stories To Learn About Amazon

193 Stories To Learn About AWS Lambda

17 of the Best Amazon Web Services (AWS) for Web Developers to Learn

3 Min Hack for Locally building a Native Extension

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps