context. getRemainingTimeInMillis() gives you a flexible way to timeout requests on the client-side based on the amount of invocation time left rather than some arbitrarily hardcoded value. With API Gateway and Lambda, you’re forced to use relatively short timeouts on the server-side: on all integration points API Gateway have a 30s max timeout Serverless framework uses a default of 6s for AWS Lambda functions However, as you have limited over a Lambda function’s cold start time and have no control over the amount of latency overhead API Gateway introduces, the actual client-facing latency you’d experience from a calling function is far less predictable. influence https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/api-gateway-metrics-dimensions.html To prevent slow HTTP responses from causing the calling function to timeout (and therefore impact the user experience we offer) we should make sure we stop waiting for a response before the calling function times out. “the goal of the timeout strategy is to give HTTP requests the , provided that doing so does not cause the calling function itself to err” — me best chance to succeed Most of the time, I see folks use fixed (either hard coded or specified via config) timeout values, which is often tricky to decide: too short and you won’t give the request the , e.g. there’s 5s left in the invocation but we had set timeout to 3s best chance to succeed too long and you run the risk of letting the request timeout the calling function, e.g. there’s 5s left in the invocation but we had set timeout to 6s This challenge of choosing the right timeout value is further complicated by the fact that we often perform more than one HTTP request during a function invocation — e.g. read from DynamoDB, talk to some internal API, then save changes to DynamoDB equals a total of 3 HTTP requests in one invocation. Let’s look at two common approaches for picking timeout values and scenarios where they fall short of meeting our goal. requests are not given the best chance to succeed requests are allowed too much time to execute and caused the function to timeout. Instead, we should , whilst taking into account the time required to perform any recovery steps — e.g. return a meaningful error with application specific error code in the response body, or return a fallback result instead. set the request timeout based on the amount of invocation time left You can easily find out how much time is left in the current invocation through the object your function is invoked with. context https://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html For example, if a function’s is 6s, but by the time you make the HTTP request you’re already 1s into the invocation (perhaps you had to do some expensive computation first), and if we reserve 500ms for recovery, then that leaves us with 4.5s to wait for HTTP response. timeout With this approach, we get the best of both worlds: allow requests the best chance to succeed based on the actual amount of invocation time we have left; and prevent slow responses from timing out the function, which allows us a window of opportunity to perform recovery actions requests are given the best chance to succeed, without being restricted by an arbitrarily determined timeout. slow responses are timed out before they cause the calling function to time out But what are you going to do AFTER you time out these requests? Aren’t you still going to have to respond with a HTTP error since you couldn’t finish whatever operations you needed to perform? At the minimum, the recovery actions should include: log the timeout incident with as much context as possible (e.g. how much time the request had), including all the relevant correlation IDs track custom metrics for so it can be monitored and the team can be alerted if the situation escalates serviceX.timedout return an application error code in the response body (see example below), along with the request ID so the user-facing client app can display a user friendly message like “Oops, looks like this feature is currently unavailable, please try again later. If this is urgent, please contact us at xxx@domain.com and quote the request ID f19a7dca. Thank you for your cooperation :-)” {"errorCode": 10021,"requestId": "f19a7dca","message": "service X timed out"} In , you can also recover even more gracefully using fallbacks. some cases Netflix’s library, for instance, supports several flavours of fallbacks via the pattern it employs so heavily. In fact, if you haven’t read its already then I strongly recommend that you go and give it a thorough read, there are tons of useful information and ideas there. Hystrix Command wiki page At the very least, every command lets you specify a fallback action. You can also chain the fallback together by chaining commands via their respective methods. getFallback For example, execute a DynamoDB read inside CommandA in the method, execute which would return a previously cached response if available getFallback CommandB if there is no cached response then would fail, and trigger its own method CommandB getFallback execute , which returns a stubbed response CommandC Anyway, check out if you haven’t already, most of the patterns that are baked into can be easily adopted in our serverless applications to help make them more resilience to failures — something that I’m actively exploring with a separate series on . Hystrix Hystrix applying principles of chaos engineering to Serverless As an aside, as Danilo mentioned in the comments, you can also use to decide when to do more work vs recurse when writing recursive functions. You can read more about that as well as other tips for writing recursive Lambda functions in this . context.getRemainingTimeInMillis() post Hi, my name is . I’m an and the author of . I have run production workload at scale in AWS for nearly 10 years and I have been an architect or principal engineer with a variety of industries ranging from banking, e-commerce, sports streaming to mobile gaming. I currently work as an independent consultant focused on AWS and serverless. Yan Cui AWS Serverless Hero Production-Ready Serverless You can contact me via , and . Email Twitter LinkedIn Check out my new course, . Complete Guide to AWS Step Functions In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. Including basic concepts, HTTP and event triggers, activities, design patterns and best practices. Get your copy . here Come learn about operational for AWS Lambda: CI/CD, testing & debugging functions locally, logging, monitoring, distributed tracing, canary deployments, config management, authentication & authorization, VPC, security, error handling, and more. BEST PRACTICES You can also get off the face price with the code . 40% ytcui Get your copy . here

Chain

Amazon

Netflix

Twitter

Ask me for help about serverless

Read My Stories

Too Long; Didn't Read

Developers: Build Less. Deliver More. [Learn how]	

AWS Lambda — use the invocation context to better handle slow HTTP responses

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Coin Mentioned

Yan Cui

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

AWS Lambda — use the invocation context to better handle slow HTTP responses

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Coin Mentioned

Yan Cui

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES