Photo by on Markus Spiske Unsplash The Problem I have a serverless process that runs in a row — one Lambda runs and before it ends, it invokes another Lambda. Due to rate limits and other errors, I needed to know if the invocation was successful, but I didn’t want to wait until the new Lambda ended. This was challenging. The other annoying thing was that the number of running Lambdas was increasing out of control, even though I should have one Lambda alive at a time (maybe two, because I start a new Lambda and only then close the current one — an overlap of few milliseconds). The Solution There are two types of lambda invocations: Sync (“RequestResponse”) and Async (“Event”). Initially, I started with the option as I didn’t want to wait for the new Lambda to end. It turns out that adds this invocation request to a queue and you have no idea whether it was invoked successfully. Furthermore, if the invocation failed then AWS may re-run it again.. Async AWS My next attempt was with the Sync invocation. In order to know if the Lambda invocation succeeded, I needed to wait for the new Lambda to end. So I used a little trick: Invoke Sync Lambda Wait for a second After one second, of three things happened: I got a successful invocation from AWS within a second (meaning that the new lambda ran for less than a second — not likely to happen) I got an error from AWS A timeout occurred, so I assume that invocation was successful. The new Lambda is running, but I no longer wait for it. Important notes from my experience: If a timeout occurred, I assumed that invoke was successful and continued with my code at some point in time the new function will end and reply with a real success or failure response. You must prepare your code to handle it. You may get two flows instead of one — watch out! BUT Same scenario as the previous, but this time the first Lambda is over before the invoked one replies with an answer — AWS will consider the invocation as failed and re-run the Lambda! Lambda socket timeout: If Lambda A invokes a new Lambda B in a synchronous way ( ), it will wait for it to end according to the Lambda socket timeout — even if your code is over and you replied with 200 or 500 response! “RequestResponse” (lambda.Config.httpOptions = {timeout: 1500};) You can force Lambda to end and not wait for the socket timeout with: . = ;” — note that you will get an error message when the invoked lambda ends (B) and it will re-invoke in this case too. “context callbackWaitsForEmptyEventLoop false Conclusion I wanted to know whether Lambda invoked successfully, but I could not find a straightforward solution. You need to use a Sync invocation and “hack” the system. Be aware of additional invocations when AWS considers your Lambda as failed just because you didn’t wait for its response. Your Lambda function will be invoked multiple times — your code should be idempotent — code logic should know how to handle multiple invocation with the same input (event). Other solutions that not discussed in this post: Lambda Dead Letter Queue — where a failed lambda is inserted to a queue that needs to be managed, the retry invocation on error is still happening in this case. Use an external DB to manage your invocation. For example, increment a dynamo counter when Lambda starts and decrease it when it ends. Amos Shahar is Director of DevOps & IT at superQuery . Via our SQL IDE for Google BigQuery, superQuery uses AI and smart caching to enable data teams to run the most efficient and cost-effective queries on Google BigQuery.
