If you're like me, a sucker for event-driven programming, you'll want to continue reading. Today we'll take a look at triggering functions from messages. I've covered a few interesting topics regarding serverless architectures and AWS already, but nothing like this yet. Dig down, and get ready. Let's go. AWS Lambda AWS SNS TL;DR What are we building? What is AWS SNS? Build the API with the Serverless Framework Deploy the API to AWS Lambda Test the API with Dashbird *: All the if you want to check out the end result right away.* Note code from this tutorial is already on GitHub What are we building? Our focus will solely be on the steps to create the infrastructure components our app will need. The code itself will just mimic the behavior of a random complex computation. I've chosen a recursive function that calculates the factorial of the number passed to it. Here's a nice diagram, because diagrams are awesome of course! The function is the only exposed function, which is hooked up to . It takes a single parameter which it validates, upon success, it publishes an SNS topic and sends along the value. init API Gateway number number The SNS topic will trigger a second function called . This function will perform the calculation and log out the result to the console. This mimics a heavy computational background task such as data processing, image manipulation or machine learning calculations. calculate If the function fails, the Dead Letter Queue SNS topic will receive a message and trigger the function. calculate error Every function invoked will twice retry its execution upon failure. Using the Dead Letter Queue as a pool for your error logs is a smart use-case. asynchronously Now you're wondering, why all the complication with SNS instead of just invoking the second lambda function from the first one with ? Lambda's invoke API First of all, it's a huge anti-pattern for asynchronous workflows, which is our case. Otherwise, it's fine if you need the response from the second lambda function right away. Another issue is hitting Lambda's concurrency limits pretty fast. It can result in losing invocations and dropping data. Sending your data through a pub/sub service like SNS or a queue like will make sure you have data integrity. SQS Makes sense now? Sweet, let's talk a bit more about SNS. What is AWS SNS? Before we start coding we need to cover the basics. We know what AWS Lambda is, but what about SNS? The AWS docs are pretty straightforward. Amazon Simple Notification Service (SNS) is a flexible, fully managed and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients. pub/sub messaging -- AWS Docs In English, this means it's a way of sending notifications between services on a publisher/subscriber basis. One service publishes some data about a and sends it along its way. SNS will then funnel it to all the subscribers of that particular . The key focus is on here, you'll see why a bit further down. topic topic topic Build the API with the Serverless Framework The first thing to do, as always, is to set up the project and install dependencies. 1. Install the Serverless Framework My go-to development and deployment tool for serverless apps is the . Let's go ahead and install it. Serverless Framework $ npm i -g serverless Note: If you're using Linux, you may need to run the command as sudo. Once installed globally on your machine, the commands will be available to you from wherever in the terminal. But for it to communicate with your AWS account you need to configure an IAM User. Jump over , then come back and run the command below, with the provided keys. here for the explanation $ serverless config credentials\
    --provider aws\
    --key xxxxxxxxxxxxxx\
    --secret xxxxxxxxxxxxxx Now your Serverless installation knows what account to connect to when you run any terminal command. Let's jump in and see it in action. 2. Create a service Create a new directory to house your Serverless application services. Fire up a terminal in there. Now you're ready to create a new service. What's a service? It's like a project. It's where you define AWS Lambda functions, the events that trigger them and any AWS infrastructure resources they require, including SNS which we'll add today, all in a file called . serverless.yml Back in your terminal type: $ serverless create --template aws-nodejs\
    --path lambda-sns-dlq-error-handling The create command will create a new . What a surprise! We also pick a runtime for the function. This is called the . Passing in will set the runtime to Node.js. Just what we want. The will create a folder for the service. service template aws-nodejs path 3. Explore the service directory with a code editor Open up the folder with your favorite code editor. There should be three files in there, but for now, we'll only focus on the . It contains all the configuration settings for this service. Here you specify both general configuration settings and per function settings. Your will be full of boilerplate code and comments. Feel free to delete it all and paste this in. lambda-sns-dlq-error-handling serverless.yml serverless.yml service: lambda-sns-dlq-error-handling

plugins:
  - serverless-pseudo-parameters

provider:
  name: aws
  runtime: nodejs8.10
  stage: dev
  region: eu-central-1
  memorySize: 128
  environment:
    accountId: '#{AWS::AccountId}'
    region: '#{AWS::Region}'
  iamRoleStatements:
    - Effect: "Allow"
      Resource: "*"
      Action:
        - "sns:*"

functions:
  init:
    handler: init.handler
    events:
      - http:
          path: init
          method: post
          cors: true
  calculate:
    handler: calculate.handler
    events:
      - sns: calculate-topic # created immediately
    onError: arn:aws:sns:#{AWS::Region}:#{AWS::AccountId}:dlq-topic
  error:
    handler: error.handler
    events:
      - sns: dlq-topic # created immediately Let's break down what's going one here. Check out the section. There are three functions here. From top to bottom they're , , and . The function will get triggered by a simple HTTP request, which we invoke through API Gateway. Familiar territory for us. functions init calculate error init However, the and functions are triggered by SNS topics. Meaning we will have logic in the function that will messages to a topic named while the function is to the same topic. calculate error init publish calculate-topic calculate subscribed Moving on, the function is subscribed to the while the function will publish messages to this topic if it fails, as you can see with the property. Now stuff makes sense, right? error dlq-topic calculate onError Make a mental note to yourself, once you add the SNS topics as events for your functions, the resources will automatically be created once you deploy the service. What else, take a look at the , they specify that our functions have permission to trigger, and get invoked by SNS topics. While the plugin lets us reference our and with the CloudFormation syntax, making it much easier to keep our SNS ARNs consistent across all resources. iamRoleStatements serverless-pseudo-parameters AccountId Region 4. Install dependencies Luckily, this part will be short. Just one package to install. First, initialize npm and then you can install . serverless-pseudo-parameters $ npm init -y && npm i serverless-pseudo-parameters That'll do. 5. Write business logic With all things considered, the configuration process was rather simple. The code we'll write now is just as straightforward. Nothing extraordinary to see, I'm sorry to disappoint. Let's keep all three functions in separate files, to keep it simple. First of all create an file and paste this snippet in. init.js // init.js
const aws = require('aws-sdk')
const sns = new aws.SNS({ region: 'eu-central-1' })

function generateResponse (code, payload) {
  console.log(payload)
  return {
    statusCode: code,
    body: JSON.stringify(payload)
  }
}
function generateError (code, err) {
  console.error(err)
  return generateResponse(code, {
    message: err.message
  })
}
async function publishSnsTopic (data) {
  const params = {
    Message: JSON.stringify(data),
    TopicArn: `arn:aws:sns:${process.env.region}:${process.env.accountId}:calculate-topic`
  }
  return sns.publish(params).promise()
}

module.exports.handler = async (event) => {
  const data = JSON.parse(event.body)
  if (typeof data.number !== 'number') {
    return generateError(400, new Error('Invalid number.'))
  }

  try {
    const metadata = await publishSnsTopic(data)
    return generateResponse(200, {
      message: 'Successfully added the calculation.',
      data: metadata
    })
  } catch (err) {
    return generateError(500, new Error('Couldn\'t add the calculation due to an internal error.'))
  }
} We have a few helper functions and the exported lambda function at the bottom. What's going on here? The lambda validates input and publishes some data to the SNS topic. That's everything this function is doing. The SNS topic will trigger the lambda function. Let's add that now. calculate-topic calculate-topic calculate Create a file and name it . Paste this snippet in. calculate.js // calculate.js
module.exports.handler = async (event) => {
  const { number } = JSON.parse(event.Records[0].Sns.Message)
  const factorial = (x) => x === 0 ? 1 : x * factorial(x - 1)
  const result = factorial(number)

  console.log(`The factorial of ${number} is ${result}.`)
  return result
} As you see this is just a simple factorial calculation implemented with a recursive function. It'll calculate the factorial of the number we published to the SNS topic from the function. init An important note here is that if the function fails a total of three times, it'll publish messages to the Dead Letter Queue SNS topic we specified with the property in the file. The Dead Letter Queue will then trigger the function. Let's create it now so it can log out errors to CloudWatch. Create an file, and paste these lines in. calculate onError serverless.yml error error.js // error.js
module.exports.handler = async (event) => {
  console.error(event)
} For now, this will do. However, ideally, you would have structured logging with detailed info about everything going on. That's a topic for another article. Deploy the API to AWS Lambda Here comes the easy part. Deploying the API is as simple as running one command. $ serverless deploy\ ![lambda deploy](https://cdn.hackernoon.com/images/cky-9-uq-08500-u-80-bs-6-b-31-s-7-jsp.jpg) You can see the endpoint get logged to the console. That's where you will be sending your requests. Test the API with Dashbird The simplest way of testing an API is with CURL. Let's create a simple CURL command and send a JSON payload to our endpoint. $ curl -H "Content-Type: application/json"\
  -d '{"number":1000}'\
  https://<id>.execute-api.eu-central-1.amazonaws.com/dev/init If everything works as it should, the result of the calculation will be logged to CloudWatch. If not, well then you're out of luck. After hitting the endpoint a few times with a couple of different values here's the result. The function works as expected. init But, what really interests us is the function. Here's what that looks like when it's successful. calculate When it fails it'll specify a crash and show the error logs. After two retries it'll send a message to the Dead Letter Queue and trigger the function. error Sweet! We've tested all the different scenarios. Hope this clears things up a bit. Wrapping up That's all folks. We've covered creating SNS triggers for Lambda while also implementing a Dead Letter Queue to catch errors from failed invocations. Using serverless for various intermittent calculations is a valid use case that will only grow in popularity in the future. There are no servers you need to worry about, and you only pay for the time it runs. Just deploy the code and rest assured it'll work. Again, here's the , if you want to take a look at the code. It can act as a starter for your own use-cases where you need SNS messages triggering Lambda functions. Give it a star if you like it and want more people to see it on GitHub. GitHub repo Also published on: https://dashbird.io/blog/triggering-lambda-with-sns-messaging/

How to Trigger Lambda with SNS Messaging: Going Serverless with AWS

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

10 Things in Engineering We Don't Spend Enough Time On

10 Lessons from 10 Years of AWS (part 2)

10 Lessons from 10 Years of AWS (part 1)

Top 10 AI Development Companies in USA

12 Strategies to Reduce Amazon S3 Costs

17 of the Best Amazon Web Services (AWS) for Web Developers to Learn

10 Things in Engineering We Don't Spend Enough Time On

10 Lessons from 10 Years of AWS (part 2)

10 Lessons from 10 Years of AWS (part 1)

Top 10 AI Development Companies in USA

12 Strategies to Reduce Amazon S3 Costs

17 of the Best Amazon Web Services (AWS) for Web Developers to Learn

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps