Zeet is sponsoring this blogpost for the next month. I tried it out the other day. It's like serverless but for running entire back ends. You can host and scale apps automagically. Pretty neat. If you’re like me, a sucker for event-driven programming, you’ll want to continue reading. Today we’ll take a look at triggering functions from messages. I’ve covered a few interesting topics regarding serverless architectures and AWS already, but nothing like this yet. Dig down, and get ready. Let’s go. AWS Lambda AWS SNS TL;DR What are we building? What is AWS SNS? Build the API with the Serverless Framework Deploy the API to AWS Lambda Test the API with Dashbird Note : All the code from this tutorial is already on GitHub if you want to check out the end result right away. What are we building? Our focus will solely be on the steps to create the infrastructure components our app will need. The code itself will just mimic the behavior of a random complex computation. I’ve chosen a recursive function that calculates the factorial of the number passed to it. Here’s a nice diagram, because diagrams are awesome of course! The function is the only exposed function, which is hooked up to . It takes a single parameter which it validates, upon success, it publishes an SNS topic and sends along the value. init API Gateway number number The SNS topic will trigger a second function called . This function will perform the calculation and log out the result to the console. This mimics a heavy computational background task such as data processing, image manipulation or machine learning calculations. calculate If the function fails, the Dead Letter Queue SNS topic will receive a message and trigger the function. calculate error Every function invoked will twice retry its execution upon failure. Using the Dead Letter Queue as a pool for your error logs is a smart use-case. asynchronously Now you’re wondering, why all the complication with SNS instead of just invoking the second lambda function from the first one with ? Lambda’s invoke API First of all, it’s a huge anti-pattern for asynchronous workflows, which is our case. Otherwise, it’s fine if you need the response from the second lambda function right away. Another issue is hitting Lambda’s concurrency limits pretty fast. It can result in losing invocations and dropping data. Sending your data through a pub/sub service like SNS or a queue like will make sure you have data integrity. SQS Makes sense now? Sweet, let’s talk a bit more about SNS. What is AWS SNS? Before we start coding we need to cover the basics. We know what AWS Lambda is, but what about SNS? The AWS docs are pretty straightforward. Amazon Simple Notification Service (SNS) is a flexible, fully managed pub/sub messaging and mobile notifications service for coordinating the delivery of messages to subscribing endpoints and clients. - AWS Docs In English, this means it’s a way of sending notifications between services on a publisher/subscriber basis. One service publishes some data about a and sends it along its way. SNS will then funnel it to all the subscribers of that particular . Key focus is on here, you’ll see why a bit further down. topic topic topic Build the API with the Serverless Framework The first thing to do, as always, is to set up the project and install dependencies. 1. Install the Serverless Framework My go-to development and deployment tool for serverless apps is the . Let’s go ahead and install it. Serverless Framework $ npm i -g serverless Note: If you’re using Linux, you may need to run the command as sudo. Once installed globally on your machine, the commands will be available to you from wherever in the terminal. But for it to communicate with your AWS account you need to configure an IAM User. Jump over , then come back and run the command below, with the provided keys. here for the explanation $ serverless config credentials \ 
    --provider aws \ 
    --key xxxxxxxxxxxxxx \ 
    --secret xxxxxxxxxxxxxx Now your Serverless installation knows what account to connect to when you run any terminal command. Let’s jump in and see it in action. 2. Create a service Create a new directory to house your Serverless application services. Fire up a terminal in there. Now you’re ready to create a new service. What’s a service? It’s like a project. It’s where you define AWS Lambda functions, the events that trigger them and any AWS infrastructure resources they require, including SNS which we’ll add today, all in a file called . serverless.yml Back in your terminal type: $ serverless create \
    --template aws-nodejs \
    --path lambda-sns-dlq-error-handling The create command will create a new . What a surprise! We also pick a runtime for the function. This is called the . Passing in will set the runtime to Node.js. Just what we want. The will create a folder for the service. service template aws-nodejs path 3. Explore the service directory with a code editor Open up the folder with your favorite code editor. There should be three files in there, but for now, we’ll only focus on the . It contains all the configuration settings for this service. Here you specify both general configuration settings and per function settings. Your will be full of boilerplate code and comments. Feel free to delete it all and paste this in. lambda-sns-dlq-error-handling serverless.yml serverless.yml service: lambda-sns-dlq-error-handling

plugins:
  - serverless-pseudo-parameters

provider:
  name: aws
  runtime: nodejs8 stage: dev
  region: eu-central memorySize: environment:
    accountId: region: iamRoleStatements:
    - Effect: Resource: Action:
        - functions:
  init:
    handler: init.handler
    events:
      - http:
          path: init
          method: post
          cors: calculate:
    handler: calculate.handler
    events:
      - sns: calculate-topic # created immediately
    onError: arn:aws:sns:#{ ::Region}:#{ ::AccountId}:dlq-topic
  error:
    handler: error.handler
    events:
      - sns: dlq-topic # created immediately .10 -1 128 '#{AWS::AccountId}' '#{AWS::Region}' "Allow" "*" "sns:*" true AWS AWS Let’s break down what’s going one here. Check out the section. There are three functions here. From top to bottom they're , , and . The init function will get triggered by a simple HTTP request, which we invoke through API Gateway. Familiar territory for us. functions init calculate error However, the and functions are triggered by SNS topics. Meaning we will have logic in the function that will messages to a topic named while the function is to the same topic. calculate error init publish calculate-topic calculate subscribed Moving on, the function is subscribed to the while the function will publish messages to this topic if it fails, as you can see with the property. Now stuff makes sense, right? error dlq-topic calculate onError Make a mental note to yourself, once you add the SNS topics as events for your functions, the resources will automatically be created once you deploy the service. What else, take a look at the , they specify that our functions have permission to trigger, and get invoked by SNS topics. While the plugin lets us reference our and with the CloudFormation syntax, making it much easier to keep our SNS ARNs consistent across all resources. iamRoleStatements serverless-pseudo-parameters AccountId Region 4. Install dependencies Luckily, this part will be short. Just one package to install. First, initialize npm and then you can install . serverless-pseudo-parameters $ npm init -y && npm i serverless-pseudo-parameters That’ll do. 5. Write business logic With all things considered, the configuration process was rather simple. The code we’ll write now is just as straightforward. Nothing extraordinary to see, I’m sorry to disappoint. Let’s keep all three functions in separate files, to keep it simple. First of all create an file and paste this snippet in. init.js aws = ( ) sns = aws.SNS({ : }) { .log(payload) { : code, : .stringify(payload)
  }
} { .error(err) generateResponse(code, { : err.message
  })
} { params = { : .stringify(data), : } sns.publish(params).promise()
} .exports.handler = (event) => { data = .parse(event.body) ( data.number !== ) { generateError( , ( ))
  } { metadata = publishSnsTopic(data) generateResponse( , { : , : metadata
    })
  } (err) { generateError( , ( ))
  }
} // init.js const require 'aws-sdk' const new region 'eu-central-1' ( ) function generateResponse code, payload console return statusCode body JSON ( ) function generateError code, err console return message async ( ) function publishSnsTopic data const Message JSON TopicArn `arn:aws:sns: : :calculate-topic` ${process.env.region} ${process.env.accountId} return module async const JSON if typeof 'number' return 400 new Error 'Invalid number.' try const await return 200 message 'Successfully added the calculation.' data catch return 500 new Error 'Couldn\'t add the calculation due to an internal error.' We have a few helper functions and the exported lambda function at the bottom. What’s going on here? The lambda validates input and publishes some data to the SNS topic. That's everything this function is doing. The SNS topic will trigger the lambda function. Let's add that now. calculate-topic calculate-topic calculate Create a file and name it . Paste this snippet in. calculate.js .exports.handler = (event) => { { number } = .parse(event.Records[ ].Sns.Message) factorial = x === ? : x * factorial(x - ) result = factorial(number) .log( ) result
} // calculate.js module async const JSON 0 const ( ) => x 0 1 1 const console `The factorial of is .` ${number} ${result} return As you see this is just a simple factorial calculation implemented with a recursive function. It’ll calculate the factorial of the number we published to the SNS topic from the function. init An important note here is that if the function fails a total of three times, it'll publish messages to the Dead Letter Queue SNS topic we specified with the property in the file. The Dead Letter Queue will then trigger the function. Let's create it now so it can log out errors to CloudWatch. Create an file, and paste these lines in. calculate onError serverless.yml error error.js .exports.handler = (event) => { .error(event)
} // error.js module async console For now, this will do. However, ideally, you would have structured logging with detailed info about everything going on. That’s a topic for another article. Deploy the API to AWS Lambda Here comes the easy part. Deploying the API is as simple as running one command. $ serverless deploy You can see the endpoint get logged to the console. That’s where you will be sending your requests. Test the API with Dashbird The simplest way of testing an API is with CURL. Let’s create a simple CURL command and send a JSON payload to our endpoint. $ curl -H \
    -d \
    https: "Content-Type: application/json" '{"number":1000}' //<id>.execute-api.eu-central-1.amazonaws.com/dev/init If everything works like it should, the result of the calculation will be logged to CloudWatch. If not, well then you’re out of luck. In cases like these, I default to using to debug what’s going on. It’s to set up. Dashbird free and doesn’t require a credit card After hitting the endpoint a few times with a couple of different values here’s the result. The function works as expected. init But, what really interests us is the function. Here's what that looks like when it's successful. calculate When it fails it’ll specify a crash and show the error logs. After two retries it’ll send a message to the Dead Letter Queue and trigger the error function. Sweet! We’ve tested all the different scenarios. Hope this clears things up a bit. Wrapping up That’s all folks. We’ve covered creating SNS triggers for Lambda while also implementing a Dead Letter Queue to catch errors from failed invocations. Using serverless for various intermittent calculations is a valid use-case that will only grow in popularity in the future. There are no servers you need to worry about, and you only pay for the time it runs. Just deploy the code and rest assured it’ll work. If something breaks, you have watching your back, alerting you in Slack or E-mail if something is wrong. You just have to love Slack integration! Dashbird Again, here’s the , if you want to take a look at the code. It can act as a starter for your own use-cases where you need SNS messages triggering lambda functions. Give it a star if you like it and want more people to see it on GitHub. GitHub repo If you want to read some of my previous serverless musings head over to or my profile join my newsletter! Or, take a look at a few of my articles right away: A crash course on serverless-side rendering with Vue.js, Nuxt.js and AWS Lambda Building a serverless contact form with AWS Lambda and AWS SES A crash course on Serverless APIs with Express and MongoDB Solving invisible scaling issues with Serverless and MongoDB How to deploy a Node.js application to AWS Lambda using Serverless Getting started with AWS Lambda and Node.js A crash course on securing Serverless APIs with JSON web tokens Migrating your Node.js REST API to Serverless Building a Serverless REST API with Node.js and MongoDB A crash course on Serverless with Node.js Hope you guys and girls enjoyed reading this as much as I enjoyed writing it. If you liked it, slap that tiny clap so more people here on Medium will see this tutorial. Until next time, be curious and have fun. Originally published at dev.to .

Amazon

Slack

Containers vs. Serverless from a DevOps Standpoint

A crash course on Serverless with AWS — Triggering Lambda with SNS Messaging

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A crash course on Serverless APIs with Express and MongoDB

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 1)

A crash course on Serverless APIs with Express and MongoDB

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 1)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps