Building an image processor on AWS Lambda Meeting at AWS re:Invent Jeff Barr I started working as a Software Developer Full-time in 2012. You build it, you run it was a core value of our small team. But, I knew nothing about servers. I soon had to learn about subnets, load balancers, database clusters and more to support the code I wrote. For me, building software that solves problems is fun, managing servers isn’t. In early 2016 I attended an AWS meetup. At this meetup, spoke about how he built with a . It blew me away. The idea of using AWS Lambda to build web apps without managing or scaling servers had me hooked. Sam Kroonenburg A Cloud Guru Serverless Architecture on AWS Since then I have dedicated my spare time to learning and blogging about Serverless. Six months after that meetup Sam offered me a job and I now build Serverless systems Full-time for . A Cloud Guru Learning about Serverless has had a big impact on the way I build software. Serverless gained a lot of momentum and maturity in 2016. Perhaps you looked at it last year but didn’t try it or maybe you’ve never heard of it. Whatever the case, there has never been a better time to get started. What is (and isn’t) Serverless? There is still a lot of debate on what is Serverless. Right now the answer depends on who you ask. There are two questions, what is a Serverless product and what is a Serverless architecture. A Serverless product is fully managed with granular scaling and granular billing. Products include but are not limited to: , and . Compute: AWS Lambda Azure Functions WebTask , and Storage: S3 DynamoDB Firebase , Authentication: Cognito Auth0 Search: Algolia Note I did not include any Platform as a Service (PaaS) products like Amazon RDS, Heroku or Google App Engine. For these services, you pay per hour and scale by adding or removing underlying servers. You might not have to manage servers but you still need to think about them. So what is a Serverless architecture? I define a Serverless Architecture as: An event driven system that utilises FaaS and other fully managed services for logic and persistence. If you are still scratching your head don’t worry. This will make more sense once you see an example. Getting started with Serverless The best way to learn is to practice. Let’s pretend that a client has asked me to reinvent anonymising faces on images. Instead of blurring a face, my client wants to replace each face with an emoji. The emoji must reflect the emotion of that face. To deliver this service I need to: Allow my client to upload images Detect the faces in each image Determine the emotion of each face Replace each face with an emoji Store the edited image Instead of building it all myself, I am going to use a Serverless architecture on AWS. Serverless Architectures There are three services I need to develop this system. : scalable object based file storage Amazon S3 : serverless compute AWS Lambda : image analysis Amazon Rekognition Using those services I developed the following architecture. A Serverless Architecture Let’s go back to the definition of a Serverless architecture. An event driven system that utilises FaaS and other fully managed services for logic and persistence. The architecture I have designed fits this definition. When a user uploads a file an event is produced and a Lambda function is invoked. ObjectCreated The Lambda function calls Amazon Rekognition to detect the faces and emotion of each face in the uploaded image. The Lambda function processes the image and persists the image in Amazon S3 The simplicity of this design is what makes Serverless compelling. What languages and tools can I use If you want to build this system yourself, you need to create the infrastructure and code. To develop the code you can use , , , or . There is an for each of these languages. If your language is missing, watch for language support to grow over 2017. Node.js Python Java C# AWS SDK To create the infrastructure there are a few options. The first option is to use the AWS console. This is a good way to start while you are learning. But this is not recommended for production services. The other option is to develop your infrastructure as code. Two good tools for this are or . CloudFormation Terraform To simplify your experience I recommend using a tool that helps you deploy your infrastructure and your code. The is the tool that I recommend using. It is an open source project with a great community of contributors (including me). Other alternatives include , , and . Serverless Framework Apex Claudia Sparta more Show me the code You’ve seen the architecture, let’s look at some code. I’m going to use the Serverless Framework and NodeJs. The first step is to define the infrastructure using the Serverless Framework. To do this I created a new service. serverless create --template aws-nodejs --path emoticon-faceswap Infrastructure This creates a folder containing a configuration file named . The functions and infrastructure for my service is defined here. serverless.yml This configuration creates my Lambda function and S3 bucket. The Lambda function will be invoked when an object is created in the folder. faceswap uploads/ Code Every Lambda function has a handler that serves as the entry point. I set the handler for my Lambda function to in . This is a combination of the file path and the function to invoke in that file . src/faceswap.handler serverless.yml src/faceswap.js handler Each handler function has three parameters: module.exports.handler = (event, context, callback) => { ... // Use callback() and return information to the caller. } : Data relating to the event. In this case it will contain information about the file that was created in S3 Event : Lambda runtime information Context : A normal node callback that you can use to return an error or result. Callback For my project the Lambda function needs to perform three tasks. Getting images from the S3 event Call the Rekognition API Process the images and save the results. You can view the code that calls Amazon Rekognition and processes the images on . GitHub The final step is to deploy the project and test it. To deploy the project I ran the following command. sls deploy --stage your_stage_name I then uploaded images from to S3 and checked the results. Pexels Photos from Pexels This was a really fun project to build. Building a project project like this is the best way to learn. It will help you understand event driven architectures and how to create Serverless systems. Where can I learn more? There are lots great resources available for you to learn more about Serverless. , , Blogs: Serverless Zone Serverless Framework A Cloud Guru Blog , Books: Serverless Architectures on AWS AWS Lambda in Action , Discussions: Serverless Forum Reddit presentations, Videos: ServerlessConf Serverless Framework Conference: Austin ServerlessConf Slack: serverless-forum Check or search meetup.com Meetup Groups: this list , Curated Resources: Awesome Serverless Serverless Resources I hope post has inspired you to look at Serverless. There are lots of great use cases to try. A HTTP API, Chatbot or Alexa Skill are just a few projects to try. If you have any questions about this project or Serverless in general you can contact me on Medium or . I’ll be running a workshop at if you want to meet me in person. Twitter ServerlessConf Austin
