In a we dis­cussed how to auto-sub­scribe a to a Lamb­da func­tion using . So that we don’t need a man­u­al process to ensure all Lamb­da logs would go to our log aggre­ga­tion ser­vice. pre­vi­ous post Cloud­Watch Log Group Cloud­Watch Events Whilst this is use­ful in its own right, it only scratch­es the sur­face of what we can do. and makes it easy to auto­mate many day-to-day oper­a­tional steps. With the help of of course ;-) Cloud­Trail Cloud­Watch Events Lamb­da I work with and heav­i­ly. When­ev­er you cre­ate a new API, or make changes, there are sev­er­al things you need to do: API Gate­way Lamb­da enable for the deploy­ment stage Detailed Met­rics set up a dash­board in Cloud­Watch, show­ing request count, laten­cies and error counts set up for p99 laten­cies and error counts Cloud­Watch Alarms Because these are man­u­al steps, they often get missed. Have you ever for­got­ten to update the dash­board after adding a new end­point to your API? And did you also remem­ber to set up a p99 laten­cy alarm on this new end­point? How about alarms on the no. of 4XX or 5xx errors? Most teams I have dealt with have some con­ven­tions around these, but without a way to enforce them. The result is that the con­ven­tion is applied in patch­es and can­not be relied upon. I find this approach doesn’t scale with the size of the team. It works when you’re a small team. Every­one has a shared under­stand­ing, and the nec­es­sary dis­ci­pline to fol­low the con­ven­tion. When the team gets big­ger, you need automa­tion to help enforce these con­ven­tions. For­tu­nate­ly, we can auto­mate away these man­u­al steps using the same pattern. In the unit of my course , I demon­strat­ed how you can do this in 3 sim­ple steps: Mon­i­tor­ing Pro­duc­tion-Ready Server­less cap­tures the request to . Cloud­Trail Cre­at­eDe­ploy­ment API Gate­way pat­tern against this cap­tured request. Cloud­Watch Events func­tion to enable detailed met­rics, and cre­ate alarms for each end­point. Lamb­da a) b) If you use the frame­work, then you might have a func­tion that looks like this: Server­less auto-create-api-alarms: handler: functions/create-alarms.handler events: - cloudwatchEvent: event: source: - aws.apigateway detail-type: - AWS API Call via CloudTrail detail: eventSource: - apigateway.amazonaws.com eventName: - CreateDeployment environment: alarm_actions: arn:aws:sns:#{AWS::Region}:#{AWS::AccountId}:NotifyMe ok_actions: arn:aws:sns:#{AWS::Region}:#{AWS::AccountId}:NotifyMe iamRoleStatements: - Effect: Allow Action: apigateway:GET Resource: - arn: aws:apigateway:#{AWS::Region}::/restapis/* - arn: aws:apigateway:#{AWS::Region}::/restapis/*/stages/${self:custom.stage} - Effect: Allow Action: apigateway:PATCH Resource: arn:aws:apigateway:#{AWS::Region}::/restapis/*/stages/${self:custom.stage} - Effect: Allow Action: cloudwatch:PutMetricAlarm Resource: "*" Cou­ple of things to note from the code above: I’m using the plu­g­in to give the func­tion a tai­lored IAM role server­less-iam-roles-per-func­tion The func­tion needs the per­mis­sion to enable detailed met­rics apigateway:PATCH The func­tion needs the per­mis­sion to get the API name and REST end­points apigateway:GET The func­tion needs the per­mis­sion to cre­ate the alarms cloudwatch:PutMetricAlarm The envi­ron­ment vari­ables spec­i­fy SNS top­ics for the Cloud­Watch Alarms The cap­tured event looks like this: { : , : , : , : , : , : , : , : [], : { : , : { : , : , : , : , : , : , : { : { : , : }
          }, : }, : , : , : , : , : , : , : { : , : { : }, : }, : { : , : , : { : , : , : }, : { : , : , : , : [ ]
          }, : { : , : , : }, : { : , : , : }
      }, : , : , : , : }
} "version" "0" "id" "dee9a69c-8166-1ad7-41d4-1dad201e29f6" "detail-type" "AWS API Call via CloudTrail" "source" "aws.apigateway" "account" "374852340821" "time" "2018-04-09T00:17:47Z" "region" "us-east-1" "resources" "detail" "eventVersion" "1.05" "userIdentity" "type" "IAMUser" "principalId" "AIDAIRMUZZEGPO27IPFYW" "arn" "arn:aws:iam::374852340821:user/yan.cui" "accountId" "374852340821" "accessKeyId" "ASIAJNZDKN26DXPZFYQE" "userName" "yan.cui" "sessionContext" "attributes" "mfaAuthenticated" "false" "creationDate" "2018-04-09T00:17:30Z" "invokedBy" "cloudformation.amazonaws.com" "eventTime" "2018-04-09T00:17:47Z" "eventSource" "apigateway.amazonaws.com" "eventName" "CreateDeployment" "awsRegion" "us-east-1" "sourceIPAddress" "cloudformation.amazonaws.com" "userAgent" "cloudformation.amazonaws.com" "requestParameters" "restApiId" "8kbasri6v7" "createDeploymentInput" "stageName" "dev" "template" false "responseElements" "id" "cj2y0f" "createdDate" "Apr 9, 2018 12:17:47 AM" "deploymentUpdate" "restApiId" "8kbasri6v7" "deploymentId" "cj2y0f" "template" false "deploymentStages" "deploymentId" "cj2y0f" "restApiId" "8kbasri6v7" "template" false "templateSkipList" "position" "deploymentDelete" "deploymentId" "cj2y0f" "restApiId" "8kbasri6v7" "template" false "self" "deploymentId" "cj2y0f" "restApiId" "8kbasri6v7" "template" false "requestID" "6e25bd56-3b8b-11e8-a351-e5e3d3161fe7" "eventID" "a150d941-7a54-4572-97b2-0614a81fd25b" "readOnly" false "eventType" "AwsApiCall" We can find the and inside the attribute. That’s all we need to fig­ure out what end­points are there, and so what alarms we need to cre­ate. restApiId stageName detail.requestParameters Inside the han­dler func­tion, which you can find , we per­form a few steps: here enable detailed met­rics with an call to API Gate­way updateStage get the list of REST end­points with a call to API Gate­way getResources get the REST API name with a call to API Gate­way getRestApi for each of the REST end­points, cre­ate a p99 laten­cy alarm in the name­space AWS/ApiGateway Now, every time I cre­ate a new API, I will have to alert me when the 99 per­centile laten­cy for an end­point goes over 1 sec­ond, for 5 minutes in a row. Cloud­Watch Alarms All this, with just a few lines of code :-) You can take this fur­ther, and have oth­er Lamb­da func­tions to: cre­ate Cloud­Watch Alarms for 5xx errors for each end­point cre­ate Cloud­Watch Dash­board for the API So there you have it, a use­ful pat­tern for automat­ing away man­u­al ops tasks! And before you even have to ask, yes I’m aware of server­less plu­g­in by the folks. It looks neat, but it’s ulti­mate­ly still some­thing the developer has to remem­ber to do. this ACloudGu­ru That requires dis­ci­pline. My expe­ri­ence tells me that you can­not rely on dis­ci­pline, ever. Which is why, I pre­fer to have a plat­form in place that will gen­er­ate these alarms instead. Hi, my name is . I’m an and the author of . I have run production workload at scale in AWS for nearly 10 years and I have been an architect or principal engineer with a variety of industries ranging from banking, e-commerce, sports streaming to mobile gaming. I currently work as an independent consultant focused on AWS and serverless. Yan Cui AWS Serverless Hero Production-Ready Serverless You can contact me via , and . Email Twitter LinkedIn Check out my new course, . Complete Guide to AWS Step Functions In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. Including basic concepts, HTTP and event triggers, activities, design patterns and best practices. Get your copy . here Come learn about operational for AWS Lambda: CI/CD, testing & debugging functions locally, logging, monitoring, distributed tracing, canary deployments, config management, authentication & authorization, VPC, security, error handling, and more. BEST PRACTICES You can also get off the face price with the code . 40% ytcui Get your copy . here

Dash

Amazon

Twitter

Highlights of Monitorama PDX day 1

You need to sample debug logs in production

Ask me for help about serverless

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

auto-create CloudWatch Alarms for APIs with Lambda

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Modeling Hierarchical Access With AppSync In 3 Simple Steps

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

10 Threats to an Open API Ecosystem

10 Indications That You Should Invest in Automation Via APIs

10 Best Practices for Securing Your API

Modeling Hierarchical Access With AppSync In 3 Simple Steps

100 Days of AI Day 1: From Newsletter to Podcast, Leveraging AI for Audio Transformation

10 Threats to an Open API Ecosystem

10 Indications That You Should Invest in Automation Via APIs

10 Best Practices for Securing Your API

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps