Check out this awesome Serverless Stack with built-in CI/CD and Blue-Green Deployments I love building CloudFormation stacks, crazy I know… I also love serverless event-driven architectures, who doesn’t… I wanted to create a reusable stack that I could easily use to build web applications. The stack consists of an , , and . This isn’t the cool part. API UI Async Tasks The cool part is the built-in CI/CD via CodePipeline and Blue-Green deployments of Lambda. CI/CD and Blue-Green deployments are very important for the stability and health of an application. It is important to deliver often and fail fast. Code The code. Check it out, run it, and let me know what you think… _serverless-stack-cicd - Serverless stack with CI/CD & blue-green deployments - API + Static UI + Async Tasks_github.com thestackshack/serverless-stack-cicd Architecture Lets take a look at the architecture before we talk about the CI/CD and Blue-Green deployments. CI/CD When you push code or infrastructure changes to this application CodePipeline builds and updates everything. Your infrastructure stacks are updated. Your code is built, tested, and deployed. You can make changes and push all day long. Development will be fast and predictable. Happy days. Oh, and there is a and a environment, so you can test out your changes in sandbox prior to pushing them to prod. It’s nice to be able to test changes in a sandbox environment with your team before you go live with them. sandbox prod Here is what the pipeline looks like: Lets take a closer look at each pipeline stage and action. A pipeline consists of stages. Each stage consists of actions. How you organize your stages and actions depends on 2 things. The order you want your stages and actions to be processed in, and the stage input and output dependencies. Stages can be organized in series or in parallel or both. This is defined by the stage input and output artifacts. Actions can also be organized in series or in parallel or both. This is defined by the action property and the input and output artifacts. order If the input artifact for each stage is the output artifact for the previous stage then they will be in series. If the input artifact for a stage is the output artifact of several previous stages, and those previous stages don’t depend on each other, then those previous stages will be in parallel. Enough about CodePipeline, lets learn more about our pipeline: Source Stage — GitHub Push The source stage is triggered on every GitHub push. - Source - CloneRepository Source ThirdParty 1 GitHub - GitSource !Ref GitHubOwner 'master' !Ref GitHubRepo !Ref GitHubToken 1 Name: Actions: Name: ActionTypeId:Category: Owner: Version: Provider: OutputArtifacts: Name: Configuration:Owner: Branch: Repo: OAuthToken: RunOrder: The pipeline uses the GitHub branch and the pipeline uses the GitHub branch. prod master sandbox develop Tasks Package Stage This stage uses CodeBuild to build & test the Lambda, then the CloudFormation template. package - TasksPackage - Package Build AWS CodeBuild 1 !Ref TasksCodeBuildProd - GitSource - TasksOutput 1 Name: Actions: Name: ActionTypeId:Category: Owner: Provider: Version: Configuration:ProjectName: InputArtifacts: Name: OutputArtifacts: Name: RunOrder: Our Lambda within the CloudFormation template uses a relative property. When using the property you have to use CloudFormation package( ). CloudFormation package does 2 things. First it uploads your Lambda at the location as a zip file to S3, then it exports the template with the replaced with the zip file location in S3. CodeUri CodeUri aws cloudformation package CodeUri CodeUri Here is the entire CodeBuild . buildspec.yml 0.2 - echo Installing source NPM dependencies...- cd ./stacks/tasks && npm install - echo Testing the code- npm test- echo Removing dev dependencies- rm -Rf node_modules- npm install --production - aws cloudformation package --template-file tasks.stack.yml --s3-bucket ${Bucket} --output-template-file tasks.stack.output.yml 'stacks/tasks' zip - tasks.stack.output.yml version: phases:install:commands:pre_build:commands: build:commands: post_build:commands: artifacts:base-directory: type: files: This CodeBuild exports the packaged CloudFormation template. This will be used in the next stage. Tasks Stack Stage This stage mutates (create/update) the tasks stack. - TasksStack - CreateChangeSet - TasksOutput Deploy AWS 1 CloudFormation "TasksOutput::tasks.stack.output.yml" CHANGE_SET_REPLACE CAPABILITY_NAMED_IAM !GetAtt CloudFormationRole.Arn !Sub "${AWS::StackName}-tasks-prod" !Sub "${AWS::StackName}-tasks-prod-cs" 1- ExecuteChangeSet - TasksOutput Deploy AWS 1 CloudFormation CHANGE_SET_EXECUTE CAPABILITY_NAMED_IAM !GetAtt CloudFormationRole.Arn !Sub "${AWS::StackName}-tasks-prod" !Sub "${AWS::StackName}-tasks-prod-cs" 2 Name: Actions: Name: InputArtifacts: Name: ActionTypeId:Category: Owner: Version: Provider: Configuration:TemplatePath: ActionMode: Capabilities: RoleArn: StackName: ChangeSetName: RunOrder: Name: InputArtifacts: Name: ActionTypeId:Category: Owner: Version: Provider: Configuration:ActionMode: Capabilities: RoleArn: StackName: ChangeSetName: RunOrder: The stage has two actions. The first action creates a CloudFormation ChangeSet. You can see that the input artifact is the output artifact of the Tasks Package stage. Take a look at the , it is from the CodeBuild package command. TemplatePath The second action executes the CloudFormation ChangeSet mutating the stack. API Package & API Stack Stages These stages are exactly the same as the Tasks stages. They package the Lambda & CloudFormation template, then create the CloudFormation ChangeSet, and finally mutate that ChangeSet. UI Stack Stage The UI stack stage does things in the reverse order. Instead of building, testing, and uploading the code, then mutating the stack. It mutates the stack and then builds, tests, and uploads the code. - UI - Stack - GitSource Deploy AWS 1 CloudFormation "GitSource::stacks/ui/ui.stack.yml" CREATE_UPDATE CAPABILITY_NAMED_IAM !GetAtt CloudFormationRole.Arn !Sub "${AWS::StackName}-ui-prod" !Sub |{"Domain": "${Domain}","TLD" : "${TLD}"} 1- Deploy Build AWS CodeBuild 1 !Ref UICodeBuildProd - GitSource 2 Name: Actions: Name: InputArtifacts: Name: ActionTypeId:Category: Owner: Version: Provider: Configuration:TemplatePath: ActionMode: Capabilities: RoleArn: StackName: ParameterOverrides: RunOrder: Name: ActionTypeId:Category: Owner: Provider: Version: Configuration:ProjectName: InputArtifacts: Name: RunOrder: The first action mutates the CloudFormation stack and creates the S3 bucket we need in the next action. The second action executes a CodeBuild that uploads the code to the S3 bucket thereby deploying the static website. Here is the CodeBuild . buildspec.yml 0.1 - aws s3 sync stacks/ui/www "s3://$(aws cloudformation describe-stacks --stack-name ${StackName} --query "Stacks[0].Outputs[0].OutputValue" --output text)" --acl bucket-owner-full-control --acl public-read --delete --cache-control "max-age=1" --exclude stacks/ui/www/assets- aws s3 sync stacks/ui/www/assets "s3://$(aws cloudformation describe-stacks --stack-name ${StackName} --query "Stacks[0].Outputs[0].OutputValue" --output text)/assets" --acl bucket-owner-full-control --acl public-read --delete --cache-control "max-age=31536000" version: phases:install:commands:pre_build:commands:build:commands: post_build:commands: This is interesting. We get the S3 bucket name from the previous step by fetching the output parameter from the stack. Here is the command: $(aws cloudformation describe-stacks--stack-name ${StackName}--query "Stacks[0].Outputs[0].OutputValue"--output text) That’s it. That’s how the pipeline performs CI/CD for our infrastructure and code. Now lets take a look at the blue-green deployments. Blue-Green Deployments Within our api CloudFormation template we have a Lambda. AWS::Serverless::Function index.handler 5 !GetAtt IamRoleLambdaExecution.Arn ./ nodejs6.10 live Canary10Percent5Minutes - !Ref 5xxAlarm- !Ref 4xxAlarm- !Ref LatencyAlarm !Sub "${TasksStack}-SNSTopic" LambdaFunction:Type: Properties:Handler: Timeout: Role: CodeUri: Runtime: AutoPublishAlias: DeploymentPreference:Type: Alarms: Environment:Variables:TasksSnsTopic:Fn::ImportValue: This Lambda uses the (SAM), which is a CloudFormation transformer. Serverless Application Model When using SAM types within a CloudFormation template you need to add the transform definition. . **Transform**: AWS::Serverless-2016–10–31 I’m not sure why SAM exists at all. Why wasn’t CloudFormation extended to include these new SAM features? I read somewhere that SAM is less verbose. Is that the only reason? If so that doesn’t make up for the fragmentation and confusion SAM brings. Just my 2 cents… Take a look at the property. This is where we define the . DeploymentPreference Safe Traffic Shifting We use which routes 10% of traffic to the new Lambda, then waits 5 minutes, if all is good (no alarms go off) the remaining traffic is routed and the deployment is done. There are 3 types of traffic shifting. Canary10Percent5Minutes : Traffic to new version will linearly increase in steps of X percentage every Y minutes. Ex: will add 10 percentage of traffic every 10 minute to complete in 100 minutes. LinearXPercentYMinutes Linear10PercentEvery10Minutes : X percent of traffic will be routed to new Version once, and wait for Y minutes in this state before sending 100 percent of traffic to new version. Some people call this as Blue/Green deployment. Ex: will send 10 percent traffic to new version and 15 minutes later complete deployment by sending all traffic to new version. CanaryXPercentYMinutes Canary10Percent15Minutes : This is an instant shifting of 100% of traffic to new version. This is useful if you want to run run pre/post hooks but don’t want a gradual deployment. If you have a pipeline, you can set Beta/Gamma stages to deploy instantly because the speed of deployments matter more than safety here. AllAtOnce The property is where you define CloudWatch alarms to monitor while your traffic is being shifted. If any of these alarms go off the deployment will be reverted. alarms - !Ref 5xxAlarm Alarms: !Ref 4xxAlarm !Ref LatencyAlarm Our alarms are based on the API Gateway traffic. Are we getting too many 5xx or 4xx errors, or did the latency spike? If so something is wrong with our new version and we don’t want to deploy it. Here are the alarms. AWS::CloudWatch::Alarm RestApi 5xx alarm for api gateway 'AWS/ApiGateway' 5XXError - ApiName !Ref RestApi Sum '60' '3' '10' GreaterThanOrEqualToThreshold AWS::CloudWatch::Alarm RestApi 4xx alarm for api gateway 'AWS/ApiGateway' 4XXError - ApiName !Ref RestApi Sum '60' '3' '10' GreaterThanOrEqualToThreshold AWS::CloudWatch::Alarm RestApi latency alarm for api gateway 'AWS/ApiGateway' Latency - ApiName !Ref RestApi Average '60' '3' '25000' GreaterThanOrEqualToThreshold 5xxAlarm:Type: DependsOn: Properties:AlarmDescription: Namespace: MetricName: Dimensions: Name: Value: Statistic: Period: EvaluationPeriods: Threshold: ComparisonOperator: 4xxAlarm:Type: DependsOn: Properties:AlarmDescription: Namespace: MetricName: Dimensions: Name: Value: Statistic: Period: EvaluationPeriods: Threshold: ComparisonOperator: LatencyAlarm:Type: DependsOn: Properties:AlarmDescription: Namespace: MetricName: Dimensions: Name: Value: Statistic: Period: EvaluationPeriods: Threshold: ComparisonOperator: That’s it. I hope you enjoyed this post and get some use out of this stack. Please if you liked this article. clap
