6,164 reads

DevSecOps Introduction: Clear Instructions on How to Build a DevSecOps Pipeline in AWS [Part 1]

by Ask Gourav DasApril 14th, 2021

Too Long; Didn't Read

DevSecOps is the new buzz and definitely a potential candidate to scare people in the software industry. In simple terms, it is all about "Shift left on Security" i.e. to introduce security as early as possible in the SDLC. The pipeline divided into four parts. The 1st part stores the source code in AWS CodeCommit, which is an AWS native private repository. The 2nd part contains the following build stages, which performs static code analysis by using AWS CodeBuild 41 Git-secrets.

Company Mentioned

Coin Mentioned

featured image - DevSecOps Introduction: Clear Instructions on How to Build a DevSecOps Pipeline in AWS [Part 1]

Note: In case if you are looking for a panacea. The following project is on 👉

Github

👈. Alternatively,

Click Here

to launch the AWS CloudFormation Create Stack Console with the prepopulated master template in the Ohio region. ~GD

Welcome to the first part of the How to Build a DevSecOps Pipeline in AWS. DevSecOps is the new buzz and definitely a potential candidate to scare people in the ever-changing software industry. When I heard the term for the first time, my inner voice said it out loud, "When just this Happened, and why SEC is sandwiched between them ?". Later on, to look cool, I started adding DevSecOps on my profile, but the dire consequence was a complete disappointment. DevSecOps isn't just restricted to understand security by heart & fit it with DevOps. A good analogy would be not to imagine your ex with someone, especially with your girlfriend/wife. In simple terms, it is all about "Shift left on Security" i.e. to introduce security as early as possible in the SDLC. Though I am not going to bore you with the definitions and concepts as I am a pragmatist.

Thus, my objective here is to demonstrate how DevSecOps works in reality.

The following series is split into two parts (refer below) with very simple and clear instructions to provision a CI/CD pipeline adhering to DevSecOps principles in AWS.

Everything covered from scratch you won't face any difficulty understanding. In case of any clarification, drop me a note on LinkedIn. Feel free to explore them with ease, skip to the one which is relevant to you.

Myth buster: Automation isn't the only thing you do in a DevSecOps workflow. It's a whole set of principles, paradigm & security best practices you introduce in every stage of the SDLC. The tutorial here focuses on the automation part of it. Thus, gear up to learn how to bring security build and test stages in a CI/CD pipeline & get continuous feedback.

Prerequisites:

You need to have an AWS Free Tier Account to run a CloudFormation Template which will do all the necessary setup for the upcoming Demo.
All the resources provisioned through CloudFormation Template comes under either Free Tier Eligibility or Free Trial.
The following Project is in Github for your reference.

How does it works:

So in this tutorial, I have used open-source tools to build the DevSecOps pipeline to make the demo more achievable. The below diagram depicts the tools and native services used along with the security control gates applied in the process.

Architecture:

You are going to build the below CI/CD Pipeline in AWS by using AWS native developer tools such as AWS CodePipeline , AWS CodeBuild & AWS CodeCommit. The entire provisioning will be done using cloudformation template. It just requires plug & play. I will be explaining the deployment process as you move forward.

The main steps are as follows:

```
1.
```
First, you would launch the entire DevSecOps pipeline using a master cloudformation template. Refer the above reference architecture.
```
2.
```
The pipeline divided into four parts.
```
3.
```
The 1st part stores the source code in AWS CodeCommit, which is an AWS native private repository. The master template would auto-populate a sample application in the repository. You need to clone the sample code from the repository to interact with the pipeline and modify it.
```
4.
```
The 2nd part contains the following build stages, which performs static code analysis by using AWS CodeBuild.
```
4.1
```
Git-secrets scans Git source repositories and finds code that may potentially include sensitive information, such as user passwords, or that has other security issues & notify of any breach via email.
```
4.2
```
Insider CLI is an open-source SAST tool that performs static code analysis.
```
4.3
```
OWASP Dependency-Check is an open-source SCA tool that attempts to detect publicly disclosed vulnerabilities contained within a project’s dependencies.
```
4.4
```
Cfn_nag (optional) is an open-source IaC scanner that looks for patterns in CloudFormation templates that may indicate insecure infrastructure. This an optional build-stage, which can be disabled while launching the master template if not required.
```
5.
```
The 3rd part builds a pre-prod environment that hosts the sample application and also performs dynamic testing, integrated with AWS CodeBuild.
```
5.1
```
CloudFormation stack will facilitate the deployment. At first, a changeset will be created & then changeset will be executed to bring up the sample application up & running. The sample application runs on an autoscaling group backed by a VPC and an ALB.
```
5.2
```
OWASP ZAP is an open-source DAST tool that performs dynamic code analysis on the running application.
```
5.3
```
Manual Approval required to move to the next stage.
```
6.
```
The 4th part (optional) is something readers can explore by themselves where they need to create a replica of the pre-prod environment to host the same sample application to bring up the production environment.

So let's see something happen now:

To set up the sample DevSecOps pipeline. Log in to the AWS account if you haven’t done so already.

Click 👉

here

👈 to launch the

AWS CFN console

 with the prepopulated master template in the

Ohio region

If you want to run the template on a different AWS region, change the region from the top right corner. Additionally, you can find the latest code on GitHub.

***Refer to the below Clean Up section to initiate decommissioning of the running resources ***

Fill in the stack parameters as shown below, acknowledge the required

capabilities: [AWS::CloudFormation::Stack]
and click create to execute the pipeline and wait for the cloudformation stack to complete.

CodeCommit Repository Name: Self-explanatory.
CodeCommit Repository Description: Self-explanatory.
TemplateFolder: The folder in your repo that contains the AWS CloudFormation templates. Keep it default.
Enable CFN-Nag (IaC Scanner): Whether to enable cfn-nag, it requires Security Hub to be enabled in your AWS account.
Click here
```
to follow the instructions to enable Security-Hub (Free-Trial) in your AWS account.
```
Weight coefficient for failing: The weight coefficient for a failing violation in the template.
```
Keep it default. 
```
Weight coefficient for warning: The weight coefficient for a warning in the template.
```
Keep it default. 
```
Fail build: Whether to fail the cfn-nag build stage when security findings are detected based on weight coefficient. Keep it false for smooth transition.
Insider CLI Technology Stack: Enter the Technology-Stack to Run Insider CLI for Vulnerabilities. Supported technologies: android, java, ios, javascript & csharp.
```
Keep it default as the demo application is using JAVA. 
```
Insider CLI Score: If the score set between 0 and 100 the exit code from Insider CLI will indicate if a vulnerability with a score equal to or higher was identified.
```
Keep it default.
```
Fail On CVSS: If the score set between 0 and 10 the exit code from dependency-check will indicate if a vulnerability with a CVSS score equal to or higher was identified.
```
Keep it default.
```
Enter the Email ID: Email ID to get critical alerts e.g. AWS Access key detected in the Code Repository.
```
NOTE: To get email alerts, subscribe to the AWS Notification mailer sent to the input email-id. 
```
The subscription's protocol: Self-explanatory.
S3 bucket with sources: This bucket contains all sources, such as the Lambda function and templates. You can keep the default text if you’re not customizing the sources.
```
Keep it default.
```
Prefix for S3 bucket with sources: The prefix for all objects. You can keep the default if you’re not customizing the sources.
```
Keep it default.
```

View the DevSecOps Pipeline:

Click here to open the AWS Pipeline console, change the region if you have run the template in a different region. Navigate to the required codepipeline, you can see something similar like below. I have explained all the Code Pipeline stages in the second part of the tutorial. Alternatively, click here to navigate to the second part of the tutorial.

How to initiate the Pipeline:

Now, if you want to trigger or modify the pipeline, you need to clone the code from the repository. Thus, go to AWS CodeCommit & select the repository e.g. AWSDevSecOpsTutorial. In the top right corner, you will see the Clone URL option. Follow the following AWS Doc, which got all the procedures to access the codecommit repository from your local system.

To test the pipeline, 👉 download 👈 the following file (it contains dummy AWS access keys), commit and push the changes to the remote repository. If you see an email alert or Secrets-Check failed in the build stage. Congratulations , you have successfully configured the pipeline on AWS (👍 ͡⚈ ͜ʖ ͡⚈)👍 .

Clean Up:

Cleanup is a bit tricky, thus follow the instructions carefully. Click here to go to the Cloudformation stacks page. Look for the stack with the name Pre-Prod-DevSecOps* & initiate the delete. With due diligence, the pre-prod stack needs to be deleted first. If you have deleted the other stack or master stack (e.g. DevSecOpsTutorial*), you won't be able to delete the pre-prod stack.

Post deletion of the first stack, you can initiate the delete of Master Stack. No need to touch the nested stack, keep it as it is. It will be removed automatically while the master template gets deleted. (Select the correct region where the master template running)

Troubleshooting:

You may encounter the following issue "The bucket you tried to delete is not empty (Service: Amazon S3; Status Code: 409; Error Code: BucketNotEmpty" while deleting the master stack. Thus, explicitly you need to go to the S3 buckets & empty the objects inside them and rerun the stack delete. Note: Versioning is enabled in the Bucket
aws-sec-build-reports-
*. Thus, you need to list all the objects and delete them.

You may encounter the pipeline didn't execute all stages successfully when the AWS CodePipeline starts automatically at the time of provisioning. You need to click the Release change in the top right corner once all the build stages are completed in the initial phase.

Reference:

https://aws.amazon.com/blogs/security/integrating-aws-cloudformation-security-tests-with-aws-security-hub-and-aws-codebuild-reports/

Congratulation for coming so far. In the next part, I will explain all the Code Pipeline stages in details, Hope to see you again😊.

We have come a long way, kindly Share and help me on my mission to educate and familiarize people in the world of digitization 💪 #This is a Free tutorial and all my upcoming tutorials will be free and accessible from Public forums# Appreciate if you drop me a note on

LinkedIn
& share your opinion. Don't worry, I don't bite 👻 so don't shy away 🏃🏻‍♀️ 🏃🏻. Your feedback will help me to come up with more awesome contents on the internet.