In your deployment phase, a crucial question to address is the cost estimation of the resources required for deploying your amazing application within the allocated or recommended budget. Failing to address this question could result in exceeding the allocated or budgeted amount due to resource creation costs. One tool that was developed to help with putting a check on the cost of cloud resources is Infracost. Infracost is a valuable tool designed to monitor and control the cost of cloud resources. Unlike tools like the AWS calculator, Infracost stands out with its automated approach. By simply parsing the format of your Terraform-created resources as an argument to the Infracost CLI command, it automatically provides you with the monthly cost estimation for the resources to be deployed, whereas the AWS calculator requires manual input of each resource to be created. JSON This article aims to guide you, the reader, on obtaining cost estimates for your cloud resources using Infracost. It covers creating an Open Policy Agent (OPA) policy to enforce monthly budget constraints and integrating the infracost command into your Jenkins pipeline. To set up Infracost, refer to the for detailed instructions on getting started. official documentation The terraform files for the resources to be created are stored in a folder which is in the same location as the This is the file location format used in this article. terrafom Jenkinsfile. As we execute the Infracost command within a Jenkins pipeline, we utilize the official Infracost docker image as the Jenkins agent, alongside configuring essential environment variables. Here is an example of how the configuration should be set up: agent {
  docker {
    image 'infracost/infracost:ci-latest'
    args "--user=root --entrypoint=''"
          }
       }
environment {
   INFRACOST_API_KEY = credentials("INFRACOST_API_KEY")
   INFRACOST_VCS_PROVIDER = 'github'
   INFRACOST_VCS_REPOSITORY_URL = 'https://github.com/Okeybukks/devops-automation'
    } To get your you can run this command in your cli. INFRACOST_API_KEY infracost configure get api_key Alternatively, you can access the Infracost dashboard, login with your credentials, navigate to " ," and click on the " " button next to the API Key. This key needs to be securely stored as secret text in Jenkins credentials, with an ID of . Considering that our Git repository is hosted on GitHub, set the value of to "GitHub." Additionally, specify the as the URL of the repository containing your Jenkinsfile and Terraform folder. Org settings Copy INFRACOST_API_KEY INFRACOST_VCS_PROVIDER INFRACOST_VCS_REPOSITORY_URL After configuring Infracost for Jenkins, the subsequent step involves executing the command. This command analyzes the Terraform plan and provides a cost estimate for the cloud resources intended for deployment. The resulting output is saved in a JSON file format, which will be utilized by our OPA policy. infracost breakdown stage("Check Financial Expense of Infrastructures Job with Infracost"){
   agent {
     docker {
       image 'infracost/infracost:ci-latest'
       args "--user=root --entrypoint=''"
             }
         }
   environment {
       INFRACOST_API_KEY = credentials("INFRACOST_API_KEY")
       INFRACOST_VCS_PROVIDER = 'github'
       INFRACOST_VCS_REPOSITORY_URL = 'https://github.com/Okeybukks/devops-automation'
            }
   steps{
       dir("./terraform") {
          sh 'echo "This is the financial check job"'     
          sh 'infracost breakdown --path . --format json --out-file infracost.json'
          archiveArtifacts artifacts: 'infracost.json'
           } 
          }
   } The terraform files are located in the terraform folder, so to change into the terraform folder, we will be using the block. dir The command takes a couple of arguments with values passed to it. infracost breakdown takes the path where the terraform files are located. Since we are already in the terraform folder, we use to reference the path. --path . takes the format of the output of the command. --format infracost breakdown takes the name of the file you wish to save the output. --out-file infracost breakdown To enable the utilization of the output file generated by the command in subsequent stages, the file is parsed to the "archiveArtifacts" command, converting it into an artifact. For a more comprehensive understanding of artifact creation and copying, I invite you to explore the I have authored on Jenkins artifacts. infracost breakdown article To establish a cost constraint on the creation of resources, an OPA policy is employed. In this article, we are imposing a limit of $100 on the allocated resources. OPA serves as a comprehensive toolset and framework for policy implementation throughout the cloud-native ecosystem. To delve deeper into OPA, you can find more information by clicking here OPA policies are written in the rego language, and as such, the policy file is suffixed with .rego. The for rego serves as a comprehensive resource to comprehend policy formulation. In this article, a straightforward policy will be presented, which verifies whether the cumulative monthly estimate of your cloud resources surpasses the predefined maximum monthly estimate that must not be exceeded. documentation package infracost

deny[out] {

    # define a variable
	maxMonthlyCost = 100.0

	msg := sprintf(
		"Total monthly cost must be less than $%.2f (Current monthly cost is $%.2f)",
		[maxMonthlyCost, to_number(input.totalMonthlyCost)],
	)

  	out := {
    	"msg": msg,
    	"failed": to_number(input.totalMonthlyCost) >= maxMonthlyCost
  	}
} The line is the infracost package or module utilized by OPA to understand the content of the output file from the command. OPA policies are written in a block. Any code written outside this block will return a error. package infracost infracost breakdown deny[out] rego_parse_error is the defined variable which is the maximum amount budgeted for our resources. maxMonthlyCost The message to be output in the Jenkins pipeline and GitHub commit message if the set condition fails is defined in the block. msg The condition logic is set using the block. The total estimate our resources will cost us is saved in variable. This variable is a default variable in the rego language. The is the total monthly cost estimated by infracost. out input.totalMonthlyCost input.totalMonthlyCost From our policy, if is true it stops the pipeline and prevents moving to the next stage of the pipeline. The message in the block explains to us that we cant go forward. The content of this policy is stored with file name. This filename can be changed though, but it must have the format appended to it. to_number(input.totalMonthlyCost) >= maxMonthlyCost msg infracost-policy.rego .rego With the policy written, it is then parsed to the command. This is what the stage looks like. infracost comment github stage("Post Infracost comment"){
  agent {
     docker {
        image 'infracost/infracost:ci-latest'
        args "--user=root --entrypoint=''"
             }
        }
  environment {
     INFRACOST_API_KEY = credentials("INFRACOST_API_KEY")
     INFRACOST_VCS_PROVIDER = 'github'
     INFRACOST_VCS_REPOSITORY_URL = 'https://github.com/Okeybukks/devops-automation'
     INFRACOST_VCS_BASE_BRANCH = 'main'
     GITHUB_TOKEN = credentials("GITHUB_TOKEN")
     GITHUB_REPO = "Okeybukks/devops-automation"
         }
  steps{
     dir('./terraform'){
         sh 'echo "This is the financial check job"'
         copyArtifacts filter: 'infracost.json', fingerprintArtifacts: true, projectName: 'test', selector: specific ('${BUILD_NUMBER}')    

         sh 'infracost comment github --path infracost.json --policy-path infracost-policy.rego \
                    --github-token $GITHUB_TOKEN --repo $GITHUB_REPO --commit $GIT_COMMIT'
         }
     }
 } Upon observation, you will notice the addition of new environment variables required for this stage. Since the infracost command will be responsible for posting the commit message to our repository, it will necessitate specific GitHub access permissions to fulfil this task effectively. Generate a classic token with repo permission ticked. If you can read how to generate a classic GitHub token using this . Once generated, save it in your Jenkins credential using secret text and save it with the ID . article GITHUB_TOKEN The GitHub repo name of this project is also needed. Store this value in the ID. GITHUB_REPO The output of the command is saved in the infracost.json file. To access the file at this stage, the is utilized. infracost breakdown copyArtifacts Now we have all the values to the arguments needed by the command to utilize our policy in creating a check for resource creation. infracost comment github value is the file created with the command, i.e infracost.json --path infracost breakdown value is the policy file we created i.e . --policy infracost-policy.rego value is our classic GitHub token stored in . --github-token GITHUB_TOKEN is the project repo name stored in --repo GITHUB_REPO is required to post on a pull request's commit. Its value is stored in variable, one of Jenkins default variables. --commit $GIT_COMMIT When you run your pipeline, you should have something similar to this if your resources estimate exceeds the set mark. + infracost comment github --path infracost.json --policy-path infracost-policy.rego --github-token **** --repo Okeybukks/devops-automation --commit a0e8fa201d81ba9d2a508e4adc55e2b8f1ffca6d
time="2023-06-12T04:29:15Z" level=info msg="Finding matching comments for tag infracost-comment"
time="2023-06-12T04:29:21Z" level=info msg="Found 0 matching comments"
time="2023-06-12T04:29:21Z" level=info msg="Creating new comment"
time="2023-06-12T04:29:21Z" level=info msg="Created new comment https://github.com/Okeybukks/devops-automation/commit/a0e8fa201d81ba9d2a508e4adc55e2b8f1ffca6d#commitcomment-117572438"
Comment posted to GitHub

Error: Policy check failed:

 - Total monthly cost must be less than $100.00 (Current monthly cost is $168.72) This concludes the article. Implementing this cost monitoring approach will not only lead to cost savings in resource usage but also prevent unintended deployments of unplanned cloud resources. Thank you for reading. Also published here. The lead image for this article was generated by HackerNoon's AI Image Generator via the prompt "A mount counting coins."

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Efficient Cloud Resource Cost Management with Infracost and Open Policy Agent in Jenkins

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Accessing Private Instances With An, Internet-Facing, Application Load Balancer in AWS

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 1)

Accessing Private Instances With An, Internet-Facing, Application Load Balancer in AWS

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 1)

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps