As companies continue to migrate their workloads from on-premise solutions to public Cloud Service Providers (CSPs) like AWS, it becomes increasingly difficult to understand the complete picture of what might be running within an organization’s cloud. Even simple applications commonly have multiple environments such as Development, Staging, and Production. For more complex workloads, there are additional environments, microservices, queues, notification services (e.g. SNS, SES), data stores, and more. Even when all this resides under a single AWS account, it is hard to conceptualize let alone manage sprawling cloud resources. Take two seemingly simple questions: “How many EC2 instances am I currently running across all my AWS accounts?” and, “Have I correctly tagged all of my production environment EC2 instances with ?”. These are very difficult to answer using the AWS console or CLI. environment: production This difficulty is compounded by common best practices such as spreading resources across regions and separating workloads across several accounts within an AWS Organization. Performing an inventory using the AWS console requires keeping a manual count while changing regions and accounts to ensure you didn’t miss anything. Unfortunately, using the AWS CLI is not much easier. For each region and account you wish to query, you need to multiple commands, calling the correct APIs for each asset type (i.e. ). Then you must manually aggregate the results and double-check you’re not missing any data. aws ec2 describe-instances Rather than give yourself a migraine dealing with all of this cross-region, cross-account manual data fetching – let’s see what this would look like using . CloudGraph is the free and open-source GraphQL API for AWS that vastly simplifies the process of answering asset inventory, compliance, and billing questions. CloudGraph So how do you get started? First, you using the single install command: install CloudGraph npm install npm install -g @cloudgraph/cli Next, you scan of your AWS accounts at once by configuring CloudGraph using the credentials you already have stored in your local ~/.aws/credentials file (see the if you’d like more information on that). You can also scan accounts using IAM roles if you prefer. Initialize CloudGraph for our AWS accounts by running the following command: all CloudGraph AWS provider README cg init aws If you have a local AWS credentials file, you should see a list of profiles to select. In my case, I will choose the profiles and so I can scan the AWS accounts running my Development and Production environments. default master Next, you can pick the regions you want CloudGraph to scan. In my case this is , , and . If you’re unsure what regions you have resources in, you can always select them all. CloudGraph will automatically scan . If necessary, this can be modified by passing the flag to the init command (see the for more information on passing flags). us-east-1 us-east-2 us-west-1 all supported resources -r CloudGraph README Now you are ready to scan your AWS accounts! Run the following command to launch a local instance of , the graph database that CloudGraph uses to locally store your data. Dgraph cg launch And finally, run the following command to scan all of your configured AWS accounts, regions, and services: cg scan aws Feel free to grab a quick cup of coffee while this runs. After the scan is complete, CloudGraph will open the you selected in the step. You can use this query tool to ask pretty much any question about your AWS resources. GraphQL query tool cg init Let’s answer the first question I posed at the start of this post: “How many EC2 instances are currently running across all my AWS accounts?”. This is simple to answer with CloudGraph. As you can see, I have 46 EC2 instances running across the two environments I chose to scan. Note that CloudGraph also provides the total of all scanned resources in the report printed at the end of each scan. Maybe I want to take this a step further and view metadata about these EC2 instances such as their . I can easily run a GraphQL query against my EC2 instances and request only the specific metadata I care about. instanceType Now let's answer the second question I posed: “Have I correctly tagged all of my production environment EC2 instances with ?”. I could actually do this in a couple of different ways. environment: production First, I could directly query the EC2 instances in Production and see their tags, like so: query {
  queryawsEc2(filter: {accountId: {eq: "632*****77"}}) {
      id
      tags {
        key
        value
      }
  }
} This output clearly shows me all my EC2 instances in account “632…” and their tags. A second and more direct way of asking this question would be to use and filter on the key and value I would like to target: queryawsTag query {
  queryawsTag(filter: {key: {eq: "environment"} and: {value: {eq: "production"}}}) {
    ec2Instance(filter: {accountId: {eq: "632*****77"}}) {
      id
      arn
    }
  }
} Now I’m only getting back EC2 instances that are tagged with “environment: production”. This would also work for other taggable AWS resources such as VPCs, IAM users, S3 buckets, or EKS clusters. CloudGraph makes it easy to inventory your assets no matter how many different regions or accounts they reside in. The sky's the limit for what you can query! If you have any questions or suggestions on how we can improve CloudGraph, or in the . please drop us a line on slack CloudGraph GitHub issues Director of Engineering at Tyler Dunkel AutoCloud

The Graph

Microsoft

A Complete Inventory of Your AWS Footprint in 5 Minutes

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 2)

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

10 Lessons from 10 Years of AWS (part 2)

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps