Recently, I encountered a task where a business was using AWS Elastic Beanstalk but was struggling to understand the system state due to the lack of comprehensive metrics in CloudWatch. By default, CloudWatch only provides a few basic metrics such as CPU and Networks. However, it’s worth noting that Memory and Disk metrics are not included in the default metric collection. Fortunately, each Elastic Beanstalk virtual machine (VM) comes with a CloudWatch agent that can be easily configured to collect additional metrics. For example, if you need information about VM memory consumption, which AWS does not provide out of the box, you can configure the CloudWatch agent to collect this data. This can greatly enhance your visibility into the performance and health of your Elastic Beanstalk environment, allowing you to make informed decisions and optimize your application’s performance. How To Configure Custom Metrics in AWS Elastic Beanstalk To accomplish this, you’ll need to edit your Elastic Beanstalk zip bundle and include a cloudwatch.config file in the .ebextensions folder at the top of your bundle. Please note that the configuration file should be chosen based on your operating system, as described in this article. By doing so, you’ll be able to customize the CloudWatch agent settings and enable the collection of additional metrics, such as memory consumption, to gain deeper insights into your Elastic Beanstalk environment. This will allow you to effectively monitor and optimize the performance of your application on AWS. Linux-Based Config: files:
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }
container_commands:
  apply_config_metrics:
    command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json Windows-Based Config: files:
  "C:\\Program Files\\Amazon\\AmazonCloudWatchAgent\\cw-memory-config.json":
    content: |
{
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }

container_commands:
  01_set_config_and_reinitialize_cw_agent:
    command: powershell.exe cd 'C:\Program Files\Amazon\AmazonCloudWatchAgent'; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a append-config -m ec2 -c file:cw-memory-config.json -s; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a start; exit As you may have noticed, I enabled only a few memory-related metrics such as mem_total, mem_available, mem_used, mem_free, and mem_used_percent. However, you can enable more metrics as needed. The complete list of available metrics can be found here. Once you have updated your application, it would be beneficial to create a CloudWatch dashboard to visualize these metrics. To do so, navigate to the AWS CloudWatch console, select Dashboards, and click on Create dashboard. From there, you can create a widget by clicking the Add widget button and selecting Line to create a line chart that displays the desired metrics. Customizing a dashboard with relevant metrics can provide valuable insights into the performance and health of your Elastic Beanstalk environment, making it easier to monitor and optimize your application on AWS. In the case of the example above, we’ll see 5 new metrics in the section CWAgent. Based on them, we may configure a memory widget and get something like this. Final Thoughts The similar approach based on YAML configuration file can be used for a bunch of other AWS metrics, that can be used together with a variety of AWS widgets available in CloudWatch, which allow you to  customize your CloudWatch dashboard as you want. It’s interesting, that AWS doesn’t provide memory (and some other) metrics out-of-box for a such  developer-friendly service as ElasticBeanStalk. Anyway, using the configuration file described in the article you can easily fix it for all yours ElasticBeanStalk applications. If you have any questions or need assistance, feel free to ask me in the comments. Recently, I encountered a task where a business was using AWS Elastic Beanstalk but was struggling to understand the system state due to the lack of comprehensive metrics in CloudWatch. By default, CloudWatch only provides a few basic metrics such as CPU and Networks. However, it’s worth noting that Memory and Disk metrics are not included in the default metric collection. AWS Elastic Beanstalk Fortunately, each Elastic Beanstalk virtual machine (VM) comes with a CloudWatch agent that can be easily configured to collect additional metrics. For example, if you need information about VM memory consumption, which AWS does not provide out of the box, you can configure the CloudWatch agent to collect this data. This can greatly enhance your visibility into the performance and health of your Elastic Beanstalk environment, allowing you to make informed decisions and optimize your application’s performance. How To Configure Custom Metrics in AWS Elastic Beanstalk To accomplish this, you’ll need to edit your Elastic Beanstalk zip bundle and include a cloudwatch.config file in the .ebextensions folder at the top of your bundle. Please note that the configuration file should be chosen based on your operating system, as described in this article. By doing so, you’ll be able to customize the CloudWatch agent settings and enable the collection of additional metrics, such as memory consumption, to gain deeper insights into your Elastic Beanstalk environment. This will allow you to effectively monitor and optimize the performance of your application on AWS. cloudwatch.config .ebextensions Linux-Based Config: Linux-Based Config: files:
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }
container_commands:
  apply_config_metrics:
    command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json files:
  "/opt/aws/amazon-cloudwatch-agent/bin/config.json":
    mode: "000600"
    owner: root
    group: root
    content: |
      {
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }
container_commands:
  apply_config_metrics:
    command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -s -c file:/opt/aws/amazon-cloudwatch-agent/bin/config.json Windows-Based Config: Windows-Based Config: files:
  "C:\\Program Files\\Amazon\\AmazonCloudWatchAgent\\cw-memory-config.json":
    content: |
{
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }

container_commands:
  01_set_config_and_reinitialize_cw_agent:
    command: powershell.exe cd 'C:\Program Files\Amazon\AmazonCloudWatchAgent'; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a append-config -m ec2 -c file:cw-memory-config.json -s; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a start; exit files:
  "C:\\Program Files\\Amazon\\AmazonCloudWatchAgent\\cw-memory-config.json":
    content: |
{
        "agent": {
          "metrics_collection_interval": 60,
          "run_as_user": "root"
        },
        "metrics": {
          "append_dimensions": {
            "InstanceId": "$${aws:InstanceId}"
          },
          "metrics_collected": {
            "mem": {
              "measurement": [
                "mem_total",
                "mem_available",
                "mem_used",
                "mem_free",
                "mem_used_percent"
              ]
            }
          }
        }
      }

container_commands:
  01_set_config_and_reinitialize_cw_agent:
    command: powershell.exe cd 'C:\Program Files\Amazon\AmazonCloudWatchAgent'; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a append-config -m ec2 -c file:cw-memory-config.json -s; powershell.exe -ExecutionPolicy Bypass -File ./amazon-cloudwatch-agent-ctl.ps1 -a start; exit As you may have noticed, I enabled only a few memory-related metrics such as mem_total , mem_available , mem_used , mem_free , and mem_used_percent . However, you can enable more metrics as needed. The complete list of available metrics can be found here . mem_total mem_available mem_used mem_free mem_used_percent here Once you have updated your application, it would be beneficial to create a CloudWatch dashboard to visualize these metrics. To do so, navigate to the AWS CloudWatch console, select Dashboards , and click on Create dashboard . From there, you can create a widget by clicking the Add widget button and selecting Line to create a line chart that displays the desired metrics. Customizing a dashboard with relevant metrics can provide valuable insights into the performance and health of your Elastic Beanstalk environment, making it easier to monitor and optimize your application on AWS. Dashboards Create dashboard Add widget Line In the case of the example above, we’ll see 5 new metrics in the section CWAgent . CWAgent Based on them, we may configure a memory widget and get something like this. Final Thoughts The similar approach based on YAML configuration file can be used for a bunch of other AWS metrics, that can be used together with a variety of AWS widgets available in CloudWatch, which allow you to  customize your CloudWatch dashboard as you want. It’s interesting, that AWS doesn’t provide memory (and some other) metrics out-of-box for a such  developer-friendly service as ElasticBeanStalk. Anyway, using the configuration file described in the article you can easily fix it for all yours ElasticBeanStalk applications. If you have any questions or need assistance, feel free to ask me in the comments.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Implementing Geo-Location Redirects with AWS CloudFront

tenv v2.0: The Importance of Explicit Behavior for the Version Manager

AWS ElasticBeanStalk Custom Metrics Configuration Based on RAM Metrics Example

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Guide on How to Use GPU Nodes in Amazon EKS

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

A Guide on How to Use GPU Nodes in Amazon EKS

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps