Stopping an EC2 instance that you don't need can save you money and improve security. In this post, I demonstrate how to use the AWS CDK to provision an EC2 instance that is automatically stopped when there are no open SSH or Session Manager connections to it.
These days I most often use an EC2 instance as a Bastion server. I use SSH or Session Manager to connect to it and access a protected database from there. This is usually just for occasional maintenance or debugging. An "always on" instance for this purpose feels wasteful since it would just sit idle most of the time. Because I only use it with an SSH (or Session Manager) connection we can automate the process of stopping it when there are no remaining connections. The primary downside of this is that I must manually start the instance (and wait for it to boot) when I need to use it. In my case, it is a tradeoff I'm happy to make.
Below I demonstrate how to use the AWS CDK to provision an EC2 instance that is automatically stopped when there are no open SSH or Session Manager connections to it. The full source code is available on GitHub at https://github.com/mpvosseller/cdk-ec2-autostop.
The approach we take is as follows:
- Install a bash script
report-metrics.shon the instance to determine whether it is "active" or not and publish the result as a CloudWatch metric. - Configure a cronjob on the instance to run
report-metrics.shonce a minute. - Configure a CloudWatch Alarm to monitor the metric and stop the instance when it is reported inactive for more than 15 minutes.
Report Metrics Script
Below is the report-metrics.sh script and it does most of the heavy lifting. It runs on the instance and determines whether it should be considered "active" or not. It then publishes a CloudWatch metric with the result. We consider the instance active if it has any open SSH or Session Manager connections to it or if the instance was recently booted.
We use various tools and services to accomplish this job:
-
The EC2 instance metadata service at 169.254.169.254 is used to determine the
availabilityZone,region, andinstanceIdof the instance. -
ec2 describe-instancesis used to get thestackNameto which the instance belongs. -
aws ssm describe-sessionsis used to count the number of Session Manager connections. -
ssis used to count the number of SSH connections. -
aws cloudwatch put-metric-datais used to publish our custom CloudWatch metrics. -
jqis used for parsing JSON API responses.
#!/bin/bash
# Publish a CloudWatch metric to report whether this instance should be considered active or not.
# We consider it active when there are any open SSH or Session Manager connections to it or if it
# was recently booted.
availabilityZone=$(curl -s http://169.254.169.254/latest/meta-data/placement/availability-zone)
region=$(echo "${availabilityZone}" | sed 's/[a-z]$//')
instanceId=$(curl -s http://169.254.169.254/latest/meta-data/instance-id)
stackName=$(aws --region "${region}" ec2 describe-instances --instance-id "${instanceId}" --query 'Reservations[*].Instances[*].Tags[?Key==`aws:cloudformation:stack-name`].Value' | jq -r '.[0][0][0]')
uptimeSeconds=$(cat /proc/uptime | awk -F '.' '{print $1}')
uptimeMinutes=$((uptimeSeconds / 60))
ssmConnectionCount=$(aws ssm describe-sessions --filters "key=Target,value=${instanceId}" --state Active --region "${region}" | jq '.Sessions | length')
sshConnectionCount=$(/usr/sbin/ss -o state established '( sport = :ssh )' | grep -i ssh | wc -l)
((totalConnectionCount = ssmConnectionCount + sshConnectionCount))
# note that "ssh over ssm" connections are double counted
isActive=0
if [ "${totalConnectionCount}" -gt 0 ] || [ "${uptimeMinutes}" -lt 15 ]; then
isActive=1
fi
metricNameSpace="${stackName}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "ConnectionCount" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${totalConnectionCount}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "UptimeMinutes" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${uptimeMinutes}"
aws --region "${region}" cloudwatch put-metric-data --metric-name "Active" --dimensions InstanceId="${instanceId}" --namespace "${metricNameSpace}" --value "${isActive}"
Crontab File
Below is the crontab file. Nothing exciting here. It is a standard crontab file that requests the report-metrics.sh script to be run every minute.
* * * * * /home/ec2-user/report-metrics.sh
CDK Application
Below is the main CDK application code. If you are familiar with the AWS CDK it should be mostly straightforward. The most important aspects are:
-
Install the
jqpackage on the instance because it is needed byreport-metrics.sh. -
Install the files
report-metrics.shandcrontabon the instance at/home/ec2-user/. -
Run the
crontabcommand to set up the cron job. -
Grant the permissions required for Session Manager and
report-metrics.sh. -
Create a CloudWatch Alarm that stops the instance when it is inactive over a 15 minute period.
import * as cloudwatch from '@aws-cdk/aws-cloudwatch'
import * as actions from '@aws-cdk/aws-cloudwatch-actions'
import * as ec2 from '@aws-cdk/aws-ec2'
import * as iam from '@aws-cdk/aws-iam'
import { CfnOutput, Construct, Duration, Stack, StackProps } from '@aws-cdk/core'
export class CdkEc2AutostopStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props)
const vpc = ec2.Vpc.fromLookup(this, 'Vpc', {
isDefault: true,
})
const keyName = '' // for ssh you must enter the name of your ec2 key pair
const instance = new ec2.Instance(this, 'Instance', {
init: ec2.CloudFormationInit.fromConfig(
new ec2.InitConfig([
ec2.InitPackage.yum('jq'),
ec2.InitFile.fromFileInline(
'/home/ec2-user/report-metrics.sh',
'./assets/report-metrics.sh',
{
owner: 'ec2-user',
group: 'ec2-user',
mode: '000744',
}
),
ec2.InitFile.fromFileInline('/home/ec2-user/crontab', './assets/crontab', {
owner: 'ec2-user',
group: 'ec2-user',
mode: '000444',
}),
ec2.InitCommand.shellCommand('sudo -u ec2-user crontab /home/ec2-user/crontab'),
])
),
instanceName: 'AutoStopInstance',
vpc,
keyName: keyName || undefined,
instanceType: ec2.InstanceType.of(ec2.InstanceClass.T3, ec2.InstanceSize.NANO),
machineImage: ec2.MachineImage.latestAmazonLinux({
generation: ec2.AmazonLinuxGeneration.AMAZON_LINUX_2,
cpuType: ec2.AmazonLinuxCpuType.X86_64,
}),
})
// WARNING: this opens port 22 (ssh) publicly to any IPv4 address
// instance.connections.allowFrom(ec2.Peer.anyIpv4(), ec2.Port.tcp(22), 'SSH access');
// permissions for session manager
instance.addToRolePolicy(
new iam.PolicyStatement({
actions: ['ssmmessages:*', 'ssm:UpdateInstanceInformation', 'ec2messages:*'],
resources: ['*'],
})
)
// permissions for report-metrics.sh script
instance.addToRolePolicy(
new iam.PolicyStatement({
actions: ['ec2:DescribeInstances', 'ssm:DescribeSessions', 'cloudwatch:PutMetricData'],
resources: ['*'],
})
)
const alarm = new cloudwatch.Alarm(this, 'Alarm', {
alarmName: `Idle Instance - ${this.stackName}`,
metric: new cloudwatch.Metric({
// this metric is generated by report-metrics.sh
namespace: this.stackName,
metricName: 'Active',
dimensions: {
InstanceId: instance.instanceId,
},
statistic: 'maximum',
period: Duration.minutes(15),
}),
comparisonOperator: cloudwatch.ComparisonOperator.LESS_THAN_THRESHOLD,
threshold: 1,
evaluationPeriods: 1,
treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
})
alarm.addAlarmAction(new actions.Ec2Action(actions.Ec2InstanceAction.STOP))
new CfnOutput(this, 'InstanceId', {
description: 'Instance ID of the host. Use this to connect via SSM Session Manager',
value: instance.instanceId,
})
}
}
Once deployed, I get an EC2 instance that can be manually started when needed and it will be automatically stopped when there are no remaining SSH or Session Manager connections to it. This lets me sleep just a tiny bit better at night.
The full code can be found on GitHub at https://github.com/mpvosseller/cdk-ec2-autostop.
If you found this helpful or have some feedback please let me know.