Automating EBS snapshots in AWS

A new feature in the AWS console lets you automate backups for all your important volumes in 5 minutes. Here’s how you do it.

Just as S3 buckets are left wide open for the world unintentionally, there are plenty of production EBS volumes out there with no backup policy in place. Many newer AWS users are surprised to find there is no automatic backup for EBS, and unfortunately the first disaster happens when they accidentally lose a volume.

Not long ago, the easy way to create EBS backups — aka snapshots — was to use a little Python scripting with a cron job on each instance. Lifecycle management required a bit more plumbing and making sure every instance was covered needed some coordinated effort.

The good news is AWS just made this much, much easier. From your EC2 dashboard, you’ll see a new option ‘Lifecycle Manager’ that recently appeared in the bottom left:

Why do you need lifecycle management for snapshots?

Snapshots are incremental backups that only save the differences between backup states — this makes it faster to create a snapshot than to carry out a full backup.

The problem is that AWS limits you to 10,000 snapshots per account, and while that’s pretty generous, it’s possible to hit the limit if you run daily snapshots across a fleet of instances. Also, depending on your backup needs, you might not need to keep more than a handful of snapshots per instance, so it makes sense to delete older snapshots.

Prepping before you start.

Thankfully, AWS has made this extremely easy for us, since you can create a policy that targets your entire inventory of volumes by using tagging. For example, you can create a tag called ‘Environment’ with the value ‘Production’ and ensure all your production volumes are included in the backup policy.

Alternatively, you could tag by project, client, department or whatever other grouping makes sense for your business. Once you have added the tags to your volumes, you are ready for the next step.

Creating the lifecycle policy.

In the Lifecycle Manager screen, you simply need to provide a description for the policy, the name of the volume tags you are targeting, and a schedule for backups. You can also include how many snapshots are retained per volume:

When you are ready, hit the Create button and your policy is live. The policy does not run instantly — you will have to wait until the first window defined in your rules.

Back in the main Lifecycle Manager window, you can then see a summary of your policy:

You are not charged for this service beyond the storage, and while your instance load and IO usage will increase during a backup, there’s nothing stopping you running snapshots every hour if really needed.

Remember: Lifecycle Manager is a per-region service — policies set in one region won’t back up volumes in another.

Do it today!

If you have EBS volumes in production, check to see how many are included in a backup schedule. In every production environment I’ve seen, there are always a handful excluded (by accident) and it can be disastrous when there is a failure or accidental instance termination.

You can also enforce tagging rules by using AWS Config to help watch for any production volumes missing tags in the future. This is a much easier way to manage large numbers of EBS volumes or more complex AWS environments, and make sure all your production volumes are always backed up. Config also makes it snap to prove to auditors that backups are actually happening.