Before we dive into the fun part of getting keys shared amongst cloud providers, there are a variety of tools required to get this tutorial working. Tools and Setup First, you’ll need to , then get it . You will also need to install and — these usually comes pre-installed with most Linux OSs, and are available via most package managers ( , , , / , etc.). download and install Vault up and running cURL OpenSSL apt yum brew choco scoop Our examples also use and which are part of the and packages under Ubuntu; you can either find a similar package for your OS or find a manual workaround for those portions. Next, install the (CLI) and make sure you to connect to your account. The last step is to and the Heroku CLI. head diff coreutils diffutils AWS command line tools configure the CLI install configure One last note — the Heroku feature to utilize keys from AWS requires a private or shield database plan, so please ensure your account has been configured accordingly. Intro In today’s hyperconnected world, the former approach of locking services behind Virtual Private Networks (VPNs) or within a demilitarized zone (DMZ) is no longer secure. Instead, we must operate on a zero-trust network model, where every actor must be assumed as malicious. This means that a focus on encryption — both at rest and in transit — along with identity and access management is critical to ensuring that systems can interact with each other. One of the most important parts of the encryption process is the keys used to encrypt and decrypt information or used to validate identity. A recent approach to this need is called Bring Your Own Key (BYOK) — where you as the customer/end user own and manage your key, and provide it to third parties (notably cloud providers) for usage. However, before we dig into what BYOK is and how we can best leverage it, let’s have a quick recap on key management. Key Management At a high level, key management is the mechanism by which keys are generated, validated, and revoked — manually and as part of workflows. Another function of key management is ensuring that the root certificate that is used as a source of all truth is kept protected at a layer below other certificates, since revoking a root certificate would render the entire tree of certificates issued by it invalid. One of the more popular tools used for key management is — specifically designed for a world of low trust and dynamic infrastructure, where key ages can be measured in minutes or hours, rather than years. It includes functionality to manage secrets, encryption, and identity-based access, provides many ways to interact with it (CLI, API, web-based UI), and can connect to many different providers through plugins. This article will not focus on how to deploy Vault in a secure fashion, but the use cases that Vault can offer around BYOK and now to consume the keys in multiple cloud environments. HashiCorp’s Vault A key feature of using Vault is that it functions in an infrastructure- and provider-agnostic fashion — it can be used to provision and manage keys across different systems and clouds. At the same time, Vault can be used to encrypt and decrypt information without exposing keys to users, allowing for greater security. BYOK on Multi Cloud At this point, we’d like to dive into a specific use-case — demonstrating how you can create and ingest your own keys into multiple clouds, focusing on Amazon Web Services (AWS) and Heroku. For our purposes, we’ll start with uploading our keys to AWS KMS using Amazon’s CLI and demonstrating how the keys can be used within AWS. We will then rotate the keys manually — ideally, this is automated in a production implementation. Finally, we will utilize the generated keys in Heroku to encrypt a Postgres database. 📝 Note Before we get started, ensure that Vault is and , and you have followed the , since we’ll use that to generate the keys we upload to AWS. Once you have validated that Vault is running and the , it is time to get started. installed running Vault transit secrets engine setup guide AWS CLI is installed Create Key in Vault First, we need to create our encryption key within Vault: vault -f transit byok-demo-key exportable= allow_plaintext_backup= write /keys/ true true To export the key, we can use the Vault UI ( ), or we’ll need to use cURL, since the Vault CLI doesn’t support exporting it directly. The 127.0.0.1 address maps to the Vault server — a production setup would not be localhost, nor unencrypted. http://localhost:8200/ui/vault/secrets/transit/actions/byok-demo-key?action=export curl — header “X-Vault-Token: &lt;token&gt;” <a = = = = = >http://127.0.0.1:8200/v1/transit/export/encryption-key/byok-demo-key/1</a> href "http://127.0.0.1:8200/v1/transit/export/encryption-key/byok-demo-key/1" data-href "http://127.0.0.1:8200/v1/transit/export/encryption-key/byok-demo-key/1" class "markup--anchor markup--pre-anchor" rel "noopener" target "_blank" This command will output a base64 encoded plaintext version of the key which we will upload to AWS KMS. Save the base64 plaintext key in a file — we used . vault_key.b64 Upload Key to AWS KMS Now, we need to generate a Customer Master Key (CMK) with no key material: # create the key — — “ — aws kms create-key origin EXTERNAL description BYOK Demo” key-usage ENCRYPT_DECRYPT # give it a nice name — — & aws kms create-alias alias-name alias/ byok-demo- key target-key- id lt; from above& gt; Copy down the key ID from the output, and download the public key and import token: — & — — aws kms get-parameters-for-import key-id lt; from above& gt; wrapping-algorithm RSAES_OAEP_SHA_1 wrapping-key- spec RSA_2048 Copy the public key and import token from the output of the above step into separate files (imaginatively, we used and as the filenames), then base64 decode them: import_token.b64 public_key.b64 openssl - -base64 -A - public_key.b64 - public_key.bin enc d in out openssl enc -d -base -A - token.b - token.bin 64 in import _ 64 out import _ With the import token and public key downloaded, we can now use them to wrap the key from vault, first by converting the key to the OpenSSL byte format, then encrypting it using the public key from KMS. # convert the vault key to bytes openssl - -base64 -A - vault_key.b64 - vault_key.bin enc d in out # encrypt the vault key with the KMS key openssl rsautl -encrypt - vault_key -oaep -inkey public_key -keyform DER -pubin -out encrypted_vault_key.bin in .bin .bin # the key KMS import encrypted into aws kms -key-material — key-id &lt; above&gt; — -key-material fileb://encrypted_vault_key.bin — -token fileb://import_token.bin — expiration-model KEY_MATERIAL_EXPIRES — - $( — iso =ns — =’ days’) import from encrypted import valid to date -8601 date 364 Now that the key has been uploaded, we can quickly encrypt and decrypt via the CLI to validate that the key is functioning properly: # some random text to a file generate head /dev/urandom | tr -dc A-Za-z0– | head -c &gt; encrypt_me.txt 9 1024 # encrypt the file aws kms encrypt — -id ; above ; — plaintext fileb://encrypt_me.txt — text — query CiphertextBlob | base64 — decode ; encrypted_file.bin key &lt from &gt output &gt # decrypt the file aws kms decrypt — -id ; above ; — ciphertext-blob fileb://encrypted_file.bin — text — query Plaintext | base64 — decode ; decrypted.txt key &lt from &gt output &gt # validate they match; should be a blank line diff encrypt_me .txt decrypted .txt At this point, the key is in place and can be used to encrypt data at rest or in transit in different parts of AWS. One of the easiest and most important places to encrypt data is in S3; files sitting around in storage mandate being encrypted. When creating a bucket, make sure you enable server-side encryption, select KMS as the key type, then choose the specific key you created from the KMS master key dropdown: At this point, any objects that are put in the bucket will automatically be encrypted and then decrypted when they are read. This same approach can be used for encrypting databases — for a complete list of the different services that integrate with KMS, you can see the list . here Use KMS Key in Heroku Our next step is to use the key we generated and uploaded to AWS in Heroku! For our purposes, we’ll encrypt a Postgres database during creation. To do this, we need to grant Heroku’s AWS account access to the key that we created. This can be done via the AWS UI when creating a key, via the CLI, or via a policy. During the key creation wizard in AWS, step four will ask to define key usage permissions; there will be a separate section that allows you to add AWS account IDs to the key: Type in the Heroku AWS account ID ( ) and finish the wizard. If you want to use the CLI to achieve this, you need to update the default policy for the key: 021876802972 # get the existing key policy aws kms get-key-policy — policy-name — key-id &lt; above&gt; — output text default from Save the output from the above into a text file called and add the following two statements: heroku-kms-policy.json { : , : , : { : }, : [ , , , , ], : }, { : , : , : { : }, : [ , , ], : , : { : { : } } } "Sid" "Allow use of the key" "Effect" "Allow" "Principal" "AWS" "arn:aws:iam::021876802972:root" "Action" "kms:Encrypt" "kms:Decrypt" "kms:ReEncrypt*" "kms:GenerateDataKey*" "kms:DescribeKey" "Resource" "*" "Sid" "Allow attachment of persistent resources" "Effect" "Allow" "Principal" "AWS" "arn:aws:iam::021876802972:root" "Action" "kms:CreateGrant" "kms:ListGrants" "kms:RevokeGrant" "Resource" "*" "Condition" "Bool" "kms:GrantIsForAWSResource" "true" Now, update the existing policy with the new statements: aws kms put-key- — key-id &lt; above&gt; — - — file://heroku-kms- .json policy from policy name default policy policy There is no output from the above, so re-run the command to validate that it worked. get-key-policy To update the policy via the UI, browse to the key under the “Customer managed keys” section in the AWS KMS console and edit to add the two statements from above. When switching from policy to default view, the “Other AWS accounts” section should show up: With Heroku granted access to the key, we now can use the Heroku CLI to attach a Postgres database to an app, specifying the key ARN (which should look something like arn:aws:kms:region:account ID:key/key ID): heroku addons: heroku-postgresql: — encryption- CMK_ARN — app your-app- create private -7 key name You can find a on encrypting your Postgres database on Heroku, including information on how to encrypt an already existing database. full set of documentation Cleaning Up At this point, given the cost of running a private or shield database in Heroku, you may want to delete any resources you have created. Similarly, removing the keys from AWS is suggested, since it will take seven days for them to be fully deprovisioned and deleted. Quitting the Vault dev server will remove all information, since the dev instance is ephemeral. Considerations While there is a lot to like about the BYOK approach to AWS and Heroku, there are a few considerations that need to be highlighted: While Vault allows for key rotation, it requires external automation — cron, a CI pipeline, Nomad/Kubernetes jobs, etc. Similarly, customer managed keys in AWS also require . manual rotation If there is a , it is possible that the key will need to be re-imported; automating, testing, and validating this should be part of any production use case. catastrophic failure in AWS If keys are deleted in AWS while being used in Heroku, . all services and servers that depend on that key will be shutdown Conclusion With all of this in place, we have demonstrated that maintaining local ownership of your encryption keys is both possible and desirable, and that ownership can then be extended into various cloud providers. This article only scratched the surface of the functionality offered by Vault, KMS, and Heroku — Vault itself can be used to manage asymmetric encryption as well, helping to protect data in transit along with identity validation. KMS is a cornerstone for encryption in AWS, and hooks into most services, allowing for an effortless way to ensure data is kept secure. Finally, Heroku’s ability to consume KMS keys directly allows for an additional level of security without adding management overhead. Read Behind a Paywall here

Amazon

Heroku

Target

Transit

Sharing Your (Encryption) Keys Across Multi/Hybrid Clouds

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

How the KubeMQ Messaging Platform Uses Operators to Succeed

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

How the KubeMQ Messaging Platform Uses Operators to Succeed

101 Stories To Learn About Cloud Infrastructure

10 Things in Engineering We Don't Spend Enough Time On

10 Things I Did To Increase CloudTrail Logs Security

10 reasons to give cloud computing a go

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps