Many software projects use secrets - usually, keys to external APIs or credentials to access an external resource such as a database. Your application needs these keys at runtime, so you need to be able to provide them when you deploy your application, or as a step in preparing your deployment environment.
In this article, I'm going to show you how to use git-crypt so that you can safely keep your application secrets in your source code repositories, even if they're public.
Most projects have some sort of secret keys or credentials. For example, if your application is hosted on Heroku, you might provide an API key to your Heroku application using a command like this:
$ heroku config:set API_KEY=my-sooper-sekrit-api-key
By running this command before you (re)deploy your application, you give it an environment variable at runtime called API_KEY with the value my-sooper-sekrit-api-key. However, keeping track of these secret values outside of Heroku (or wherever you deploy your application) is still a challenge.
I always try to set up my projects so that I can run a single command to deploy them from scratch without any separate, manual steps. For our example, this means I need to store the value my-sooper-sekrit-api-key somewhere so that my deployment code can use it (in this case, to run the heroku config:set... the command above).
My project source code is always stored in git and usually hosted on github.com or bitbucket.com or some other source code hosting service. I could store my API_KEY value in my source code repository, however, there are some downsides to this:
I could store my secrets somewhere separate from my application source code, but this has its own problems:
Git-crypt aims to solve this problem by encrypting your secrets whenever you push them to your git repository, and decrypting them whenever you pull them. This happens transparently, from your point of view. So the secrets are in cleartext as far as you and your deployment code are concerned, but nobody else can read them, even if your source code is in a public Github repository.
Let's look at an example.
1. Install git-crypt.
There are instructions for Linux, Mac, and Windows on the git-crypt install page
If like me, you're using a Mac with Homebrew installed, you can run:
$ brew install git-crypt
2. Create a new git repository.
$ mkdir myproject $ cd myproject $ git init $ echo "This is some text" > file.txt $ git add file.txt $ git commit -m "Initial commit"
Now we have a git repository containing a single text file.
$ git-crypt init
You should see the output:
Before we do anything else, please run the following command:
$ git-crypt export-key ../git-crypt-key
This command creates a copy of the git-crypt symmetric key that was generated for this repository. We're putting it in the directory above this repository so that we can re-use the same key across multiple git repositories.
By default, git-crypt stores the generated key in the file .git/git-crypt/keys/default so you can achieve the same result by running cp .git/git-crypt/keys/default ../git-crypt-key
This git-crypt-key file is important. It's the key that can unlock all the encrypted files in our repository. We'll see how to use this key later on.
Imagine our application needs an API key, and we want to store it in a file called api.key.
Before we add that file to our repository, we will tell git-crypt that we want the api.key file to be encrypted whenever we commit it.
We do that using the .gitattributes file. This is a file we can use to add extra metadata to our git repository. It's not specific to git-crypt, so you might already have a .gitattributes file in your repository. If so, just add the relevant lines—don't replace the whole file.
In our case, we don't have a .gitattributes file, so we need to create one. The .gitattributes file contains lines of the form:
[file pattern] attr1=value1 attr2=value2
For git-crypt, the file pattern needs to match all the files we want git-crypt to encrypt, and the attributes are always the same: filter and diff, both of which we set to git-crypt.
So, our .gitattributes file should contain this:
api.key filter=git-crypt diff=git-crypt
Create that file, and add and commit it to your git repository:
$ echo "api.key filter=git-crypt diff=git-crypt" > .gitattributes $ git add .gitattributes $ git commit -m "Tell git-crypt to encrypt api.key"
I've used the literal filename api.key in my .gitattributes file, but it can be any file pattern that includes the file(s) you want to encrypt, so I could have used *.key, for instance. Alternatively, you can just add a line for each file you want to encrypt.
It can be easy to make a mistake in your .gitattributes file if you're trying to protect several files with a single pattern entry. So, I strongly recommend reading this section of the git-crypt README, which highlights some of the common gotchas.
Now that we have told git-crypt we want to encrypt the api.key file, let's add that to our repository.
It's always a good idea to test your setup by adding a dummy value first, and confirming that it's successfully encrypted, before committing your real secret.
$ echo "dummy value" > api.key
We haven't added api.key to git yet, but we can check what git-crypt is going to do by running:
$ git-crypt status
You should see the following output:
encrypted: api.key not encrypted: .gitattributes not encrypted: file.txt
So, even though the api.key file has not yet been committed to our git repository, this tells you that git-crypt is going to encrypt it for you.
Let's add and commit the file:
$ git add api.key $ git commit -m "Added the API key file"
We've told git-crypt to encrypt, and we've added api.key to our repository. However, if we look at, nothing seems different:
$ cat api.key dummy value
The reason for this is that git-crypt transparently encrypts and decrypts files as you push and pull them to your repository. So, the api.key file looks like a normal, cleartext file.
$ file api.key api.key: ASCII text
One way to confirm that your files really are being encrypted is to push your repository to GitHub. When you view the api.key file using the GitHub web interface, you'll see that it's an encrypted binary file rather than text.
An easier way to see how the repository would look to someone without the decryption key is to run:
$ git-crypt lock
Now if we look at our api.key file, things are different:
$ file api.key api.key: data $ cat api.key GITCRYPTROܮ7y\R*^
You will see some different garbage output to what I get, but it's clear the file is encrypted. This is what would be stored on GitHub.
To go back to having a cleartext api.key file, run:
$ git-crypt unlock ../git-crypt-key
The ../git-crypt-key the file is the one we saved earlier using git-crypt export-key...
Let's do a quick review of where we are now.
The git-crypt-key the file is very important. Without it, you won't be able to decrypt any of the encrypted files in your repository. Anyone who has a copy of that file has access to all of the encrypted secrets in your repository. So you need to keep that file safe and secure.
We used git-crypt init and git-crypt export-key to create our git-crypt-key file. But, if we have to have a separate key file for each of our repositories, then we haven't improved our secret management very much.
Fortunately, it's very easy to use the same git-crypt key file for multiple git repositories.
To use an existing key file, just use git-crypt unlock instead of git-crypt init when you set up your git repository to use git-crypt, like this:
$ mkdir my-other-project # At the same directory level as `myproject` $ cd my-other-project $ git init $ echo "Something" > file.txt $ git add file.txt $ git commit -m "initial commit" $ git-crypt unlock ../git-crypt-key
If you run the git-crypt unlock command before adding any files to your git repository, you will see a message like this:
fatal: You are on a branch yet to be born Error: 'git checkout' failed git-crypt has been set up but existing encrypted files have not been decrypted
This still works just fine, but it's a bit confusing, so I made sure to add and commit at least one file before running git-crypt unlock...
Re-using your git-crypt key file is convenient, but it does mean that if anyone else gets a copy of your key file, all of your encrypted secrets are exposed.
This is the same kind of security trade-off as using a password manager like LastPass or 1password. Rather than managing multiple secrets (passwords), each with its own risk of exposure, you keep them all in a secure store and use a single master password to unlock that.
The idea here is that it's easier to manage one important secret than many lesser secrets.
Git-crypt is a great way to keep the secrets your applications need right in the git repository, alongside the application source code. However, like every other security measure, it's not always going to be appropriate or advisable.
Here are some things to consider to decide whether it's the right solution for your particular project:
There is more information in this section of the git-crypt README.
Rather than managing your git-crypt key file directly, there is a better way to manage encrypted repositories by integrating git-crypt with gpg, so that you can use your gpg private key to decrypt the git repository. This also allows you to add multiple collaborators to a git repository without transmitting any secrets between the parties. However, this requires a more complicated setup, so we'll save that for another article.