: I wrote this article first for , to whom it belongs and it is shared in this blog under . Disclaimer semaphoreci.com community blog by-nc-sa 4.0 licence Introduction Docker changed how development, testing and running software are done, and that’s why software engineers and companies should develop new habits and follow best practices. On the other hand, orchestration technologies like Docker Swarm, K8S or Mesos are also changing depending on the industry demands. Security management, including secret management in Docker Swarm, is one of the things that have evolved as a reaction to these changes. When working with Docker and orchestration tools, following security best practices is important. Software is becoming more distributed and with the rise of new development models, e.g. Microservices Architecture, communication between software components is adopting new patterns like message-based communication. In a production environment, containers running different services may exchange some critical information. When deploying a software to different environments, it is evident that storing critical information like production passwords in the development environment is not a good practice. There are multiple scenarios where we can notice the need of a standardized interface for accessing secrets. Datacenter Operating System, Kubernetes and other orchestration technologies have their own integrated secrets management solutions. In this tutorial, we re going to discover Docker native secret management. Managing Secrets in Swarm Mode In this part of this tutorial, we are going to see how to create, attach and rotate sensitive data using Docker Secrets. How Docker Secrets Work Starting from the version 1.13, Docker users can use Docker Secrets in a Swarm cluster. Managers in Docker Swarm act as an authoritative delegation to coordinate secrets management. Secrets could be passwords, keys, certificates, sensitive environment data or any other custom data that a developer wants to protect e.g a database name, the administrator username, etc. Docker Secrets is only available in the Swarm mode, so standalone containers can not use this feature. Swarm Mode allows you to centrally manage sensitive data and messages by encrypting and transmitting them in a secure way and only to containers that need to access to it ( the principle of least-privilege). When a user adds a new secret to a Swarm cluster, this secret is sent to a manager using a TLS connection. Note: TLS is a cryptographic protocol that provides communications security over a network by providing communication encryption, privacy and data integrity. In order to make all managers up to date with a new secret, when one manager node receives the secret, it saves it to a Raft store with a 256-bit key. Creating a Secret In order to see how everything works, let’s create a new Docker machine: ```docker-machine create --driver virtualbox secrets-testing > Running pre-create checks...> (secrets-testing) Default Boot2Docker ISO is out-of-date, downloading the latest release...> (secrets-testing) Latest release for github.com/boot2docker/boot2docker is v17.05.0-ce> (secrets-testing) Downloading /home/eon01/.docker/machine/cache/boot2docker.iso from > (secrets-testing) 0%....10%....20%....30%....40%....50%....60%....70%....80%....90%....100%> Creating machine...> (secrets-testing) Copying /home/eon01/.docker/machine/cache/boot2docker.iso to /home/eon01/.docker/machine/machines/secrets-testing/boot2docker.iso...> (secrets-testing) Creating VirtualBox VM...> (secrets-testing) Creating SSH key...> (secrets-testing) Starting the VM...> (secrets-testing) Check network to re-create if needed...> (secrets-testing) Waiting for an IP...> Waiting for machine to be running, this may take a few minutes...> Detecting operating system of created instance...> Waiting for SSH to be available...> Detecting the provisioner...> Provisioning with boot2docker...> Copying certs to the local machine directory...> Copying certs to the remote machine...> Setting Docker configuration on the remote daemon...> Checking connection to Docker...> Docker is up and running!> To see how to connect your Docker Client to the Docker Engine running on this virtual machine, run: docker-machine env secrets-testing``` https://github.com/boot2docker/boot2docker/releases/download/v17.05.0-ce/boot2docker.iso... You should see the created machine using : ```docker-machine ls > NAME              ACTIVE   DRIVER       STATE     URL                         SWARM   DOCKER        ERRORS> secrets-testing   -        virtualbox   Running   tcp://192.168.99.100:2376           v17.05.0-ce``` Let’s see the machine variables: ```docker-machine env secrets-testing > export DOCKER_TLS_VERIFY="1"> export DOCKER_HOST="tcp://192.168.99.100:2376"> export DOCKER_CERT_PATH="/home/eon01/.docker/machine/machines/secrets-testing"> export DOCKER_MACHINE_NAME="secrets-testing" ``` If everything is ok, we can connect the current shell session to the new machine: ```eval "$(docker-machine env secrets-testing)"``` We can now start a Swarm cluster to run our tests. Using the IP address of the created machine, initialize the Swarm: ```docker swarm init --advertise-addr 192.168.99.100``` You should see something similar to: ```> Swarm initialized: current node (sqd5o5vj5c2r9gn53jd1uc4ig) is now a manager.>> To add a worker to this swarm, run the following command:>>     docker swarm join \>     --token SWMTKN-1-0atbw3c3geeyievi62dkgrw03pauh93rhfz41mdveo21ei1fb6-abf6azeaezaea \>     192.168.99.100:2377>> To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.``` Our manager is created, and you can create new machines and add new managers or workers of your choice. Now, let’s create a new random Docker secret. While the secret will be created using , a Docker secret is created using this command: date |md5sum|awk '{print $1}' docker secret create [OPTIONS] SECRET file|- We can run a single command to create a Docker secret: ```date |md5sum|awk '{print $1}' | docker secret create my_secret -> qk4k9adgri6b3ubjua9nca5fp``` The created secret is called . my_secret When you type, you can get a view of the used secrets: docker secret ls ```> ID                          NAME                CREATED              UPDATED> qk4k9adgri6b3ubjua9nca5fp   my_secret           About a minute ago   About a minute ago``` qk4k9adgri6b3ubjua9nca5fp is a random and unique identifier of the recently created secret. If you want to have more information about , you can use: my_secret docker secret inspect  my_secret and you will see a JSON output with different other information such as the creation or the update date. ```> [>     {>         "ID": "qk4k9adgri6b3ubjua9nca5fp",>         "Version": {>             "Index": 10>         },>         "CreatedAt": "2017-05-22T13:25:23.815462834Z",>         "UpdatedAt": "2017-05-22T13:25:23.815462834Z",>         "Spec": {>             "Name": "my_secret",>             "Labels": {}>         }>     }> ]``` Our secret is now stored in the encrypted Raft logs. Consuming Docker Secrets Let’s create a new service that will be using the created secret ( ). my_secret docker service create --name my_test_app --secret my_secret eon01/infinite eon01/infinite is a Docker image that I created for this kind of tests that uses Alpine to run an infinite loop. Now, we have a running service named using the secret my_secret. We can view it using or even . my_test_app docker service ls docker ps ```docker ps > CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS               NAMES> db38ae5f19fa        eon01/infinite:latest   "/bin/sh -c 'tail ..."   10 minutes ago      Up 10 minutes                           my_test_app.1.cc71i2rm0cl6pghn7z5ht7b98``` Only the created service is using the password, and since our service is running one container, only this container can see the secret. In other words, when a secret is attached to a service, only this service and its tasks have access to this sensitive data. The secret is now stored to a temporary file system (tmpfs) under /run/secrets/my_secret Tmpfs is the temporary file storage facility on many Unix-like operating systems stored in volatile memory instead of a persistent storage device. Since our container id is db38ae5f19fa, we can check the tmpfs content using: ```docker exec -it db38ae5f19fa cat /run/secrets/my_secret > 21ac8406497dded57ceafb7767e53633``` If you create other secrets for the same services, you can find them in the same directory under /run/secrets/<secret_name> Sharing Docker Secrets In order to see how sensitive data could be shared between two applications, we can create a new service called my_other_test_app and send it the created secret. In our case, my_other_test_app will use the same password but with a different name (my_other_secret): ```docker service create --name my_other_test_app --secret source=my_secret,target=my_other_secret,mode=0400 eon01/infinite``` We have set the mode to share the secret (0400) but it is possible also to add other options like UID or GID. Let’s get inside the container to verify the content of the create secret: ```docker exec -it 51fe104e416b cat /run/secrets/my_other_secret``` And we will find the same output as the previous command which is the first secret 21ac8406497dded57ceafb7767e53633. docker exec -it db38ae5f19fa cat /run/secrets/my_secret Rotating Docker Secrets Rotating sensitive data like passwords is a security best practice and it is possible using the Docker Swarm Secrets feature. Start by creating a new secret: ```date |md5sum|awk '{print $1}' | docker secret create my_new_secret -``` After that, the old secret should be deleted and updated by the new one and this is done in one command: docker service update --secret-rm my_secret --secret-add  source=my_new_secret,target=my_secret  my_test_app If you want to verify, you will find that the old secret (21ac8406497dded57ceafb7767e53633 in my case) has now a new value: docker exec -it 5152ba89ea7f cat /run/secrets/my_secret You can also inspect the new secret: ```docker secret inspect my_new_secret > [>     {>         "ID": "loame64czkbto3z87s37iy2ds",>         "Version": {>             "Index": 34>         },>         "CreatedAt": "2017-05-22T14:03:46.627206229Z",>         "UpdatedAt": "2017-05-22T14:03:46.627206229Z",>         "Spec": {>             "Name": "my_new_secret",>             "Labels": {}>         }>     }> ]``` Conclusion We have seen a native feature in Swarm, it was introduced in Docker 1.13 after realizing how much important secret management is. All recent releases of Docker CLI and API support this feature but only in the Swarm mode where secrets are stored in the Raft log of the associated Swarm cluster and stored in a temporary filesystem inside a container. Secret management with Docker is useful: sensitive data is immutable, never written to a disk and never sent in clear text to the network. Other orchestration technologies like Kubernetes have different models of security and secrets management. If security is important in your project, you should compare these different models and choose the best fit for you. Connect Deeper If you resonated with this article, you might be interested in my book: . Painless Docker: Unlock The Power Of Docker & Its Ecosystem You can also subscribe to : and : DevOpsLinks An Online Community Of Diverse & Passionate DevOps, SysAdmins & Developers From All Over The World Shipped An Independent Newsletter Focused On Containers & Orchestration. You can find me on and you can also check other books like : Twitter SaltStack For DevOps Extremely fast and simple IT automation and configuration management. If you liked this post, please recommend and share it to your followers.

Initialized

Target

Twitter

Read My Stories

Too Long; Didn't Read

In your car, at home, or at work — Bosch technology shapes many areas of life.

Managing Secrets in Docker Swarm

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Aymen (@eon01)

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

Managing Secrets in Docker Swarm

Too Long; Didn't Read

People Mentioned

Companies Mentioned

Aymen (@eon01)

About Author

TOPICS

THIS ARTICLE WAS FEATURED IN...

RELATED STORIES