

cloud architect, open-source developer
Congratulations! You know how to build a Docker image, and are able to compose multiple containers into a meaningful application. Hopefully, youโve already created a Continuous Delivery pipeline and know how to push your newly created image into production or testing environment.
Now, the question is
How do we test our Docker containers?
There are multiple testing strategies we can apply. In this post, Iโll highlight them presenting benefits and drawbacks for each.
This is the default approach for most people. It relies on a CI server to do the job. When taking this approach, the developer is using Docker as a package manager, a better option than the jar/rpm/deb approach. The CI server compiles the application code and executes tests (unit, service, functional, and others). The build artifacts are reused in Docker build to produce a new image. This becomes a core deployment artifact. The produced image contains not only application โbinariesโ, but also a required runtime including all dependencies and application configuration.
We are getting application portability, however, we are loosing the development and testing portability. Weโre not able to reproduce exactly the same development and testing environment outside the CI. To create a new test environment, weโll need to setup the testing tools (correct versions and plugins), configure runtime and OS settings, and get the same versions of test scripts as well as perhaps, the test data.
To resolve these problems leads us to the next one.
Here, we try to create a single bundle with the application โbinariesโ including required packages, testing tools (specific versions), test tools plugins, test scripts, test environment with all required packages.
The benefits of this approach:
This approach has significant drawbacks:
Hereโs a simplified Dockerfile. It illustrates this approach.
There has to be a better way for in-container testing and there is.
Today, Dockerโs promise is
Build -> Ship ->ย Run
build the image, ship it to some registry, and run it anywhere. IMHO thereโs a critical missing stepโโโTest.
The right and complete sequence should be:
Build -> Test -> Ship ->ย Run
Letโs look at a โtest-friendlyโ Dockerfile syntax and extensions to Docker commands. This important step could be supported natively. Itโs not a real syntax, but bear with me. Iโll define the โidealโ version and show how to implement something thatโs very close.
ONTEST [INSTRUCTION]
Letโs define a special ONTEST instruction, similar to existing ONBUILD instruction. The ONTEST instruction adds a trigger instruction to the image to be executed at a later time when the image is tested. Any build instruction can be registered as a trigger.
The ONTEST instruction should be recognized by a new docker test command.
docker test [OPTIONS] IMAGE [COMMAND] [ARG...]
The docker test command syntax will be similar to docker run command, with one significant difference: a new โtestableโ image will be automatically generated and even tagged with <image name>:<image tag>-test tag (โtestโ postfix added to the original image tag). This โtestableโ image will generated FROM the application image, executing all build instructions, defined after ONTEST command and executing ONTEST CMD (or ONTEST ENTRYPOINT). The docker test command should return a non-zero code if any tests fail. The test results should be written into an automatically generated VOLUME that points to /var/tests/results folder.
Letโs look at a modified Dockerfile belowโโโit includes the new proposed ONTEST instruction.
We believe Docker should make docker-test part of the container management lifecycle. There is a need to have a simple working solution today and Iโll describe one thatโs very close to the ideal state.
As mentioned before, Docker has a very useful ONBUILD instruction. This instruction allows us to trigger another build instruction on succeeding builds. The basic idea is to use ONBUILD instruction when running docker-test command.
The flow executed by docker-test command:
Why not create a new Dockerfile.test without requiring the ONBUILD instruction?
Because in order to test right image (and tag) weโll need to keep FROM always updated to image:tag that we want to test. This is not trivial.
There is a limitation in the described approachโโโitโs not suitable for โonbuildโ images (images used to automatically build your app), like Maven:onbuild
Letโs look at a simple implementation of docker-test command. It highlights the concept: the docker-test command should be able to handle build and run command options and be able to handle errors properly.
Letโs focus on the most interesting and relevant part.
Letโs say we have an application built from tens or hundreds of microservices. Letโs say we have an automated CI/CD pipeline, where each microservice is built and tested by our CI and deployed into some environment (testing, staging or production) after the build and tests pass. Pretty cool, eh? Our CI tests are capable of testing each microservice in isolationโโโrunning unit and service tests (or API contract tests). Maybe even micro-integration testsโโโtests run on subsystem are created in ad-hoc manner (for example with docker compose help).
This leads to some issues that we need to address:
There should be a better way than just dropping a new microservice version into production and tightly monitoring it for a while.
There should be a special Integration Test Container. These containers will contain only testing tools and test artifacts: test scripts, test data, test environment configuration, etc. To simplify orchestration and automation of such containers, we should define and follow some conventions and use metadata labels (Dockerfile LABEL instruction).
The Integration Test Container is just a regular Docker container. It does not contain any application logic and code. Its sole purpose is to create repeatable and portable testing. Recommended content of the Integration Test Container:
Integration Test Containers should run in an operational environment where all microservices are deployed: testing, staging or production. These containers can be deployed exactly as all other services. They use same network layer and thus can access multiple services; using the selected service discovery method (usually DNS). Accessing multiple services is required for real integration testingโโโwe need to simulate and validate how our system is working in multiple places. Keeping integration tests inside some application service container not only increases the container footprint, but also creates an unneeded dependency between multiple services. We keep all these dependencies at the level of the Integration Test Container. Once our tests (and testing tools) are packaged inside the container, we can always rerun the same tests on any environment including the developer machine. You can always go back in time and rerun a specific version of Integration Test Container.
WDYT? Your feedback, particularly on standardizing the docker-test command, is greatly appreciated.
Video and slides are available at Codefresh site
Create your free account to unlock your custom reading experience.