Express Setup: Golang and Testcontainers Unwrapped

Hi there!

What is this article about?

In this article, I want to discuss test containers and Golang, how to integrate them into a project, and why it is necessary.

Testcontainers review

Testcontainers is a tool that enables developers to utilize Docker containers during testing, providing isolation and maintaining an environment that closely resembles production.

Why do we need to use it? Some points:

Importance of Writing Tests

— Ensures code quality by identifying and preventing errors.

— Facilitates safer code refactoring.

— Acts as documentation for code functionality.

Introduction to Testcontainers

— Library for managing Docker containers within tests.

— Particularly useful when applications interact with external services.

— Simplifies the creation of isolated testing environments.

Support Testcontainers-go in Golang

— Port of the Testcontainers library for Golang.

— Enables the creation and management of Docker containers directly from tests.

— Streamlines integration testing by providing isolated and reproducible environments.

— Ensures test isolation, preventing external factors from influencing results.

— Simplifies setup and teardown of containers for testing.

— Supports various container types, including databases, caches, and message brokers.

Integration Testing

— Offers isolated environments for integration testing.

— Convenient methods for starting, stopping, and obtaining container information.

— Facilitates seamless integration of Docker containers into the Golang testing process.

So, the key point to highlight is that we don't preconfigure the environment outside of the code; instead, we create an isolated environment from the code. This allows us to achieve isolation for both individual tests and all tests at once. For example, within integration tests, we can set up a single MongoDB for all tests and work with it. However, if we need to add Redis for a specific test, we can do so through the code.

Let's try.

Let’s explore its application through an example of a portfolio management service developed in Go.

Service Description

The service is a REST API designed for portfolio management, utilizing MongoDB as the data storage and Redis for caching queries. This ensures fast data access and reduces the load on the primary storage.

Technologies

Go — Programming language used for developing the service.

MongoDB — Document-oriented database employed for storing portfolio data.

Docker and Docker Compose — Used for containerization and local deployment of the service and its dependencies.

Testcontainers-go — Library for integration testing using Docker containers in Go tests.

Configuration and Launch

Make sure you have:

- Docker

- Docker Compose

- Go (version 1.16 or higher)

Testing using Testcontainers

Testcontainers allows integration testing of the service under conditions closely resembling a real environment, using Docker containers for dependencies. Let’s provide an example of a function to launch a MongoDB container in tests:

func RunMongo(ctx context.Context, t *testing.T, cfg config.Config) testcontainers.Container {
 mongodbContainer, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
  ContainerRequest: testcontainers.ContainerRequest{
   Image:        mongoImage,
   ExposedPorts: []string{listener},
   WaitingFor:   wait.ForListeningPort(mongoPort),
   Env: map[string]string{
    "MONGO_INITDB_ROOT_USERNAME": cfg.Database.Username,
    "MONGO_INITDB_ROOT_PASSWORD": cfg.Database.Password,
   },
  },
  Started: true,
 })
 if err != nil {
  t.Fatalf("failed to start container: %s", err)
 }

 return mongodbContainer
}

And a part of the example:

package main_test

import (
 "context"
 "testing"

 "github.com/testcontainers/testcontainers-go"
 "github.com/testcontainers/testcontainers-go/wait"
)

func TestMongoIntegration(t *testing.T) {
 ctx := context.Background()

 // Replace cfg with your actual configuration
 cfg := config.Config{
  Database: struct {
   Username   string
   Password   string
   Collection string
  }{
   Username:   "root",
   Password:   "example",
   Collection: "test_collection",
  },
 }

 // Launching the MongoDB container
 mongoContainer := RunMongo(ctx, t, cfg)
 defer mongoContainer.Terminate(ctx)

 // Here you can add code for initializing MongoDB, for example, creating a client to interact with the database

 // Here you can run tests using the started MongoDB container
 // ...

 // Example test that checks if MongoDB is available
 if err := checkMongoAvailability(mongoContainer, t); err != nil {
  t.Fatalf("MongoDB is not available: %s", err)
 }

 // Here you can add other tests in your scenario
 // ...
}

// Function to check the availability of MongoDB
func checkMongoAvailability(container testcontainers.Container, t *testing.T) error {
 host, err := container.Host(ctx)
 if err != nil {
  return err
 }

 port, err := container.MappedPort(ctx, "27017")
 if err != nil {
  return err
 }

 // Here you can use host and port to create a client and check the availability of MongoDB
 // For example, attempt to connect to MongoDB and execute a simple query

 return nil
}

How to run tests:

go test ./… -v

Now, it is necessary to relocate the operation of the MongoDB Testcontainer into the primary test method. This adjustment allows for the execution of the Testcontainer a single time.

var mongoAddress string

func TestMain(m *testing.M) {
	ctx := context.Background()
	cfg := CreateCfg(database, collectionName)

	mongodbContainer, err := RunMongo(ctx, cfg)
	if err != nil {
		log.Fatal(err)
	}
	defer func() {
		if err := mongodbContainer.Terminate(ctx); err != nil {
			log.Fatalf("failed to terminate container: %s", err)
		}
	}()

	mappedPort, err := mongodbContainer.MappedPort(ctx, "27017")
	mongoAddress = "mongodb://localhost:" + mappedPort.Port()

	os.Exit(m.Run())
}

And now our test should be:

func TestFindByID(t *testing.T) {
	ctx := context.Background()
	cfg := CreateCfg(database, collectionName)

	cfg.Database.Address = mongoAddress

	client := GetClient(ctx, t, cfg)
	defer client.Disconnect(ctx)

	collection := client.Database(database).Collection(collectionName)

	testPortfolio := pm.Portfolio{
		Name:    "John Doe",
		Details: "Software Developer",
	}
	insertResult, err := collection.InsertOne(ctx, testPortfolio)
	if err != nil {
		t.Fatal(err)
	}

	savedObjectID, ok := insertResult.InsertedID.(primitive.ObjectID)
	if !ok {
		log.Fatal("InsertedID is not an ObjectID")
	}

	service, err := NewMongoPortfolioService(cfg)
	if err != nil {
		t.Fatal(err)
	}

	foundPortfolio, err := service.FindByID(ctx, savedObjectID.Hex())
	if err != nil {
		t.Fatal(err)
	}

	assert.Equal(t, testPortfolio.Name, foundPortfolio.Name)
	assert.Equal(t, testPortfolio.Details, foundPortfolio.Details)
}

Ok, but if we already have everything inside makefile?

Let's figure it out – what advantages do test containers offer now? Long before, we used to write tests and describe the environment in a makefile, where scripts were used to set up the environment. Essentially, it was the same Docker-compose and the same environment setup, but we did it in one place and for everyone at once. Does it make sense for us to migrate to test containers?

Let's conduct a brief comparison between these two approaches.

Isolation and Autonomy

Testcontainers in tests ensures the isolation of the testing environment. Each test launches its container, guaranteeing that changes made by one test won’t affect others.

Ease of Configuration and Management

Testcontainers provides simplicity in configuring and managing containers. You don’t need to write complex Makefile scripts for deploying databases; instead, you can use the straightforward Testcontainers API within your tests.

Automation and Integration with Test Suites

Utilizing Testcontainers enables the automation of container startup and shutdown within the testing process. This easily integrates into test scenarios and frameworks.

Quick Test Environment Setup

Launching containers through Testcontainers is swift, expediting the test environment preparation process. There’s no need to wait for containers to be ready, as is the case when using a Makefile.

Enhanced Test Reliability

Starting a container in a test brings the testing environment closer to reality. This reduces the likelihood of false positives and increases the reliability of tests.

Conclusion

In conclusion, it's worth mentioning that delaying transitions from old approaches to newer and simpler ones is not advisable. Often, this leads to the accumulation of significant complexity and requires ongoing maintenance. Most of the time, our scripts set up an entire test environment right on our computers, but why? In the test environment, we have everything - Kafka, Redis, and Istio with Prometheus. Do we need all of this just to run a couple of integration tests for the database? The answer is obviously no.

The main idea of such tests is complete isolation from external factors and writing them as close to the subject domain and integrations as possible. As practice shows, these tests fit well into CI/CD under the profile or stage named e2e, allowing them to be run in isolation wherever you have Docker!

Thank you for your time, and I wish you the best of luck! I hope the article proves helpful for you!

Code:

Read more:

https://testcontainers.com

https://java.testcontainers.org/modules/databases/mongodb/

https://testcontainers.com/modules/mongodb/

Also published here.