586 reads

Application Testing In Go: Do It Right And Don't Create a Mess

by Mohammed GadiwalaMay 4th, 2020

Too Long; Didn't Read

Using Docker, we can easily solve this problem by spawning a Docker container just to run test cases and killing it after it’s purpose is solved. No need to worry about the collateral damage done by our test cases since killing that container handles that for us. I have created a simple todo app that basically allows a user to save, fetch, and delete the item. I will be exposing the REST endpoint for all these functionalities and will be using MySQL as my database. The main file initializes services and boots up the server.

Company Mentioned

featured image - Application Testing In Go: Do It Right And Don't Create a Mess

Problem - How many times have we faced this problem of worrying about cleaning up after running test cases. When some of our test cases run, they might add data to our database or add files to our directory which we don’t want to worry about every time.

Solution -We can easily solve this using Docker. Imagine spawning a Docker container just to run test cases and killing it after it’s purpose is solved. Poof! our problem is solved, no need to worry about the collateral damage done by our test cases since killing that container handles that for us.

What is docker?

There are many resources out there which will explain what is docker and containerization. To explain you in the context of this in layman terms, it can be considered a free computer that we get to do some stuff and you’re given to liberty to do whatever you want with it because once you’re done it will be destroyed and next time you’ll get a brand new one.

Step 1: Creating our app

I have created a simple todo app that basically allows a user to save, fetch, and delete the item. I will be exposing the REST endpoint for all these functionalities and will be using MySQL as my database.


func main() {
	// Initialize a connextion to DB. Sourcename contains the auth information, host information
	// obtained from the env.
	db, err := sql.Open("mysql", sourceName)
	if err != nil {
		panic(err)
	}

	Init(db)
}

// Init initializes the dependecies and boots up the server.
func Init(db *sql.DB) {
	// Initialize our model
	repo := todo.NewListRepository(db)
	err := repo.Init()
	if err != nil {
		panic(err)
	}

	// Pass our model to our service which will handle business logic
	listService := todo.NewListService(repo)

	// start server
	http.HandleFunc("/save", listService.AddItem)
	http.HandleFunc("/delete", listService.DeleteItem)
	http.HandleFunc("/find", listService.FindItem)
	http.ListenAndServe(":8080", nil)
}

The main file initializes services and boots up the server

As we see we connect to MySQL and pass the connection to our model/repository which is a dependency of our service which handles our business logic.

type ListService struct {
	repo ListRepository
}

func NewListService(repo ListRepository) (service ListService) {
	return ListService{
		repo: repo,
	}
}

func (service ListService) AddItem(w http.ResponseWriter, r *http.Request) {
	param1 := r.URL.Query().Get("item")
	if param1 != "" {
		ID := uuid.New().String()
		err := service.repo.SaveItem(ID, param1)
		if err != nil {
			w.Write([]byte(err.Error()))
			w.WriteHeader(400)
			return
		}

		w.Write([]byte(ID))
	} else {
		w.WriteHeader(500)
	}
}

func (service ListService) DeleteItem(w http.ResponseWriter, r *http.Request) {
	ID := r.URL.Query().Get("ID")
	if ID != "" {
		err := service.repo.DeleteItem(ID)
		if err != nil {
			w.Write([]byte(err.Error()))
			w.WriteHeader(400)
			return
		}

		w.Write([]byte("delete success"))
	} else {
		w.WriteHeader(500)
	}
}
...

Above is our ListService which is essentially our controller, having logic for 3 endpoints.

// ListRepository is an interface
type ListRepository interface {
	SaveItem(ID string, name string) (err error)
	FindItem(ID string) (res string, err error)
	DeleteItem(ID string) (err error)
	Init() (err error)
}

// MySQLListRepository impl of Listrepo
type MySQLListRepository struct {
	// conn sql.Conn
	db *sql.DB
}

// NewListRepository is a constructor
func NewListRepository(db *sql.DB) (repo ListRepository) {
	repo = MySQLListRepository{db: db}
	return repo
}

func (repo MySQLListRepository) SaveItem(ID string, name string) (err error) {
	_, err = repo.db.Exec("Insert into `items` (`ID`, `name`) values (?, ?)", ID, name)
	return
}

func (repo MySQLListRepository) FindItem(ID string) (res string, err error) {
	var row *sql.Row
	row = repo.db.QueryRow("SELECT `name` FROM `items` WHERE `ID` = ?; ", ID)
	err = row.Scan(&res)
	return
}

func (repo MySQLListRepository) DeleteItem(ID string) (err error) {
	var res sql.Result
	var affected int64
	res, err = repo.db.Exec("DELETE FROM `items` WHERE `ID` = ?; ", ID)
	affected, err = res.RowsAffected()
	if err != nil {
		return
	}

	if affected == 0 {
		return errors.New("invalid ID")
	}

	return
}

func (repo MySQLListRepository) Init() (err error) {
	// var result sql.Result
	_, err = repo.db.Exec(`CREATE TABLE IF NOT EXISTS items (ID VARCHAR(255), name VARCHAR(255)) ENGINE=INNODB;`)
	return err
}

This is our model handling database operations.

Step 2: Start testing it using Docker

Unit tests

First, we will start unit testing. We can directly test our models. This is where the importance of cleanup comes in. As we can see we are modifying essentially our database after every operation. Let’s say sometimes there are 100s of inserts running in each test. After every test run unnecessarily after DB will start filling.

We will use a MySQL docker container to spawn a database and pass that connection to our ListRepository. After unit testing our repo methods, we will purge the container thus avoiding the hassle of cleanup.

Starting and stopping the docker containers via code

There are essentially docker clients available for each language

We will be using a Go client to handle docker operations.

func TestMain(m *testing.M) {

	// Create a new pool of connections
	pool, err := dockertest.NewPool("myPool")

	// Connect to docker on local machine
	if err != nil {
		log.Fatalf("Could not connect to docker: %s", err)
	}

	// Spawning a new MySQL docker container. We pass desired credentials for accessing it.
	resource, err := pool.Run("mysql", "5.7", []string{"MYSQL_ROOT_PASSWORD=secret"})
	if err != nil {
		log.Fatalf("Could not start resource: %s", err)
	}

	connected := false
	// Try connecting for 200secs
	for i := 0; i < 20; i++ {
		// Try establishing MySQL connection.
		Conn, err = sql.Open("mysql", fmt.Sprintf("root:secret@(localhost:%s)/mysql?parseTime=true", resource.GetPort("3306/tcp")))
		if err != nil {
			panic(err)
		}

		err = Conn.Ping()
		if err != nil {
			// connection established success
			connected = true
			break
		}

		// Sleep for 10 sec and try again.
	}

	if !connected {
		fmt.Println("Couldnt connect to SQL")
		pool.Purge(resource)
	}

	// Run our unit test cases
	code := m.Run()

	// Purge our docker containers
	if err := pool.Purge(resource); err != nil {
		log.Fatalf("Could not purge resource: %s", err)
	}
	
	os.Exit(code)
}

I’ll explain what’s happening

We create a new connection pool to docker and try connecting it.After the docker connection is successful we spawn a new MySQL container hosting our database.We try connecting to that container in a loop with a given timeout since spawning the container will take some time. If a timeout occurs then purge the container and exit.If the Conn.ping() doesn't return an error, congrats we have successfully connected to SQL container.Run the test cases and after that's done stop the SQL container.Voila! no hassle of cleanup our db since purging the container will take care of it.

Below is the example of our test case

func TestRepo(t *testing.T) {
	assert := assert.New(t)
	listRepo := todo.NewListRepository(Conn)
	err := listRepo.Init()
	assert.NoError(err, "error while initializing repo")

	ID := uuid.New().String()
	name := "itemName 1"

	// Save Item
	err = listRepo.SaveItem(ID, name)
	assert.NoError(err, "error while saving item")

	// Find Item
	foundName, err := listRepo.FindItem(ID)
	assert.NoError(err, "error while saving item")
	assert.Equal(foundName, name)

	// Delete Item
	err = listRepo.DeleteItem(ID)
	assert.NoError(err, "error while saving item")
	foundName, err = listRepo.FindItem(ID)
	assert.Error(err, "delete unsuccessful")
}

Hence we have successfully tested our repository using Docker MySQL.

E2E test using Docker

Above, we are only testing our repository. What if we want to test our business logic and server as well with docker. Say if our business makes some file changes on our system. On running it’s test cases, we will also need to do the cleanup.

What we essentially need to do is start our app server in a separate docker container and run our e2e tests.

Setup

FROM golang:latest

WORKDIR $GOPATH/src/todo

ENV GO111MODULE=on

COPY . .

RUN go build -o main .

EXPOSE 8080

CMD ./main

Above is a simple docker file for our application. But this docker container will not have MySQL installed with it. We need to start SQL in a separate container and bridge the network. We can do this using Docker Compose.

services:

  mysql_db:
    image: mysql:5.7
    environment:   # Set up mysql database name and password
      MYSQL_ROOT_PASSWORD: password
      MYSQL_DATABASE: todo
      MYSQL_USER: user
      MYSQL_PASSWORD: password
    ports:
    - 3306:3306
    networks:
      - my-network

  todo_app:
    image: todo_app
    container_name: todo_app
    environment:   # Set up mysql database name and password
      host: 0.0.0.0:3306 # Passing MySQL host
    build:
      context: .
      dockerfile: Dockerfile
    depends_on:
      - mysql_db
    ports:
      - "8080:8080"
    networks:
      - my-network


networks:
  my-network:
    driver: bridge

This Docker compose files can help us running both the service(SQL, todo_app) together and connecting them via a network bridge

We start our service using:

docker-compose up -d

This will ensure that our server is up and running and we can do our e2e tests below.

Our server will be up on localhost:8080 and we can directly hit it.

func TestE2E(t *testing.T) {
	assert := assert.New(t)
	client := &http.Client{}

	req, err := http.NewRequest("GET", "localhost:8080/save?item=test122", nil)
	assert.NoError(err, "error while initializing request")

	res, err := client.Do(req)
	assert.NoError(err, "error while making request")
	assert.Equal(res.StatusCode, 200)
	... // Other E2E requests
}

E2E test

We can test it by making HTTP requests to docker server and asserting responses.

After we have completed the test we will have to manually shutdown todo server and SQL containers using `docker-compose down`. In this case, we have to manually start and stop the docker service but this can be automated as well by using commands via code to start and stop the containers.

Hence by doing this, we ensure that if our app makes changes in our system or our database those are not persisted and we don't have to clean those up.

Conclusion:

Hence we saw using docker we can run e2e and unit test without creating a mess of our system rendered by our application. Full code available here.

Previously published at https://medium.com/@mohdgadi52/testing-application-without-creating-a-mess-af1a26e9c2c4