The Never-Ending Debate: For Loops in Test Cases or Multiple Assertions

Testing has been a subject for developers for a while now, the agile Manifesto is more than 20 years old. Before that, we had eXtreme Programming (XP), and XP was the precursor for testing, meaning: having testable code as a practice, shifting left the need for writing the test first.

This is not to say that, before, XP developers weren't testing applications - but writing test code became popular.

Along those lines came different concerns, for example, any developer that follows TDD (Test Driven Development) or is familiar with that, follows the flow: write a failing test, make it pass with the simplest possible change, and then, refactor.

Refactoring is a crucial step. Although practitioners have been practicing TDD for a few years, as there is no formal education, some questions come to mind:

1. Should I use logic inside my tests?
2. Should I use multiple assertions inside my tests?

Here, we will focus on the assertions aspect of things as well as the logic inside the tests, using for loops for example.

To For Loop or Not?

If you think about that at first glance, having logic (of any kind inside a test case) is something that looks like a smell. Diving a bit on the subject, the xUnit book mentions that tests should have no logic inside the test case.

For example, the classic logic is the if/else statement: if variable X is equal to true, I will assert that X is true, otherwise I will assert that Y is a string. With that, the test has two different reasons to fail.

If you do that, you will have to spend some extra time understanding the logic behind the test and dedicate time to debugging which branch failed.

In the end, if you have dynamic code in the test case, who will test the test?

Therefore, we still have the question “Is it ok to use for loops in test cases?“ there. Let’s start with some survey data, as it could provide insights into what approach practitioners follow.

The Twitter poll didn't get much attention but, from the ones that answered, it seems that developers don't care much about that (as interesting as it could sound, but in the comments, there are strong opinions).

On the one hand, the results show just a minor difference. The "no", you should not use for loops, won the poll, but with a difference of 2.5% of the votes compared to “yes”. Not that much difference.

On the other hand, If you take the GitHub repositories used in the TDD anti-patterns series, and navigate inside the source code, the probability of seeing a for loop inside the test case is not that high; at least, it wasn't for me. The pattern is the AAA without for loops.

Last but Not Least, What About Multiple Assertions?

There is a grey area in the comments on the tweet, but as far as the poll data on Twitter goes, there is no rule of having only one assertion per test case; the winner was “As many as I want“.

Developers said that for the testing in place, if you are asserting multiple times the same behavior, why shouldn't you use multiple assertions?

Elliotte Rusty Harold lists, as a best practice, to use one assertion per test method:

All in all, assertions are a tricky subject to tackle, and it depends on the context in which the test was written, as such the takeaway from this section for me is the following:

listen to your tests (Steve Freeman and Nat Pryce).

Listening to the tests and the feedback it gives is an indicator of whether you are doing something good or not - good in the sense that it gives your ease of keeping the test running fast and gives the correct feedback when it fails.