Code Coverage: How to Measure You've Done Enough Testing

Testing is an important part of the software development cycle. It gives you confidence that the code you have written works as expected.

You may be thinking about how you can measure if you have done enough testing on your code. Are you confident your tests cover all the different scenarios your code will run? This is where code coverage comes in.

You may have heard the term code coverage mentioned already in your workplace, and you are not quite sure what it means or where it fits in.

In this article, you will learn what code coverage is, how code coverage is calculated, and how much code coverage to aim for in your tests.

What is Code Coverage?

Code coverage is a metric that can help you understand how much of your source code is tested.

A program with high code coverage, measured as a percentage, has more of its source code executed when you run its test suite, suggesting it has a lower chance of containing undetected software bugs than a program with low code coverage.

Code coverage is a very useful metric that can help you assess the quality of your test suite.

How is Code Coverage Calculated?

To measure what percentage of your code is tested, one or more coverage criteria are used. Coverage criteria are usually defined as rules or requirements, which a test suite needs to satisfy.

A code coverage report for one file (source: https://www.atlassian.com/continuous-delivery/software-testing/code-coverage).

Code coverage tools will use one or more criteria to determine how your code was exercised or not during the execution of your test suite. The common metrics that you might see mentioned in your coverage reports include:

• Function coverage: how many of the functions defined in your source code have been called.
• Statement coverage: how many of the statements in the source code have been executed.
• Branches coverage: how many of the branches of the control structures (if statements for instance) have been executed.
• Condition coverage: how many of the boolean sub-expressions have been tested for a true and a false value.
• Line coverage: how many lines of your source code have been tested.

These metrics are usually represented as a coverage percentage - the number of items actually tested divided by the total items found in your source code (items tested / items found).

A code coverage report for a full source code (source: https://www.atlassian.com/continuous-delivery/software-testing/code-coverage).

A good code coverage tool will give you the percentage of the source code that is tested, and also allow you to dig into the coverage reports to see the actual items that weren't covered by tests and then use that to identify critical parts of your application that still need to be tested.

There are many code coverage reporting tools. Examples such as Istanbul for the Javascript programming language and SimpleCov for the Ruby programming language. A simple search for your “[programming language/framework] code coverage” will present you with options. You might find several options to create coverage reports depending on the programming language/framework you use. Select your preferred option for your project.

You can then integrate code coverage into your daily development workflow - when you run your tests locally and in your Continuous Integration (CI) reports on pull requests or merge requests.

Is 100% Code Coverage the Goal?

Having a high code coverage means most of your source code is tested. This is a good thing. However, you need to be careful about only looking at code coverage in isolation. A high percentage of code coverage could still be problematic if critical parts of your application are not being tested, or if the existing tests are not robust enough to properly capture failures upfront.

It is generally accepted that having 80% code coverage is a good goal to aim for. Trying to reach a higher coverage might turn out to be costly while not necessarily producing enough benefit.

If you are just starting with testing and you run your code coverage tool you will likely find that you have a low percentage of code coverage. This is normal, and you can gradually increase the coverage over time. As you add new features or are refactoring existing code, make sure you are adding tests to cover your code and increase your code coverage.

While code coverage is a good metric of how much testing you are doing, it is not necessarily a good metric of how well you are testing your product. Achieving great coverage is an excellent goal, but it should be paired with having a robust and high-quality test suite that can ensure that the source code behaves as expected.

The New Developer

This article was originally published on The New Developer. Head over there if you like this post and want to read others like it.

Conclusion

To recap - In this article, you learned that code coverage is a measure that tells you how much of your source code is tested. It helps you measure the efficiency of your test implementation.

The five code coverage methods are - Function coverage, Statement coverage, Branches coverage, Condition coverage, and Line coverage.

You should aim to have high code coverage. However, 100% coverage may not be realistic or necessary. You should also ensure that your tests are good quality tests that give you confidence about your source code.

Are you measuring code coverage in your day-to-day work? How is that working for you?

Also published here.