The test pyramid introduced by Mike Cohn in 2009 describes a standard to help developers plan and prioritize automated software tests.

It follows the philosophy that tests that run quickly and do not require a lot of resources should be included in the test suite in large numbers, whereas costly testing methods should be used more sparsely.

This article explores the pyramid in detail and how it can be applied to the testing of iOS applications.

What is the test pyramid?

The test pyramid is a framework introduced to deliver high-quality software and a robust test suite.

It describes the different types of tests to be done by the development and quality assurance teams and defines their order and frequency. The purpose of utilizing this framework is to execute tests in the most efficient way possible and to ensure that code changes do not impact existing functionality.

The pyramid consists of three layers that visually represent the order of the different test types.

Unit tests, which appear in the bottom layer, are cost-effective, take the least amount of time to run, and should therefore make up the majority of tests.

UI tests, which appear at the top of the pyramid, are costly, take the most time to run, and should therefore make up the minority of tests.

How are tests executed in iOS applications?

In iOS development, using the XCTest framework is typically recommended. This tool enables developers to automate tests and is integrated into the testing workflow of Xcode.

XCTest makes sure that conditions that the developer defines are met during code execution and also records test failures. Additionally, XCTest can measure performance and interact with the UI.

Let us take a closer look at the different types of tests in the pyramid and how each can be executed using XCTest.

Unit Tests

Unit tests ensure that isolated units of code work as expected. They do not use external dependencies and therefore have a limited scope. When unit tests are executed on a module that calls an external service, this service is mocked.

Executing these tests on virtual devices such as emulators is sufficient to save time and resources.

Unit tests should consider each possible scenario to ensure high test coverage. For example, if a method accepts an optional parameter, then the tests should be written with and without this input. To add a test class, the developer will create a subclass of XCTest containing unit tests for a given class. The name of each test method should start with “test.”

Each test method contains three parts:

1. Arrange: Any objects needed to test the method functionality are created, such as class variables and objects fed to the method as parameters. Dependencies on external services are replaced with stubs.
2. Act: The method or function that should be tested is called using parameters configured in the “Arrange” step.
3. Assert: The test assertions provided by XCTest are used to compare the code’s behavior in the “Act” step with the expected results, which the developer defines. If any assertion is false or an uncaught exception occurs during execution, the test fails.

Integration Tests

While unit tests verify the correct functionality of isolated fragments of the codebase, integration tests check how the code interacts with external components, such as third-party services and databases.

Integration tests ensure that accurate data is retrieved when calling external systems and that the application continues to run efficiently.

As they need to interact with external interfaces, integration tests are slower than unit tests and require a pre-production environment.

Some types of integration testing, such as acceptance testing, can be done on virtual devices. On the other hand, security and performance tests must be executed on real devices for the results to be meaningful.

Another vital point to keep in mind is to use device settings that match the metrics of the target user.

The development of integration tests works similarly to that of unit tests, the difference being that while unit tests only cover isolated parts of the code, an integration test looks at the behavior of a combination of several classes and functions. Therefore, fewer stub objects are used.

Ultimately, integration tests aim not to cover every scenario but to assert that different components work together as intended. For example, an integration test may check that a value received from a controller is expectedly stored in the model or that an error returned by an external system gets passed to the user interface.

UI Tests

UI tests, or end-to-end tests, test the entire application. Test data and a test environment are used to simulate real-world functionality. They are the most expensive and time-consuming of the three test types.

UI tests are executed from the user’s perspective. The tester considers how the user may engage with the application and what mistakes they might make. Tests that simulate these actions are then developed.

These types of tests can be fragile due to unpredictable external dependencies. They also require using physical rather than virtual devices to mimic the issues of users accurately. Similarly, it is essential to test using various device settings and different degrees of network latency and throttling.

Xcode provides a UI Test Case class template with a starting point for UI tests. UI tests are also organized as subclasses of XCTest but work a bit differently than unit and integration tests.

To create a UI test, one can record an interaction with the app using Xcode‘s “Record UI Test” feature. Xcode will run the app and allow the developer to interact with the UI. The app’s interface should be used the same way a user would to determine whether specific tasks can execute properly. To use the more expensive and time-consuming UI tests effectively, they should replicate only the most essential user flows with the highest impact.

When using the “Record UI Test” feature, the corresponding code will be written into the test method, and the developer must add the assertions required for the test to pass.

What are the benefits of the test pyramid?

The purpose of the test pyramid is to save time and resources. A large amount of automated tests reduces the risk of human error and enables the development team to reuse and scale application tests to meet changing requirements.

The test pyramid has also become a staple approach in the agile methodology, prioritizing efficiency and effectiveness. The simplest tests are run first to catch errors early, enabling the testers to use their time more effectively.

Quality assurance teams use the pyramid as a benchmark for the correct prioritization of the different types of tests. For instance, if the number of UI tests is disproportionately high, the test coverage of the backend is subpar. Bugs may only be found once UI tests are run when they could have been detected much earlier through unit tests.

In many projects, the test pyramid is used without the team necessarily setting out to do so. When development is just starting, it is infeasible to write UI tests, and by the time a prototype of the application exists, the team will have had lots of time to write unit and integration tests.

Limits of the test pyramid for iOS applications

The principle of running a large amount of low-level, cost-effective unit and integration tests and a smaller amount of time-consuming UI tests can be applied to many software applications and serves as a solid base for iOS development.

However, some counter that one cannot ensure a fully functional UI without some degree of manual testing.

As discussed in this article, automated UI tests should only cover the most critical and high-impact workflows. If there were an edge case where, for instance, a view was covering a button, making the button untappable, this bug would likely not be detected through automated testing alone. Therefore, some manual testing in the final stages before a release should always be a part of iOS development.

Furthermore, the test pyramid should not be looked at too dogmatically. Every project is different, and there are scenarios in which UI tests are easy to maintain and can frequently be executed. As with every part of the software development process, it is crucial to maintain the flexibility of adapting the test suite to changing requirements.