What Product Managers Should Know About App Testing

As a product manager, testing is second nature. The build-measure-learn principle from the lean startup methodology encourages teams to test everything they build to get insights to improve subsequent iterations and the product as a whole.

The dynamics of how testing is done or how involved product managers are in testing differs for teams, throughout the product development process. Large teams have dedicated QA and testing teams but we may not say the same for small startups. However, the focus of this article is to discuss the common types of tests that are run at various parts of the development cycle, and why product managers should have a grasp of them.

1. Unit Tests

   
   

When software developers write code they write it in lines, and a unit of code can be about 15 lines, less or more. But the smaller the unit, the easier it is to test. So often, long units of code are broken into multiple smaller units to improve maintainability.

Unit testing is testing a coding unit to verify that it is working correctly. This test is done while a software developer is coding. To perform this test, a piece of code [unit test] is written to test a unit. The same concept works for component testing.

Unit tests can be automated by integrating tools like reSharper, NUnit, etc, with developer tools to write and run the tests. Since mostly for a project, all tests can be run at once in no particular order, it’s best that there is no dependency between the tests.

Let’s use a mobile phone as an analogy to give more clarity. A phone has a battery, a sim slot, a charging port, an earpiece port, etc; right? Unit testing is like testing these phone parts individually to ensure that they work fine.

2. Integration Tests

   
   

It’s one thing to test different units of code in isolation and they pass, and another when combined as a group. Integration testing is performed after unit testing by testing different units, modules, or components of an application together, to check that these units combined work smoothly. This is not the same as system testing.

Integration testing focuses on the interfacing of units and identifying interface errors, while system testing tests the finished product to help identify system errors. Also, integration testing is generally limited to the functional aspects of the interconnected units, while system testing covers both the functional and non-functional aspects of the application.

3. Smoke Tests

Smoke testing also known as the build verification test is used to check that the product is stable. The word “stable” in testing springs up these common questions:

• Does stable mean ready to be released?

• Does stable mean it no longer contains a bug?

  
  

I would like to think that the standard for what stable is will differ in companies. But generally, stability should portray ‘bug-free enough’ to be pushed to production, such that the product’s behavior does not change anymore. Hence, stable!

Smoke testing is a type of acceptance testing that is done to ensure a build is stable enough to undergo other tests like regression and functional testing.

Typically, smoke testing is done manually, but automating it in a CI pipeline helps to prevent an unstable build from being pushed into the production environment. To make the best use of UAT, smoke testing should be conducted on every new build.

Also, conducting smoke tests first before asking select users to test, saves a lot of time. Else, the bug list would constitute bugs that should have been otherwise fished out earlier and fixed. Sanity regression test is another type of testing that provides a gate check or indication that a build is ready for more testing, just like with smoke testing.

4. Regression Tests

When changes are made to the codebase, regression testing is done to ensure the overall stability of the existing features to verify that the change did not adversely affect any existing functionality. This entails re-running already executed test cases.

The extent of the regression test depends on the scope of the newly added code to the codebase; whether it is going to be a unit regression test, partial or complete regression.

Some regression testing tools to explore: Avo Assure, Subject 7, Digivante, Testsigma, etc.

5. Acceptance Tests

   
   

Towards the end of a software testing cycle, the new application is tested in line with the real-time scenarios to verify that it meets all the specifications, expected quality, and design before it is pushed to commercial production. This test commonly called User Acceptance Testing or End-to-End testing is an acceptance test.

As the name implies, an acceptance test is a test for acceptability. If an application does not meet the specifications, then it has to be modified, debugged, or what else is required to suit the needs of the users and business.

Acceptance testing is done in stages, first, there is the Alpha testing, and next, the Beta testing. Users or other authorized entities can determine whether to accept the application or not when they test. Let’s dive a little into some types of acceptance testing.

• [ ]undefinedAlpha Tests

Alpha testing is a type of User Acceptance Testing performed by internal team members who put themselves in the user’s shoes while testing. They are knowledgeable of the product but not directly involved in the development. This test is done in the development environment and has a longer test cycle—The testers give feedback to the development team, and test again once the bugs or feedback are implemented.

• [ ]undefinedBeta Tests

Beta testing serves as a soft launch and is done by a select number of potential-real users called beta testers, in the production environment. This test enables you to get unbiased feedback about the user experience and usability of the application. It also helps you identify critical bugs to be fixed before release to a wider user base.

In summary,

Commonly, product managers are focused on testing at the end of the development cycle, but it makes more sense to create a culture of product testing that keeps the whole team on the same page about QA and testing.

Even if you are not directly concerned with QA and testing, it is recommended that you at least know the types of tests that need to be run at various aspects of the development life cycle of your product. This will help improve the quality of your product.

Know a non-technical person who is trying to get into the product management space? You may want to share my previous article on the topic Explore Must-Know Technical Concepts for Product Managers: Your Guide to the Mind of a Dev with them. Thanks for sharing!

Feel free to connect with me on LinkedIn.