Introduction As a product manager, I’ve lost count of the times I need to work with the QA team to draft repetitive test cases with a few variation tweaks. I understand that it’s part of the job, but it’s energy draining and time-consuming because you are basically copying, pasting, changing a few lines, then repeating. A few months ago, I was pulled into a project to work with a GPU giant company that owns labs across the globe. By coincidence, the main problem they encountered was test case management in their massive library with 100,000+ cases. The crazy number isn’t accidental, since the company needs to work with multiple product lines and each of them has a unique configuration, environment variables, and compliance rules. Even small changes like swapping a part, adding a new sensor, or adjusting a voltage spec could force the teams to spend hours setting up everything again.  At such a large scale, two critical problems surfaced: Test case duplication: It’s easy to hide coverage gaps with a massive and unstructured test library. As the number goes up, duplicate test cases multiply,Lack of traceability: Traceability was missing from the workflow. There was no strong link between requirements, test cases, and executions. That means when an execution went wrong, it’s difficult to trace whether the root cause lay in the requirement, the test itself, or the test run. Test case duplication: It’s easy to hide coverage gaps with a massive and unstructured test library. As the number goes up, duplicate test cases multiply, Test case duplication: Lack of traceability: Traceability was missing from the workflow. There was no strong link between requirements, test cases, and executions. That means when an execution went wrong, it’s difficult to trace whether the root cause lay in the requirement, the test itself, or the test run. Lack of traceability: Hardware Testing: Like Software, But Angrier and Heavier Software testing could cause headaches, but one can usually run, debug, and execute test runs again in minutes. However, hardware testing is slower due to physical constraints and resource distribution. Take GPU testing execution as an example. Here is the process broken down: Instance grid setups: Placing GPUs in respective testing slots and checking if the system detects them. Then, push workloads at different speeds to test the PCIe bandwidth, which is the amount of data transfer between the GPU and the motherboard.Environment Configuration: Installing GPUs in different server environments. For example, it could be a Dell workstation or a SuperMicro rackmount. The point is that each environment embodies unique CPU, memory, or power supplies. Resource bottlenecks: It's important to plan the allocation of resources ahead of time because testing slots are limited. If all slots are full, the leftover test runs pile up in line. Instance grid setups: Placing GPUs in respective testing slots and checking if the system detects them. Then, push workloads at different speeds to test the PCIe bandwidth, which is the amount of data transfer between the GPU and the motherboard. Instance grid setups: Environment Configuration: Installing GPUs in different server environments. For example, it could be a Dell workstation or a SuperMicro rackmount. The point is that each environment embodies unique CPU, memory, or power supplies. Environment Configuration: Resource bottlenecks: It's important to plan the allocation of resources ahead of time because testing slots are limited. If all slots are full, the leftover test runs pile up in line. Resource bottlenecks: Getting “Drowned” in Test Cases An industrial study from Sciencedirect caught my attention while I was doing desk research on the customer issue. It analyzed regression test suites across six large companies and revealed that enterprises often maintain tens of thousands of test cases. The problem?  Many of them are redundant, showing how duplication can bloat test libraries, eventually increasing maintenance costs. Sciencedirect Sciencedirect This outcome aligns with the pain point of our customers. For their testing teams, life was like this: Manually create thousands of new test cases, even if they were 95% similar to an earlier case.Keep typing environment configurations for each small change.Diving deep into multiple spreadsheets to match the pertinent test cases with the respective requirements. Manually create thousands of new test cases, even if they were 95% similar to an earlier case. Keep typing environment configurations for each small change. Diving deep into multiple spreadsheets to match the pertinent test cases with the respective requirements. It would be unreasonable to blame the testers without a structured testing library that supports memory, templates, or parameterized configurations. If copy and paste is the only "automation" available, it can definitely drive people insane in a huge library with more than 100,000 test cases. A Flawed Workflow Is at Fault If the system doesn’t support reuse, repetition will take over. And in a high-volume, high-variety area such as GPU hardware testing, management malfunction is inevitable. Three questions waiting to be answered by the solution: Traceability: Does the system help visualize test coverage? Is each test case mapped directly to the requirements? Without a link between them, teams can easily duplicate work or miss critical coverage. Modularity: Are there built-in test case templates (base case)? If that's true, can they be reused for different product lines, with minor parameter overrides? Scalability: After finishing setting up test execution configurations, can the system quickly bulk-create all the matching test runs without manually rebuilding each one? Traceability: Does the system help visualize test coverage? Is each test case mapped directly to the requirements? Without a link between them, teams can easily duplicate work or miss critical coverage. Traceability: Modularity: Are there built-in test case templates (base case)? If that's true, can they be reused for different product lines, with minor parameter overrides? Modularity: Scalability: After finishing setting up test execution configurations, can the system quickly bulk-create all the matching test runs without manually rebuilding each one? Scalability: Reusability To the Rescue The solution we provided is a structure that introduced the concept of test case recycling. Here is how it answers the questions above: Traceability Built In Linking each test case directly to its related requirements and the associated parameters.Integrating reference materials and configuration details so testers could see why a case existed and what it was validating. Linking each test case directly to its related requirements and the associated parameters. Integrating reference materials and configuration details so testers could see why a case existed and what it was validating. Modular Test Components Inventing the idea of combination grids, which is a table that maps each base case to the products/SKUs it can be applied to. Allowing overrides for specific parameters so the same base case could apply to different products/SKUs without creating a separate copy. Inventing the idea of combination grids, which is a table that maps each base case to the products/SKUs it can be applied to. Allowing overrides for specific parameters so the same base case could apply to different products/SKUs without creating a separate copy. Adaptable Instance Grid and Configuration Handling Enabling testers to adjust GPU-slot assignments, server types, and performance parameters in one place.Adding bulk creation so a single configuration change could generate all the matching test runs automatically, instead of rebuilding each from scratch. (eg. testing PCIe x16 versus x8 across three different PCs (6 test runs in total) Enabling testers to adjust GPU-slot assignments, server types, and performance parameters in one place. Adding bulk creation so a single configuration change could generate all the matching test runs automatically, instead of rebuilding each from scratch. (eg. testing PCIe x16 versus x8 across three different PCs (6 test runs in total) Combined together, this solution injected flexibility into the test library, as it could now handle many products/SKUs without creating new cases. Different GPU models, server types, and other parameters could be mixed and matched during test runs. More importantly, test cases are traced back to requirements, making testing coverage transparent. When the Painkiller Pays Off According to Deloitte, 74% of organizations reported efficiency gains from automation, while 59% achieved up to 30% cost reductions. This study demonstrates that structured automation improves both quality and consistency. Deloitte Deloitte Once the new system was in place, the difference was obvious: Once the new system was in place, the difference was obvious: Setup time dropped by 40% for new product combinations.Testers could launch executions in minutes, even across different GPU and server configurations.Improved test consistency because the same base cases were adapted, not rebuilt.Fewer delays from waiting on slots, servers, or approvals. Setup time dropped by 40% for new product combinations. Testers could launch executions in minutes, even across different GPU and server configurations. Improved test consistency because the same base cases were adapted, not rebuilt. Fewer delays from waiting on slots, servers, or approvals. Lesson Learned: If You’re Repeating Yourself, Your Process Is Broken Repetition is an urgent sign of implementing an automation mechanism. In complicated systems, such as a test library with well over 100,000 test cases, having to cover thousands of hardware configurations will cause issues. However, if there is a way to reuse and adapt what’s already there, it will fix the broken process. Keep in mind that if the system remembers, you don’t have to.

Dell

Keep

Read My Stories

Too Long; Didn't Read

Download free Tech Leaders Productivity Report!

How I Stopped Copy-Pasting My Sanity Away in Product Testing

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

4 Ways to Manage the Configuration in Python

5 Reasons We Use Open Source FreeBSD as Our Enterprise OS

Guide To Organizing Your iOS Debug, Development, and Release States With .xcconfig Files

A Non-Standard Way to Use Apache Kafka Message Broker

Automating open source project configuration with Mrm

4 Ways to Manage the Configuration in Python

5 Reasons We Use Open Source FreeBSD as Our Enterprise OS

Guide To Organizing Your iOS Debug, Development, and Release States With .xcconfig Files

A Non-Standard Way to Use Apache Kafka Message Broker

Automating open source project configuration with Mrm

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps