Hi there!
I'm thrilled to share how I managed to improve our release flow by 20% through the implementation of a system QA role.
Given that my company is a typical product company, the teams are divided not by product but by components. Thanks to this, on the one hand, we managed to form powerful teams with "holders" of knowledge. But on the other hand, the roles within the units are relatively isolated, and different sets of hard skills and expertise impose their limitations. For example, I sometimes need to move a tester from the backend team to the frontend or vice versa.
This led to the fact that there was a question of effective orchestration within QA teams and management of cross-team interaction. Which, of course, ultimately affected the release flow.
Release flow before changes
Before the changes, our release flow looked like this:
- A Business Requirements Specification (BRS) is a document that outlines the detailed needs, expectations, and specifications for a specific feature, system, product, or project within a business. It serves as a bridge between business stakeholders and the development or implementation teams, ensuring a clear understanding of what is to be achieved and how it aligns with the overall business goals.It should contain detailed scenarios that describe how users will interact with the feature. The Feature Owner (FO) is tasked with the duty of formulating business requirements.
- A Software Requirements Specification (SRS) is a comprehensive document that outlines the detailed description of what a software system is expected to accomplish. For example, how and which commands interact, by which protocol, and what data will be transmitted. If the team working on the feature utilizes a graphical interface, the SRS should include layouts of the feature being developed. The feature Architect is responsible for writing SRS.
- Quality Action Plan (QAP) – a set of cases that a feature owner checks before accepting a feature. After the documents are described, we proceed to implement the feature. When the first team finishes developing and testing its part of the feature, it's transferred to the second team. The second team performs integration/development/testing and passes it to the next unit. And so on until all component teams participating in the feature development pass through this flow. FO validates the feature, and it's sent to the release.
Release flow problems
So, everything seems to be okay – documents, applications, acceptance cases. However, we encountered the following difficulties in this process:
Problems on the part of QA:
- Requirements testing begins only after development. Tasks that have already been implemented by developers, along with their requirements, are handed over to the QA team. But, as we know, there may be errors in the requirements.
- Finding the team responsible for a bug takes quite a bit of time because it's not always clear which cases have already been tested by other teams.
- No reuse of test artifacts. As part of testing one feature, QA teams prepare similar test data sets. But due to the isolation and narrow specialization of the teams, they could not transmit this data to each other.
Problems on the part of FO:
- Due to the many features, writing QAP takes a lot of time.
- Validation of the feature occurs after development by all teams. In our case, this significantly lengthens the release flow due to many integration issues.
- The preparation of the test environment is also precise due to the complexity of the product and the volume of integrations between teams. Different teams are testing their components simultaneously, increasing the risks of mashing, changing, or deleting data.
System QA in the updated release flow
To facilitate effective cross-team interaction among QA teams and reduce the release flow, we introduced the role of system QA.
This helped alleviate the workload in the form of writing acceptance cases with FO and speed up the writing of test scenarios, introduce intermediate testing of the feature component before passing it to the next team, and also shift the time-consuming work of preparing the test environment to the system QA, taking into account all the nuances and requirements of the teams for integrations and test data.
System QA has become a link between the technical and business requirements for each feature and the product as a whole.
Onboarding for system QA
To understand the entire release cycle, System QAs need to understand how a specific release cycle works in each team. Onboarding typically lasts about three months as the system QA spends 2-3 weeks in each team, understanding their specific release cycles.
Results of the new process
-
We are now testing the BRS/SRS requirements from feature owners and architects. Early bug detection leads to cost savings for the business.
-
We've established a cross-team QA space, where test artifacts are attached to each feature – business requirements, technical requirements, acceptance cases, cases of other teams, test data. This significantly helped all QA teams to be in a single context and effectively reuse data.
-
Accelerated the process of localization of bugs because the system QA has sets of test cases from all teams.
-
Since the system QA is writing acceptance cases for each team, this is an excellent hint for speeding up and improving testing quality.
-
The integration process has become painless since the feature is validated by acceptance cases after each command.
-
Having removed a significant part of the load from the FO, the acceptance of features and the preparation of an integration stand with test data accelerated.
Overall, accelerated the release flow by 15-20% and reduced the number of integration bugs by almost half since now we catch them both at the stage of writing BRS and SRS requirements and during team integrations within the framework of feature development.
Happy and productive testing!