An Intro to QAOps in Continuous Delivery Systems

In the common paradigm, dedicated QA teams focus solely on product quality. As the competition between products and teams that deliver them has grown, QA teams have started enhancing the existing process. QAOps is the new approach in testing that enables an efficient quality assurance process.

Similar to DevOps, which involves tightly connected development and IT infrastructure operation teams, QAOps becomes essential as it integrates quality assurance, IT operations, and software development processes into one highly effective and cohesive process. It's about ensuring that QA forms a part of your CI/CD pipeline and their ongoing collaboration throughout the entire software development lifecycle.

QAOps: What is it?

To explain this trend in quality assurance in more detail, let’s look at a few true and false definitions of QAOps.

The crucial difference between DevOps and QAOps could be defined in the following way: the latter focuses exclusively on the infrastructure for remote test case execution. This infrastructure allows teams to run autotests on CI/CD platforms and get reports. One cannot build a QAOps infrastructure and process separately from CI/CD pipeline.

QAOps is not a specialization, nor a team role*.* QAOps is a practice, a methodology that cultivates healthy quality product creation culture. “Healthy” in this context means that the complexity of the product doesn’t have a negative impact on process efficiency. With the growth of complexity, QAOps becomes more and more necessary, which may require the involvement of automation testing specialists or a dedicated QAOps professional to address the issue.

The responsibilities of an engineer who does QAOps work include not only the development of the QA automation framework but also configuring remote automated testing run, gathering data about its flow, reporting, and distributing of results via communication channels.

QAOps process specifics

The effect described above stems from the inability of the traditional QA framework to stay efficient within Agile product teams. The QAOps-driven process aims to integrate testing practices at every stage of the product delivery pipeline.

A QAOps framework is based on a shift-left paradigm. Unlike in the Waterfall-like process, testing comes into play one stage earlier than in conventional paradigms. Elements that contribute to the success of the shift-left paradigm include environment standardization, build and deployment automation, and testing automation.

When to introduce QAOps?

There are two common situations when teams decide to introduce QAOps.

1. The CI/CD product infrastructure isn’t optimized for automating quality assurance. When one needs to introduce and configure the QAOps process, either an automation QA engineer skilled in QAOps or a tandem structure including a DevOps team and an Automation QA team may fit well. The combination of CI/CD proficiency with QA automation lays the groundwork for great QAOps processes.
2. The testing infrastructure is very complex, which requires QA automation engineers to have deep knowledge about configuring, maintaining, or enhancing it. This complexity becomes a bottleneck for product growth. In this case, a QAOps engineer or any automation engineer with the knowledge of DevOps spearheads the process transformation.

Techstack case

We don’t have dedicated QAOps engineers on our product teams as a common practice. However, we’re always ready to jump in when it’s necessary as we have QA Automation, DevOps expertise, smooth communication within teams, and a thirst for new challenges.

For most cases, a distinct QAOps role has limited involvement time in a project, whereas we look for long-term relationships with both partners and team members. That’s why we tend to ensure QAOps results from the hard work of different collaborative functions instead of from hiring a professional for a short time. In addition, the constant need in process configuration is more favorable for products with very large product teams and numbers of processes to run.

QA Automation engineers at Techstack can configure CI/CD infrastructure on their own when only consulting with a DevOps team: deploy, update, adjust, etc.

QAOps architecture and stack

Technology stack for each QAOps framework layer, usually employing the following tools:

• CI/CD pipeline: Jenkins, Team City.
• Cloud infrastructure: Docker containers, AWS, Digital Ocean, Azure.
• Remote browser access: LambdaTest, Selenium, Katalon, BrowserStack.
• Performance: JMeter, K6, Octoperf.
• Reporting: Allure, Report Portal, Data Dog.

The four pillars QAOps stands on

Besides the architecture and tech stack, the framework is founded on several core principles.

• Scalability. Hidden bottlenecks in an application's performance should be revealed to ensure it can scale. To adjust testing for custom needs of the application, QA engineers and developers define testing types together, as well as their frequency and amount.
• Automation. If you can’t automate, you can’t scale. Save time on different stages of product development using an automated testing approach, allowing you to reduce the need to run frequently used tests such as regression, non-functional tests by hand.
• Parallelization. You may do several test processes simultaneously and provide better quality assurance speed. Together with manual and automated testing, you parallelize testing across system features and modules.
• Integration. Now as you applied three previous pillars, the last step towards implementing QAOps is to imply quality assurance activities into the continuous delivery pipeline. Among the most common ways to incorporate development and infrastructure operations with QA is to have test cases written by developers.

Summary

QAOps is now essential for teams that automate their delivery pipelines. With the growth of the product and the number of people and processes, it becomes increasingly valuable to choose the right infrastructure and the right system architecture to scale. Once introduced, the QAOps process in CI/CD pipeline saves effort and shrinks costs of products used on testing.