Building QA Processes From Scratch: QA in SDLC, Collaboration and AI

Being a QA engineer for over four years, I decided to share my view of building the processes of software testing in companies and sum up everything that is inevitable to know for a good tester. BASIC TERMS First and foremost, let’s define the basic terms – what is ? What’s the main difference between and ? Where do they all stem from? Quality Assurance Quality Assurance Quality Control Well, has been around for quite a while. Its origins can be traced back to the early 20th century. One of the pioneers in the field was , an American physicist. Quality Control Walter A. Shewhart He developed statistical methods for industrial quality control while working at Bell Telephone Laboratories in the 1920s. Over time, these principles have been applied and adapted across various industries to enhance processes and deliver better products and services. …And of course, it touched software development where Quality Control lies in identifying defects or the behaviour which contradicts or do not align with business and functional requirements. Going further with , it’s slightly different process. It became popular in 1970s as organizations began to recognize the need for a more proactive and comprehensive approach to ensuring quality. Quality Assurance involves establishing processes, standards, and methodologies to prevent defects and errors rather than simply detecting and correcting them after production. Quality Assurance (or QA) To draw a conclusion here: Quality Control Quality Assurance We look for the existing bugs such as unexpected behaviour, bad user experience or misunderstanding requirements. We apply all best practices to prevent defects in future by building and enhancing QA processes. QA IN SOFTWARE DEVELOPMENT LIFE CYCLE (SDLC) Quality Assurance engineer is undoubtedly involved in the vast majority of stages of Software Development Life cycle. Analysis + UI/UX design: QA starts by ensuring a clear understanding of project requirements to set the foundation for quality development. How can we test if we don’t know the expected behaviour? This is why continuous collaboration with the Analysts and Project Management team is crucially important. The same with UI/UX designers. When application is not responsive, or it’s hard to identify the purpose of any UI element, it all makes user experience really bad. Would you want to order pizza from the website which froze for minutes last time you used it? I bet you wouldn’t! Implementation During implementation, developers can direct their questions about functional requirements to QA, particularly in subsequent iterations of development. Mostly, Agile methodologies assume ongoing collaboration among QA, Analysts and developers at each stage of this cycle. Testing itself It’s quite evident that QA plays an important role in test execution making it smoothly due to the prior gathering of requirements and thorough test planning. During testing, we verify that the product aligns with both business and functional requirements. I also incorporate what’s known as in each iteration of testing, involving no planning or documentation, just random scenarios that come to mind. And guess what? Each time it gives me at least a couple of new issues. Ad hoc testing Maintenance QA’s work does not conclude with the , we maintain the product and user feedbacks even after releases. In collaboration with the technical support department, QA remains knowledgeable about all processes and issues raised from the inception to the delivery of the product. deployment QA PROCESS First let’s remember what is as it’s the most commonly used methodology software development in modern companies. The framework is an iterative methodology. After every sprint, teams reflect and look back to see if there was anything that could be improved so they can adjust their strategy for the next sprint. Here’s the most common QA processes chain. Agile Agile Requirements Requirements should be tested as well as the product itself because they can contradict each other, not respond the best practices of user experience, or the expected behaviour of some scenarios can be simply missing. Test planning The information gained during the requirements analysis phase is used for test planning. The test plan should comprise the software testing strategy, the scope of testing, the project budget, and established deadlines. It should also outline the types and levels of testing required, methods, and tools for tracking bugs and allocate resources and responsibilities to individual testers if there are more than one tester on the product. Preparing test documentation Here we are talking about test cases first of all. Just to remind you the main parts of a good test case are: (like testing environment, specific actions that need to be done with your system), , . Writing test cases can be time-consuming that’s why it’s better to allocate enough time for preparing them. In case of lack of time there can be the following options: preconditions steps expected behaviour You can write test cases only for the basic functionalities, cover smoke scenarios You can limit your documentation by checklists which are clear and understandable for professional testers but take quite fewer time than creating complete test cases coverage You can ask for AI help. If you generate a correct request to ChatGPT, for example: ‘ . It will give a good output which you can use at least while there’s no time for doing it properly and manually. Write smoke test cases for testing a 1-on-1 messenger as a support chat on the website. Include steps, actual behaviour and preconditions.’ Testing So it’s time to execute tests and if we didn’t miss all previous stages, we are good to execute testing the entire feature or any bug that has been pushed to testing. If QA finds a serious issue thanks to the flexibility of Agile methodology the bug can be fixed while testing is being performed. Reporting Depending on the bug tracking system selected on the project, or whether any bug trackers are used, reporting defects requires a solid understanding and proficiency in working with this system. The key here is to know the steps to reproduce a mistake and clearly describe actual vs expected behaviours. Automation And last but not least, automation stage. Ideally, between testing iterations, QA should have enough time to cover necessary scenarios with automation tests, but in practice, the lack of time often delays the creating automation scripts. Since automation is time-consuming process, it’s advisable to automate only basic scenarios in the early stages of product development. Start by covering all smoke scenarios initially, and then, if time permits, add the scenarios from the extended path. What to automate? Just to remind you the principle of choosing scenarios for automation, let’s look at the following image. What would I cover here with automation tests? we are going through the most and user case when a new user is being registered with valid data. This includes filling all inputs, clicking Sign up button. Additionally, I would include the verification of – we need to make sure that user is created and added to a database. Automation frameworks are also capable to pull data from . First of all, basic positive backend databases that must be included to (whether it’s manual or automated), login if user already has an account. Even it’s the other feature but we are referring to it from our tested form. That’s why we need to make sure that we are able to open login form and authorise with valid credentials. The second scenario smoke testing Everything the rest does not go to the first iteration of automation. What will be included on the next iterations in automation? Basically everything that you will repetitively be testing each time for : invalid data input like special symbols, some kind of Javascript and HTML injections, or any other data not mentioned in product requirements. Here also goes filling the form with one missing input, with all missing inputs, registering the existing user and etc. Everything where your fantasy leads you and it makes sense in terms of regression testing. regressions Choosing automation framework The market proposes lots of paid and unpaid options. Here are the main factors which you should pay attention to: As I said there are pretty many free open-source options which are good for the starting small products but can cause extensibility or technical support. On the other hand, open source code is better in terms of debugging any issues that work incorrectly. Price: It’s good if an automation framework has a complete and clear documentation describing the main features and workflow, or at least support where you can address all related issues. Documentation and support: Scripts and data separation Automation test scripts separated from input data store (for example: XML, Excel files). No modification is required to the test scripts. Only input data gets manipulated for testing with multiple input values. Assume your application is so dynamic and your test scenarios are based on multiple logics (Ex:- If you click on A then B should be visible, else C should be visible), then you will need to develop your automation suite with your own methods which satisfy this requirement. To develop your own methods, the framework should provide its base methods. Opportunity to automate multiple logics This is a very interesting point. Some projects have specific requirements on testing features visually such as 3D engines, all games and web and desktop design tools, basically everything that has meaningful and editable visual assets. Visual testing: I personally used Cypress and its modules for web applications and Squish framework for desktop applications. How does visual regression testing works? In the first run, the script captures the actual image which we expect to be a reference image. In the next runs, the script captures the actual image and pixel by pixel compares the actual and the reference images. As the output, it creates a diff image showing which areas of actual image do not correspond with the reference. Before producing the output image, the tool considers whether there is any sensitivity factor or any other values that the user has specified. Sensitivity is also called . The or value defines how strictly comparison should take place. The higher the value, the lower the sensitivity and the comparison will ease out. threshold sensitivity factor BUILDING A TEAM The bare minimum that the QA team should cover is manual testing which includes both functional and non-functional. Automation engineer qualifications If we are talking about the necessity of hiring a separate automation engineer, that means that our product is not at its early stage of development, as having an automation engineer along with manual QAs can cost quite much money. Anyway, if we hire an automation engineer he or she should have: Working experience as a manual tester. It’s inevitable as these two close areas are interrelated. Background in programming ideally but at least solid understanding of what programming is. Expertise in any of automation framework And here I do not highlighting any particular framework as I personally think if a software tester knows how to work with one automation framework and has expertise in programming, they will be able to familiarize themselves with any other framework. Therefore, we have the following team: responsible for Quality Assurance in the company; Team Lead: responsible for the Quality Assurance of any project; Test manager: the quantity may vary but usually a good option is one QA per two developers; Manual specialists: depends on the product; Automated engineers: To sum up, to build a QA team you need to: Prepare project documentation for an easier onboarding Asses your resources and needs: expertise level, requirements for new hires, the number of new hires etc. Pick the right tools for the job: bug tracking system, test documentation platform, automation framework, CI/CD. Collaborate with recruiters and interviewing SELECTING TOOLS All members of development team, stakeholders and project management, everybody should be on the same page. By collecting all related issues in the same place, task management tools allow team members to be in the context of what’s going on and collaborate on planning and execution. Such collaboration is useful as it keeps priorities of current tasks visible and help prevent the majority of issues from being pushed to release by mistake. Criteria of a good management tool The first criterion which should be considered when choosing a management tool is . What does it mean? It means that the selected tool should ideally have any training options like tutorial or tech support on how to get started, be ease of use, satisfy all project’s needs: for example, the feature to create sprint boards, adapt them to Kanban, Agile or any other methodology which your team uses or the fields for creating bug, the types of issues that are available to be created and lots of more issues which you face on a daily basis. integrability Tasks statuses, adjustable boards & sprints, tags, labels, assignees etc. – so basically all custom need that you’re applying in your software development cycle. Even if you are working according to one of the popular development methodology, you are not doing everything according to the set rules. The length of sprints can differ, the logic of moving tasks between projects can vary and so on and so forth. Customisation: Moving to a test management platform other than Excel is often driven by the need for to collaborate efficiently. However, when multiple users access the tool, challenges arise from tracking and security perspective. For instance, one of the critical things is whether the test management tool locks a particular test case when it is being accessed, preventing simultaneous edits by two individuals. multiple users Finally and most importantly, as other tools, the cost of a solution should be looked into while selecting a test management tool. This involves taking into account not only the upfront amount it costs to buy a tool, but also training, implementation, and support costs. Pricing: Besides managing tasks, it’s important to fill your environment with supporting tools and frameworks. What tools do you also need for a productive work? a must-have for those who maintain automation tests or stores test documentation in repos. The most popular version control system is Git for sure. It allows to track changes in any set of computer files, usually used for coordinating developers during software development but also widely used by software testers to develop and maintain automation tests and test documentation. Version control system: Depending on the programming language, tools like Visual Studio Code, IntelliJ IDEA, or Eclipse provide a comfortable environment for writing and debugging code. IDEs also support Git embedded to them. Integrated Development Environment: Automation testing tools have a significant impact on the final quality of product. We touched the criteria for choosing the right automation tool in the previous section but what should also be noticed here is that choosing automation framework depends on what type of application you’re developing: web application, mobile or desktop – they all have slightly different approach to testing and automating as a result. I personally use Postman. To make my work more comfortable, I create an hierarchy of folders with different types of requests depending on what feature they relate to. This all is called workspace and it’s easily shared between team members. API testing tools: I guess most of us agree that AI became irreplaceable during every day work: writing the skeleton of test cases, asking for automation help and advices, language support. ChatGPT: video and chat like Slack, Zoom, or Google Meet and tools controlling your timetable and ideas like Apple Notes or Microsoft OneNote By the way, I became way more productive since I started using notes because all my tasks are listed there, I am not stuck thinking what I forgot to do, who I should reply to, what else is on my plate. These thoughts are destructive and don’t allow to focus on your job. Communication tools: Conclusion Quality Assurance is an integral part of software development that ensures the delivery of a high-quality product to its users. Therefore, setting and optimizing the testing environment and processes is a priority for anyone leading QA team, as well as for each team member. (Header image by pikisuperstar on Freepik)