Test Automation Suite: Cabot’s Solution to Minimize Software Testing Time

Technology has pushed boundaries that have been unbeknownst to man until now. Automation, Artificial Intelligence, Advanced Robotics, Virtual, Reality — we have indeed come real far with technology, and with frightening speed and precision. Through automation, the aim of eliminating errors and saving time has been successful so far.

And the good news is that automation is no longer the monopoly of big companies like IBM (Watson) or Google (driverless cars) anymore. It has permeated into the lives of everyone. You enter the airport, you feel it; you shop in a brick and mortar store, you feel it.

This brings forth another question — will most of the jobs be automated? According to a report by McKinsey, about 45% of the jobs that people perform now can easily be automated by technologies. And this could mean not only in the low-skill, low-wage areas, but also in the top positions like senior executives and CEOs.

With the best technologies that you have in hand now, it is now possible to redefine jobs and set complicated business goals and achieve them.

This is exactly why more and more companies have begun to embrace automated technologies. What Atlassian says regarding this is worth noting here. According to them, about 69% of the high performing companies admit that their use of automation has accelerated over the past two years.

Unfortunately, several companies are still stuck in the past and even though they admit that they are focused on risk reduction, only 29% of them believe that it is their IT team that brings forth critical business success.

High performing teams are 2.5 times likely to indulge in problem management, while others still waste precious time trying to discover what went wrong and where. In most cases, the root problem doesn’t even get fixed, it will only keep cropping up, affecting their business goals.

All these statistics make it evident that automated testing is the norm, and it can be much better than manual testing in more ways than one. We will go into that later.

First let’s examine the importance of automated testing.

Tests are conducted regularly during each development cycle, and sometimes tests have to be repeated. Software testing is done on different supporting operating systems and hardware configurations. This could take a whole lot of time and money, because it could get repetitive.

Automated testing improves the accuracy of these tests, while saving time in the process, and it doesn’t miss a mistake. Even the most conscientious (human) tester is likely to make a mistake, but through automated testing, this can be completely eliminated.

Automated testing is often done for Regression testing, to seek new bugs in a software program and to pinpoint the dangers it can bring. Automated software testing can be categorized into different types:

Unit testing — For testing the smallest piece of code.

Functional testing — For testing how applications function, with the system on one side and users on the other.

Regression testing — For testing essential functions to ensure they work, even when they are modified.

Integration testing — For testing how two systems work together.

Keyword testing — Testing based on keywords like mouse-click, opening or closing window, keystrokes, etc.

Data Driven testing — Entering data in a particular scenario and conducting the tests to check for performance.

Blackbox testing — Test for detecting gaps in the system.

Manual Testing vs. Automated Testing

Manual testing is done by a human, preferably an experienced software tester or developer. Each part of the code will be tested and compared with the standard parameters to check and rectify defects and the final observations are recorded for future reference.

However, there are some disadvantages of manual testing that makes it very unpopular.

1) Physically Demanding

The tester has to be present at the location and watch the tests to ensure all the codes are tested and monitored. He will have to be alert at all times, during all phases of the test cycle. There is a chance this could lead to errors, and eventually, inconsistent or misleading results.

2) Extremely Time-Consuming

Running the tests can be a very time consuming job, especially as it is not done by a machine, but by a human. And this could be a concern if you are planning to make lots of changes to your app in the future.

After running the script, the test results should be analyzed thoroughly, and if failures are detected at the checkpoints, investigations will have to be done. And, if you are handling a big project, manual testing will, indeed, be a highly time consuming activity.

3) Highly Repetitive

It could turn out to be disaster if you assign the same developer who developed the app to do the manual testing. As he/she is already familiar with the app, he/she might miss the error.

And the tests could get repetitive; it is called testing fatigue and chances are the testers would get weary (with the job), and errors could slip through the cracks.

4) Involves Extra Costs

You need more people to do manual testing, so you need to pay people for their time. Is it worth saving up for purchasing automation tools, than paying for something that is not 100% foolproof? Could be food for thought!

In Automated testing, you need to download automation tools and frameworks to run the tests and improve the functionalities and features of your software.

You have to write and run tests on the basis of predefined actions and then examine whether things fall in place or if there are hitches. The moment bugs come up, the testing tool would alert you and you can get them fixed.

Automated testing definitely fares better than manual testing in more ways than one. Let’s run through the main points here:

1) No Human Effort Required

Automated testers design testing scripts that would go through each script in your page. It can analyze the different states through which users can access your app. A human tester will not be able to cover all the states, hence errors are likely to occur.

2) Faster and Money Saving

Humans can get slow, but not scripts. You can easily get more things done in the shortest time possible with automated testing. Testing always has time constraints and it has to cover not just the tests, but the quality of work, writing and even executing and reviewing the test cases. Automated testing saves costs as well.

3) Wide Test Coverage

Apart from the wide coverage of tests, test automation frameworks are usually extensible so if you have new test cases and components, those can be easily completed. Additionally, you can also create large number of complex automated tests and execute them seamlessly.

4) Improved Accuracy

Humans are likely to make mistakes; they tend to get easily distracted. Scripts are not likely to make mistakes, and the chances are very, very low for any errors to be missed, irrespective of the complexity of the project.

5) Reduced Repetitive Work

With automated testing, mundane and repetitive tests can be reduced to a certain extent and this saves time. What earlier took days to compete will now be finished within hours.

This means the testers are also free to concentrate and focus on solving the bugs, while saving the repetitive and redundant tasks to the automated test suite.

6) Improved Efficiency for Developers and Testers

Of course, the initial role of developers is to make programs. With testing becoming automated, developers and testers can concentrate on polishing and sharpening their skills. They no longer have to worry about the testing part of their program.

7) Useful for Large and Time-bound Projects

The type of testing required would, of course, be dependent on the size of the project. Automated testing would be a feasible option for larger projects, because you can save a lot of time on testing; especially when you are working round the clock, chasing deadlines.

Test Automation Suite — the Answer to Cabot’s Manual Testing Woes

We, at Cabot Technology Solutions, felt that it was imperative to properly address issues that our team faced with manual testing. Our developers and testers were experts in the field, but they still had to suffer the following bottlenecks whilst performing testing:

1. Heavy time consumption for testing the application

2. Writing test scripts for each programming language

3. Huge pile-up of regression test cases in latter sprints

Repetitive testing is a really time consuming activity. Each time a change is made to the code, it is important to check

(1) that it doesn’t affect the functioning of the program in any way,

(2) there are no new bugs and

(3) the previous functionalities are all working in the way it was meant to be.

All these prompted us to look for a permanent solution to manual testing — developing a flexible test automation suite.

The main hurdle that we faced during manual testing was time crunch especially while performing full regression tests before each release.

A full regression test done manually is painstaking because it is almost like going through an entire haystack to ensure no needle is present there. It means spending countless person hours and dedicating your testers’ skills on an activity that gives the shareholders the “feeling” that everything is covered.

Hence, we had to work our way to find a solution so that both functional and non-functional testing was completed on time. We worked very hard and tried different approaches to ensure the softwares were tested error free before the product launch.

The Solution

After months of research and analysis, Cabot, successfully, built a Test Automation Suite that is integrated with Selenium and Appiumfor performing Regression testing and Loadrunner and jMeter for Performance testing. The entire system functioned under Behavior Driven Development (BDD) framework managed by the tool, Cucumber.

The automation suite paved the way for parallel testing where we were able to use the test resources in the most efficient way, while reducing the execution time of the testing and procuring the quality of the software within the shortest time possible. Our testing suite was able to reduce the tiresomeness of repetitive testing, while test consistency and test coverage were sure guarantees.

The most impressive development that emerged with this automation suite was time management. Deadlines were met with absolute precision and the overall testing time got reduced from 5–6 hours to 1 hour — almost 80% savings in time!

Now, tests were run on a daily basis. Regression testing was a nightmare for the testers during the days of manual testing. This got automatically translated into a cakewalk job.

In the days of manual testing, it took days and about 4–5 testers to complete regression testing; now a single tester can easily finish it in just a few hours.

With the test automation suite, performance testing and sanity testing could be run parallel to regression testing. This not only saved time, but also helped the team to focus on critical areas of the software and successfully analyze and test complex features.

The test automation suite helped us to develop test scripts in the design and development phases of SDLC (software development life cycle). As the suite supports multiple test tools, it reduced the maintenance of test scripts without, in any way, affecting the quality of testing.

The higher reuse of scripts reduced effort during maintenance, as the automated test cases were based on the business processes. It was also possible to constantly generate new test cases, adding them to the existing automation so that they could run parallel to the development of the software.

At Cabot, we always stress on the Agile approach, where the coding and testing are performed iteratively and incrementally in sprints to ensure each feature adds to the quality and stability of the product.

Sprints are set timings during which the specific work must be completed and readied for review. After each sprint is completed, the team will determine which defects are to be corrected, which is done on a priority basis.

This entire process is executed through the development phase of the software with test cases running on a daily basis. Even though the number of test cases in the lower sprints could be initially less, it could drastically shoot up in the latter sprints if testing is done manually.

Such a scenario got eliminated with the implementation of the test automation suite. Test cases were run in an orderly manner and all the test cases that came up in a particular sprint were executed in that sprint itself. The automation suite, thus, provided consistency in testing by preventing the piling up of test cases in the latter sprints thereby saving time and human effort.

Our automation suite is integrated to Jenkins, a reliable open source tool which can perform Continuous Integrationand ensures continuous delivery. The tests would be conducted automatically (you can set the time for this) after each build and the results will be mailed to the development and testing teams.

In this way, errors can, instantly, be noted and changes made, without any wastage of time. Jenkins will also send out notifications of failure in the same fashion.

Behavior Driven Development (BDD) Approach

It’s a policy of Cabot to ensure transparency in product development. Not only do we collaborate with our cross functional teams, but we also update our stakeholders about the course of the product development on a regular basis. That’s why we decided to adopt the Behavior Driven Development (BDD) framework for our test automation suite.

Behavior-driven Development is an extension of test-driven development and uses domain specific scripting language to convert structured natural statements into executable tests. Cucumber acts as a tool for BDD and helps in understanding your software in a better way.

In the BDD methodology, the software is considered/viewed as a system with a behavior. BDD is all about managing both the technical insight of the software in correlation with business interests; we have been able to do this seamlessly.

Through BDD, specialized software tools are employed to accelerate the development process as planned. BDD helps you to plan a number of things like where to start the test process, what to include and what not to, what to name the tests, reason for a test to fail (if there is), how much to test at a single time and so on.

The main purpose of testing is to understand whether each unit of the software functions in the way they are meant to be. BDD dictates a semi-formal format for behavioral specifications which will be mentioned specifically in terms of user stories and then written down in a dedicated document. There will be a specific structure for each user story.

BDD details on how each desired behavior should be in different test scenarios and the expected outcome. It helps the Cabot team to communicate in a better way with our stakeholders, who can analyze and control the regression workflow with XML files.

Credits: Cucumber

The best feature of integration of Cucumber with BDD is that even people who don’t have technical knowledge can easily understand it as it uses plain text instead of complex codes.

Cucumber runs automated acceptance tests in the BDD style. It has evolved to be a flagship BDD tool and helps in understanding your software in a better way. The plain language parser known as Gherkin shows software behaviors to be specified in a language that non-technical persons can follow.

Initially, Cucumber was used for Ruby testing as a complement to the BDD framework, RSpec. The style of writing tests in Cucumber is so easy and the codes can be reused later. The main focus here is on end-user experience.

Conclusion

Automation has evolved to be the most important aspect in software testing and has been deemed critical to the success of any software development team. Speed, reliability, complete regression cycles with tight deadlines and reducing redundancy are some of the benefits of automated testing.

With the test automation suite developed by Cabot Technology Solutions, you can make use of the pre-built functions to conduct automated test authoring. To know more about our offering, get in touch with us today.

Facing problems with manual testing your software product? Don’t worry, we can guide you to implement a Test Automation Suite.

[Contact Us Today!](http://<a href="javascript:void(0);" onclick="getQuote();">Contact Us Today!</a>)

Originally published in Cabot Solutions Blog on July 26, 2017.