Explore the Top 5 Test Management Solutions for Banking

Quality Assurance in Banking is a tricky matter.

Releasing without major bugs is not a competitive advantage, but a regulatory requirement. Not every general-purpose test management vendor fits them, so finding good software is that much harder.

Let’s have a look at test management solutions that specifically advertise themselves as banking-ready and a popular tool that doesn’t.

Mistakes to Avoid

As I mentioned, banks are under greater scrutiny than most businesses when it comes to the quality of their software. Mistakes cost money not just for the bank, but its customers and potentially the country's central bank as well.

Regulatory compliance is the biggest caveat when it comes to test management solutions for banking. Here are certifications that your solution should obtain and/or not block you from obtaining:

Test Maturity Model integration.
ISO 9001.
ISO 27001.
ISO 26262.
ISO 13485.
FDA 21 CFR Part 11.

The depth of traceability is also a major concern. Should something ever go wrong, you need to find who made a negligent or malicious change to your software and/or approved an unstable version of the app.

Implementation of traceability functionality is a great point of distinction between suitable tools: the features can be useful in more ways than passing regulatory audits.

Not scrutinising on-premise offerings can prove costly. Information security requirements make it impossible to use Cloud-only solutions, so some of the test management market’s hip newcomers are out of the question.

On a similar note, you should pay extra attention to feature sets of solutions that have both Cloud and On-Premise versions. The latter could be lagging behind due to development nuances and/or the client portfolio of the vendor.

Neglecting capacity for large testing volumes can be pretty dire. It’s not uncommon for banks to reach 6-digit and even 7-digit number of tests. Some test management solutions are not built to handle that much, due to the database architecture and/or suboptimal client experience.

Banking Tools Landscape

I won’t speculate why, but surprisingly few test management solutions explicitly claim they are banking-ready. These include QA Cube, Polarion, and aqua. These services have dedicated success cases and claim to meet all the regulatory requirements.

Most popular test management tools are not as straightforward. A good example here would be Zephyr. Although they do not specifically talk about banking, they feature Pfizer as one of their clients. If Zephyr went through even bigger trouble of complying with medical industry requirements, they surely can work with banks, can’t they? Well, this is the kind of question you will have to ask yourself and/or clarify with sales for most test management solutions.

Picking an explicitly banking-ready solution or going through options that could be banking-ready are two valid approaches. In this article, however, we will focus on the first group.

Best Test Management Solutions for Banking

aqua ALM

aqua ALM was launched in 2013 by the German andagon group, which provides IT consulting services since 2001.

The company specialises in highly regulated industries, reflected by dedicated solutions pages for both banking, insurance, and government agencies. Some of the clients are Nürnberger Versicherung and BaFin, the regulatory body that a prominent competitor Polarion helps pass audits from.

Naturally, aqua uses traceability as one of the main selling points. The logs are ISO 9001, BRSG, and IFRS 17 compliant. Traceability both meets regulatory needs and brings its own features, e.g. comparing revisions and reverting changes through the project’s lifespan.

Security is another prominent aspect of aqua. It has over 100 separate user permissions rather than predefined access levels; both SAML and LDAP are supported for user management.

On-premise instances can be entirely handled by the client and completely isolated from aqua. The extensively documented REST API implementation uses oauth2 tokens for authentication, which means you can connect any third-party tool without breaching security regulations.

The main criticism for aqua is mostly about reporting and interface-based feature differences. Reports, while offering great depth, can be a bit tricky to set up and require the ALM licence to create new templates. Some features of the desktop client may be unavailable in the web version, but patch notes indicate progress there (most recently adding Agile functionality).

QA Cube

QA Cube (Swiss company founded in 2014) is a test management solution with a strong emphasis on visualisation.

They do not think that “merely” elevating from Excel spreadsheets to well-organised projects is good enough. The bread and butter of this solution are customisable dashboards, not a database of tests or Kanban boards.

Another selling point of QA Cube is predictive analytics. The company claims it will help identify potential points of failure and make testing more efficient. Alas, the same tech cannot be used for software development, since QA Cube is a test management but not ALM solution.

Other than that, QA Cube boasts customisable reports, integrations with test automation solutions, and Agile methodology support. These features are not exclusive compared to other test management solutions, so it comes down to implementation. Unfortunately, you will have to see that for yourself: QA Cube has a total of 0 reviews across G2 and Capterra.

As far as obvious flaws go, QA Cube lacks a future-proof (or legacy-friendly) solution to integrations. Even though they offer 25 integrations with mainstream and niche software, you’re limited just to these.

Some sort of an API solution would help potential banking clients connect their in-house software and, should QA Cube seize active development, hook new third-party tools as well. Some last-minute grievance: a 48-hour turnaround for customer support is pretty slow.

Polarion

Founded in 2003 and acquired by Siemens 12 years later, Polarion is a staple of test management. Their biggest banking client is the central bank of Germany, and Polarion also showcases about a dozen of European private banks and FinTech companies.

Polarion strongly emphasise their traceability features. The software logs all changes so you can demonstrate them to regulators and even automatically self-audits the log. Polarion specifically mentions helping to pass regular BaFin audits and complying with MaRisk requirements.

Another highlighted point is reusability. Polarion suggests cross-linking projects and requirements to reduce redundant work, automating workflows on the project and/or release level, and reusing tests. Polarion is an ALM solution, meaning that your developers can benefit from some of this functionality too.

Some relatively minor complaints about Polarion include the lack of documentation for some integrations and poor reusability of reports. The fundamental issue, however, seems to be performance at scale, as pointed out by several G2 reviews and explored in this one:

“Even if it is one of the best tools I have ever used, what I personally love and also dislike at the same time is the backend based on the Subversion. I love it, because thanks to its nature Subversion saves everything in a real concrete storage, with no possibility of data loss. My personal dislike is based on the fact that even [though] Subversion can store millions of revisions, it is file-based, so it may become quite slower than expected after some years”.

Summing up, Polarion is great and most reviews corroborate that. Performance, however, can be a deal-breaker. If you’re a bank looking for a test management solution, I strongly suggest that you discuss this potential issue with the sales team before proceeding.

Zephyr

Zephyr is a test management solution for Jira available since 2012 and created by US-based SmartBear. It is primarily an app for Jira rather than a standalone test management solution, although Enterprise has a separate client. Featured users include Microsoft and Expedia; no banks are listed on the website.

Zephyr’s strengths are in enhancing existing Jira’s functionality. It helps adapt Agile features to quality assurance work and makes Jira reports serviceable for test management purposes. The integration suite includes both third-party tools for test automation and REST API for connecting anything else.

Part of SmartBear’s portfolio, Zephyr preaches Behavioural-Driven Development. Shifting the testing optics to the end-user's experience is a valid approach, so Zephyr appears to be the go-to pick for this methodology.

Some persistent criticism for Zephyr concerns performance and stability. While “it takes too long to load” is subjective, multiple reviews claim that new releases also bring new bugs. These would take a few days to fix and can be disruptive. Although such reviews are from a couple of years back, stability is a great point to bring up when exploring Zephyr as a test management solution for banking.

QMetry

QMetry is, too, a test management solution from an American company launched in 2012. They list BFSI as one of the key industries and boast a diverse portfolio there. Perhaps their most notable client is Allianz.

QMetry’s focus is regulatory compliance. Logging, although not adding any extra value, still ticks all the boxes for audits. Alas, QMetry is a test management solution and not ALM, so regulatory compliance for development is out of the scope.

One rare strength of QMetry is the eSignature functionality. It allows users to sign off tests for management to later see who approved what. Built-in approval workflow reports make this feature even better. Other than that, QMetry offers out-of-the-box and REST API integrations and comes with decent reporting features.

The main con of QMetry seems to be the user experience. Reviews say that admin screens are hard to navigate; integrations take quite a bit of training and are generally a bit confusing. Just like with aqua, there is no negative feedback on performance — and that’s what matters in banking.

Banking test management solutions checklist

To recap, here is what you should look for in a test management solution as a bank or financial organisation:

☐ Fits banking-specific requirements

☐ Demonstrates that banking-specific features are executed well

☐ Has efficiency-driven general features (Reporting, Agile, Reusability, etc.)

☐ Handles large testing volumes well on the fundamental level

☐ Provides REST API implementation for in-house and third-party tools

☐ Continues to receive regular updates of the On-Premise version