Testing the Requirements: Classifications, Criteria, and Checklist

While testing software products, quality engineers face diverse defects that might greatly affect the release date. They can lead to delays in customer delivery dates and increased cost of development for a company.

Lots of issues result from ambiguous, incompleteness, or incorrect functional requirements. Fuzzy requirements lead not only to higher costs but also may result in:

• Bugs and errors during or after implementation
• A distorted view of functionality or product goals
• Double work for every part of the team (system, software, quality engineers)

To prevent such requirements from entering the development and testing cycle it is essential that a specialist on the team, such as a system analyst, a QA engineer, a product manager or other role, tests the requirements before your team starts the development of the project.

Classification of Requirements

Let’s consider basic notions and start with the definition and types of requirements. Despite the fact that there are plenty of definitions, I give my preference to the next one.

According to Ian Sommerville, “The requirements themselves are the descriptions of the system services and constraints that are generated during the requirements engineering process.”

This means that all types of documentation, such as user stories, diagrams, and scenarios are viewed from system analysts’ perspective as requirements that should describe the behavior of a system that your team is going to launch for clients.

There are four types of such requirements:

• Business requirements: a high-level business goal of a product or team that builds a product.

• User requirements: tasks or goals that customers want to achieve using the software and features necessary for that purpose.

• Functional requirements, which describe the features that a system will display or perform under specific conditions.

• Non-functional requirements, which limit services or functions that a system offers for clients.

The most popular chart that illustrates relationships among various types of requirements is provided in the book “Software Requirements” written by Karl Wiegers and Joy Beatty. I will cite a simple version of it below.

Requirements Quality Criteria

Collecting and writing down all necessary requirements for the system is vitally important, however, these requirements should also meet quality criteria. The following principles may help to identify which feature requests for the product require additional elaboration from the business’s or system analysis’ perspective.

• Completeness

Each requirement must contain all necessary information for readers to understand it, and there must be enough information for implementation by a development team. Obviously, it is not possible to elaborate on every single requirement for your product, however, it is important to check that documentation does not have ambiguity or missing requirements.

• Correctness

Each request must be correct. This means it must be worthwhile and truthful in relation to system functionality and stakeholder requirements. In addition to that, low-level requirements should not conflict with high-level or parental ones.

• Consistency

Project requirements must not conflict with any other requirements, inner and external documentation, and standards. You should pay attention to the relationships between the requirements and compare them to identify similar or partial duplicates before starting the development.

For example, if you use user stories in feature requests for your project, make sure they don’t contain duplicates as there may be cases when the same statement is used for different roles and the same goal is achieved in different ways.

• Unambiguity

Regardless of the fact that ambiguity is a basic property of natural language, requirements must be written clearly and explicitly. They must be easily understandable by the reader and must not contain possible different interpretations.

If the requirement is ambiguously formulated, it should be clarified to the extent that it is clearly understood by all participants in the development process.

• Viability

Requirements should be possible to implement within the project. Most projects have different limitations such as budget, employees’ skill level, development language features, etc. Before the requirements are set, they should be evaluated by business and development teams to ensure that their implementation does not exceed the costs of the project and does not require excessive effort on the part of developers.

However, before striking out the requirements, the product owner must evaluate the impact on the project’s or client’s needs.

• Testability

If requirements cannot be verified, it is hard to identify that they were implemented correctly. Testable requirements are key to building a product that can be easily maintained in the future and would provide clients with the functionality that fits their needs.

Moreover, performing requirements examination by a QA team to identify their testability also helps to avoid incorrect, incomplete, and ambiguous requirements in the project.

• Priority

Generally, prioritization describes the importance of each feature request and allows the project manager to identify the most valuable ones in order to plan the release schedule and find resources for development.

In addition, marking requirements as important, desirable, and nice to have helps stakeholders to reveal hidden user needs and helps developers correctly allocate the efforts.

• Traceability

All requirements must have unique identifiers that link them to the original business needs, specifications, code that implements them, and verification tests. This allows us to facilitate work with requirements throughout the development life cycle and to ensure their traceability.

After the analysis phase, but before the implementation, all requirements should be additionally reviewed by quality engineers to ensure that documentation meets all quality specifications. This helps prevent possible bugs before development starts.

Furthermore, it is good practice if your team has a checklist for evaluating the quality of requirements on the project. This greatly simplifies the requirements collection stage and reduces the time spent on the task by every party involved.

Requirements Testing Checklist

The following list can help you identify requirements that don't provide enough information about the customer's needs, describe the system environment, prevent the IT team from implementing the product, or cover it with tests.

If, during the review of the requirements, the answer to many of these questions is “no,”  then there is an increased risk that the requirements are implemented incorrectly.

Completeness:

• The requirements cover all system and customer needs.

• All requirements that need additional elaboration are marked as TBC (To Be Confirmed).

• The requirements cover all characteristics of the product, such as inputs/outputs interfaces, software/hardware, operation system, etc.

• The requirements describe all assumed errors (business and system faults).

• The requirements contain a description of the acceptance criteria (inspection, demonstration, analysis, testing, etc.).

• The specification provides safety and security claims correctly.

  
  

Correctness:

• None of the requirements have spelling and grammatical errors.
• Each requirement is stated in a clear, unambiguous, and precise manner.
• The requirements avoid statements about the solution to the problem.
• The requirements can be implemented within known limitations.
• The terminology is clear and consistent.
• The requirements don’t contain signs of multiple requirements such as conjunctions “and,” “but,” “or” or similar linkers.

Consistency:

• Each requirement is defined only once.
• None of the requirements have logic and temporal conflicts with others.
• The output data of one requirement is consistent with the input data of another respective requirement.
• All formalized and schematic material is consistent with the text of the requirements.

Unambiguity:

• The requirements are written in natural language using the customer's domain vocabulary.
• The requirements have only one possible interpretation, and everyone consents to their meaning.
• All terms mentioned in the requirements are defined in the glossary.

Viability:

• Required specialists, tools, software, and other technology available for implementation.
• Technologies and tools allow your team to fulfill every requirement.
• The requirements don’t limit the implementation method.
• The end product characteristics described in the requirements are realistic and achievable.

Testability:

• Each requirement can be testable or verifiable by some methods (inspection, demonstration, analysis, testing).
• Each requirement is quantifiable.
• Test results of the requirements can be estimated against predefined acceptance criteria.

Priority:

• Each requirement has an implementation priority.
• Priorities are justified in terms of product functionality.

Traceability:

• Each requirement is uniquely and clearly identified.
• Each requirement is traceable correctly to an origin high-level requirement, external documentation, and other project specification.
• The requirements are expressed independently and don’t mix with other ones.
• The requirements are structured in a hierarchic logical way that is clearly understandable for readers.

Requirements are as important a development artifact as the code based on them. So, the processes applied to feature development—review, test, refine—should also be applied to requirements.

Properly written requirements allow a company to involve new employees in a project more efficiently. Also, they provide technical specialists with a complete picture of the tasks being implemented.

Moreover, an appropriate level of requirements allows you to plan product improvements and changes without using a codebase and build a traceability matrix for your product, which directly affects its quality.

An efficient traceability matrix shows test coverage of the system, therefore it allows us to determine which services need more attention from the QA team.

All these measures could reduce the cost of implementation by finding solutions and answers to questions at an early stage when the requirements have not yet been implemented or tested.