REPORT is for bugs what INVEST is for user stories
There will be bugs.
This is a reality we all need to deal with. So you better handle bugs effectively. Bug reports are time sinks, as they often lack the right information to start working on them.
I came up with a handy mnemonic called REPORT that functions for bugs like INVEST for user stories. You can use it to check if your bug contains the right information to be picked up effectively.
Before elaborating on REPORT, I will explain why bug reports waste a lot of time.
Making sense of bug reports
When you receive a bug report, you might think: great! The development team has a problem to solve. It is not that simple. The development team now has two problems: a bug report and the actual problem.
Bug reports are the start of an investigation. As with any investigation, it might go smooth or lead to nowhere. And it all starts with the facts. Even Ace Ventura needs facts to figure out what happened.
When the police broadcasts an all-points bulletin (APB) to apprehend a suspect, the information provided is crucial. We are searching for a suspect that is 40–50 years old, black hair, athletic build and 1.75 to 1.80 meters tall. Better information in the APB, increases the chance to catch a suspect.
It is the same with bug reports. Having a bug report does not guarantee you will find the actual offender. Having the right information in a bug report is paramount. It helps the investigation go smoothly. If information is missing developers will need to put on their detective hat on to obtain the facts.
Getting the right information in a bug report is hard. There is a big disconnect between the bug reporters and the developers who pick up the bug. Bug reports are a place where technical and non-technical interact. It sometimes seems like they are both coming from a different galaxy.
Bug reporters often have no clue what developers need to effectively pick up a bug. Developers get tired and frustrated from repeating the same questions over and over. Bug report interactions often end in disappointment, leaving both parties disheartened.
Fed up with those damn bug reports!
If you ever worked as part of a software development team, then the following comments in bug reports will sound familiar:
- ”This is not a bug. This is how it is supposed to work!”
- “I cannot reproduce, therefore assigning back to you.”
- “It does not work, great. But can you tell me how you expect it to work?”
- “There are 10 different issues in this bug report. Can you please split it up?”
- “I did not fix this bug yet. If it was that urgent why didn’t you tell me?”
All these frustrations can be lessened by using a handy acronym called REPORT.
REPORT stands for:
- Reproducible —What are the exact steps to reproduce without requiring prior knowledge?
- Expected results — What do you expect?
- Priority — What is the impact of the issue? Used to determine priority.
- Observed results — What is happening?
- Relevance — Keep bug reports atomic and limited to the information relevant to reproduce a single issue.
- Traceable — If it does not work as agreed, link it to the description of the scope where how it is supposed to function is described. Context matters.
Reproducible
As a bug reporter, it is important to realise that developers who will pick up your bug might know less about the application than you from a functional perspective. A development team is normally responsible for just a part of the application. Knowledge in the team about their part of the application is often unevenly spread between the developers in the team.
So when a developer picks up a bug report, reproducing the issue might require using functionality they know little about. This is why bug reports need to be as easily reproducible as possible, preferably without requiring any prior knowledge.
All steps necessary to reproduce the issue should be present. These steps should be so clear a developer without background knowledge can perform them, regardless of the environment they are using. It is important to mention if the issue is consistently reproducible or not.
If there are no reproduction steps but you know the issue exists, then just mention you do not have a reproduction path. Do provide the evidence you have that it exists.
Common pitfalls:
- Necessary preconditions are missing such as the browser used or specific configuration settings.
- The issue depends on the environment where it was reported. Sometimes the cause cannot be figured out and this cannot be prevented.
- Reproduction steps are missing or described in a vague way. The developer has to guess or play around to reproduce the issue.
- The variable nature of the issue is missing. The result is that the developer gives up on reproducing the issue too early and sends it back to the reporter.
Expected results
Expected results are often missing from bug reports. The bug reporter writes:”It does not work the way I expect”. This expectation is never made explicit. The developer pulls out his crystal orb and rubs on it to guess what the bug reporter expects. With varying degrees of success.
If you do not want the resolution of your bug report up to chance, it is necessary to state how you expect it to work. The chance you get what you want is higher and the chance of rework is lower.
It could also be that how you expect it to work conflicts with original requirements which were agreed upon. Making your expectation explicit creates space for discussion.
Priority
All relevant information should be present to determine the priority of an issue. The priority of an issue results from the impact. By knowing the impact, the priority can be determined and validated by the customer and developers. If you provide priority and no impact, then nobody can double-check the priority.
- Who are affected? All users or only a specific subgroup of users? Are they always affected?
- How are they affected? Can they still use the application for the most part? Is a suitable work-around available?
- What do we expect the impact on the business to be?
I once worked for a customer who reported every bug as blocking or critical. This was the result of us not picking up issues with lower priority. As a result the product missed deadlines and the customer was not happy with the delivered quality.
Not all bugs are equal and almost always it is impossible to fix all bugs before a deadline. Being clear about the impact allows to prioritize effectively. If everything has equal priority, then nothing has priority. By stating the impact, the priority becomes negotiable. With a higher-quality product and a happier customer as the result.
Observed results
A description of what happens when you perform the steps as described in the bug report. Information about when the bug does not happen may help to pinpoint the root cause.
Common pitfalls:
- Observed results are incomplete. The user does not see a value and checks if it is present in the CMS system. The CMS check is not mentioned in the bug report even though it was performed.
- The cases where it work as expected are not mentioned. These might provide useful information about the root cause of the issue.
Relevant
A bug report needs to be atomic. Bug reports should only contain the information relevant to reproduce a single issue. If many different issues are present in the same bug report this introduces noise. Remove all unnecessary steps and superfluous information to prevent confusion.
Common pitfalls:
- The user describes twenty steps to reproduce an issue. Investigation results in only two of those steps actually required to reproduce the issue.
- A bug report contains a collection of different issues. The issues originate from different pieces of code. Mentioning them in the same bug report restricts developers from working in parallel. Testing becomes more complicated. If a single issue is not fixed, then the whole bug report needs to be reopened. What is working or not becomes unclear.
Traceable (optional)
Tracing bug reports to the original requirements gives developers valuable background information. Developers can check if what they are fixing conflicts with original requirements and may cause new issues. In fixed-price contracts traceability is vital. It allows to determine if an issue is in scope or must be treated as a change not part of the original contract.
Sometimes traceability is not possible, for example when you use software for which the customer does not have any requirements documentation. If you do not run fixed price contracts or want to perform root cause analysis, then traceability may add little value. The context determines if traceability matters. In the words of the agile manifesto: “Customer Collaboration over Contract Negotiation”.
Conclusion
When bug reporters uses REPORT to judge the quality of their bug reports it will help developers to:
- Pick up issues efficiently.
- Reduce rework.
- Decrease the chance of picking up something that conflicts with requirements.
It will help your customer to understand what developers need without a lengthy explanation. It provides a handy acronym to fall back on and increase understanding in a simple yet effective manner.