Bringing Effectiveness in Bug Reporting
Bringing Effectiveness in Bug Reporting was originally written to address some fundamental issues that surface during bug reporting. This article discusses a few typical problems that surface and a few recommendations that have worked.
Problems in Bug Reporting –
- Reproducibility Issues: This is the most common among problem. The Testers log bugs and the developers discard them claiming that they are not able to reproduce it.
- Works as Design Issues: Its very common when a Tester logs a bug and the developer marks it as work as design where in many times a tester is right but its just this that he does not know how to justify him.
- Not enough Information: Many times the developers discard the bugs claiming that not enough information is provided with the bug. It can be with respect to the Use Case/Requirement ID/Environment details/and steps to reproduce.
- Duplicate Issues: Many a times bugs are marked as duplicate by the developers which is not always true.
- Severity and Prioritization Issues: There are issues which a Tester will find Severity 1 and the dev Team will change it as a last Severity/Priority.
Recommended Solutions to bring in Effectiveness in Bug Reporting:
1. Handling Reproducibility Bugs: This is a most common problem and at times the development could be right that they are not able to reproduce it. There can be zillions of reasons because of which a developer is not able to reproduce it but as a Tester there are certainly some things you can ensure that you do your best to ensure that your bug is not marked with Unable to Reproduce. Here are the steps:
- Mentioning Percentage of Reproducibility: There are certain bugs which we as testers log where the reproducibility of bug is very low, which we should log but in this case we should mention the exact environment details, exact steps to reproduce, screen caps of any, and the most important mention the Percentage of Reproducibility. Mentioning the percentage of reproducibility will ensure that the developer knows it before hand and will take it seriously before changing the status.
- Using Registry Activated Logging Service: There are some techniques in Microsoft Windows which you can use to generate crash/event logs. Capturing them and attaching them along with the bug will also help the developer to point to the exact location where the problem has encountered. Here are the steps how to do it.
- Enable Windows Installer Logs: Windows Installation logs can be generated by Adding Registry EntriesWord of Caution: If you use Registry Editor incorrectly, you may cause serious problems that may require you to reinstall your operating system.Start Registry Editor (Regedt32.exe), and then create the following path and keys in the registry:
Reg_SZ: Logging Value: voicewarmup
The letters in the value field are the options that are available to use with Windows Installer logging. You can use the options in any order. Each option turns on a specific logging mode. For MSI version
1.1, the function of each option is as follows : v – Verbose output
o – Out-of-disk-space messages
i – Status messages
c – Initial UI parameters e – All error messages w – Non-fatal warnings a – Start up of actions
r – Action-specific records
m – Out-of-memory or fatal exit information u – User requests
p – Terminal properties
+ – Append to existing file
! – Flush each line to the log
* – Wildcard, to log all information except for the v option. To include the v option, specify /l*v.
It is recommended that you use this service only for troubleshooting. Leaving the service turned on creates a new Msi*.log file every time you use the Add/Remove Programs tool in Control Panel. This activity adversely affects system performance and disk space.
Enable Windows Installer Logging by Modifying Group Policy You can use Group Policy to enable logging by modifying the appropriate organizational unit (OU) or Active Directory Group Policy:
Click StartR -> Run.
In the Open box, type gpedit.msc to start the Group Policy Editor.
Expand Computer Configuration, expand Administrative Templates, expand Windows Components, and then click Windows Installer.
Double-click Logging, and then click Enabled.
In the Logging box, specify the options for what you want to log. The
log file, Msi.log, appears in the Temp folder of the system volume.
For more information about MSI logging, refer to Windows Help. Search Help by using the phrase “msi logging,” and then click the “Managing options for computers through Group Policy” topic.
d. Uploading Video Files: Recently we have started incorporating it. Whenever bugs start going back and forth between dev and test team, the best thing is to record the video while reproducing the bug with a tool such as Camtasia and upload it along with the bug
- Handling WAD (Works as Design Issues): At times there are bugs which get marked as Works as Design by the dev team. There are many reasons for that, the best case is check with the Requirement Specifications itself before logging the bug. And in case you are sure, log it and in case the Dev Team rejects it, include the BA and include his comments in the bug details section. I have also seen bugs related to negative testing resulting in unhandled exception errors which the developers marked it as WAD (works as design). Initially we just raised it to the technical management to make sure that it no longer remains Works as Design. Later, we included a checklist itself during the beginning of the Test Cycle which ensured the compliance by the development Team.
- Bug Scrubs: Bug Scrubs is one of the practices which was used in one of my ex-organization and I found it extremely effective to avoid bugs going back and forth between development and Testing Teams. Bug Scrub refers to a meeting scheduled where in you have a dev lead, QA Lead, BA, and a Product Manager to prioritize the bugs and handle/update the status of WAD bugs. This ensures everyone is on the same page and has everyone’s consent on the status of the bug.
- Requirements Analysis Checklist: Testers can have a Requirement Analysis Checklist and should review it thoroughly clearing out any amount of ambiguity in the document. This ensures the early catching of the bugs and helping to fine tune the document to produce effective and more usable Test Cases.
- Other Solutions: Some other tips for Effective Bug Reporting are:
- Have a crisp and concise subject line bug still should be able to deliver/communicate themessage about the bug details. For example, using a term like “Using euro characters/long file names in the User Name field on xyz page results in Application Crash”.
- In the Summary section, always write point wise: accurate environment details right from browser/application version, service pack (if any), OS details, hardware details, network details (if applicable), and whatever is applicable from the Application perspective.
- Write the exact steps to reproduce the bug. Remember, not one more word/sentence more or less.