Or should you refrain unless and until specific steps to reproduce the problem can be determined?
To me, the answer is obvious - report the bug! Let me tell you why.
When we talk about non-reproducible errors, there are several possibilities:
A. There is actually no bug, and what I thought I saw didn't represent a bug
B. There was a bug, but it has been fixed since I last observed it
C. There is a bug, but I don't currently know how to make it appear on demand
If I don't ever report the bug, then it's probably Ok for case A and B. copyrightjoestrazzere
But if I don't report the bug in case C - I have failed to report a real bug. And now, the possibility exists that this bug will not get fixed, and will ultimately be discovered by the customer. I consider this "a bad thing"!
If I report the bug, in case A some investigation will be needed, and it will ultimately be determined that the bug report is invalid. If my company likes to (misguidedly, in my opinion) penalize testers for invalid bug reports, I may get penalized
If I report the bug, in case B some investigation will be needed, which will (hopefully) result in the bug report being marked as Fixed. Then the usual fix verification can occur.
If I report the bug, in case C the developer may be able to determine the cause by examining the source code, or other testers may encounter the same problem. In either of these cases, the bug report would be updated with additional findings and the bug will be fixed as usual.
Here's the table of possibilities:
(A) There is no bug
|
(B) There was a bug, but it has since been fixed
|
(C) There is a bug
| |
Don’t write a bug report
|
Ok.
|
Ok.
|
FAILURE!
Bug may get to customer undetected and unfixed.
|
Write a bug report
|
Ok.
Some wasted effort.
Risk of penalty for writing an invalid bug report.
|
Ok.
Normal verification process.
|
Ok.
Normal verification process.
|
To me, the risk of not reporting this potential bug far outweighs the additional investigation that might unnecessarily be triggered.
On my teams, I encourage my skilled testers to use their talents wisely. If they see what they believe to be a bug, they should attempt to reproduce it.
If, after their investigation, they still feel they found a bug but are unable to reproduce it, then they should file a bug report, indicating that it isn't yet reproducible.
I never let my testers get penalized for bug reports like these that eventually turn out to be invalid. If there are complaints about these bug reports, I am more than happy to take the hit.
From James Bach's Blog comes this excellent post "How to Investigate Intermittent Problems":
http://www.satisfice.com/blog/archives/34
