Peter Seebach ([mailto:crankyuser@seebs.plethora.net?cc=htc@us.ibm.com&subject=To err(or) is human] crankyuser@seebs.plethora.net) Freelance writer 06 Oct 2004
Abstract: It is not enough to say the software's awful. How do you help developers get it right on the next iteration? Cranky user Peter Seebach illustrates the difference between good and bad error reports and demonstrates how you can craft an effective error brief.
One of the recurring problems that users face is that computers...well...don't always work quite as well as expected (and sometimes not at all.) But that doesn't make it any easier for developers and vendors to fix computer problems.
For developers to fix the problems, they need to understand what the problem is. Here's where the error report comes in. When a vendor is willing to address a problem, a report that describes the problem well can be the difference between a quick turnaround and a quick runaround.
I'm going to provide it in easy-to-print, cut-and-paste form so you can print it out and paste it up near your computer:
In this month's column, I'll review some of the things that make for good and bad error reports. Your goal is to report an error in a way that the vendor has a head start on finding and fixing the problem. (The ultimate goal, of course, is to persuade vendors to not release software before its time, but that is beyond the scope of this column: Quite possibly, it's beyond the capacity of human endurance.)
What is an error, anyway? It's when you try to make the computer do one thing and it does another.
These three things form the core of an error report. If you don't include them, you simply haven't described the error sufficiently.
It's not an error that my computer hasn't saved this file in the last five minutes when I haven't told it to. But, when my chat program just suddenly closes without warning, that's an error. Just like with my pets, I expect it to wait until I tell it to do what I want it to do.
In describing errors, the more specific you are, the better. I think nearly every system administrator has at one point or another been told that someone's e-mail doesn't work. What does this mean?!?! Perhaps the user can't send e-mail. Perhaps the user doesn't receive e-mail. Perhaps the user sees an error message; if so, the phrase "doesn't work" discloses none of that information. Perhaps the software went to a party last night and just didn't show up on the desktop this morning (and didn't call in). Be specific.
One purpose of a detailed description of user actions is to make it easier for developers to reproduce an error. When developers can reliably reproduce an error, the error is a lot easier to debug. If the problem only happens some of the time, say so! If it happens under specific circumstances, include those as part of your report.
One famous bad error report involves a flashlight. The user complained that the computer wouldn't work. After a few minutes of troubleshooting, the tech support rep asked the user to make sure the power cord was plugged in. "Okay, but I'll have to get a flashlight. The power's out and I can't see behind the computer."
While this story might be true or not, incidents like it happen all the time. These problems can be the most difficult to figure out. If the user understood why the information was relevant, they wouldn't report a problem in the first place.
Often users find it hard to recognize when information is missing. If the apocryphal user with the flashlight had understood that the computer used electricity and things which use electricity don't run when the power's out, the call would never have happened.
This type of problem mostly involves users who know very little about computers. Unfortunately, it's easy for people to mistakenly believe that they know more than they do -- after all, what if you don't know enough to know how much there is to know?
This problem is hardly unique to users; everyone I know has had at least one conversation with a technical support person with the same problem.
The best preparation that you, as a user, can make is to get a good, basic understanding of how your computer is supposed to work. It takes a little time, but that knowledge can make it a lot easier for you to offer better reports when you experience computer problems -- and sometimes save you the trouble of sending a report to the wrong vendor.
A related issue is forgetting to mention things that are relevant to the developer. If I'm having problems running a video game under Windows, the technical support staff will want to know whether I'm talking about Windows 98 or XP. If my word processor is crashing, the vendor probably needs to know which version I'm running.
Many things that vendors frequently ask about may seem irrelevant at first, but likely do matter. A basic assumption is that the specifications of a computer and the software running on it always matters. Maybe it doesn't matter for a given problem, but including this information is a good idea anyway.
Of particular relevance is whether or not you're having other problems. It's a different matter to say that a single program is crashing frequently than to say that all your programs are crashing frequently. If multiple programs are crashing, that's probably a sign that the individual programs that are not at fault.
Another thing people forget to mention is recent changes. Even worse is a tendency to paper over them and try to extend blame to the vendor. Under careful examination, "But I didn't change anything!" often turns into "well, admittedly, I put in a new motherboard last week and come to think of it, I installed an OS upgrade."
Always tell the support person what's changed. Did what you're trying to do ever work? When did it stop? Did anything change? Anything at all? Users are often a little casual (and embarrassed) about changes that shouldn't matter -- sometimes the changes do matter. If your printer stopped working right after you moved your computer, the number one reason might be that you forgot to plug it back in. Trust me.
Vendors can do other things to help get good reports. The corollary to the endless claims of "but I didn't change anything" is that a lot of vendors try to blame everything on any change you made. No matter how silly.
I had a bad laptop battery once. The first support rep I talked to decided that the real problem was that the power management code in NetBSD was probably buggy and that if I only ran Windows, everything would be fine. I pointed out that the bogus data showed up even in the BIOS, but he wouldn't hear of it. Because of his diagnosis, the vendor sent me two replacement laptops before they tried replacing the battery. The support rep's fixation on a change that made no difference just made progress toward a resolution more difficult.
Accept bug reports and support requests through e-mail. Web forms don't cut it and never will because a Web form is an invitation to the user to skimp on details. It could be worse: The form might demand details that aren't appropriate or be buggy.
I never was able to send in a support request for one program I used because a bug in the form's JavaScript made it impossible to fill out the form successfully -- so I could never submit it. Plain old e-mail works, and should be used by preference.
This week's action item: Write the best problem report you can for a problem you have. See if you get a response. If you do, ask how you can improve your report.
Photo of Peter Seebach Application errors got you down? Peter Seebach says that the more you write vendors with error reports, the better the chances that they'll figure out their software has errors! Please don't send your error reports to [mailto:crankyuser@seebs.plethora.net] crankyuser@seebs.plethora.net