[wp-hackers] User Feedback and Testing
elharo at metalab.unc.edu
Tue Jan 3 11:04:42 GMT 2006
David House wrote:
> I think this will be a great alternative to automated testing.
> Although on the face of it it seems like we're going to need hundreds
> of people to fill this out, we really only need five or six per
> feature group, and the staffing can overlap so that we'd only need,
> say, 50 testers. This will be a comfortable expansion up from our
> current numbers and should really direct the bugfixing efforts.
This keeps getting mixed up with searching for and finding bugs. This
isn't about that. Allow me to summarize. There are two separate issues
here, both of which need to be addressed:
1. Bug hunting: finding areas where the code does not behave as it is
intended too. This can be addressed by unit testing, static code
analysis, edge condition analysis, code inspection, and various other
techniques. This is often the domain of professional testers and does
not require end user participation, though if an end user finds such a
big, great. This sort of testing can and should be automated.
2. Usability testing: finding areas where the code does exactly what the
programmer intended it to do, and yet this is the wrong thing to do.
This is where we need end users, because we cannot anticipate all the
areas where they are likely to encounter problems. The question here is
not whether the user interface behaves as the programmer intended it to.
It is whether the design of the user interface fells natural to the end
user, and allows them to accomplish the overall tasks in a
straightforward, obvious, and convenient manner. This sort of testing
really can't be automated, though it can be formalized.
Another difference: bugs are often binary. Either the bug is present or
it isn't. Usability is more of a continuum. It's a question of more or
less, not yes or no.
Elliotte Rusty Harold elharo at metalab.unc.edu
XML in a Nutshell 3rd Edition Just Published!
More information about the wp-hackers