[wp-hackers] Wordpress 1.5 downloads

[Elliotte Harold]

Aaron Brazell wrote:

> Anybody can sign up for beta testing, downloading SVN updates, downloading nightlies. I've been using 1.6a since June. We probably do need to make it easier for people but we don't want people downloading nightlies and thinking they're going to get official support.  They are, after all, for testing and development.

In other words, no.

This is a real problem for open source software. Nobody does any formal 
or informal user testing; but this is critical for producing usable 
software, especially when you're designing something new and innovative 
like WordPress and not something that's just a copy of an existing 
closed source program that did do user testing.

Effective user testing requires going out and finding users. The users 
who come to you are not representative. Then you have to sit them down 
and watch them operate the system, not rely on what they choose to 
report. Only by doing this can you find out where the real pain points are.

For example, grab your father or your sister (assuming they're not 
techies) and assign them tasks such as:

1. Install WordPress
2. Write  a Post
3. Delete an existing post
4. Write a post in a new category.
5. Comment on an existing post

Then silently watch over their shoulder as they attempt to complete each 
task. Resist the urge to "help" or "explain" to them how they should 
proceed.

You will be astonished at what proves hard to do and what doesn't. You 
may want to ask them to verbalize their thought process in which case 
the mental models they use will be nothing like you expect. But resist 
the urge to correct them. You want to know how users think about 
software so you can design software that flows with them rather than 
fights against them.

Programmers are extremely poor judges of usability. No one can 
reasonably judge the usability of their own program without seeing 
actual users attempt to use it.

In many cases they will not be able to complete the task. They will not 
know how to file a bug. They will not even be able to articulate what 
went wrong. They may very well blame themselves and feel stupid, though 
the bad design that prevents them from completing the task is in now way 
their fault. (It's sometimes a good idea to offer testees chocolate 
after they've finished a test so they don't feel too bad. Be sure to 
emphasize after they've finished the test that any problems they had are 
not their fault; that you know the system is buggy and that's why you're 
testing it. You don't want them to leave feeling bad.)

But you're never going to get this sort of valuable feedback from people 
who figure out how to download and install a beta and then login to the 
bug tacker and write up a bug or user interface glitch. It took me a 
good half hour to figure out where and how to report the first bug I 
found in WordPress. If it takes an ubergeek half an hour to do this, how 
do you expect a non-programmer end user to accomplish this? That's not 
going to happen.

If you want to make usable software, you need to recruit users to test 
the software and then you have to watch them. Do not rely on 
self-reporting.

-- 
Elliotte Rusty Harold  elharo at metalab.unc.edu
XML in a Nutshell 3rd Edition Just Published!
http://www.cafeconleche.org/books/xian3/
http://www.amazon.com/exec/obidos/ISBN=0596007647/cafeaulaitA/ref=nosim