[wp-hackers] User Feedback and Testing

[Owen Winkler]

David House wrote:
> As for usability, we don't need a centralised usability-bugs-finding
> service. If you want to do a usability report (we've had one in the
> past [1]), go ahead.
> 
> [1]: http://comox.textdrive.com/pipermail/wp-hackers/2005-April/000602.html

I think the intent of my original message got lost.

There is no escaping the importance of thorough bug testing, but let's 
set aside that issue for now.

I think that usability testing like that provided in the link above can 
be useful, but it's not what I'm talking about.  "Usablilty" in this 
sense is how well a user is able to navigate the interface.  That is 
useless when the features that the user wants aren't present or don't 
work.  WordPress developers need to take a harder look at what actual 
in-the-trenches WordPress users are doing with WordPress and then 
respond to their needs.

The primary response to the 2.0 release from people who are not 
upgrading is essentially this:  Version 2.0 offers me nothing I want 
that I don't already get from 1.5.2, and makes some of my more important 
plugins break, so why should I bother?

I suppose that many of the enhancements to WordPress are under the 
surface.  But it's pretty clear from the comments I've received that 
those features that are visible to users are not enticing them to 
upgrade.  Those users who were pleased enough by the upgrade that they 
felt compelled to comment were often vague in their reasons for doing so.

Amusingly, most people who have upgraded and commented that WordPress 
2.0 is good are people who describe the goodness as "The install went 
fine."  They don't provide the reason that they upgraded besides there 
being a new version to try.  I would expect that some would say, "I 
upgraded because I wanted the WYSIWYG editor."  I didn't see much of 
that at all.  The features that they liked surprised me, too.

The most-lauded new features of WordPress were the new included plugins. 
  Most folks didn't seem to care much about many other of the software 
enhancements.  Asking users not just what they want but how they want it 
to be done could have been useful in the development process.

What I'm suggesting is a complete rethinking of the WordPress 
development lifecycle.

First, we need to discover what the users want WordPress to do, and 
perhaps what users will want WordPress to do in the future.  Then decide 
on features to implement those requirements.  Code those features. 
Provide enough time for testing of those features, while instituting a 
support structure for it.  Getting thorough bug testing is only a part 
of a development plan.

I think these initial ideas about how to go about instituting better bug 
testing are pretty good, and we should follow through on them.  But we 
should do this as part of an overall development plan, not just 
something we slap onto whatever output the project ends up producing. 
I'm sure that a dedicated testing group would rather test bugs on 
features that they would want to have for themselves, and would produce 
better reports as a result.

I recommend the book "Software Project Survival Guide" by Steve C 
McConnell as reading for such an endeavor, which gives some advice on 
forming requirements for a project, and producing estimates for project 
duration.  This could be useful for producing a plan of attack for 
WordPress 2.5 or 3.0.

Owen