[wp-hackers] User Feedback and Testing
ringmaster at midnightcircus.com
Sun Jan 1 23:42:28 GMT 2006
Ryan Boren wrote:
> On Sun, 2006-01-01 at 15:17 -0500, Elliotte Harold wrote:
>> I'm curious. Has any actual user testing been done on WordPress? In
>> other words, has anyone put non-developer users in front of a WordPress
>> system, asked them to perform tasks, and watched them to see how they
>> do, and what confuses them? I suspect the results would be informative.
>> There are some other problems I've noticed in the UI, but so far those
>> are all common to 1.5 and 2.0.
> wordpress.com always tracks the latest development, and we have many
> thousands of users on wp.com who love to give feedback. They worked
> with all the iterations of the RTE and the inline uploader, and their
> suggestions shaped development. The RTE is a major draw to wp.com.
Testing is the fundamental reason why I think there is so much churning
on these topics, and it's not just the kind of bug/patch testing that
we're accustomed to hearing about on these lists.
I've heard more than once that this is the best-tested version of
WordPress yet. Maybe more people have downloaded and tried pre-release
versions of WordPress than before, and maybe there has been a lot of
feedback from WP.com (none of which seems to be made public like it is
in Trac, now that I think about it). Does that imply that user testing
was done? Have we really asked uninvolved users what they think of the
software? Whether they think something could be improved? Or are we
basing what we think works on how well they receive features that they
have no influence over?
People say that WordPress was released too quickly. I don't think this
is phrased correctly. I think that an inadequate testing duration was
allotted for those last minute patches that were applied. Nevermind if
every single patch works flawlessly - What kind of policy is it to
release as large a project as this without letting the code burn-in on
testers machines for a few days?
We could keep pushing back the release date forever because the bugs
would never end. We would never release a new version. Yeah, I get the
idea. Likewise, we could push the release back two weeks and allow a
burn-in of frozen code to make sure those last-minute patches are
working as expected. Or, we could push the release back a month and
have more complete documentation. Clearly, a four-day release delay was
valuable to the support folks. In a delay's worst case, if none of the
remaining bugs are addressed during that period, you have two things you
didn't have before: 1) A more thorough idea of what is wrong with the
release for use by support. 2) A firm, organized release date.
Real products have development cycles, where development is broken into
phases that allow for discussion of new feature requirements,
development of planned new features, testing of those features, and
providing documentation and support for those features upon release.
These cycles are not necessarily periodic (as in using a regular release
date), but if are planned properly can be assigned a best-guess timeline
that can be used for planning useful things like press releases, site
design updates, and support ramp-up.
With a project this large affecting so many people, it seems about time
that the development cycle of WordPress becomes a bit more openly
scripted. Otherwise we can continue as is: Guessing at new features,
wondering if the new features are addressing the needs of users,
speculating on release dates at the expense of complete documentation
and support, and watching a growing unease among the project's most
My belief is that nothing positive can happen if nobody says anything.
I hope I have contributed positively by airing my thoughts on this
matter, and I hope that my thoughts are considered and reasoned before
hasty or pat responses are returned.
More information about the wp-hackers