[theme-reviewers] Public Redress of Theme Reviewers

[esmi at quirm dot net]

on 01/09/2010 20:58 Chip Bennett said the following:

<snip>

>>> And, please don't tell me to go and do a few reviews.

> Sorry, but that's a cop-out.

It's not! It's sheer lack of time! Plus I've been doing (and still do)
an almost identical design review/assessment for one community for 6
years. And I undertake *much* longer, more comprehensive, site/design
reviews with another team and have been doing that for 3 years.

I don't need to carry out yet more reviews to understand how the general
process works and where it can hit problems. I've been directly
responsible for refining and redeveloping the reviews on the other sites
I work with.

> If you do not understand our point of view, how can you admonish *us*
>  regarding a lack of respect?

The point I was trying to make is that I *do* understand your position
but I also think you're making some critical mistakes. Been There. Done
That. Got The T-Shirt.

 > It is rather easy to criticize. The Theme
> Review process is a classic Pareto example: over 80% of tickets are
> closed by four people.

And? One of of the sites I work with 100% of all tickets are closed by
(currently) 2 people. So what? These are just numbers. They say nothing
about the quality of the review or the relationship that can be
developed with the submitter. On this same site, a review is normally
left open for at least 4 weeks to allow the developer to come back and
discuss the issues. Many don't bother but some do and we work *with*
them. We also don't assume that there is only ever one way to achieve a
given end and invite submitters to argue their design decisions. For us
it's all about promoting thoughtful & aware design and never about
ticking boxes.

> An average of 10 Themes are submitted per day. We have a system and
> tools in place that currently are less than ideal for processing 
> Themes in the most efficient and effective manner.

Correct me if I'm wrong but I've not seen many complaints about the time
it takes for a theme to go through the process. The concerns I've seen
raised have all been about the manner in which the review is handled.
And you don't need special tools to say "thank you" or to invite the
author to discuss the issues further. This about about being *seen* to
value people and their efforts - not the software being used.

> We started out with NO clear, concise review guidelines - we had to
> consolidate redundant and conflicting information spread across two
> dozen Codex pages before we could even really start reviewing Themes
> effectively.

Then, with all due respect, more fool you for jumping in at the deep
end. Given the task at hand, a phased approach that allowed the Review
Guidelines to develop more organically (for want of a better term) with
input from developers, and even, dare I say, users, would perhaps have
been more appropriate. Jumping into the deep end with both feet and a
huge list of do's and don'ts that appears to change relatively
frequently is, to say the least, highly risky. And the objections being
raised are the result of that.

> I'm sorry that we're not yet able to meet our goal of having every
> Theme reviewed within one day of submission.

I never said *anything* about the review timescales! Where did that come
from? For what it's worth, I think a more extended - and human - review 
period would actually work a lot better.

> I'm equally sorry that we're still
> not to the best level of consistency and correctness with our reviews.

But if you're acting as the gatekeepers, you *have* to be the best. You 
have to be 110% sure that *every* item on your list is *absolutely* 
correct. You have to temper the language that is used, replace words 
like "must" with "recommended" (after all, there is often more than 1 
valid way to skin any cat), be open to new ideas/approaches rather than 
just citing guidelines that *your* team wrote. And above all, you have 
to approach each review under the impression that the developer at the 
other end is busting with enthusiasm and that one wrong word from you 
will send him/her scuttling for shelter - never to return.

In short, you need to be approaching sainthood. It ain't fair. It ain't 
right but that's how it works. Anything less and you'll drive people away.

esmi