[theme-reviewers] Theme Review Feedback at WPTavern Forum

[Chip Bennett]

On Wed, Aug 25, 2010 at 11:30 AM, Otto <otto at ottodestruct.com> wrote:

> On Wed, Aug 25, 2010 at 11:15 AM, Chip Bennett <chip at chipbennett.net>
> wrote:
> > Semantics. We're not "rejecting" a Theme (or its developer); we're
> > suggesting that a Theme *as submitted* not be approved for inclusion in
> the
> > repository.
>
> In other words, "go away, we don't want you to be a part of our community".
>

If you're saying that's what the Theme developer is *inferring*, then fair
enough: let's fix it so that they get the proper inference. Because that's
certainly not what a "not-accepted" Theme review is *implying*.

>
> Not allowing somebody into wordpress.org is rejection in every sense
> of the term. It's a rejection of their theme, a rejection of their
> code, and a rejection of that person.
>
> > We have a very limited implementation of Trac to work with.
>
> Just tell me what you need the tools to do in order to make this work,
> and we'll try to work together towards making the tools do what they
> need to do. I'm a programmer, I can do that sort of thing.
>

Probably summarized elsewhere and throughout, but I'm sure we can come up
with a full list, if need be:

1) a way to keep a Theme associated with a given ticket, all the way through
final approval
2) a way for Theme developers to update and resubmit directly to their SVN
directory, rather than continually through the uploader, for any given
revision/Trac ticket.
3) more appropriate resolution options, to allow the back-and-forth to take
place within one single ticket.

>
> > You'll have to ask Cais for exact details, but what you're asking for, I
> > believe, would require a LOT of manual intervention in order to make it
> > work.
>
> Then let's do that intervention. This process simply does not work as
> it stands, and if we need to change it, then that's fine. But before
> we change it, let's decide on the right way, then change it to do it
> that way.
>

That manual intervention would probably be asking way too much (Cais?). So,
let's focus on improving the process.

>
> > If you can have some influence in improving our ability to take advantage
> of
> > Trac, then I think every one of us would be supportive of and grateful
> for
> > that improvement.
> > For example:
> > 1) Find a way to link new revisions to the already open Ticket for a
> given
> > Theme
> > 2) Add new options for ticket resolution, such as "needs further
> revision"
> > or whatever
> > Every one of us, I believe, would LOVE such improvements.
>
> Done. I'll put those on the list.
>
> > We have to balance. I like to give very thorough reviews. But, if a Theme
> > has been submitted 2,3,4 or more times and *still* fails the review, then
> > IMO it is clear that the Theme developer isn't making the effort to
> ensure
> > that the Theme meets the Guidelines. We don't want to incentivize such
> > behavior.
>
> Also understand that it might also be an indication that the theme
> author is not understanding the purpose or reason for some of the
> feedback he's being given. If we are going to be looking at themes for
> some set of predefined things to be there, then we should have a well
> written and highly verbose explanation of why we think it should be
> there that should be given to the author directly, every single time.
>
> More than a sentence. Somebody lacking "body_class" should get three
> or four paragraphs on why it's a good idea to use it.
>

I disagree. Link to the Codex references, and improve the Codex entries if
need be.

I suppose we can draft boilerplate text for each function/tag/hook, if need
be, to put in a Trac ticket comment. If that is the best use of my time, for
the better of the Theme developer community, then I'd be happy to work on
that.

>
> In my experience, most people *can't* educate themselves from mere
> documentation. You do have to hold their hand through it. Once they
> get over that initial learning curve problem, *then* you can let them
> go on their own. But you never get anybody to improve until you guide
> them *really* well through those first steps.
>

If the Codex isn't written well enough to be a useful educational tool, then
we need to look at improving the Codex. But reinventing that wheel with each
and every Theme review would not be useful.

>
> > But, again: Trac
> > is a mostly foreign concept to a lot of Theme developers. Not only are we
> > having to educate them regarding new review Guidelines, but we're also
> > having to educate on how to make use of the Trac system, and the tickets
> > generated for their Theme submissions.
>
> Okay. Then let's dump trac.
>
> Or let's put a new face on trac to make it easier.
>
> Or *something*. I don't claim to know what would work, and I don't
> claim to know all the answers. I just know that it doesn't work now,
> and I'm willing to help in changing to make it better. Because it
> *has* to get better, somehow.
>

So let's make it better! Tell us what our options are, to improve the system
with which we are working. Trac is pretty powerful. Help us make better use
of it.

>
> > As for helping Theme developers understand why certain criteria are
> > required: we've been working very hard on improving the Theme Review
> Codex
> > page, to make it more clear, understandable, and concise. We've added
> Codex
> > cross-references for darn-near everything. It is not too much to expect
> > Theme developers to keep abreast of the information available in the
> Codex,
> > especially regarding standard WordPress features, functions, tags, and
> > hooks. We're doing everything we can to point them in the right
> direction.
>
> I do appreciate and understand this, and don't want anybody to take it
> as criticism of them. I'm being critical of the process here, not the
> people.
>
> > Have you actually "experienced" it? Have you reviewed a Theme? Have you
> > submitted a Theme for review?
>
> Yes. You reviewed one of them. You may not have known it was me, as it
> wasn't under my name. :)
>

And if you have any feedback or constructive criticism of my review, I'd
love to hear it - on the mail list, or to me personally: I have no
preference. I'm sure my own reviews can be improved, and others may benefit
from that feedback also. So please: give me your honest opinion.

>
> > Agreed, regarding eventual approval. If we can make that process easier
> for
> > both the Theme developers and the review team, then by all means, let's
> do
> > it.
> > But just because the *process* has issues, doesn't mean that the
> Guidelines
> > aren't sound.
>
> I think that some of the guidelines are a problem in that they are
> being used as cause for rejection when they should not be, however I
> also have an issue with the main idea of "rejection" itself. You may
> not think of it as actual literal rejection, but it very much seems
> that way to the outsider.


So let's drop the use of the term "reject" with respect to reviews of
submitted Themes. Help us implement a system in which our only option is
"suggest approval" or 'suggest not-approval" (reject). If we're giving the
impression that we're "rejecting" community contributions, then we need to
fix that impression.

Chip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wordpress.org/pipermail/theme-reviewers/attachments/20100825/95d0f7fe/attachment.htm>