[theme-reviewers] Payment Option for Theme Reviews

[Chip Bennett]

Adding more admins, or getting a paid full-time admin, would merely be a
band-aid. The issue won't truly be resolved until the second review becomes
unnecessary. What will help mark Themes live is ensuring that they don't
need an admin re-review. As Otto pointed out: the final admin review is
there, because it is needed, unfortunately. There are too many approved
Themes that have significant issues, that need to be resolved before those
Themes are placed in users' hands.

Theme review is hard, it has a steep learning curve, and it takes a while
to get really comfortable with it. The biggest hindrance is reviewer
training/education. The admins are doing what we can to mitigate that, but
time spent on education/training is time not spent on auditing Themes and
pushing them Live.

I have seen some improvement overall, but it has been marginal (I have to
reopen probably 80% of tickets, where before I had to reopen 90% of tickets
- give or take).

What I suspect is still happening is that most reviews are conducted like
so:

1) Install Theme
2) Run Theme Check, dump output into the ticket
3) Look at the Theme on the front end

Unfortunately, that should be the final 10% of the review, not not bulk of
the review. But I really think that's what's happening.

Reviews should look more like:

1) Review style.css for license/keyword information
2) Review readme.css for license information and look for custom
features/functionality
3) Review header.php
4) Review footer.php
5) Review functions.php (and included sub-files)
6) Review bundled/included resources, and ensure they're listed with
copyright/license information in the readme

80% of important issues will be found with just those 5 steps.

If the Theme uses a base with which you're familiar (Underscores, Twenty
Twelve, etc.), you can most likely ignore the rest of the
template/template-part files.

>From there:

7) Install the Theme, run Theme Check. Ensure there are no critical or
required issues. Review (but DO NOT POST) the recommended and info output
(INFO might include hard-coded links, that you'll need to check to ensure
appropriateness, but you wouldn't put anything in the ticket, unless you
find an inappropriate link).
8) Look at the Theme on the front end and in the back end. Ensure there is
no PHP error/notice output. Ensure there are no deprecated notices. Verify
custom functionality works (if applicable) - assign a menu to a theme
location, add a Widget to a dynamic sidebar, etc.
9) Ensure all style.css keyword tags are appropriate.

That will catch almost all of the rest of the required issues.

Notice that this implies a code-review and functionality-focused review. We
tried to emphasize this point by making the Theme Unit Tests officially
only *recommended* instead of *required*: we're primarily reviewing code
and functionality, not design and aesthetics.

(Yes, all of this is going into a re-write of my guide to reviewing Themes;
but it's taking a while to complete.)


On Thu, Jul 31, 2014 at 3:01 PM, Michael Hebenstreit <michael at mhthemes.com>
wrote:

> That doesn’t help the review queue and also won’t mark themes as live. :-)
>
>
> Am 31.07.2014 um 21:58 schrieb Emil Uzelac <emil at uzelac.me>:
>
> And you can get paid for reviews too, just go to
> http://jobs.wordpress.net/ and problem solved ;)
>
>
> On Thu, Jul 31, 2014 at 2:57 PM, Otto <otto at ottodestruct.com> wrote:
>
>> On Thu, Jul 31, 2014 at 2:52 PM, Michael Hebenstreit <
>> michael at mhthemes.com> wrote:
>>
>>> At the moment it seems that a lot of unexperienced reviewers are being
>>> motivated on WordCamps around the world. They review a few themes during
>>> the contribution day and then it’s over (in most cases).
>>>
>>> WordCamps aren't recruitment drives. Those sessions should be entirely
>> for the benefit of the people there, not for any other purpose. Our goal at
>> WordCamps, overall, is to educate and inform (and meet and greet and BBQ
>> and so on).
>>
>> If showing somebody how the theme review works and having them do one for
>> hands on experience helps them in any way, then that was the point. Whether
>> the result is useful to us or not is irrelevant, we only care whether or
>> not it was useful for that person.
>>
>> Wordcamps are for the Wordcampers. If they decide to continue reviewing,
>> great. If not, that's okay too.
>>
>> -Otto
>>
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wordpress.org/pipermail/theme-reviewers/attachments/20140731/76b71b75/attachment-0001.html>