[theme-reviewers] Payment Option for Theme Reviews

Thu Jul 31 21:00:32 UTC 2014

Relating to the admin bottle-neck: What about appointing trusted /
experienced theme reviewers? They can perform second reviews of the
"approved themes but not marked live" list. Since they are experienced they
will find issues quickly. They should not be allowed to mark themes as
live, but are able to reopen tickets.

As result outstanding issues would be reported back to the theme author
faster than now and the quality of the "approved themes but not marked
live" ticket list would improve. The final decision of marking a theme live
would still be made by the admins, but we could reduce the work load of
Chip, Emil, Otto and Justin a little bit. The need to reopen 80% of all
approved tickets is really too high in my opinion.

Of course this would still not solve our major problem that there are too
many themes and not enough reviewers.

2014-07-31 22:26 GMT+02:00 Chip Bennett <chip at chipbennett.net>:

> Adding more admins, or getting a paid full-time admin, would merely be a
> band-aid. The issue won't truly be resolved until the second review becomes
> unnecessary. What will help mark Themes live is ensuring that they don't
> need an admin re-review. As Otto pointed out: the final admin review is
> there, because it is needed, unfortunately. There are too many approved
> Themes that have significant issues, that need to be resolved before those
> Themes are placed in users' hands.
>
> Theme review is hard, it has a steep learning curve, and it takes a while
> to get really comfortable with it. The biggest hindrance is reviewer
> training/education. The admins are doing what we can to mitigate that, but
> time spent on education/training is time not spent on auditing Themes and
> pushing them Live.
>
> I have seen some improvement overall, but it has been marginal (I have to
> reopen probably 80% of tickets, where before I had to reopen 90% of tickets
> - give or take).
>
> What I suspect is still happening is that most reviews are conducted like
> so:
>
> 1) Install Theme
> 2) Run Theme Check, dump output into the ticket
> 3) Look at the Theme on the front end
>
> Unfortunately, that should be the final 10% of the review, not not bulk of
> the review. But I really think that's what's happening.
>
> Reviews should look more like:
>
> 1) Review style.css for license/keyword information
> 2) Review readme.css for license information and look for custom
> features/functionality
> 3) Review header.php
> 4) Review footer.php
> 5) Review functions.php (and included sub-files)
> 6) Review bundled/included resources, and ensure they're listed with
> copyright/license information in the readme
>
> 80% of important issues will be found with just those 5 steps.
>
> If the Theme uses a base with which you're familiar (Underscores, Twenty
> Twelve, etc.), you can most likely ignore the rest of the
> template/template-part files.
>
> From there:
>
> 7) Install the Theme, run Theme Check. Ensure there are no critical or
> required issues. Review (but DO NOT POST) the recommended and info output
> (INFO might include hard-coded links, that you'll need to check to ensure
> appropriateness, but you wouldn't put anything in the ticket, unless you
> find an inappropriate link).
> 8) Look at the Theme on the front end and in the back end. Ensure there is
> no PHP error/notice output. Ensure there are no deprecated notices. Verify
> custom functionality works (if applicable) - assign a menu to a theme
> location, add a Widget to a dynamic sidebar, etc.
> 9) Ensure all style.css keyword tags are appropriate.
>
> That will catch almost all of the rest of the required issues.
>
> Notice that this implies a code-review and functionality-focused review.
> We tried to emphasize this point by making the Theme Unit Tests officially
> only *recommended* instead of *required*: we're primarily reviewing code
> and functionality, not design and aesthetics.
>
> (Yes, all of this is going into a re-write of my guide to reviewing
> Themes; but it's taking a while to complete.)
>
>
> On Thu, Jul 31, 2014 at 3:01 PM, Michael Hebenstreit <michael at mhthemes.com
> > wrote:
>
>> That doesn’t help the review queue and also won’t mark themes as live.
>> :-)
>>
>>
>> Am 31.07.2014 um 21:58 schrieb Emil Uzelac <emil at uzelac.me>:
>>
>> And you can get paid for reviews too, just go to
>> http://jobs.wordpress.net/ and problem solved ;)
>>
>>
>> On Thu, Jul 31, 2014 at 2:57 PM, Otto <otto at ottodestruct.com> wrote:
>>
>>> On Thu, Jul 31, 2014 at 2:52 PM, Michael Hebenstreit <
>>> michael at mhthemes.com> wrote:
>>>
>>>> At the moment it seems that a lot of unexperienced reviewers are being
>>>> motivated on WordCamps around the world. They review a few themes during
>>>> the contribution day and then it’s over (in most cases).
>>>>
>>>> WordCamps aren't recruitment drives. Those sessions should be entirely
>>> for the benefit of the people there, not for any other purpose. Our goal at
>>> WordCamps, overall, is to educate and inform (and meet and greet and BBQ
>>> and so on).
>>>
>>> If showing somebody how the theme review works and having them do one
>>> for hands on experience helps them in any way, then that was the point.
>>> Whether the result is useful to us or not is irrelevant, we only care
>>> whether or not it was useful for that person.
>>>
>>> Wordcamps are for the Wordcampers. If they decide to continue reviewing,
>>> great. If not, that's okay too.
>>>
>>> -Otto
>>>
>>>
>>> _______________________________________________
>>> theme-reviewers mailing list
>>> theme-reviewers at lists.wordpress.org
>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>
>>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wordpress.org/pipermail/theme-reviewers/attachments/20140731/d95a55e4/attachment-0001.html>