[theme-reviewers] Prioritizing The Review Queue by Developer

[Greg Priday]

I'm not sure that this would promote favoritism, especially if the
ordering is completely automated. It also wouldn't change anything
about the review process, only the order in which the themes are
reviewed. If I found an issue in one of @automattic's themes, I'd
gladly let them know ;)

I agree that first come, first serve is ideal. The problem is that we
can't view our pool of reviewers as a black box that gobbles up
tickets and spits out feedback. There are very few active reviewers
working against a constantly growing queue. My solutions might not be
the best ones, but we do need to come up with a solution to the
problem.





On Wed, Apr 10, 2013 at 1:12 PM, Bryan Hadaway <bhadaway at gmail.com> wrote:
> This would promote favoritism, whether intended or not.
>
> Review order and queue should remain relevant to the theme itself on a
> "first come, first serve" basis.
>
> And the review of a theme should only be based on the merit of that specific
> theme and have nothing to do with the developer, whether they're new,
> seasoned, have zero themes in the repo or a hundred.
>
>
>
> Bryan Phillip Hadaway
>
>
> Web & Graphic Designer
> calmestghost.com
> bhadaway at gmail.com
>
>
> Socialize: Facebook | Twitter | LinkedIn | Google+
>
>
>
> On Wed, Apr 10, 2013 at 5:04 AM, Greg Priday <greg at siteorigin.com> wrote:
>>
>> If you look at the new themes (#4) queue, Around 75% of the tickets
>> (at the time of my research) are from new theme developers, without
>> any approved themes on .org. These themes are waiting in the same
>> queue as established developers with lots of approved themes like
>> @automattic (23 themes), @d5creation (9 themes), etc.
>>
>> In my experience, established developers have a much higher chance of
>> having a high quality issue-free theme. I'm sure someone could analyze
>> all the tickets on track find the exact relationship of number of
>> themes to approval rate.
>>
>> I'm not hating on new developers. We all had to start somewhere, and
>> there definitely are diamonds in the rough.
>>
>> I burned out a little on reviewing themes because searching through
>> poorly written code for the same handful of issues is mind numbing.
>> The highlight of my day was finding someone who actually checked the
>> theme review guidelines instead of just tweaking their theme until
>> they made it past the theme check. I'm sure that's the case for a lot
>> of us.
>>
>> We simply don't have the reviewer resources to make this a priority.
>>
>> Solutions:
>>
>> * Order the theme review queues first by number of approved themes
>> from a given developer, then date. This will allow us to focus on
>> established developers first, then get around to finding the
>> diamond-in-the-rough new developers (I'm not sure this is possible in
>> trac though).
>>
>> * Screen new developers with a quiz that ensures they understand the
>> guidelines. Even 20 multiple choice questions covering the most common
>> issues would go a long way.
>>
>> Side Notes:
>>
>> Interestingly, if you look at the #3 queue (previously reviewed), only
>> 50% of the tickets are from new developers. Much less than the 75% in
>> #4. This is some indication of the drop out rate of new devs vs
>> established devs.
>>
>> --
>> I make free WordPress themes
>> http://siteorigin.com
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>



-- 
I make free WordPress themes
http://siteorigin.com