[theme-reviewers] Prioritizing The Review Queue by Developer

[Greg Priday]

If you look at the new themes (#4) queue, Around 75% of the tickets
(at the time of my research) are from new theme developers, without
any approved themes on .org. These themes are waiting in the same
queue as established developers with lots of approved themes like
@automattic (23 themes), @d5creation (9 themes), etc.

In my experience, established developers have a much higher chance of
having a high quality issue-free theme. I'm sure someone could analyze
all the tickets on track find the exact relationship of number of
themes to approval rate.

I'm not hating on new developers. We all had to start somewhere, and
there definitely are diamonds in the rough.

I burned out a little on reviewing themes because searching through
poorly written code for the same handful of issues is mind numbing.
The highlight of my day was finding someone who actually checked the
theme review guidelines instead of just tweaking their theme until
they made it past the theme check. I'm sure that's the case for a lot
of us.

We simply don't have the reviewer resources to make this a priority.

Solutions:

* Order the theme review queues first by number of approved themes
from a given developer, then date. This will allow us to focus on
established developers first, then get around to finding the
diamond-in-the-rough new developers (I'm not sure this is possible in
trac though).

* Screen new developers with a quiz that ensures they understand the
guidelines. Even 20 multiple choice questions covering the most common
issues would go a long way.

Side Notes:

Interestingly, if you look at the #3 queue (previously reviewed), only
50% of the tickets are from new developers. Much less than the 75% in
#4. This is some indication of the drop out rate of new devs vs
established devs.

--
I make free WordPress themes
http://siteorigin.com