[theme-reviewers] Review-Continuation Tickets, and Reviewing Previous Tickets

Mon Jan 7 16:08:32 UTC 2013

As I see it, the Theme Review Team was never meant to be and should not be
opening the door to become "the" Quality Control mechanism for Theme
Authors; that responsibility still falls squarely on their shoulders.

I can see merit to continuing with a reviewer-prerogative method of keeping
tickets open but as others have noted in the past and once again in this
conversation, this could lead to the WPTRT becoming even more of a QC
system than it already is. Themes should be approved within two iterations
of the initial submission as far as I am concerned, beyond that let them go
to the back of the line. Themes previously approved should continue to be
approved (with the exception of minor over-sights directly related to
"REQUIRED" items as this "open-ticket" idea allows for), or they should
simply go back into the pool like new themes as well.

Perhaps I am being a bit on the harsh side (and I do not want to sidetrack
this topic), but I still see themes submitted that simply should be
resolved as "not-approved" as soon as they pass the upload check
(especially with one of the most obvious items: inappropriate screenshots).

Edward Caissie
aka Cais.

On Mon, Jan 7, 2013 at 10:45 AM, yulian yordanov <yul.yordanov at gmail.com>wrote:

>  When I was more active reviewer, it such cases I discussed in the ticket
> if the author is able to make fixes quickly. And of course we talk about
> some minor issue or an oversight which can be fixed easily.
>
> Fingli
>
>
> On 7.1.2013 г. 17:27 ч., Mario Peshev wrote:
>
> It's a very gray territory there, probably, I would just be happy to hear
> other opinions as well (both admins and reviewers). There's the compromise
> between the reviewers (volunteers) time and the very long cycle for a theme
> to get in due to the hundreds of requirements whilst a number of them are
> subjective.
>
>  Unless other group participants take a position here, I'll postpone the
> idea for now since it could get messy and involve specific theme review
> rules, their significance, reviewer's time, type of authors (regular
> contributors vs. new ones, and large companies vs. solo devs).
>
>  Thanks,
>
> Mario Peshev
> WordPress Engineer, Open Source Consultant
> http://www.linkedin.com/in/mpeshev
> http://me.peshev.net
>
>
>
> On Mon, Jan 7, 2013 at 5:17 PM, Chip Bennett <chip at chipbennett.net> wrote:
>
>> At the moment, we leave that determination entirely up to the Reviewer:
>> any ticket, at the sole discretion of the Reviewer, may be held open to
>> allow for a revision to be submitted, in order to continue the review on a
>> subsequent ticket.
>>
>>  Personally, I would like to see this stay at the informal,
>> discretion-of-the-Reviewer level, rather than try to formalize the criteria
>> for review continuation. But, if you think it merits further consideration,
>> we can certainly discuss!
>>
>>  Thanks,
>>
>>  Chip
>>
>>
>> On Mon, Jan 7, 2013 at 10:11 AM, Mario Peshev <mario at peshev.net> wrote:
>>
>>> Yes, I know we partially do it already (given the two examples of yours)
>>> but I think that it might be a good practice to set a rule and extend this,
>>> let's say (just as a sample) - up to 5 required and 10 recommended issues
>>> that apparently might be fixed in a few hours, we give 48 hours (or 2
>>> business days) for the author to fix them, if not, the ticket is closed and
>>> not approved.
>>>
>>> I know that agencies, companies and teams with resources spending their
>>> time completely in the WordPress ecosystem could react and we could speed
>>> up the process instead of getting the new version on the next day and
>>> waiting for another month and another close.
>>>
>>> Best,
>>>
>>>
>>> Mario Peshev
>>> WordPress Engineer, Open Source Consultant
>>> http://www.linkedin.com/in/mpeshev
>>>  http://me.peshev.net
>>>
>>>
>>>
>>>   On Mon, Jan 7, 2013 at 5:04 PM, Chip Bennett <chip at chipbennett.net>wrote:
>>>
>>>> Mario,
>>>>
>>>>  I'm not sure I'm completely following your question.
>>>>
>>>>  I'm talking specifically about two cases:
>>>>
>>>>  1. Where in the current ticket, the reviewer says, "hey, I found
>>>> these one or two, really minor, but required issues; I'm approving the
>>>> Theme, but please fix them in the next revision"
>>>> 2. Where in the current ticket, the reviewer says, "hey, you missed
>>>> this required thing; please fix it and re-submit. I'll hold your ticket
>>>> open so you don't have to wait in line again"
>>>>
>>>>  Both of these things do help expedite the process, and make it less
>>>> frustrating for the developer. But, we have to make sure that we verify
>>>> that the *required* issues identified in each case are resolved in the
>>>> subsequent ticket.
>>>>
>>>>  I *think* you're talking about "holding open" tickets in general? If
>>>> so, that's not something that we've really addressed. Might be worth a
>>>> discussion, perhaps?
>>>>
>>>>  Thanks,
>>>>
>>>>  Chip
>>>>
>>>>
>>>>  On Mon, Jan 7, 2013 at 9:59 AM, Mario Peshev <mario at peshev.net> wrote:
>>>>
>>>>>  Chip, isn't it 'good to have' to keep tickets open? In my opinion
>>>>> most themes need approx. 3-4 iterations to get in and given the stats, that
>>>>> might take few months even though fixes might take a few hours.
>>>>>
>>>>>  IMO theme authors that prioritize theme submissions should have a
>>>>> fast lane open when the feedback could apparently be resolved in a few
>>>>> hours. This is in case reviewers conduct a complete review and not a quick
>>>>> look only.
>>>>>
>>>>> Mario Peshev
>>>>> WordPress Engineer, Open Source Consultant
>>>>> http://www.linkedin.com/in/mpeshev
>>>>> http://me.peshev.net
>>>>>
>>>>>
>>>>>
>>>>> On Mon, Jan 7, 2013 at 4:11 PM, Chip Bennett <chip at chipbennett.net>wrote:
>>>>>
>>>>>> This is especially important for previously approved (Priority #1)
>>>>>> tickets, since such tickets already receive an expedited, diff-only review.
>>>>>>
>>>>>>  Two, we have a great system in place, that allows a reviewer to
>>>>>> hold a ticket open if only minor issues need to be addressed, such that
>>>>>> once a revision is submitted, the reviewer can continue the current review
>>>>>> in the new ticket. If the previous ticket is still open, the new ticket
>>>>>> should be left for the reviewer of the previous ticket, so that the
>>>>>> previous review may be continued.
>>>>>>
>>>>>>  I had two such tickets over the weekend: both had previous-ticket
>>>>>> "required" issues that were not addressed. I left the tickets open to allow
>>>>>> for a review continuation. Both developers uploaded revisions, but when I
>>>>>> went in this morning to take the tickets, both had been assigned, reviewed,
>>>>>> and closed. Unfortunately, in both cases, the review failed to indicate
>>>>>> whether previous-ticket "required" issues had been addressed.
>>>>>>
>>>>>
>>>>>
>>>>>  _______________________________________________
>>>>> theme-reviewers mailing list
>>>>> theme-reviewers at lists.wordpress.org
>>>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> theme-reviewers mailing list
>>>> theme-reviewers at lists.wordpress.org
>>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>>
>>>>
>>>
>>> _______________________________________________
>>> theme-reviewers mailing list
>>> theme-reviewers at lists.wordpress.org
>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>
>>>
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
>
>
> _______________________________________________
> theme-reviewers mailing listtheme-reviewers at lists.wordpress.orghttp://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wordpress.org/pipermail/theme-reviewers/attachments/20130107/30448b7e/attachment-0001.htm>