[theme-reviewers] Review-Continuation Tickets, and Reviewing Previous Tickets

Mon Jan 7 16:17:18 UTC 2013

Using the Theme Review Team as an adjunct "Quality Control" unit is a very
real issue, and I agree with everything you say in that regard. But, it is
also a limited-scope issue, and (generally speaking) not one that should
drive Theme Review guidelines or WPTRT policies. So, as with most things,
we end up with a balancing act.

And as others have expressed: I too want to avoid the WPTRT becoming
unnecessarily bureaucratic, which is why I favor a "Reviewer's discretion"
approach, both to early termination of a review of an obviously failing
Theme, and to continuing the review in a subsequent ticket for "almost
there" Themes.

Chip

On Mon, Jan 7, 2013 at 11:08 AM, Edward Caissie <edward.caissie at gmail.com>wrote:

> As I see it, the Theme Review Team was never meant to be and should not be
> opening the door to become "the" Quality Control mechanism for Theme
> Authors; that responsibility still falls squarely on their shoulders.
>
> I can see merit to continuing with a reviewer-prerogative method of
> keeping tickets open but as others have noted in the past and once again in
> this conversation, this could lead to the WPTRT becoming even more of a QC
> system than it already is. Themes should be approved within two iterations
> of the initial submission as far as I am concerned, beyond that let them go
> to the back of the line. Themes previously approved should continue to be
> approved (with the exception of minor over-sights directly related to
> "REQUIRED" items as this "open-ticket" idea allows for), or they should
> simply go back into the pool like new themes as well.
>
> Perhaps I am being a bit on the harsh side (and I do not want to sidetrack
> this topic), but I still see themes submitted that simply should be
> resolved as "not-approved" as soon as they pass the upload check
> (especially with one of the most obvious items: inappropriate screenshots).
>
> Edward Caissie
> aka Cais.
>
>
> On Mon, Jan 7, 2013 at 10:45 AM, yulian yordanov <yul.yordanov at gmail.com>wrote:
>
>>  When I was more active reviewer, it such cases I discussed in the
>> ticket if the author is able to make fixes quickly. And of course we talk
>> about some minor issue or an oversight which can be fixed easily.
>>
>> Fingli
>>
>>
>> On 7.1.2013 г. 17:27 ч., Mario Peshev wrote:
>>
>> It's a very gray territory there, probably, I would just be happy to hear
>> other opinions as well (both admins and reviewers). There's the compromise
>> between the reviewers (volunteers) time and the very long cycle for a theme
>> to get in due to the hundreds of requirements whilst a number of them are
>> subjective.
>>
>>  Unless other group participants take a position here, I'll postpone the
>> idea for now since it could get messy and involve specific theme review
>> rules, their significance, reviewer's time, type of authors (regular
>> contributors vs. new ones, and large companies vs. solo devs).
>>
>>  Thanks,
>>
>> Mario Peshev
>> WordPress Engineer, Open Source Consultant
>> http://www.linkedin.com/in/mpeshev
>> http://me.peshev.net
>>
>>
>>
>> On Mon, Jan 7, 2013 at 5:17 PM, Chip Bennett <chip at chipbennett.net>wrote:
>>
>>> At the moment, we leave that determination entirely up to the Reviewer:
>>> any ticket, at the sole discretion of the Reviewer, may be held open to
>>> allow for a revision to be submitted, in order to continue the review on a
>>> subsequent ticket.
>>>
>>>  Personally, I would like to see this stay at the informal,
>>> discretion-of-the-Reviewer level, rather than try to formalize the criteria
>>> for review continuation. But, if you think it merits further consideration,
>>> we can certainly discuss!
>>>
>>>  Thanks,
>>>
>>>  Chip
>>>
>>>
>>> On Mon, Jan 7, 2013 at 10:11 AM, Mario Peshev <mario at peshev.net> wrote:
>>>
>>>> Yes, I know we partially do it already (given the two examples of
>>>> yours) but I think that it might be a good practice to set a rule and
>>>> extend this, let's say (just as a sample) - up to 5 required and 10
>>>> recommended issues that apparently might be fixed in a few hours, we give
>>>> 48 hours (or 2 business days) for the author to fix them, if not, the
>>>> ticket is closed and not approved.
>>>>
>>>> I know that agencies, companies and teams with resources spending their
>>>> time completely in the WordPress ecosystem could react and we could speed
>>>> up the process instead of getting the new version on the next day and
>>>> waiting for another month and another close.
>>>>
>>>> Best,
>>>>
>>>>
>>>> Mario Peshev
>>>> WordPress Engineer, Open Source Consultant
>>>> http://www.linkedin.com/in/mpeshev
>>>>  http://me.peshev.net
>>>>
>>>>
>>>>
>>>>   On Mon, Jan 7, 2013 at 5:04 PM, Chip Bennett <chip at chipbennett.net>wrote:
>>>>
>>>>> Mario,
>>>>>
>>>>>  I'm not sure I'm completely following your question.
>>>>>
>>>>>  I'm talking specifically about two cases:
>>>>>
>>>>>  1. Where in the current ticket, the reviewer says, "hey, I found
>>>>> these one or two, really minor, but required issues; I'm approving the
>>>>> Theme, but please fix them in the next revision"
>>>>> 2. Where in the current ticket, the reviewer says, "hey, you missed
>>>>> this required thing; please fix it and re-submit. I'll hold your ticket
>>>>> open so you don't have to wait in line again"
>>>>>
>>>>>  Both of these things do help expedite the process, and make it less
>>>>> frustrating for the developer. But, we have to make sure that we verify
>>>>> that the *required* issues identified in each case are resolved in the
>>>>> subsequent ticket.
>>>>>
>>>>>  I *think* you're talking about "holding open" tickets in general? If
>>>>> so, that's not something that we've really addressed. Might be worth a
>>>>> discussion, perhaps?
>>>>>
>>>>>  Thanks,
>>>>>
>>>>>  Chip
>>>>>
>>>>>
>>>>>  On Mon, Jan 7, 2013 at 9:59 AM, Mario Peshev <mario at peshev.net>wrote:
>>>>>
>>>>>>  Chip, isn't it 'good to have' to keep tickets open? In my opinion
>>>>>> most themes need approx. 3-4 iterations to get in and given the stats, that
>>>>>> might take few months even though fixes might take a few hours.
>>>>>>
>>>>>>  IMO theme authors that prioritize theme submissions should have a
>>>>>> fast lane open when the feedback could apparently be resolved in a few
>>>>>> hours. This is in case reviewers conduct a complete review and not a quick
>>>>>> look only.
>>>>>>
>>>>>> Mario Peshev
>>>>>> WordPress Engineer, Open Source Consultant
>>>>>> http://www.linkedin.com/in/mpeshev
>>>>>> http://me.peshev.net
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Mon, Jan 7, 2013 at 4:11 PM, Chip Bennett <chip at chipbennett.net>wrote:
>>>>>>
>>>>>>> This is especially important for previously approved (Priority #1)
>>>>>>> tickets, since such tickets already receive an expedited, diff-only review.
>>>>>>>
>>>>>>>  Two, we have a great system in place, that allows a reviewer to
>>>>>>> hold a ticket open if only minor issues need to be addressed, such that
>>>>>>> once a revision is submitted, the reviewer can continue the current review
>>>>>>> in the new ticket. If the previous ticket is still open, the new ticket
>>>>>>> should be left for the reviewer of the previous ticket, so that the
>>>>>>> previous review may be continued.
>>>>>>>
>>>>>>>  I had two such tickets over the weekend: both had previous-ticket
>>>>>>> "required" issues that were not addressed. I left the tickets open to allow
>>>>>>> for a review continuation. Both developers uploaded revisions, but when I
>>>>>>> went in this morning to take the tickets, both had been assigned, reviewed,
>>>>>>> and closed. Unfortunately, in both cases, the review failed to indicate
>>>>>>> whether previous-ticket "required" issues had been addressed.
>>>>>>>
>>>>>>
>>>>>>
>>>>>>  _______________________________________________
>>>>>> theme-reviewers mailing list
>>>>>> theme-reviewers at lists.wordpress.org
>>>>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>>>>
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> theme-reviewers mailing list
>>>>> theme-reviewers at lists.wordpress.org
>>>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>>>
>>>>>
>>>>
>>>> _______________________________________________
>>>> theme-reviewers mailing list
>>>> theme-reviewers at lists.wordpress.org
>>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>>
>>>>
>>>
>>> _______________________________________________
>>> theme-reviewers mailing list
>>> theme-reviewers at lists.wordpress.org
>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>
>>>
>>
>>
>> _______________________________________________
>> theme-reviewers mailing listtheme-reviewers at lists.wordpress.orghttp://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.wordpress.org/pipermail/theme-reviewers/attachments/20130107/89cc06e8/attachment-0001.htm>