[theme-reviewers] Review Process [was: Simple-Blue-Dashed 1.0]

Fri Jun 11 18:29:31 UTC 2010

A couple of other ideas (though perhaps not popular enough to make the
list below):

1) More precise, objective definitions for some TDC criteria that are
currently rather subjective

2) Tag nomenclature standardization

3) Creation of some tool for collection/retention/sharing of theme review
data (GoogleDocs, database, etc.)

> I concede entirely... security should, of course, go last like said -
> scratch my "security first" ordering from the recording.
>
> General would at a guess be the most popular group to volunteer for, can
> personally envisage a 24hr turn around working just fine.
>
> Thanks for the Wiki page updates thus far Tom.   :)
>
> I think the most popular ideas to come out of today have been:
>
> - Splitting the review process into "groups".
>
> - Not reviewing design/aesthetics at all, leaving that up to WP end
> users to vote for.
>
> - A simple checklist/standards based review procedure, that results in
> themes being of a *consistently* good quality (whether we should use a
> simpler yes/no, or more analytical 1-5 based system wasn't fully
> decided).
>
> Joseph/Joe may have some better input at this stage on what is expected.
>
> Gav
> //gavinpearce.com
>
> -----Original Message-----
> From: theme-reviewers-bounces at lists.wordpress.org
> [mailto:theme-reviewers-bounces at lists.wordpress.org] On Behalf Of Tom
> Lany
> Sent: 11 June 2010 18:19
> To: theme-reviewers at lists.wordpress.org
> Subject: Re: [theme-reviewers] Simple-Blue-Dashed 1.0
>
> I also like the idea of splitting the process into groups, and would
> agree with the idea of looking at general issues before security.  While
>
> it would probably be hard to complete the who process within 24 hours, I
>
> am sure that an initial contact from a "general" group could be
> completed within that time frame.  Would that meet the 24 hour goal?
>
> My vote for group review order and naming (at this point) is:
>
> 1 - General
> 2 - Design
> 2 - Security
>
> Tom Lany
> http://tomlany.net/
>
>
> On 6/11/10 12:11 PM, chip at chipbennett.net wrote:
>> Three thoughts:
>>
>> 1) I think the General review should go first, and the theme cleaned
> up as
>> appropriate, before it is reviewed for security. In order for an
> effective
>> security review, a theme needs to be as standards-compliant as
> possible,
>> with as efficient code as possible. Otherwise, the security-related
>> needles get harder and harder to find in the haystack.
>>
>> 2) Would having three separate teams negatively impact Joseph's goal
> of
>> completing theme review within 24 hours? Would the benefits of such
>> approach outweigh the benefits of theme turnaround in 24 hours? (I
> think
>> yes - within reason, of course.)
>>
>> 3) I really like the division-of-labor approach that could, ideally,
> suit
>> each person's best skills to the tasks at hand (while also providing
> some
>> cross-training opportunities). I would love to focus on "General"
> review,
>> while learning what I can from the Security and Design teams' reviews,
> if
>> we decide to go that route.
>>
>> Cheers,
>> Chip
>>
>>
>>
>>> Hi Chip,
>>>
>>> That's not a bad idea at all you know ... Maybe we should split each
>>> part of the review of each theme into different groups, and each
> Theme
>>> review passes from one group to the next.
>>>
>>> 1st) Theme must pass security team review with no major fails, then
>>> passes onto General;
>>>
>>> 2nd) Theme must pass functionality team with no major fails, then
> passes
>>> onto CrossBrowser/HTML/CSS testing;
>>>
>>> 3rd) Theme must pass browser team browser testing to reasonable
> levels
>>> (saw Tim Golen mention this a minute ago, personally I think this
> falls
>>> under "technical" rather than "design" - there are plenty of
> standards
>>> for this already defined).
>>>
>>> 4th) If at this point the total number of "advisories/minors" is
> below
>>> X, the theme gets approved.
>>>
>>> If one of the groups finds anything critical along the way the author
> is
>>> advised once that groups "checking" is complete, and theme doesn't
> get
>>> passed onto the next group. This saves everyone in every group
> testing
>>> everything twice, just because of a security fail.
>>>
>>> Then, all the volunteers who've offered here recently can decide what
>>> team their skills best fit into, and be put to best use. I know some
>>> people here would prefer to test cross-browser than security, and
>>> vice-versa of course.
>>>
>>> Gav
>>> //gavinpearce.com
>>>
>>>
>>> -----Original Message-----
>>> From: theme-reviewers-bounces at lists.wordpress.org
>>> [mailto:theme-reviewers-bounces at lists.wordpress.org] On Behalf Of
>>> chip at chipbennett.net
>>> Sent: 11 June 2010 17:50
>>> To: theme-reviewers at lists.wordpress.org
>>> Subject: Re: [theme-reviewers] Simple-Blue-Dashed 1.0
>>>
>>>
>>>> Security is a big item, themes mis-use any external data ($_GET,
>>>> $_POST, $_REQUEST, $_COOKIE, $_SERVER) must be addressed, no two
> ways
>>>> about it.  Direct DB queries must properly escape data in the query
>>>> (and if there is a WP function to do the same thing the direct DB
>>>> query should be replaced with the function call).  Those are the
>>>> basic, *minimum* things that every theme needs to address security
>>>> wise.
>>>>
>>> For those of us who are less-than-expert in the SQL aspects of theme
>>> reviewing, can those who are more adept create a reasonably
>>> easy-to-follow
>>> security checklist?
>>>
>>> Or, maybe we should have some security-ninja theme reviewers, who can
>>> focus on the security aspects of themes? If so, we could divy up the
>>> review work, such that security concerns are handled separately -
> after
>>> the theme is initially reviewed (and cleaned up, if necessary) based
> on
>>> the "normal" criteria?
>>>
>>>
>>>> Sometimes the theme author just isn't aware of specific functions or
>>>> services in WordPress, so some hints and reference URLs for more
> info
>>>> are helpful there.
>>>>
>>> Agreed. I tried to add in Codex references, where appropriate, in my
>>> first
>>> review.
>>>
>>> Speaking of which: the Theme Development Checklist entry in the Codex
> is
>>> sorely in need of cross-referencing to Codex entries for functions,
>>> template tags, and hooks/filters.
>>>
>>> _______________________________________________
>>> theme-reviewers mailing list
>>> theme-reviewers at lists.wordpress.org
>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>> _______________________________________________
>>> theme-reviewers mailing list
>>> theme-reviewers at lists.wordpress.org
>>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>>
>>>
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>