[theme-reviewers] Review Process [was: Simple-Blue-Dashed 1.0]

Fri Jun 11 17:55:19 UTC 2010

I concede entirely... security should, of course, go last like said -
scratch my "security first" ordering from the recording. 

General would at a guess be the most popular group to volunteer for, can
personally envisage a 24hr turn around working just fine.

Thanks for the Wiki page updates thus far Tom.   :)

I think the most popular ideas to come out of today have been:

- Splitting the review process into "groups".

- Not reviewing design/aesthetics at all, leaving that up to WP end
users to vote for.

- A simple checklist/standards based review procedure, that results in
themes being of a *consistently* good quality (whether we should use a
simpler yes/no, or more analytical 1-5 based system wasn't fully
decided).

Joseph/Joe may have some better input at this stage on what is expected.

Gav
//gavinpearce.com

-----Original Message-----
From: theme-reviewers-bounces at lists.wordpress.org
[mailto:theme-reviewers-bounces at lists.wordpress.org] On Behalf Of Tom
Lany
Sent: 11 June 2010 18:19
To: theme-reviewers at lists.wordpress.org
Subject: Re: [theme-reviewers] Simple-Blue-Dashed 1.0

I also like the idea of splitting the process into groups, and would 
agree with the idea of looking at general issues before security.  While

it would probably be hard to complete the who process within 24 hours, I

am sure that an initial contact from a "general" group could be 
completed within that time frame.  Would that meet the 24 hour goal?

My vote for group review order and naming (at this point) is:

1 - General
2 - Design
2 - Security

Tom Lany
http://tomlany.net/

On 6/11/10 12:11 PM, chip at chipbennett.net wrote:
> Three thoughts:
>
> 1) I think the General review should go first, and the theme cleaned
up as
> appropriate, before it is reviewed for security. In order for an
effective
> security review, a theme needs to be as standards-compliant as
possible,
> with as efficient code as possible. Otherwise, the security-related
> needles get harder and harder to find in the haystack.
>
> 2) Would having three separate teams negatively impact Joseph's goal
of
> completing theme review within 24 hours? Would the benefits of such
> approach outweigh the benefits of theme turnaround in 24 hours? (I
think
> yes - within reason, of course.)
>
> 3) I really like the division-of-labor approach that could, ideally,
suit
> each person's best skills to the tasks at hand (while also providing
some
> cross-training opportunities). I would love to focus on "General"
review,
> while learning what I can from the Security and Design teams' reviews,
if
> we decide to go that route.
>
> Cheers,
> Chip
>
>
>    
>> Hi Chip,
>>
>> That's not a bad idea at all you know ... Maybe we should split each
>> part of the review of each theme into different groups, and each
Theme
>> review passes from one group to the next.
>>
>> 1st) Theme must pass security team review with no major fails, then
>> passes onto General;
>>
>> 2nd) Theme must pass functionality team with no major fails, then
passes
>> onto CrossBrowser/HTML/CSS testing;
>>
>> 3rd) Theme must pass browser team browser testing to reasonable
levels
>> (saw Tim Golen mention this a minute ago, personally I think this
falls
>> under "technical" rather than "design" - there are plenty of
standards
>> for this already defined).
>>
>> 4th) If at this point the total number of "advisories/minors" is
below
>> X, the theme gets approved.
>>
>> If one of the groups finds anything critical along the way the author
is
>> advised once that groups "checking" is complete, and theme doesn't
get
>> passed onto the next group. This saves everyone in every group
testing
>> everything twice, just because of a security fail.
>>
>> Then, all the volunteers who've offered here recently can decide what
>> team their skills best fit into, and be put to best use. I know some
>> people here would prefer to test cross-browser than security, and
>> vice-versa of course.
>>
>> Gav
>> //gavinpearce.com
>>
>>
>> -----Original Message-----
>> From: theme-reviewers-bounces at lists.wordpress.org
>> [mailto:theme-reviewers-bounces at lists.wordpress.org] On Behalf Of
>> chip at chipbennett.net
>> Sent: 11 June 2010 17:50
>> To: theme-reviewers at lists.wordpress.org
>> Subject: Re: [theme-reviewers] Simple-Blue-Dashed 1.0
>>
>>      
>>> Security is a big item, themes mis-use any external data ($_GET,
>>> $_POST, $_REQUEST, $_COOKIE, $_SERVER) must be addressed, no two
ways
>>> about it.  Direct DB queries must properly escape data in the query
>>> (and if there is a WP function to do the same thing the direct DB
>>> query should be replaced with the function call).  Those are the
>>> basic, *minimum* things that every theme needs to address security
>>> wise.
>>>        
>> For those of us who are less-than-expert in the SQL aspects of theme
>> reviewing, can those who are more adept create a reasonably
>> easy-to-follow
>> security checklist?
>>
>> Or, maybe we should have some security-ninja theme reviewers, who can
>> focus on the security aspects of themes? If so, we could divy up the
>> review work, such that security concerns are handled separately -
after
>> the theme is initially reviewed (and cleaned up, if necessary) based
on
>> the "normal" criteria?
>>
>>      
>>> Sometimes the theme author just isn't aware of specific functions or
>>> services in WordPress, so some hints and reference URLs for more
info
>>> are helpful there.
>>>        
>> Agreed. I tried to add in Codex references, where appropriate, in my
>> first
>> review.
>>
>> Speaking of which: the Theme Development Checklist entry in the Codex
is
>> sorely in need of cross-referencing to Codex entries for functions,
>> template tags, and hooks/filters.
>>
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>> _______________________________________________
>> theme-reviewers mailing list
>> theme-reviewers at lists.wordpress.org
>> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>>
>>      
>
> _______________________________________________
> theme-reviewers mailing list
> theme-reviewers at lists.wordpress.org
> http://lists.wordpress.org/mailman/listinfo/theme-reviewers
>    
_______________________________________________
theme-reviewers mailing list
theme-reviewers at lists.wordpress.org
http://lists.wordpress.org/mailman/listinfo/theme-reviewers