[theme-reviewers] The purpose of automated theme checking

[Otto]

Hi again! I've gotten a few emails about the Theme Check tool that
show something of a misunderstanding about the underlying purpose
here. So I thought I'd clarify, for all parties, to make sure we're on
the same page.

Yes, many of the checks are simplistic and easily "worked around". But
they're not supposed to be perfect. Nor, in fact, can they be. I can
think of at least 2 ways to work around any and all of them right off
the top of my head.

It is not the purpose of the automated theme checker to detect
everything that could be in there. Nor is it to protect against
malicious action. We have theme reviewers that are perfectly capable
of scanning a theme, by eye, and spotting weirdness along those lines.

*The purpose of the theme checker is to help theme authors produce
better themes.*

By this, I mean that creating a theme is hard. Yeah, it's easy to
knock together a custom theme for one site and have it do exactly what
you want. But creating a theme for public consumption, with a proper
upgrade path, that causes few problems, and "just works" for all the
different possible installs out there is darned tricky.

The review guidelines, as they stand, are created in that spirit. To
provide a minimal set of things that a theme must support in order to
be capable of general usage by the public. So the theme checker's
purpose is to scan a theme that is presumably made *in good faith*,
and to notify the author of known issues, problems, oversights, etc.
Some of it is to notify the author of possible problems, such as known
malware, that they may not even know is in there.

Yes, we have theme "viruses" out there, which can propagate from theme
to theme when on the same WP installation. Authors have uploaded
themes before, themes which they created,  without knowing that the
theme had been "infected" with one of these. This is the purpose of
the "malware" and "bad things" scans, to tell authors about things
they might not even know.

So the point of the theme check is not to be an obstacle for theme
authors to work around, but a tool to help them make their themes
better. Having to wait a week for a human reviewer to get around to
telling you the problems with the theme is frustrating (believe me),
but an automated scan can give you results instantly. Okay, so the
results aren't perfect, and they're never going to be perfect. But if
it helps to improve the themes to the point where the reviewers only
have to find the more tricky problems with the themes they're
reviewing, then it'll still help.

Just wanted to make sure that everybody understood that. The review
team already gets it, so this is for everybody else reading the list.
:)

-Otto