[Wp-unit-tests] some thoughts

[Daniel Convissor]

Hi Folks:

Kurt asked me the other day for some things I picked up while whipping
PEAR's unit tests into shape.  I figured it'd be good take what was said
to on IRC and share it here, with some edits, for the consideration
during planning of future WP improvements.

Once $this->fail() is called, the given test stops.  Iaw a lot of tests
in PEAR assuming that execution continued, with loads of if statements,
etc.

If there are a lot of evaluations of similar things in a particular
test, put in the message parameter to the assertion, so it's clear which
one failed.  Similarly always put messages into fail() and
markTestSkipped/Incomplete() calls.

Structure the tests using the file system.  One class per file.  Group
tests in directories and sub-directories.  Name the file for the
containing class.  All class / file names should end in Test / Test.php.
http://www.phpunit.de/manual/3.7/en/organizing-tests.html#organizing-tests.filesystem

Have every test file require once a centralized helper file.  The helper
file then establishes the environment by including the other files
needed.  The only PHPUnit file the helper needs to include manually is
PHPUnit/Autoload.php.

The combination of the prior two suggestions permit calling any test or
combination of tests very easily.  Plus it makes finding tests easy.

If all tests in a class need a particular extension, or have similar
optional requirements, do the skip test in a setUp() function in that
class rather than checking it in each test method.  The class' setUp()
needs to manually call parent::setUp() of course.

When gathering data sets (from queries, etc), first check that the query
didn't return a failure before testing the data inside the results.
This produces a clear indication of what's going on rather than some
field not matching the expected result, which obfuscates the problem.
Using a centralized method in the test class to do this validation is
helpful.  See the checkResultForErrors() method here for an example:
http://svn.php.net/viewvc/pear/packages/MDB2/trunk/tests/Standard/Abstract.php?view=markup

There should be no error_reporting() calls in the code.  The error
reporting should be set on the command line when calling the test suite.
This permits people running the tests to choose what to look for.  In
some scenarios, one is looking for E_STRICT.  In others one needs to
find all E_DEPRECATED.  While in others, nothing should be seen.  If the
level is set inside the code, that's impossible.

There are some limited circumstances where turning down error reporting
is necessary (such as when using external packages to perform a task).
In times like those, call $old = error_reporting(new level) before that
particular task, then set error_reporting($old) after it.

Hope that helps,

--Dan

-- 
 T H E   A N A L Y S I S   A N D   S O L U T I O N S   C O M P A N Y
            data intensive web and database programming
                http://www.AnalysisAndSolutions.com/
 4015 7th Ave #4, Brooklyn NY 11232  v: 718-854-0335 f: 718-854-0409