[wp-hackers] Who on the commit team or contributors actually run the Automattic WordPress Tests?

[Jacob Santos]

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Nikolay Bachiyski wrote:
> | 2008/4/1, Jacob Santos <wordpress at santosj.name>:
> |> I don't want to step out of line with a statement I was going to make
> on
> |>  my blog, but seriously how many core developers which commit patches
> |>  actually test it with the Automattic WordPress Tests [1]?
> |>
> |>  How many contributors have the Subversion checkout of the Automattic
> |>  WordPress Tests and run it with patches?
> |
> | I do.
> |
>
> I don't ........ yet.
>
> It is on my todo list to get this setup so that I can run the test suite
> against patches.


So, it is basically you checkout the SVN repository, add your test suite
apply the patch, and then run the CLI file in the Automattic WordPress
Tests repository.

I was able to do that in several minutes. Give or take modifying the CLI
to work with the web, which was a two weekend project.


> It is also my intent to push for test cases before committing when
> people want to change the behaviour of complex code - autop for example.


Hooray! I think the intent should be on only committing code which has
documentation (even if it is from a core commit member) and has a test
case. It'll be a pain, so I do think it should be a recommendation until
which time more contributors are comfortable with it. I say that from how
much I originally fought such a requirement, but the only reason I did it
was because I really wanted to work on the project.

I'm not sure if I would work on WordPress, because it is not as fun as
game development (but somehow holds my interest better than game
development, so figure that out for me).


> I already have a number of useful scripts I use to increase my
> productivity when reviewing patches (which I must share sometime) and
> adding one which allows me to run all the (relevant) test cases against
> a patch is where I would like to be :-)

Yeah, you are looking for a Continuous Integration package. There is
Bitten, which probably isn't intelligent enough at this moment and there
is phpUnderControl, which uses CruiseControl. The Automattic WordPress
Tests do not currently do that and my tool does not do that. With as many
tests that could be when all of the functions are tested, it will be an
excellent idea to do that in the future.

I'm currently looking at Continuous Integration packages, which are free
and wouldn't be difficult to do for WordPress.

> Some questions for those of you more up-to-speed with the test cases.
>
> What are the prerequisites to getting the tests up and running over and
> above what WordPress itself requires?

Automattic WordPress Tests
Linux (PHP 5.3 for Windows)
PHPUnit, might need an older version if using PHP4.
PHP4 or PHP5 with the current version of PHPUnit.

WPTests @ http://wptests.dragonu.net
Web Server
PDO_SQLite
PHP5.1+
Add yourself to Allowed Hosts in the .htaccess file.

WPTests is actually sort of a continuous integration like system, but not,
so it isn't meant to do much of what another project will allow you to do.
It is meant to just be on a web server and hit with a cron job or
scheduled script and go to once in a while to see the results.


> How long do the tests take to run?

Just go to the web site and it will tell you. It is off by a minute, which
I plan on fixing, but the PHPUnit results will tell you.

It is anywhere (on my host) from 2 minutes to 7 minutes. On my machine it
usually takes the same amount of time.