[wp-trac] [WordPress Trac] #43432: Streamline tests with installation skips

[WordPress Trac]

#43432: Streamline tests with installation skips
------------------------------+------------------
 Reporter:  soulseekah        |       Owner:
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:  5.0
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:                    |     Focuses:
------------------------------+------------------

Comment (by soulseekah):

 Indeed, some of the tests are non-repeatable on a dirty install as they
 either rely on clean state, or get messed up due to state left by
 following tests. This can be partially checked by adding `--repeat 2` when
 running `phpunit`, which runs every test twice in a row.

 However, the purpose of this skip install is to save time while developing
 a patch for a test case (or suite maybe, group). At least that's what I
 do, I don't run the whole test suite every while developing a patch, I
 just run a specific test. After I'm satisfied with the test, I run the
 whole suite.

 So while developing I'd be doing `WP_TESTS_SKIP_INSTALL=1 phpunit --filter
 test_ticket_43432`, which takes a second or two, and then when I'm ready I
 run a final regular `phpunit` that takes 5-6 minutes making sure my patch
 doesn't break anything.

 Over the years I've found myself adding variations of the skip install
 flag time and time again  to help streamline development for core tests
 and plugin tests. In fact I wrote patches for GravityView and Gravity
 Forms fixing non-repeatable tests, and I've been running both suites
 without reinstalling core for months at a time without any issues
 whatsoever.

 Coincidentally, using a dirty install while writing tests will ensure that
 a new test a developer is writing is repeatable and doesn't depend on a
 clean database or some quirky global state.

 In short, full tests: long run, full reinstall, not much gain. Single
 test: short run, no reinstall, potentially around 50-80% time saved per
 run. Hope this makes sense.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/43432#comment:3>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform