[wp-trac] [WordPress Trac] #30284: Remove failing automated tests from the suite

[WordPress Trac]

#30284: Remove failing automated tests from the suite
------------------------------+----------------------------
 Reporter:  boonebgorges      |      Owner:
     Type:  task (blessed)    |     Status:  new
 Priority:  normal            |  Milestone:  Future Release
Component:  Build/Test Tools  |    Version:
 Severity:  normal            |   Keywords:
  Focuses:                    |
------------------------------+----------------------------
 Our previous convention was to write unit tests to demonstrate bugs,
 annotate those tests with a `@ticket` flag, and commit them to the repo.
 Then, when running the suite, `WP_UnitTestCase::knownWPBug()` would get a
 list of open tickets from Trac, and skip tests against open tickets,
 allowing the build to pass. This convention is not ideal for a couple
 reasons:

 * It requires fetching a report from Trac on every run of the suite. This
 adds a couple seconds to each run of `phpunit`, which can be a real drag
 when using TDD techniques. (It also causes miscellaneous problems with
 transports and offline access - see
 https://buddypress.trac.wordpress.org/changeset/9053 for an illustration.)
 * When you skip failed tests by default, you have to do things like
 `phpunit --group 12345` to get them to run during development. This, in
 turn, can mask certain kinds of pollution between new and old tests,
 pollution that may not show up until after commit. See [30112] and [30148]
 for a couple of recent examples.
 * In terms of developer psychology, committing a failing test is a way of
 washing one's hands of an issue: "look, I've documented the bug, and now I
 can move on".

 The new convention is to commit tests only when fixing the bug. But this
 leaves dozens of failed tests in the repo. These tests should be pulled
 out, converted to patches, and uploaded to their relevant Trac ticket with
 a note indicating what's just happened. Once all the failed tests are out
 of the repo, we can remove the `knownWPBug` behavior from the test runner.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/30284>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform