[wp-trac] [WordPress Trac] #53009: Tests: review all calls to markTestSkipped() and @requires annotations

[WordPress Trac]

#53009: Tests: review all calls to markTestSkipped() and @requires annotations
-------------------------------+-----------------------------
 Reporter:  jrf                |      Owner:  (none)
     Type:  task (blessed)     |     Status:  new
 Priority:  normal             |  Milestone:  Awaiting Review
Component:  Build/Test Tools   |    Version:  trunk
 Severity:  minor              |   Keywords:
  Focuses:  docs, performance  |
-------------------------------+-----------------------------
 PHPUnit allow for skipping tests when certain conditions are not met.

 There are multiple ways in which test skipping can be done:
 * Via an annotation at test class level.
 * Via an annotation at test function level.
 * Via an inline condition in a test or fixture function with a call to
 `markTestSkipped()`.

 The annotations can mostly be used when a particular PHP version /
 extension / PHPUnit version / Operating System etc is needed and will
 result in a descriptive "test skip" notice in the verbose test output.
 See: https://phpunit.readthedocs.io/en/9.5/incomplete-and-skipped-
 tests.html#incomplete-and-skipped-tests-requires-tables-api

 Inline conditions with a call to `markTestSkipped()` should be used for
 any situation for which annotations can not be used.

 While reviewing some tests, @hellofromtonya and me noticed situations
 where the test skipping can be improved.
 To that end, we are recommending a full review of all test skipping being
 done in the complete WP Core test suite.

 Such a review should:
 * Check whether an annotation uses the right condition and format.
 * Check whether annotations are at the right level - class vs function.
 * Check whether an inline condition can be replaced by an annotation.
 * Check whether an inline condition is at the right level - class vs
 fixture vs function
 * Check that all calls to `markTestSkipped()` contain a verbose
 explanation of why the test is being skipped.

 This list should function as a starting point and should be enhanced with
 any additional checks which may be needed as discovered once the review is
 started.

 Also see the following section of the PHPUnit manual:
 https://phpunit.readthedocs.io/en/9.5/incomplete-and-skipped-tests.html
 #skipping-tests

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/53009>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform