[wp-trac] [WordPress Trac] #31130: WP_INSTALLING causes leakage between unit tests

[WordPress Trac]

#31130: WP_INSTALLING causes leakage between unit tests
------------------------------+------------------------------
 Reporter:  boonebgorges      |       Owner:
     Type:  defect (bug)      |      Status:  new
 Priority:  normal            |   Milestone:  Awaiting Review
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:                    |     Focuses:
------------------------------+------------------------------
Description changed by boonebgorges:

Old description:

> A number of multisite unit tests invoke
> `WP_UnitTest_Factory_for_Blog::create()`, which uses `wpmu_create_blog()`
> to create a blog. Whenever this happens, the `WP_INSTALLING` constant is
> defined as `true` for all later tests. This has a number of
> ramifications, especially in the Options API, which skips the cache when
> `WP_INSTALLING`. See [28965], [31280], #28706.
>
> I see four strategies for working around the problem:
>
> 1. For any test that breaks when `WP_INSTALLING`, mark it skipped on
> multisite. This is what we're currently doing, and is lame.
> 2. Create wrapper functions for `WP_INSTALLING` so that it can be toggled
> on and off (unlike a constant). Either `apply_filters()`, or a global
> toggle like `wp_suspend_cache_invalidation()`. Then, in
> `WP_UnitTest_Factory_For_Blog::create()`, toggle it off after
> `wpmu_create_blog()` has turned it on. The main downside here is that
> `WP_INSTALLING` is not really the kind of thing we want to be toggleable
> by devs, so we'd have to document it accordingly.
> 3. Run all blog-creating tests in separate processes that don't preserve
> the global state. That way, the constant would be set in the other
> process, but not when returning to run the rest of the tests. I played
> with this a little, and it looks like it's not going to be trivial to
> make it work - we may need to bootstrap WordPress for every test or
> something like that.
> 4. Make sure that all tests that create a blog are run at the very end of
> the suite.
>
> Option (2) seems like the most natural fix, but it's also the only one
> that touches `src/`. Any additional thoughts are welcome.
> 3.

New description:

 A number of multisite unit tests invoke
 `WP_UnitTest_Factory_for_Blog::create()`, which uses `wpmu_create_blog()`
 to create a blog. Whenever this happens, the `WP_INSTALLING` constant is
 defined as `true` for all later tests. This has a number of ramifications,
 especially in the Options API, which skips the cache when `WP_INSTALLING`.
 See [28965], [31280], #28706.

 I see four strategies for working around the problem:

 1. For any test that breaks when `WP_INSTALLING`, mark it skipped on
 multisite. This is what we're currently doing, and is lame.
 2. Create wrapper functions for `WP_INSTALLING` so that it can be toggled
 on and off (unlike a constant). Either `apply_filters()`, or a global
 toggle like `wp_suspend_cache_invalidation()`. Then, in
 `WP_UnitTest_Factory_For_Blog::create()`, toggle it off after
 `wpmu_create_blog()` has turned it on. The main downside here is that
 `WP_INSTALLING` is not really the kind of thing we want to be toggleable
 by devs, so we'd have to document it accordingly.
 3. Run all blog-creating tests in separate processes that don't preserve
 the global state. That way, the constant would be set in the other
 process, but not when returning to run the rest of the tests. I played
 with this a little, and it looks like it's not going to be trivial to make
 it work - we may need to bootstrap WordPress for every test or something
 like that.
 4. Make sure that all tests that create a blog are run at the very end of
 the suite.

 Option (2) seems like the most natural fix, but it's also the only one
 that touches `src/`. Any additional thoughts are welcome.

--

--
Ticket URL: <https://core.trac.wordpress.org/ticket/31130#comment:1>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform