[wp-trac] [WordPress Trac] #31245: Replace alloptions with a key cache

[WordPress Trac]

#31245: Replace alloptions with a key cache
--------------------------------+-----------------
 Reporter:  rmccue              |      Owner:
     Type:  enhancement         |     Status:  new
 Priority:  normal              |  Milestone:  4.2
Component:  Options, Meta APIs  |    Version:
 Severity:  normal              |   Keywords:
  Focuses:  performance         |
--------------------------------+-----------------
 Let's talk about `alloptions`. `alloptions` is a particularly fragile
 piece of WordPress, since it's exceptionally vulnerable to accidental
 concurrency issues due to its design.

 A primer on `alloptions`, for those who haven't dug into it: `alloptions`
 is a cache key that stores all the options marked with autoload, thereby
 reducing the number of external calls out to an object cache from hundreds
 down to 1. It does this by running `SELECT * FROM wp_options WHERE
 autoload = 'yes'`, then storing that array (map of `option_name =>
 option_value`) under the `alloptions` cache key. (Non-autoloaded options
 are stored as individual cache items instead, with a key of `option_name`
 in the `options` group.)

 When you add an option, you can pick the `autoload` flag. This flag is
 `'yes'` (`true`) '''by default''', meaning most calls to `add_option` will
 set options to autoload. Additionally, if you add an option via
 `update_option`, there's no way to set this flag. These combined mean that
 the vast majority of options are stored in the `alloptions` cache.

 == Why is this an issue? ==

 The `alloptions` cache is loaded very early in the WordPress request
 lifetime. From this moment until the end of a request, option reads are
 taken from this cache rather than the database.

 In addition, every time an autoloaded option is updated or deleted,
 `alloptions` is similarly updated. However, this means that on '''any'''
 autoloaded option being updated, '''every''' autoloaded option has its
 value set to the value at load time.

 This is a huge concurrency bug, as it's very easy to run into
 accidentally. As a proof of concept, try updating two different options in
 two different requests at the same time, and you'll see that whichever ran
 first will lose the changes. (See #25623)

 (We're running into this issue a few times a week, due to certain plugins
 running an `update_option` on a high percentage of requests. These plugins
 are obviously being dumb in doing so, but they're revealing the core bug
 quite nicely.)

 == How do we fix this? ==

 The biggest issue is that `alloptions` is a single cache item containing
 every option. If we reduce this to one cache item per option, the
 concurrency problem is then reduced to individual options (and can be
 mitigated in the places it actually matters, typically in plugins). Rather
 than storing all the options, we can instead cache the keys that matter,
 then grab the values for all of those at once.

 This means that we may still have minor concurrency issues when adding and
 removing options, however these happen much less often than updating
 existing ones. In addition, the cache would now only be used to decide
 which options to load at the start of the request, so worst case scenario,
 you end up with extra object cache `get` calls. This is much better than
 the current case, where a concurrency problem can return incorrect data.

 This change would require adding a `wp_cache_get_multi` to take full
 advantage of the cache, however compatibility could be shimmed here with a
 loop and individual calls to `wp_cache_get`. This '''would''' cause worse
 performance for object caches that don't support multi-get (or haven't
 been updated), but at the benefit of no longer returning potentially bad
 data.

 There's a few things that need to be done for this:

 * Add `wp_cache_multi_get`, and talk to object cache maintainers to update
 the popular ones to support it (Memcache, APC)
 * Change `wp_load_alloptions` to pull from `alloptionskeys`, then run a
 multi-get on those
 * Ensure `alloptionskeys` is a protected option name
 * Store value caches per-key

--
Ticket URL: <https://core.trac.wordpress.org/ticket/31245>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform