[wp-trac] [WordPress Trac] #32474: Facilitate widgets to be stored in posts instead of options

[WordPress Trac]

#32474: Facilitate widgets to be stored in posts instead of options
-------------------------+------------------
 Reporter:  westonruter  |       Owner:
     Type:  enhancement  |      Status:  new
 Priority:  normal       |   Milestone:  4.3
Component:  Widgets      |     Version:  2.8
 Severity:  normal       |  Resolution:
 Keywords:  has-patch    |     Focuses:
-------------------------+------------------

Old description:

> Widget instance data has always been stored in `wp_options`. This worked
> fine when there were only a few widgets used on a site. When there are
> hundreds—or thousands—there are big performance problems. When
> initializing widgets, Core loads the entire settings arrays of all
> widgets even when they are not used (#23909). Additionally, widget
> settings options are by default autoloaded: this means that for widget-
> heavy sites using Memcached Object Cache will reach the 1MB limit and
> crash since the autoloaded options will no longer be cacheable
> (WordPress.com even [https://github.com/Automattic/vip-
> quickstart/blob/master/www/wp-content/mu-plugins/alloptions-limit.php
> block a site from loading] in this scenario). Storage in autoloaded
> options also means widgets are susceptible to alloptions cache corruption
> (#31245). Bottom line: widgets stored in options are not scalable.
>
> As I've [https://core.trac.wordpress.org/ticket/31436#comment:2 mentioned
> elsewhere], widgets should ideally be stored in posts instead of options.
> In addition to improving the performance problems above, there are many
> advantaged to storing widgets as posts, including user attribution via
> `post_author`, revision history, import/export, querying, widget drafts,
> scheduled widgets,  etc.
>
> Changing the storage mechanism for widgets is a major change, but with a
> couple small changes to `WP_Widget`, Core can easily support alternative
> widget instance storage systems, all through the
> `pre_option_widget_{$id_base}` and `pre_update_option_{$id_base}`
> filters.
>
> The `WP_Widget` change is to allow settings to be array-like objects,
> specifically `ArrayIterator`. A plugin then can implement the
> `offsetGet()` and `current()` methods for an `ArrayIterator` subclass to
> lazy-load data from posts only when it is needed. Otherwise, the only
> data needed for WordPress at `widgets_init` are shallow arrays of multi-
> widget numbers mapped to their corresponding widget instance post IDs.
> When accessing an item in the array, the `offsetGet()` method can then
> look-up the widget instance data and cache it for future lookups.
>
> This backwards-compatible shim approach is similar to what @nacin did in
> #20103 for `WP_Theme` (see also [https://vip.wordpress.com/2014/08/26
> /how-wordpress-evolves-full-transcript/ How WordPress Evolves Without
> Breaking Everything]).
>
> Let's make the widget a first-class citizen in WordPress.

New description:

 Widget instance data has always been stored in `wp_options`. This worked
 fine when there were only a few widgets used on a site. When there are
 hundreds—or thousands—there are big performance problems. When
 initializing widgets, Core loads the entire settings arrays of all widgets
 even when they are not used (#23909). In the Customizer, changes to widget
 settings is very expensive due having to unserialize and serialize option
 arrays repeatedly (#32103). Additionally, widget settings options are by
 default autoloaded: this means that for widget-heavy sites using Memcached
 Object Cache will reach the 1MB limit and crash since the autoloaded
 options will no longer be cacheable (WordPress.com even
 [https://github.com/Automattic/vip-quickstart/blob/master/www/wp-content
 /mu-plugins/alloptions-limit.php block a site from loading] in this
 scenario). Storage in autoloaded options also means widgets are
 susceptible to alloptions cache corruption (#31245). Bottom line: widgets
 stored in options are not scalable.

 As I've [https://core.trac.wordpress.org/ticket/31436#comment:2 mentioned
 elsewhere], widgets should ideally be stored in posts instead of options.
 In addition to improving the performance problems above, there are many
 advantaged to storing widgets as posts, including user attribution via
 `post_author`, revision history, import/export, querying, widget drafts,
 scheduled widgets,  etc.

 Changing the storage mechanism for widgets is a major change, but with a
 couple small changes to `WP_Widget`, Core can easily support alternative
 widget instance storage systems, all through the
 `pre_option_widget_{$id_base}` and `pre_update_option_{$id_base}` filters.

 The `WP_Widget` change is to allow settings to be array-like objects,
 specifically `ArrayIterator`. A plugin then can implement the
 `offsetGet()` and `current()` methods for an `ArrayIterator` subclass to
 lazy-load data from posts only when it is needed. Otherwise, the only data
 needed for WordPress at `widgets_init` are shallow arrays of multi-widget
 numbers mapped to their corresponding widget instance post IDs. When
 accessing an item in the array, the `offsetGet()` method can then look-up
 the widget instance data and cache it for future lookups.

 This backwards-compatible shim approach is similar to what @nacin did in
 #20103 for `WP_Theme` (see also [https://vip.wordpress.com/2014/08/26/how-
 wordpress-evolves-full-transcript/ How WordPress Evolves Without Breaking
 Everything]).

 Let's make the widget a first-class citizen in WordPress.

--

Comment (by westonruter):

 Replying to [comment:1 nacin]:
 > I think we're likely going to see alloptions go away in #31245,
 something I'm willing to try in 4.3. I'd be curious to see arguments here
 that don't involve alloptions.

 As mentioned in the description, non-alloptions arguments for not using
 options for storing widget instances include:

 * Loading all instance settings data for all widgets even when they are
 not ever used is wasteful and inefficient. Imagine if all posts in the
 database had to be loaded into memory each time WordPress renders a
 template.
 * Updating a single widget instance is expensive when settings are stored
 in options because for each instance change, the entire dataset for the
 widget type has to be unserialized. (Especially in the Customizer,
 #32103.)
 * For Memcached Object Cache specifically, when the number of widgets of a
 given type grows large enough it will breach the 1MB limit and no longer
 be cacheable.

 Using a custom post type for storing widget instances seems like an
 elegant WordPress-way to store this data and it brings with it several
 benefits, as noted in the ticket's description.

 I'm working on finishing a proof of concept plugin that makes use of
 widget instances being passed around as `ArrayIterators` instead of
 intrinsic `array`s.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/32474#comment:2>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform