[wp-trac] [WordPress Trac] #35669: Store widgets in a custom post type instead of options

Sun Jan 31 05:08:41 UTC 2016

#35669: Store widgets in a custom post type instead of options
--------------------------+------------------------------
 Reporter:  westonruter   |       Owner:
     Type:  enhancement   |      Status:  new
 Priority:  normal        |   Milestone:  Awaiting Review
Component:  Widgets       |     Version:  2.8
 Severity:  normal        |  Resolution:
 Keywords:  dev-feedback  |     Focuses:
--------------------------+------------------------------
Description changed by westonruter:

Old description:

> Widget instances are stored in options. For a multi-widget (`WP_Widget`)
> the widget instances of a given type (`id_base`) are stored in a
> serialized array of instance arrays. A widget ID is comprised of a
> widget's `id_base` followed by a number which is the array index for that
> widget instance. For example, the third-created Text widget would have
> the ID `text-4` (note that multi-widget numbering starts at 2). Old
> single widgets do not include the numeric index after the `id_base`, and
> technically they could be stored anywhere (see #35656 for suggestion to
> deprecate old single widgets).
>
> == Issues
>
> There are several problems with how widgets are currently stored as
> options.
>
> '''Scalability:''' For sites with a large number of widget instances, the
> entire collection of widgets must be unserialized with each request to
> access only one widget of a given type. (Note #23909 for how all widget
> instances get registered with every request.) For sites that use
> Memcached as an external object cache where cache buckets have a 1MB
> limit, since all widget instances of a given type are stored in a single
> option, sites with a huge number of widgets will overrun this limit.
> What's more is that widget options get registered as autoloaded, so all
> widget options will get combined together in the `alloptions` key, making
> widgets even more liable to overrun the 1MB cache bucket limit in
> Memcached.
>
> '''Concurrency:''' Since all widget instances of a given type are stored
> in a single option, if two users attempt to update two separate widgets
> at the same time, it is possible that one of the updates will get lost
> (see #31245). Additionally, the widgets admin page and widgets in the
> Customizer both get loaded with the max number (array index) for each
> widget type. When a new widget instance is created, this maximum number
> is incremented in memory and used in the new widget ID which is then
> passed to the server for saving. If two users have loaded the UI at the
> same time, when they both create a widget of a given type and save their
> widget changes, the one who saves last will overwrite the other user's
> widget since the two widgets would have the same ID. (See #32183 for more
> about the widget ID collisions, and see [https://wordpress.org/plugins
> /customize-widgets-plus/ Customize Widgets Plus] for a “Widget Number
> Incrementing” component which uses Ajax to generate new widget IDs in a
> more concurrency-safe manner.)
>
> '''Addressability:''' As noted above, widget instance IDs are comprised
> of the widget type's `id_base` followed by the array index `number`. Two
> different widget instances can have the same `number`, such as `search-3`
> and `text-3`, since the `number` is incremented in the scope of the
> instances of the given type. No other objects in WordPress are identified
> by strings in this way, that is as of now: taxonomy terms actually used
> to have to be addressed by a numeric term ID and taxonomy name until term
> splitting happened in 4.2 (see #5809). Now, however, a term can be
> uniquely identified by a single integer ID.
>
> All of the above issues would be resolved by switching to store widget
> instances in a custom post type, where each widget instance has a single
> unique auto-incremented post ID.
>
> == Advantages
>
> Storing widgets in custom post type has several benefits beyond fixing
> the above issues, including:
>
> * widget authorship attribution
> * revision history
> * import/export
> * querying
> * widget drafts
> * scheduled widgets
>
> == Data Migration
>
> Migrating widgets from options to a custom post type would involve some
> tedious data migration to update all references to current `id_base-
> number` widget IDs to their new integer IDs. The old widget ID could
> actually be copied directly into the `post_name` field for the
> `widget_instance` posts. Backwards compatibility for the
> `sidebars_widgets` option containing the old-style IDs may be necessary.
> Newly created widget IDs could have `post_name` fields populated with the
> `id_base` followed by the post ID. This switch would also necessitate
> discontinuing to register all widget instances with every request
> (#23909).
>
> == Sidebars and Widget Groups
>
> Perhaps out of scope for this ticket, but the way that widgets get
> associated with sidebars should also perhaps be changed to follow the
> pattern of how nav menu items are associated with a nav menu via a
> taxonomy term. The implementing of widget groups (#19912) could be the
> right opportunity to do this, where a `widget_grouping` taxonomy could be
> introduced, and when a grouping is assigned to a sidebar, the backwards-
> compatible widget IDs could be copied into the existing
> `sidebars_widgets` option. Otherwise, backwards compatibility might
> entail adding `pre_option_sidebars_widgets` filter.
>
> == REST API Impacts
>
> For more on widgets and now they relate to nav menu items in the context
> of a harmonized interface via the REST API, see https://github.com/WP-API
> /wp-api-menus-widgets-endpoints/issues/10

New description:

 Widget instances are stored in options. For a multi-widget (`WP_Widget`)
 the widget instances of a given type (`id_base`) are stored in a
 serialized array of instance arrays. A widget ID is comprised of a
 widget's `id_base` followed by a number which is the array index for that
 widget instance. For example, the third-created Text widget would have the
 ID `text-4` (note that multi-widget numbering starts at 2). Old single
 widgets do not include the numeric index after the `id_base`, and
 technically they could be stored anywhere (see #35656 for suggestion to
 deprecate old single widgets).

 == Issues

 There are several problems with how widgets are currently stored as
 options.

 '''Scalability:''' For sites with a large number of widget instances, the
 entire collection of widgets must be unserialized with each request to
 access only one widget of a given type. (Note #23909 for how all widget
 instances get registered with every request.) For sites that use Memcached
 as an external object cache where cache buckets have a 1MB limit, since
 all widget instances of a given type are stored in a single option, sites
 with a huge number of widgets will overrun this limit. What's more is that
 widget options get registered as autoloaded, so all widget options will
 get combined together in the `alloptions` key, making widgets even more
 liable to overrun the 1MB cache bucket limit in Memcached.

 '''Concurrency:''' Since all widget instances of a given type are stored
 in a single option, if two users attempt to update two separate widgets at
 the same time, it is possible that one of the updates will get lost (see
 #31245). Additionally, the widgets admin page and widgets in the
 Customizer both get loaded with the max number (array index) for each
 widget type. When a new widget instance is created, this maximum number is
 incremented in memory and used in the new widget ID which is then passed
 to the server for saving. If two users have loaded the UI at the same
 time, when they both create a widget of a given type and save their widget
 changes, the one who saves last will overwrite the other user's widget
 since the two widgets would have the same ID. (See #32183 for more about
 the widget ID collisions, and see [https://wordpress.org/plugins
 /customize-widgets-plus/ Customize Widgets Plus] for a “Widget Number
 Incrementing” component which uses Ajax to generate new widget IDs in a
 more concurrency-safe manner.)

 '''Addressability:''' As noted above, widget instance IDs are comprised of
 the widget type's `id_base` followed by the array index `number`. Two
 different widget instances can have the same `number`, such as `search-3`
 and `text-3`, since the `number` is incremented in the scope of the
 instances of the given type. No other objects in WordPress are identified
 by strings in this way, that is as of now: taxonomy terms actually used to
 have to be addressed by a numeric term ID and taxonomy name until term
 splitting happened in 4.2 (see #5809). Now, however, a term can be
 uniquely identified by a single integer ID.

 All of the above issues would be resolved by switching to store widget
 instances in a custom post type, where each widget instance has a single
 unique auto-incremented post ID.

 == Advantages

 Storing widgets in custom post type has several benefits beyond fixing the
 above issues, including:

 * widget authorship attribution
 * revision history
 * import/export
 * querying
 * widget drafts
 * scheduled widgets

 == Data Migration

 Migrating widgets from options to a custom post type would involve some
 tedious data migration to update all references to current `id_base-
 number` widget IDs to their new integer IDs. The old widget ID could
 actually be copied directly into the `post_name` field for the
 `widget_instance` posts. Backwards compatibility for the
 `sidebars_widgets` option containing the old-style IDs may be necessary.
 Newly created widget IDs could have `post_name` fields populated with the
 `id_base` followed by the post ID. This switch would also necessitate
 discontinuing to register all widget instances with every request
 (#23909).

 == Sidebars and Widget Groups

 Perhaps out of scope for this ticket, but the way that widgets get
 associated with sidebars should also perhaps be changed to follow the
 pattern of how nav menu items are associated with a nav menu via a
 taxonomy term. The implementing of widget groups (#19912) could be the
 right opportunity to do this, where a `widget_grouping` taxonomy could be
 introduced, and when a grouping is assigned to a sidebar, the backwards-
 compatible widget IDs could be copied into the existing `sidebars_widgets`
 option. Otherwise, backwards compatibility might entail adding
 `pre_option_sidebars_widgets` filter.

 == REST API Impacts

 For more on widgets and now they relate to nav menu items in the context
 of a harmonized interface via the REST API, see https://github.com/WP-API
 /wp-api-menus-widgets-endpoints/issues/10

 == Feature Plugin

 See the [https://github.com/xwp/wp-customize-widgets-plus Customize
 Widgets Plus] feature plugin's “Widget Posts” module for an initial
 implementation of storing widgets in a `widget_instance` custom post type.
 This plugin depends on #32474 which facilitated plugins to store widgets
 in posts instead of options.

--

--
Ticket URL: <https://core.trac.wordpress.org/ticket/35669#comment:2>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform