[wp-meta] [Making WordPress.org] #8009: Phased releases and roll-outs of plugins

[Making WordPress.org]

#8009: Phased releases and roll-outs of plugins
------------------------------+---------------------
 Reporter:  matt              |       Owner:  (none)
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:
Component:  Plugin Directory  |  Resolution:
 Keywords:  has-patch         |
------------------------------+---------------------

Comment (by dd32):

 > @tw2113: If a plugin has opted into phased releases, would sites that
 haven't been actively touched in say 1 year+ be potentially first in line,
 leaving the available update in limbo?
 > or would sites that are at least relatively actively maintained be
 considered first?

 Unfortunately WordPress.org has no idea about any given WordPress install
 - We don't know if it's been logged into, if it's a spam site, if it's
 updated a plugin, etc.

 While we could look at the version of WordPress it's using, or the
 previous version of the plugin it's using (ie. Is it using a 1month old
 version of the plugin / last stable, or is it 5 years out of date) that's
 going to significantly complicate calculations and processing speed (The
 update-check API for example receives 10's of thousands of requests a
 minute, even 10ms adds up)

 > @nerrad: I wonder if we could add some additional functionality to the
 WP Fatal Error Handler:

 I'd love to see this, and it would greatly help plugin authors and the
 stability of WordPress sites long-term.

 This probably should happen regardless, and should be implemented on it's
 own merits, rather than attached to this feature.

 I know some people will have concerns about privacy (even if it's opt-in)
 and access to the raw data though. Performing anything significant in a
 PHP fatal error handler has also been problematic in the past from a PHP
 perspective.

 > @joshuaflow: Rollback - When something more critical is discovered, we
 need a way to quickly revert those changes. Leaving sites affected with a
 critical issue for a length of time is not acceptable.


 Updating the WordPress plugin updates system to allow a forced downgrade
 ''would'' be useful in certain circumstances, however we are limited in
 what we can do here IMHO.

 When you're controlling the hosting environment you can rollback very
 easily.

 When it's WordPress.org, we have to rely upon the fact that the WordPress
 site will check for new updates in 12 hours time for another update, and
 if we tell it something then, it'll (maybe) install that update anywhere
 from 5minutes to 12hours later.

 In other words; there's no choice in it, the site WILL be running the
 aborted version for likely at least 12 hours, and potentially up to 24hrs
 before it performs a follow-up action.

 For many plugin authors, they'd be able to release a patched version in
 that time instead. So much so that I suspect many authors would not abort
 a release until they knew what was wrong (ie. to prevent one persons
 plugin conflict stopping the release that works for a million others).

 > @namithjawahar: A new dimension to the downloads graph can be a quick
 solution to reporting showing downloads vs errors, if its crosses a
 threshold

 The problem is what is "an error"? If we're talking PHP Fatal errors
 that's one thing, but we can't exactly detect Javascript errors, plugin
 conflicts, or other similar broken functionality.

 If plugins could have a "health check" (similar to docker containers) that
 indicated they're operating correctly that might help here, but still
 often a plugin doesn't know it's operating badly - and that's the entire
 reason the bad plugin releases exist.

 > @azaozz In that terms think there is no need of buckets or storing any
 site identifiers. Only thing that needs to be stored is how many sites
 were actually updated.

 The main reason I have against the counting method, is that while we can
 count based on ZIP downloads, or post-update pings, if we were limiting to
 say 5k sites to start with, that doesn't help that we've returned update
 responses to 10k other sites already that will perform their updates in
 the next 5 hours time.

 In other words: WordPress does not apply plugin updates when it finds out
 about an update being available.

 3rd-party tooling (Such as Managed WordPress hosts) however likely DO
 update as soon a plugin release is made, but I'm basing my thoughts on
 core WordPress, as these may change over time.


 The counting could of course be at the update-check serving level "OK,
 I've told 50k sites to update, let's put the brakes on for an hour" but
 there's no guarantee that they'll update - that 50k might see 5k or 45k
 updates. I think this is what you might be getting at.
 We're a little hamstrung by how to store that data, Memcache doesn't
 guarantee that the data will remain (and we sometimes have to reset it),
 and database writes are expensive. Memcache would be "good enough" for
 most people even if it reset, but would be frustrating to users and
 authors when 50% of the way through an update it reset back to starting
 over again.

 Definitely solvable problem, but I personally prefer statistics, and it's
 what we've used on WordPress.orgs APIs for this type of bucketing for a
 long time in the core updates.

 > @azaozz: Seems this will be true for plugins with less active installs.
 Plugins with lots of installs (>1m) will probably get a lot of manual
 updates too so the rollout will be quite fast.

 I don't have any data on the manual-vs-auto update information for
 plugins, but I don't think user-initiated updates within the first day or
 so are likely to be significant enough.
 While they 100% occur, based on the data I'm looking at, I expect in the
 order of 99% of updates that would be affected by this would be automated
 in one way or another. The downloads-per-minute graphs are just too
 consistent to be anything else.

 One thought is that we could hold back auto-updates for the first 24hrs
 and just let manual updates occur, users that update plugins are more
 likely to report issues and to actually be using the site. The problem
 with this suggestion is that we have no way of preventing a site
 performing an auto-update or signaling to it at present. We could add this
 to WordPress 6.9+ but again, what do you do about existing and yesterdays
 WordPress's.

 > @azaozz: Thinking that monitoring the forums after a new release is a
 fairly standard practice. Right, there can be better ways, but they all
 require the plugin's users to act.

 That's my point :) Support forums ARE a good way to gauge things, but it
 is not foolproof AND we're talking about a very small percentage of users
 who actually create threads verses the number of installs a plugin has.

 > @azaozz: Maybe having a "Report a problem" button on each plugin row in
 the Plugins list table would help somewhat.

 Originally that's kind of what I was thinking for with the Ratings/reviews
 from the plugins dashboard, being able to say "Nah this has broken my
 site, sad-face 1 star." but you're right, ratings aren't necessarily the
 important thing here, being able to report an issue and get it resolved
 is.

 It might need to be considered whether `Create a support thread` is
 appropriate for the purpose though. It might be that `Report a problem`
 which feeds into anonymous aggregated plugin feedback would be better..
 but at that point, you might as well collect positive feedback too and
 just call it Reviews...

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/8009#comment:21>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org