[wp-meta] [Making WordPress.org] #8009: Phased releases and roll-outs of plugins

[Making WordPress.org]

#8009: Phased releases and roll-outs of plugins
------------------------------+---------------------
 Reporter:  matt              |       Owner:  (none)
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:
Component:  Plugin Directory  |  Resolution:
 Keywords:  has-patch         |
------------------------------+---------------------

Comment (by joshuaflow):

 Much of this has already been mentioned but I'll add my 2 cents since a
 phased rollout is part of our release process.  The rollout itself is
 extremely helpful when it comes to discovering bugs due to a broader range
 of plugins, environments, and settings than those used during testing.

 A few items that we've found important for this rollout:

 - **Gradual release** - Releasing to a small number of pools and gradually
 ramping up is very important.  This allows us to slowly uncover errors
 instead of crashing all the sites if a more severe issue is found.
 - **Rollback** - When something more critical is discovered, we need a way
 to quickly revert those changes.  Leaving sites affected with a critical
 issue for a length of time is not acceptable.
 - **Quick feedback** - Timely feedback on issues is important in both
 addressing them and rolling back if necessary.


 Some of the challenges we've encountered with these rollouts include:

 - **Reproducing issues** - In order to be able to reproduce issues, we
 need as much environmental context as possible.  This is much easier on
 your own testing infrastructure where you have full access to logs
 compared to issues that occur in the wild.
 - **Plugin conflicts** - This point may be controversial, but when you
 encounter a critical issue, the onus to resolve the issue (seemingly)
 falls back to the plugin doing the rollout even when those issues may stem
 from other plugins incorrectly using hooks/APIs.  This tends to affect
 monolith plugins and those that are extended by other plugins far more
 often.
 - **Monitoring** - We need to monitor proactively, which has come with a
 pretty steep learning curve.  Dashboards and specific error filtering are
 critical to diagnosing problems quickly.

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/8009#comment:18>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org