[wp-meta] [Making WordPress.org] #8009: Phased releases and roll-outs of plugins

[Making WordPress.org]

#8009: Phased releases and roll-outs of plugins
------------------------------+---------------------
 Reporter:  matt              |       Owner:  (none)
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:
Component:  Plugin Directory  |  Resolution:
 Keywords:  has-patch         |
------------------------------+---------------------

Comment (by azaozz):

 Replying to [comment:21 dd32]:
 > The main reason I have against the counting method, is that while we can
 count based on ZIP downloads, or post-update pings, if we were limiting to
 say 5k sites to start with, that doesn't help that we've returned update
 responses to 10k other sites already that will perform their updates in
 the next 5 hours time.
 >
 > In other words: WordPress does not apply plugin updates when it finds
 out about an update being available.
 > ''
 > The counting could of course be at the update-check serving level "OK,
 I've told 50k sites to update, let's put the brakes on for an hour" but
 there's no guarantee that they'll update - that 50k might see 5k or 45k
 updates. I think this is what you might be getting at.

 Right, as the initial updates number is not critical was thinking it would
 be acceptable to have some variations, for example "5k means 4.5 to 5.5k".
 This would still allow for the functionality to work correctly, and will
 simplify the rest of the code/logic quite a lot.

 For example, if we wanted to initially update 5k sites we'll probably want
 to send the update notification to 6 or 7k sites. That number will of
 course depend on the ratio between manual and auto-updates, and will
 continue to grow (slower) for the next few hours (because of the
 continuing manual updates on sites that use cached data from the API). Of
 course we'll be able to tweak the number of extras once this is released
 and we're able to monitor the updates flow.

 > We're a little hamstrung by how to store that data, Memcache doesn't
 guarantee that the data will remain (and we sometimes have to reset it),
 and database writes are expensive.

 Perhaps a good compromise would be to store the data in Memcache and flush
 it to the DB every minute or so? (At flush time it can check if the new
 number looks okay, i.e. is not smaller than the DB number, and if yes it
 can be restored from the DB). This is again based on the premise that the
 numbers are not critical and some variations are acceptable.

 Of course if this can be made to work similarly to, or even reuse some
 exiting code it would be very nice.

 > I don't have any data on the manual-vs-auto update information for
 plugins, but I don't think user-initiated updates within the first day or
 so are likely to be significant enough.

 I don't have any data either but have monitored updates for a 2k active
 installs plugin few times before there were auto-updates in WP. It
 somewhat depends on the release time but generally (as far as I remember)
 the first 10k downloads happen in less than 10 minutes, reaching 100k in
 about 2 hours, and about 1m in 24 hours.

 > One thought is that we could hold back auto-updates for the first 24hrs
 and just let manual updates occur, users that update plugins are more
 likely to report issues and to actually be using the site.

 Yea, this is a good idea.

 > The problem with this suggestion is that we have no way of preventing a
 site performing an auto-update or signaling to it at present.

 Right, has to be fixed in core. The API should be able to set whether to
 do an auto-update. Thinking this would be a fairly simple enhancement.

 > Support forums ARE a good way to gauge things, but it is not foolproof
 AND we're talking about a very small percentage of users who actually
 create threads verses the number of installs a plugin has.
 > ..
 > It might need to be considered whether `Create a support thread` is
 appropriate for the purpose though. It might be that `Report a problem`
 which feeds into anonymous aggregated plugin feedback would be better..
 but at that point, you might as well collect positive feedback too and
 just call it Reviews...

 Thinking more about this, you're right. A standard support thread may not
 be best. Also afaik one of the difficulties with that form of feedback is
 the requirement for the user to long in at wp.org as that may affect
 privacy, etc. But perhaps anonymous feedback would be better? That would
 not only solve any privacy related issues/questions but will also make it
 easier to use as the "reporter" will not need to log in.

 I kind of imagine something like this: on clicking "Report a (plugin)
 problem" a modal opens with the site health info pre-filled. The user will
 be able to add few words to describe it. Submitting it would go through
 their WP install (TBD, but seems preferable not to be directly to wp.org;
 privacy, etc. and there can be filters for plugins to access it) and WP
 will send it through the API.

 Then these semi-automated reports can be visible only by the plugin
 authors, not public so they are unattractive for spammers. Or perhaps we
 can try to aggregate some of the data and post to the plugin's support
 forum so other users are aware (if they ever look there..).

 Thinking this is worth a core trac ticket at this point :)

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/8009#comment:25>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org