[wp-meta] [Making WordPress.org] #1181: Enhancement for Reviews/Ratings: Use a Rolling Time Frame for more current reviews

[Making WordPress.org]

#1181: Enhancement for Reviews/Ratings: Use a Rolling Time Frame for more current
reviews
----------------------------+---------------------
 Reporter:  cais            |       Owner:  (none)
     Type:  enhancement     |      Status:  new
 Priority:  low             |   Milestone:  Q1
Component:  Support Forums  |  Resolution:
 Keywords:                  |
----------------------------+---------------------

Comment (by JeffPaul):

 Cross-posting my thoughts on the duplicate #7968 to consolidate ideation
 here:
 I believe the existing star rating for a plugin (i.e., "4.2 out of 5
 stars.") is based on the lifetime average of reviews. Since some of those
 reviews may be many years old and not reflective of more recent updates
 for a plugin, I propose that we consider an update to this calculation
 that takes into account the time since a review.

 Instead of a simple average, I propose that we calculate a time-weighted
 average where newer reviews contribute more to the final score and older
 reviews gradually fade in influence.

 A formula for this could be something like:

 > Time-weighted average = (sum of (rating x weight)) / (sum of weights)

 ...where the rating is each individual review (e.g. 1 - 5 stars) and
 weight is determined by how recent the review is.

 Potential weightings to consider could be:

 > Reviews less than 6 months old: weight = 1.0
 > 6-12 months old: weight = 0.75
 > 1-2 years old: weight = 0.5
 > 2-3 years old: weight = 0.25
 > 3-5 years old: weight = 0.1

 To ensure some statistical relevance and to try and avoid manipulation, we
 could only show the time-weighted average if a plugin has at least 10
 reviews in the past 12 months ...OR... If it has at least 30 total reviews
 across all time periods. A potential fallback here could be something like
 "Not enough recent reviews to generate a current rating" and instead
 display the historical average as is currently displayed.

 Some other potential adjustments to consider:

 * Release-Based Reset Weighting: When a plugin releases a major version,
 consider re-weighting reviews so that pre-major-release reviews carry less
 weight. I'm not a huge fan of this as it potentially allows for
 manipulation and the fact that versioning is already a loaded topic (e.g.,
 SemVer vs. WPver).
 * Display Two Ratings: a Current Rating (i.e., time-weighted 12-month
 view) and a Lifetime Rating (i.e., simple historical average). This
 mirrors how Amazon sometimes displays "recent reviews" and "overall
 ratings".

 Some of the benefits of an updated calculation are that it encourages
 plugin maintainers to improve their plugins, it prevents stale or no-
 longer-relevant reviews from skewing perception, and it can help users
 make better-informed plugin decisions.

 I'm curious what folks think of an algorithm like this and if there are
 any iterations to the weighting time ranges, the weights, and the minimum
 review threshold... thoughts?

-- 
Ticket URL: <https://meta.trac.wordpress.org/ticket/1181#comment:33>
Making WordPress.org <https://meta.trac.wordpress.org/>
Making WordPress.org