[wp-trac] [WordPress Trac] #62148: Add Twenty Twenty-Five to the Performance Tests

[WordPress Trac]

#62148: Add Twenty Twenty-Five to the Performance Tests
------------------------------+--------------------------
 Reporter:  joemcgill         |       Owner:  pbearne
     Type:  task (blessed)    |      Status:  closed
 Priority:  normal            |   Milestone:  6.8
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:  fixed
 Keywords:  early has-patch   |     Focuses:  performance
------------------------------+--------------------------

Comment (by joemcgill):

 Just wanted to follow up to provide more context about the purpose of
 these baseline measurements to clear up some potential confusion that I
 noticed in [https://github.com/WordPress/wordpress-
 develop/pull/7688#discussion_r1886293496 the PR discussion] about whether
 these should be bumped to 6.7.1 instead—specifically:

 > I think it should be fine to use 6.7 and not 6.7.1. After all, we're
 comparing performance between major releases, not minor ones. If we start
 with 6.7.1, it's possible (though unlikely) a performance positive change
 in 6.7.1 would not be included in the next report.

 In this workflow, the whole purpose of including a baseline test is to
 normalize the data when comparing these metrics for multiple commits over
 time, as we do when when we log and display this data in the
 [https://www.codevitals.run/project/wordpress WordPress Code Vitals
 dashboard]. The theory is that by collecting performance metrics for the
 same commit over time, we can control for the performance impact
 introduced by non-code factors, like the difference in performance of the
 GitHub workers, and get a better indication that a performance change was
 introduced by the specific code changes of a commit.

 The way this works in practice is that when we record the performance data
 to the dashboard, the metrics for the latest commit get normalized based
 on the difference recorded between the baseline metrics from that workflow
 run with the previously recorded baseline
 ([https://github.com/youknowriad/codevitals/blob/16542dcaf71f448fd0c29f19b18144284ce00106/pages/api/log.js#L48
 reference]).

 We've maintained the same baseline since this was introduced so that our
 normalization stayed consistent, so after this change it is expected that
 the trends on the dashboard will take time to even out (as you can already
 see when visiting the dashboard after this commit). Since we really only
 use this method as a tool to spot potential commits that introduced a
 performance cahnge, I think starting fresh each release is likely fine, as
 it only affects how we're observing this data, not actually ''changing''
 the way this is measured, which can be confirmed by looking at the summary
 data displayed on the workflows [https://github.com/WordPress/wordpress-
 develop/actions/runs/12371242023 before] and [https://github.com/WordPress
 /wordpress-develop/actions/runs/12378113523 after] this change.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/62148#comment:16>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform