[wp-trac] [WordPress Trac] #57687: Add automated performance testing CI workflow MVP

[WordPress Trac]

#57687: Add automated performance testing CI workflow MVP
--------------------------------------+--------------------------
 Reporter:  adamsilverstein           |       Owner:  mukesh27
     Type:  task (blessed)            |      Status:  assigned
 Priority:  normal                    |   Milestone:  6.2
Component:  Build/Test Tools          |     Version:  trunk
 Severity:  normal                    |  Resolution:
 Keywords:  has-patch has-unit-tests  |     Focuses:  performance
--------------------------------------+--------------------------

Comment (by desrosj):

 Replying to [comment:7 joemcgill]:

 Thanks so much for both your work on this!

 > 1. The longest running step in the
 workflow is by far installing npm dependencies, assuming they're uncached,
 but I'm also not seeing these get cached between workflows and want to
 make sure we're not missing something.

 The way that caching works within the `setup-node` action is the global
 package data is cached, not the `node_modules` directory. We could add a
 separate step after that uses `actions/cache` to cache the actual
 `node_modules` directory, but this is generally recommended against for a
 few reasons:
 - It can cause breaks when using multiple node versions (not currently a
 real issue for us in this workflow).
 - According to the
 [https://github.com/actions/cache/blob/main/examples.md#node---npm npm
 specific documentation] in the `actions/cache` action, it can break when
 using `npm ci` (which we do use here).

 There is a [https://github.com/WordPress/gutenberg/pull/45932 PR in the
 Gutenberg repository] suggesting that we also cache the `node_modules`
 directory. I've been meaning to dive in and review that PR as well. The
 TLDR thinking there is to call `npm ci` when a cache hit does not occur
 (calling `npm ci` will delete `node_modules` before installing the
 modules, causing problems), and `npm install` when one does.

 For the time being, I think I'd prefer to match how the other `wordpress-
 develop` workflows currently are set up with `npm ci` and no direct
 caching of the `node_modules` directory. The time for that step to
 complete is ~2 minutes (give or take ~30 seconds), which inline with what
 we see in the other `wordpress-develop` workflows. And if the Gutenberg PR
 ends up being merged with good results/no problems, I'd like to bring that
 over here for all of our workflows at once.

 > 2. The publish step requires that we set up a secret token in the
 WordPress repo, and I'm unsure who has access to add the necessary token
 there.

 Just to publicly document this, @joemcgill and I discussed this within a
 Slack DM and he shared the repository secret that needed to be set. I've
 set that and workflows should now be able to access the
 `CODEVITALS_PROJECT_TOKEN` secret for reporting results on `wordpress-
 develop`.

 One final thing that came up through some discussion on the PR I think is
 worth documenting here is a decision was made to target the current
 version of WordPress being built in `trunk` (WP 6.2  once branched) and
 future branches for running this testing, similar to how PHP Compatibility
 testing was introduced. A separate discussion would need to be had
 weighing level of effort, maintenance burden (any required changes to the
 workflow will need to be backported as necessary), and whether it's worth
 merging this to any branches only receiving security updates (and
 technically are unsupported). I think there could also be an argument to
 stop running performance testing on unsupported branches as new ones are
 released. But that's also a separate discussion for future us.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/57687#comment:9>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform