[wp-trac] [WordPress Trac] #53737: Create a way to autogenerate end-to-end test code from manual testing steps for WordPress core

[WordPress Trac]

#53737: Create a way to autogenerate end-to-end test code from manual testing steps
for WordPress core
------------------------------+------------------------------
 Reporter:  psykro            |       Owner:  (none)
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:  Awaiting Review
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:                    |     Focuses:
------------------------------+------------------------------

Comment (by youknowriad):

 While this sounds great on principle, I'd like to share some concerns
 about this: e2e tests are tests that require constant monitoring and
 attention, meaning any subtle change in something that is potentially
 unrelated can result in flaky or failing tests. Debugging this flaky tests
 take time and the reason for these failures is very different from failure
 to another. A lot of e2e tests will most likely depend on testing plugins
 and themes, which need to be coded and can't be generated.

 This means, that there's no reliable way to generate tests and e2e tests
 need to be crafted carefully and follow some guidelines to reduce the risk
 for flakiness.

 I guess what I'm trying to say is that this is jumping steps, initially,
 we should focus on crafting a good set of e2e tests manually, ensure these
 tests are stable enough, learn from them.

 I think having a way to generate tests might be useful for developers to
 "bootstrap" their tests but I don't think we'll be able to get something
 working without any manual change. I think the value of auto-generated e2e
 tests is largely overrated.

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/53737#comment:2>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform