[wp-trac] [WordPress Trac] #34693: Add browser-based testing and automated acceptance tests (BrowserStack)

[WordPress Trac]

#34693: Add browser-based testing and automated acceptance tests (BrowserStack)
------------------------------+----------------------------
 Reporter:  westonruter       |      Owner:
     Type:  enhancement       |     Status:  new
 Priority:  normal            |  Milestone:  Future Release
Component:  Build/Test Tools  |    Version:
 Severity:  normal            |   Keywords:  needs-patch
  Focuses:                    |
------------------------------+----------------------------
 Automated testing in WordPress is primarily focused on PHPUnit testing,
 with some JavaScript QUnit tests. While the PHP tests are run against
 every version of PHP since 5.2, the JavaScript tests are limited to one
 single browser environment (PhantomJS). This has meant that browser-
 specific bugs have been introduced (e.g. #30781) because manual testing
 isn't always performed in all the browsers that WordPress supports. We
 should connect WordPress unit tests with something like BrowserStack to
 run the QUnit tests in the various browsers.

 While we're at it, the unit/integration testing in PHPUnit and QUnit only
 take us so far in WordPress as it is. What WordPress is missing are
 acceptance tests that fire up an actual browser to do actual tasks that
 users would perform when using WordPress. Such acceptance tests would make
 WordPress much more resilient against regressions, and it would open up a
 lot of new areas for automated testing that are currently very difficult
 to test using unit/integration tests. For instance, the Customizer heavily
 uses data from both PHP and JavaScript along with Ajax requests between
 the two. To create unit/integration testing mocks for PHP, JS, and Ajax
 requests adds a lot of testing overhead and is also difficult to maintain.

 I believe acceptance testing would allow much more of WordPress to be
 tested automatically, and it would allow tests to be written by more
 people.

 See also https://wordpress.slack.com/archives/core/p1419407324003594

--
Ticket URL: <https://core.trac.wordpress.org/ticket/34693>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform