[wp-trac] [WordPress Trac] #47622: The E2E test has a wrong assertion

[WordPress Trac]

#47622: The E2E test has a wrong assertion
------------------------------+-----------------------------
 Reporter:  hideokamoto       |      Owner:  (none)
     Type:  defect (bug)      |     Status:  new
 Priority:  normal            |  Milestone:  Awaiting Review
Component:  Build/Test Tools  |    Version:
 Severity:  normal            |   Keywords:
  Focuses:  javascript        |
------------------------------+-----------------------------
 The E2E test succeeds even if we enter whatever value
 https://github.com/WordPress/wordpress-
 develop/blob/master/tests/e2e/specs/hello.test.js#L4-L10


 **Expect behavior**

 The test should fail when we put invalid text condition.
 And I tried to run the following test code, the E2E test has been passed.

 **Code**
 {{{
 const { visitAdminPage } = require('@wordpress/e2e-test-utils');

 describe( 'Hello World', () => {
         it( 'Should load properly', async () => {
                 await visitAdminPage( '/' );
                 const title = await page.$x(
                         '//h2[contains(text(), "aaaaaaaaaaa")]'
                 );
                 expect( title ).not.toBeNull();
         } );
 } );
 }}}


 **Result**
 {{{
 > wp-scripts test-e2e --config ./jest.config.js

  PASS  __tests__/specs/index.test.js
   Hello World
     ✓ Should load properly (1645ms)

 Test Suites: 1 passed, 1 total
 Tests:       1 passed, 1 total
 Snapshots:   0 total
 Time:        3.169s, estimated 4s
 }}}

 ** Why?**
 I think the page.$x() methods return an array.
 So the `title` variable should not be null.

 ** Debug code / result**
 {{{
 const { visitAdminPage } = require('@wordpress/e2e-test-utils');

 describe( 'Hello World', () => {
         it( 'Should load properly', async () => {
                 await visitAdminPage( '/' );
                 const title = await page.$x(
                         '//h2[contains(text(), "aaaaaaaaaaa")]'
                 );
                 expect( title ).not.toBeNull();
         } );
 } );
 }}}

 {{{
 > wp-scripts test-e2e --config ./jest.config.js

  PASS  __tests__/specs/index.test.js
   Hello World
     ✓ Should load properly (1645ms)

   console.log __tests__/specs/index.test.js:9
     []

 Test Suites: 1 passed, 1 total
 Tests:       1 passed, 1 total
 Snapshots:   0 total
 Time:        3.169s, estimated 4s
 Ran all test suites.
 }}}

 ** Possible  Solution**

 We should change the assertion.
 The page.$x methods return an array, so we need to check the array length.

 ->  Should fail the test
 {{{
 const { visitAdminPage } = require('@wordpress/e2e-test-utils');

 describe( 'Hello World', () => {
         it( 'Should load properly', async () => {
                 await visitAdminPage( '/' );
                 const nodes = await page.$x(
                         '//h2[contains(text(), "aaaaaaaaaaa")]'
                 );
                 expect( nodes.length ).not.toEqual( 0 )
         } );
 } );
 }}}

 -> Should pass the test
 {{{
 const { visitAdminPage } = require('@wordpress/e2e-test-utils');

 describe( 'Hello World', () => {
         it( 'Should load properly', async () => {
                 await visitAdminPage( '/' );
                 const nod[]es = await page.$x(
                         '//h2[contains(text(), "Welcome to WordPress!")]'
                 );
                 expect( nodes.length ).not.toEqual( 0 )
         } );
 } );
 }}}

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/47622>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform