[wp-trac] [WordPress Trac] #41641: Unit testing reaching the memory limit

[WordPress Trac]

#41641: Unit testing reaching the memory limit
------------------------------+-----------------------------
 Reporter:  Hristo Sg         |       Owner:
     Type:  enhancement       |      Status:  new
 Priority:  normal            |   Milestone:  Future Release
Component:  Build/Test Tools  |     Version:
 Severity:  normal            |  Resolution:
 Keywords:                    |     Focuses:
------------------------------+-----------------------------
Changes (by danielbachhuber):

 * milestone:  Awaiting Review => Future Release


Comment:

 I spent some time profiling and researching this morning.

 Running the test suite locally, here's observed memory usage:

 '''75% test suite completion == 2 GB resident memory'''

 [[Image(https://user-
 images.githubusercontent.com/36432/36432869-1336403e-1610-11e8-844d-
 1bcc66ab9db9.png)]]

 '''99% test suite completion == 3.1 GB resident memory'''

 [[Image(https://user-
 images.githubusercontent.com/36432/36432929-37c4cb5a-1610-11e8-8c4b-
 c7141f2b5cf6.png)]]

 Generally, memory usage scaled in a relatively linear way throughout the
 test suite. However, the
 [https://blackfire.io/profiles/ab3145d8-f75f-4272-99d9-337291f353a9/graph
 Blackfire report for the same test run] only reports 505 MB memory usage.

 [https://stackoverflow.com/questions/38640461/how-much-memory-is-php-
 actually-using From StackOverflow], I learned:

 > Regarding `memory_get_usage()`, this reports the heap memory actually
 allocated by systems using PHP's internal memory manager. This means,
 libraries which directly use other memory managers of the system
 (`mmap(2)` or `malloc(3)`) do not expose their memory usage here. [This is
 for example why mysqlnd does show much memory usage while libmysqlclient
 does not - the latter uses `malloc()` internally.]

 My guess at this point is that some PHP extension (ImageMagick?) is
 holding on to memory as the test suite progresses.

 The REST API tests are definitely a source of slowness. In particular,
 `WP_Test_REST_Attachments_Controller` calls `copy()` twice before each
 test, and there's a lot of image resizing going on.

 I also rediscovered how slow `remove_added_uploads()` is when you already
 have files in your uploads directory.

 Not quite sure of next steps yet. Open to suggestions.

--
Ticket URL: <https://core.trac.wordpress.org/ticket/41641#comment:7>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform