[wp-trac] [WordPress Trac] #55443: Create WebP sub-sizes and use for output

Tue Mar 22 23:15:33 UTC 2022

#55443: Create WebP sub-sizes and use for output
-------------------------+-------------------------------------------------
 Reporter:               |      Owner:  adamsilverstein
  adamsilverstein        |
     Type:  enhancement  |     Status:  assigned
 Priority:  normal       |  Milestone:  6.0
Component:  Media        |    Version:  trunk
 Severity:  normal       |   Keywords:  has-patch has-unit-tests needs-
  Focuses:  performance  |  testing needs-dev-note
-------------------------+-------------------------------------------------
 === Summary
 This ticket introduces core support for generating and using additional
 mime types for sub-sized image uploads, with a focus on the WebP format.
 In addition, it filters frontend content to use the modern format when
 available for an image. After this change, WordPress will generate and use
 WebP sub sizes by default when available.

 Currently, when users upload images in a supported mime type like JPEG or
 WebP, WordPress creates sub-sized images of the same type for use on the
 front end (WordPress does not currently create sub-sized images for TIFF,
 GIF or PNG images). In WordPress 5.8 we introduced the
 `image_editor_output_format` filter to enable filtering the mime type used
 to save images.

 This ticket introduces the capability to generate more than a single mime
 type. For example, when users upload JPEG images, WordPress can
 automatically generate both WebP and JPEG sub-sized images. Then WordPress
 can use the WebP images when serving up the front end. JPEG images are
 still being generated by default, to support other use-cases outside of
 common web browsers (e.g. email newsletters) where WebP may not be
 supported yet.

 The above is only the default though - the behavior of which image formats
 are generated and used can be customized via filters.

 === New Filters

 This patch introduces two new filters, one to control the mime types
 WordPress generates when you upload an image and one to control which
 image mime type is used for displaying images on the front end of the
 website.

 ==== Mime type image generation filter

 This patch adds a new filter named `wp_upload_image_mime_transforms` (and
 similarly named function) that maps the upload image mime type to an array
 of one or more output mime types:

 By default, the mapping is:
 {{{
         array(
                 'image/jpeg'        => array( 'image/jpeg', 'image/webp'
 ),
                 'image/webp'      => array( 'image/webp', 'image/jpeg' ),
         );

 }}}
 The attachment id is also passed to the filter for context.

 ==== Mime type front end output filter

 This patch adds a new filter `wp_content_image_mimes` for the front end
 that specifies the preferred format(s) to use for content output (as long
 as the image file is available). When outputting images in the content,
 the first mime type available will be used. The attachment id and context
 are also passed to the filter.

 === WebP as a default output format for images
 On the front end, sites will benefit from using WebP in place of JPEG for
 their images. Research on
 https://github.com/WordPress/performance/issues/7 indicates images will
 average around a 30% overall file size reduction. This work also includes
 verifying the visual image quality of the output using
 [https://github.com/kornelski/dssim dssim].

 === Developers Note
 The function `make_subsize`      in both of core’s bundled WP_Image_Editor
 implementations (WP_Image_Editor_Imagick and WP_Image_Editor_GD) has been
 updated to accept a `$mime_type` parameter which in turn passed the mime
 type to the `_save` function (which already supports a `$mime_type`
 parameter). If your code implements `WP_Image_Editor::make_subsize()`
 make sure you update your function to support the `$mime_type` parameter.
 Plugins that modify image use (CDN, S3) or already create alternate mime
 types should test integrating with the code on this ticket and consider
 supporting the new sources meta data.

 === How to Test
 Test with this patch or the Performance Lab [https://wordpress.org/plugins
 /performance-lab/ plugin]
 Create a post and upload a jpeg image, placing in the post
 Publish the post and view it’s source code, the image URLs should be
 “.webp” images

 === Technical Details
 * In src/wp-admin/includes/image.php::wp_create_image_subsizes:
 loop through all of the mime types returned from the
 `wp_upload_image_mime_transforms` filter, generating sub-sized images
 for every mime format supported by the system.  Default when a type isn’t
 mapped in the array  is outputting in the mime type of the uploaded image.
 The first type is the “primary” mime type and is stored exactly like
 current WordPress uploads (no change)
 “Full” sized images are created for each mime type supported by the system
 with a `-{mime extension}` appended to the file name
 When images over `big_image_size_threshold` are uploaded, the primary
 image gets a `-scaled` suffix (unchanged), alternate types get a `-{mime
 extension}-scaled` suffix
 Similarly, alternate mime rotated images get a  `-{mime
 extension}-rotated` suffix
 New image meta `sources` array stores each mime type created for each full
 sized image and for each sub-sized image - both the file name (`file`) and
 size (`filesize`) are stored.
 In src/wp-admin/includes/image.php::_wp_make_subsizes:
 Using a mapping (from the input file mime type) provided by
 `wp_upload_image_mime_transforms`, output sub sized images in one or more
 output mime types

 * In src/wp-admin/includes/image.php
 Add a new function `wp_upload_image_mime_transforms` as well as a matching
 filter named `wp_upload_image_mime_transforms` . Returns an array with the
 list of valid mime types that a specific mime type should be converted
 into.
 In src/wp-includes/class-wp-image-editor-gd.php and src/wp-includes/class-
 wp-image-editor-imagick.php:
 Update `make_subsize` to accept a new `mime_type` parameter that gets
 passed to the `_save` function when set (which already has a mime_type
 parameter). Add the new parameter to internal function calls as well.

 * In src/wp-includes/media.php: Adds  new function
 `wp_image_use_alternate_mime_types` that is called for every image in
 `wp_filter_content_tags`. This function:
 Adds a new filter `wp_content_image_mimes` that returns the mime type(s)
 to try to use for output, in the order they should be tried. The default
 order to try is array( 'image/webp', 'image/jpeg' ), meaning if a WebP
 version of the image exists, WordPress will use that image for the image
 tag output
 Replaces all image components (including urls in the `src` and `srcset`
 attributes) when the same image is available in the preferred mime type
 Returning an empty array will skip any image mime type substitution ( ie.
 `add_filter( ‘wp_content_image_mimes’, ‘__return_empty_array’ );`

 * In src/wp-includes/post.php::wp_delete_attachment_files:
 Delete additional mime type images referenced in the image meta root
 ‘sources’ array as well as each sizes->sources array when present. This
 ensures that additional mime images are cleaned up if the original image
 is deleted.

 * In src/js/_enqueues/vendor/plupload/wp-plupload.js and
 src/js/_enqueues/vendor/plupload/handlers.js retry count is raised from 4
 to 8. Should this be filterable? Note: needs to be changed in Gutenberg as
 well, see
 https://github.com/WordPress/gutenberg/blob/23f11a18726bf69717595451ab3250cb67b426f6/packages
 /api-fetch/src/middlewares/media-upload.js#L38

 === PHPUnit test changes
 New sources data added to expected results in `
 test_wp_generate_attachment_metadata_pdf`, `test_crop_setting_for_pdf`
 Remove new `wp_content_image_mimes` filter for tests in
 tests/phpunit/tests/media.php

 === Todo items remain for this patch:
 - Add tests for new filters `wp_content_image_mimes` and
 `wp_upload_image_mime_transforms`
 - Add full sized alternate mime creation for PDF upload special handling,
 see https://github.com/WordPress/wordpress-
 develop/pull/2393/files#r832651580
 - Add REST support from https://github.com/WordPress/performance/pull/224
 (or in follow up ticket)
 - Improve naming of rotated/cropped alternate mime images

-- 
Ticket URL: <https://core.trac.wordpress.org/ticket/55443>
WordPress Trac <https://core.trac.wordpress.org/>
WordPress publishing platform