Client Side Media iteration for WordPress 7.2 and beyond

[adamsilverstein]

Top Tickets for WordPress 7.2

• Enable client-side media in the Media Library:
  • Enable client-side media uploads in the Media Library  wordpress-develop#12585
• Enable client-side media in the Media Library modal (in Gutenberg):
  • Media: Handle editor's media modal uploads with client-side pipeline #82473

Nice to have in 7.2

• Client-side media: process image edits (crop, rotate, flip) with wasm-vips instead of the server /edit endpoint #82358
• Local poster generation during video upload #79366 - Generate the poster image locally during video upload.
• Add support for live photo / burst mode (aka HEIC/HEIF image sequence) #79642 - Convert HEIC/HEIF image sequences (Live Photos, bursts) to short video images.
• Client-side media processing: add an opt-in filter to generate animated GIF sub-sizes #80383 - Add a filter to opt-in to generating animated GIF sub-sizes.

Bug fixes/refinements for 7.2

• Media: EXIF and IPTC metadata is lost when HEIC images are converted by client-side media #82033
• External image sideload skips parts of the REST attachment creation path #82034

Goals for WordPress 7.3 or later

New Capabilities

• Optimize previously uploaded media #79364 - Optimize previously uploaded media.
• Add a before/after comparison with a quality slider when optimizing media #82036
• Video transcoding to web-safe formats #79363

Infrastructure

• Resume uploads after a crash or reload (durable upload queue) #79365 - Make the upload queue durable so it resumes where it left off after a reload.
• Improve architecture of client-side media upload finalize #82613

Image format support

• Add JPEG XL (JXL) support #76981 - Add JPEG XL support when browsers fully support it.
• Video transcoding to web-safe formats #79363 - Transcode video to web-safe formats in the browser.

What shipped during the 7.1 development cycle

The 7.1 cycle closed out most of the goals from #76756, including:

• HEIC image support with canvas fallback, so iPhone photos are transcoded in the browser before upload (Media: Add HEIC canvas fallback for client-side processing #76732)
• Ultra HDR (ISO 21496-1) gain map preservation through the resize and encode pipeline (Add UltraHDR isupport #74874)
• Upload error handling and retry logic (Add comprehensive error handling  #74366)
• Upload progress tracking and progress UI snackbars (Track progress during media uploads #74362, Images: Implement upload progress UI #74363)
• Concurrent uploading of images and sub-size sideloads (Investigate concurrent uploading of images #75257)
• Performance metrics for client-side media processing (Add performance metrics for client-side media processing #76789)
• Playground compatibility fixes (Client Side Media does not work in Playground #75941)
• image_editor_output_format filtering moved to the upload response (Move image_editor_output_format data to the full sized image upload response #75784)
• Sideload REST API endpoint requirements completed (Ensure REST API endpoint for side-loaded media covers all requirements #74360)
• Crash and console error fixes during image upload processing (Investigate and fix crashes and console errors during client-side image upload processing #76706)
• e2e test coverage for client side media (Ensure complete e2e testing for client side media #74367)
• Animated GIF to video conversion (Add animated GIF to web video conversion #76942)

Future plans

Down the road, the roadmap also includes audio transcoding, image editing, BlurHash placeholders, per-size format optimization, and saliency-based cropping.
If there is something you would like to see on the client side media roadmap, please leave a comment.

History

Previous tracking issues:

• Client Side Media iteration for WordPress 7.1: stabilization, docs, and Core backports #76756
• Client Side Media iteration for WordPress 7.0 #74333