Beyond Images: Unleash Media Magic in WP with WebAssembly

This experimental WordPress plugin aims to bring improved media capabilities to WordPress and specifically the block editor, all powered by new web platform features like WebAssembly.


Reminder: this is an experimental WordPress plugin. While you can totally install it on your website, it is recommended to try it on a development environment first.

You can download the latest nightly build directly from GitHub. Afterwards, you should be able to use a plugin like Git Updater to receive updates. Alternatively, simply download the nightly build again every now and then.


Run npm install and npm run build to build all the JavaScript and CSS.


A non-exclusive list of things that have been implemented so far:

  • Local poster generation during video upload
  • Preview image generation for PDFs
  • Converting GIFs to videos
  • Video transcoding and compression (e.g. MOV to MP4)
  • Image transcoding and compression (e.g. HEIF to WebP)
  • Audio transcoding (e.g. OGG to MP3)
  • Video audio track detection and muting
  • Optimization of existing media with a comparison screen
  • Self-recording using webcam / microphone
  • Import external media to library
  • Client-side image downsizing and thumbnail generation
  • Preferences dialog to change default settings

These features can be tested either by dropping media files into the editor or by editing existing media blocks in the editor.

Note: not all of these have been fully implemented yet.

Automatic Poster Generation

Poster images are important for UX as they are shown before a video is fully loaded and started playing. During upload, a poster image (thumbnail) of the video is automatically taken and added to the video in both the editor and the media library.

Preview image generation for PDFs

In addition to that, poster images are automatically generated for PDFs as well (powered by PDF.js).

Converting GIFs to Videos

Videos are the better GIFs, at least when it comes to file types. Animated GIFs should really not be used anymore.

This project automatically converts animated GIFs to an actual video file during upload. The image block, upon receiving the updated media item, transforms into a video block. Note: Right now, that last part (block transformation) is achieved by patching Gutenberg.

HEIF Image Transcoding

Uses libheif-js to automatically convert HEIC images (typically taken on iPhone) to a more common format (WebP or JPEG).

Media Optimization and Transcoding using FFmpeg

This project uses ffmpeg.wasm to unlock video/image/audio transcoding and compression during upload. For example from MOV to MP4, HEIF to WebP, or OGG to MP3.

Video Audio Channel Detection

During upload, detect whether a video actually has audio. If not, hide/disable “Muted” option in the editor.

Note: Right now, this is achieved by patching Gutenberg.

Video Muting

This allows completely removing the audio channel from a video, so it’s actually muted.

During upload, it’s automatically detected whether the video actually has any audio channels. If not, it’s marked as muted.

(Improved) Import of External Media

While the image block in Gutenberg already supports importing external images into the site’s own media library, other blocks don’t. This project changes this, while also hooking the import into the same upload logic so that imported files automatically get compressed and transcoded if needed.

Self Recording

Adds self-recording capabilities to the video/image/audio blocks in Gutenberg so that one can record videos, GIFs (which are basically muted looped videos), audio transcripts, or simply take a still picture using your camera and/or microphone.

This also supports automatically blurring backgrounds.

Improved Media Placeholders

Automatically extracts the dominant color of an image and generates a BlurHash during upload. This can be used to improve perceived performance by using them as placeholders during loading — both in the editor and on the frontend.

Whenever available, the BlurHash is used to generate a blurred placeholder image using just CSS, with a fallback to just a solid color.

Caveat: no placeholder is displayed if the image has alpha transparency, as that would lead to undesired results.

This is more advanced than the Dominant Color Images feature plugin, which does not generate blurred placeholders.

Block sidebar controls showing BlurHash and dominant color of a video Blurred placeholder vs actual image

Optimize Existing Media

Allows optimizing existing media items in a post by converting them to a more modern format. Either individually or in bulk. After conversion, the old and new versions can optionally be compared with a slider. The file size savings in % are shown next to it.

Single optimization with approval/comparison step:

Bulk optimization without the approval step:

Client-Side Image Downsizing

When a new image is uploaded, WordPress will detect if it is a “big” image by checking whether it exceeds a certain threshold (currently 2560px width). Images exceeding the threshold will be scaled down on the server to prevent users from using unnecessarily large images in their content. However, the original file is still retained just in case.

While the server-side downsizing is admirable, it’s still wasteful to upload the full original image. Doing this client-side improves upload time and reduces bandwidth consumption.

Client-Side Thumbnail Generation

When uploading an image to WordPress, it generates many thumbnail versions of it. This often leads to very slow upload processes, to the point where WordPress had to build a workaround to resume failed attempts. Besides the performance concern, the thumbnails would also not benefit from the same client-side image optimizations.

To address this, the thumbnail generation is done entirely client-side as well. Less server stress, less failed uploads, and thumbnails with the same traits as the original.

Saliency Detection

Detect the most relevant part of an image to improve thumbnail cropping. For example using MediaPipe’s object detection solution.

MozJPEG Support

Oftentimes, the MozJPEG encoder provides better or equal results than WebP. It’s appealing to be able to generate lightweight images without the UX downsides of WebP. MozJPEG is typically not available in the server-side image libraries (GD and Imagick) used by WordPress.

AVIF support

Same as above, useful for comparison.

Preferences / Settings

Adds a preferences modal to the block editor where users can choose their preferred image formats (e.g. MozJPEG, WebP) and quality levels.

The default camera and microphone to use for media recording can be set here as well.

Media preferences modal in the block editor

Regenerate Media Library

Ability to convert all existing images in the media library client-side. Ideally without having to stay on the page.

Generate video captions for videos

Use AI to automatically generate video captions for videos. Nice accessibility and UX enhancement.

Alt Text Generation

Use AI to automatically generate alt text for images and videos to improve accessibility.

Upload Manager

Show all in-progress items for easier inspection, and allow cancelling individual uploads.

Upload from another device

Allows uploading media to the current post from an entirely different device, such as a mobile phone, even when not logged in.

Download Details:

Author: swissspidy

Official Github: 

License: GPL-2.0 license


Beyond Images: Unleash Media Magic in WP with WebAssembly
1.65 GEEK