Video optimization for web

Web users love videos (high-quality videos). In fact, 4x as many customers would rather watch a video about a product than read about it. But adding videos to your website can cause a lot of performance problems, and it needs to be optimized to handle them.

Video optimization refers to a set of technologies used to improve consumer viewing experience improving video quality and reducing video start times or re-buffering events, and the amount of network bandwidth consumed. Taking special consideration to play videos on the wide variety of mobile devices and networks.

Video formats

Video optimization starts with the video format. In fact, MP4 and WebM have become the standard formats for platforms like YouTube and Vimeo. This formats pack high-quality videos into small file sizes. Moreover, HTML5 can now deliver video content without the aid of a plugin, using this two formats (MP4 and WebM).

It is important to know that MP4 and WebM files are containers. This containers may contain one or more streams and use different codecs. This distintion between a container and a codec is important as files with the same container can have their contents encoded with different codecs.

Video codecs

Nowadays, there are three main options for video encoding:

  • H264 (also known as AVC): Packaged in a MP4 file is the universally supported standard for web and mobile devices.

  • VP9: Supported by Chrome, Firefox and Android is the current open source codec used in WebM files. As an advantage, WebM VP9 encoded videos tend to be lighter than H264 encoded files.

  • H265 (also known as HEVC): Packaged as a MP4 file it is about 30 percent more efficient than the H264 (lighter files with the same visual quality). Currently H265 works only on IE, Edge, Safari, and iOS Safari 11 and up.

Extension Video Codec Audio Codec
mp4 H264 aac
webm VP9 vorbis
webm VP9 opus
mp4 H265 aac

Modern codecs, such as HEVC (H265) and VP9, render sharing pixels across frames more efficient, encoding the key frames while maintaining the visual fidelity of videos. In fact, they are about 30-50 percent more efficient than the H264 standard. However, HEVC and VP9 do not enjoy universal browser or device support, so these options must be provided with a fall back H264 version.

Video optimization

To convert your videos to one of this formats, one of the best tools available is HandBrake. This open source application provides good presets to reduce video files without degrading image quality.

hand brake capture transcoding a video file

For instance, we can optimize the Sintel trailer just using the default options to produce a high quality MP4 file ready for the web, reducing the original size from 28.2MB to 19.2MB. (Note that the file is saved by HandBrake with the m4v extension, but this is an standard mp4 file renamed)

This generated file can be directly delivered from your server, as a progressive video using the HTML5 video tag standard for playback in the user's browser.

<video width="100%" controls>
  <source src="Sintel_Trailer.1080p.DivX_Plus_HD.m4v" type="video/mp4">
</video>

The HTML5 video tag also supports multiple sources to allow the browser to choose from. The browser then starts at the top of the list (H265) and goes down (VP9) until it finds a version it can play. This way, you can get optimal cross-browser support for video compression.

Check for H265 support first and then VP9 support. If neither of them works, fall back to the H264 manifest.

<video width="100%" controls>
  <source src="Sintel_Trailer.1080p.DivX_Plus_HD.mp4" type="video/mp4; codecs=hevc">
  <source src="Sintel_Trailer.1080p.DivX_Plus_HD.webm" type="video/webm; codecs=vp9">
  <source src="Sintel_Trailer.1080p.DivX_Plus_HD.m4v" type="video/mp4">
</video>

This approach can be great for short-form videos (under 20 seconds) and for videos that are displayed at a low resolution (such as ads and previews). But since your videos are delivered as a single file, this video optimization is not enough to provide an optimal experience to all users, especially in the case of mobile users.

For these users, videos must follow the principles of responsive web design to make your website adaptable to different screens sizes and display resolutions. Hence, sending down a video too large for the screen is not only a waste of resources and increased bandwidth costs on your part, but also a waste of your users' bandwidth.

In that case, the solution appears as Adaptive Bitrate Streaming to deliver a lightweight and optimal experience to all users.

Adaptive Bitrate streaming

Adaptive Bit-Rate (ABR) technology solves some of the challenges with video streaming. These techniques use multiple versions of the video available at different resolutions and bitrates to provide the quality / bitrate that works best for every user, considering their network bandwidth (like 3G or LTE 4G) and visualization resolution.

ABR takes into account the network conditions and the available resolution of the viewer to choose the best video version to be displayed every time. It switches dynamically to a lower-bitrate (lower-quality) version of the video when encountering slow network conditions.

Although ABR are excellent methods for packaging and delivering content streams that will perform well on a variety of networks and devices, they can be expensive to implement due to the equipment, storage capacity and expertise needed. In particular, ABR streaming requires that video streams be encoded at multiple bit rates, screen sizes and quality levels, a process that complicates content preparation and management workflows.

HLS and MPEG-DASH

Two main protocols have becoming standard for video web streaming:

  • HTTP Live Streaming (HLS) is an HTTP-based media streaming communications protocol implemented by Apple Inc. It is natively supported by Safari, OS X, and iOS software.

  • Dynamic Adaptive Streaming over HTTP (MPEG-DASH) is an HTTP-based media streaming protocol either. But is more recent than HLS, it is codec agnostic, and supports some additional characteristics.

Both protocols work by breaking the overall stream into a sequence of small HTTP-based file downloads, each download loading one short chunk of an overall potentially unbounded transport stream. As the stream is played, the client may select from a number of different alternate streams containing the same material encoded at a variety of data rates, allowing the streaming session to adapt to the available data rate.

At the start of the streaming session, the client software downloads a master playlist file containing the metadata for the various sub-streams that are available. The client software then decides what to download from the available media files, based on predefined factors such as device type, resolution, current bandwidth, size, etc.

This means to deliver videos using adaptive streaming, you need multiple copies of your video prepared at different resolutions, qualities, and data rates. These are called video representations (or variants).

Video representations

Bitrate and resolution are the properties that correlate to the amount of data in a media file. Selected a codec, the number of pixels (video resolution) and bitrate set the video quality.

Bitrate = total pixels/second * bytes/pixels

Thus, a 10-second video with a 10MB file size has a bitrate of 8 mbps (1 megabyte per second * 8 megabits per megabyte).

  • Bitrate is the measure of how many bits are used to encode one second of a stream. The more bits used to encode a second of stream, the higher the quality. For instance, the table bellow compares web bitrates with common home video formats. Unsurprisingly, bitrates for the web are low. Values are given in megabits per second (Mbs).
Format Bitrate
Blu-ray 20 Mbs
DVD 6 Mbs
Desktop web video 2 Mbs
4G mobile video 0.7 Mbs
3G mobile video 0.35 Mbs
  • Resolution is the amount of information in a single frame of video, given as the number of logical pixels in each dimension. For instance, the following list is common for 16:9 video resolutions.
Abbreviation Dimensions
2160p 3840x2160
1440p 2560x1440
1080p 1920x1080
720p 1280x720
480p 854x480
360p 640x360
240p 426x240

There is no universal best parameters to adjust video bitrates. They should be selected for every video content, testing the result and tweaking the bitrates to your requirements. Keep in mind that a given bitrate at a constant resolution would result in a video of higher visual quality in H265 and VP9 than in H264.