When talking about media file types, we’re really talking about two separate components: the software used to encode and decode the audio or video stream (the codec, which is short for compressor-decompressor or coder-decoder), and the container, which is a wrapper format that contains the media streams and information about how the data and metadata co-exist. An example of a container is the open source Ogg (from Xiph.Org), while an example of a codec is VP8, a lossy video compression format from On2 (and Google). Technically, a codec could be used with many different containers, and containers could wrap many different codecs, but we tend to think of pairs of container/codecs when talking about browser support.
Audio files are containers wrapping one type of media data, the audio stream, but video files typically wrap two different media streams: the video and the audio data streams. In addition, containers can also support subtitles and captions, as well as the information to keep all data tracks in sync.
Note
Though you can embed subtitles directly into the file with some containers, HTML5 video provides a means of incorporating external subtitle files. More on incorporating subtitles and other accessible features in Chapter .
Just like with image containers, such as JPEG and PNG, audio and video codecs can either be lossless or lossy. A lossless video or audio codec preserves all of the original media file’s data when it’s compressed. Lossy compression techniques, however, lose data each time the data is encoded.
Though most of us have the bandwidth to download lossless images
such as PNGs, lossless video is beyond even the most generous of
broadband capacity, so the only codecs supported for HTML5 video are
lossy codecs. Audio, however, is different. The audio element supports uncompressed audio
files, as well as audio files with both lossless and lossy
codecs.
One of the older and more familiar audio file formats is the Waveform Audio File Format (WAVE), commonly known as WAV for the extension the audio files are given (.wav). Though WAV files can support compression, most WAV files contain audio in an uncompressed Pulse-Code Modulation (PCM) representation, which means the files tend to be quite large.
Safari, Chrome, Firefox, and Opera support uncompressed WAV files. However, the size of the WAV files preclude their being a popular HTML5 audio file format.
Another well known and common audio file format is the MPEG-1 Audio Layer 3, commonly known as MP3 because of the extension given MP3 files (.mp3). It is neither a container or codec, as we know these things. Instead, it’s an all-in-one lossy compressed audio file with metadata strategically inserted.
At this time, the only audio format that Microsoft supports in IE9 and up, by default, is MP3. In addition, the format is also supported by Safari and Chrome. However, Firefox and Opera refused to support MP3s right from the start, because of patent issues and royalty requirements.
MP3 is supported in most operating system environments, and MP3 files are a popular fallback when linked into the page. Though the file won’t play natively in the browser, clicking the link will trigger some media player in most environments:
<audio id="background" autoplay loop>
<source src="audiofile.mp3" type="audio/mpeg" />
<source src="audiofile.ogg" type="audio/ogg" />
<p><a href="audiofile.mp3">Your audio file fallback</a></p>
</audio>Note
Safari requires the installation of QuickTime and supports whatever media types QuickTime natively supports in the system. Since QuickTime supports MP3 and WAV, Safari supports MP3 and WAV.
When the media elements were first added to HTML5, the specification included a requirement that all user agents support the Ogg open source container. The Ogg container was developed by the Xiph.Org foundation, which also developed an associated audio codec, called Vorbis. The Vorbis codec is a lossy compression technique that is free for everyone to use and is, according to the folks at Xiph.Org, free of patents (to the best of their determination). The hope at the time the media elements were first defined was that this tower of babble that we have for audio and video could be avoided by ensuring support for one container and one codec, neither of which are encumbered by patents or royalty requirements.
Note
Find out more about the Ogg Vorbis container/codec at the official support site at http://www.vorbis.com/.
Apple and other companies, though, objected to the Ogg Vorbis requirement because of lack of hardware support, their belief that the Vorbis codec was inferior to other codecs, and concerns of potentially hidden patents (known as submarine patents) related to the codec.
Though the Xiph.Org foundation has done their best to search among patents to ensure Vorbis is patent free, there’s no way to guarantee that unless it is challenged in a court of law. It’s a catch-22 situation without any viable solution, so the section in the specification that required support for Ogg Vorbis was removed.
Note
For an interesting historical perspective, the email from Ian Hickson, HTML5 editor, about dropping support for both Ogg Vorbis and Ogg Theara can be found online at http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2009-June/020620.html.
Though Ogg Vorbis is no longer a requirement, several browsers do support it. Firefox, Opera, and Chrome support Ogg Vorbis, while Safari and IE do not.
The Advanced Audio Coding (AAC) lossy compression codec was originally considered to be a successor to MP3, though it didn’t get broad acceptance. It languished, little known, until Apple picked it as the format for the files in its iTunes store. The container it’s most used with is the MPEG-4 Part 14 container, known as MP4 for the .mp4 file extension. Though most of us assume that MP4 files are video, they can be audio only. In fact, another common file extension used with MP4 audio files is .m4a, again primarily because of Apple’s influence. Safari, Chrome, and IE support MPEG-4 AAC.
WebM is a container based on the profile for the Matroska Multimedia Container. WebM was designed from the beginning to be patent and royalty free. Google was instrumental in forming the organization behind WebM, but has given up any and all patent claims to the container.
Codec support in WebM is quite simple: WebM only supports Vorbis for audio, and VP8 for video (which I’ll cover in the next section). The reasons for such simple codec support are given in a FAQ at the WebM web site:
We decided to define WebM files in this way because we wanted to do what’s best for users. Users just want video to work, they don’t want to worry about supported codecs, file formats, and so on. After much discussion with browser makers, tool developers and others, we reached a consensus that a narrowly defined format would cause the least confusion for users. If a user has a .webm file, he or she can be confident that it will play in any browser or media player that supports WebM.
WebM is supported by Chrome, Firefox, and Opera. It is not currently supported by IE and Safari. However, people can ensure that WebM files work in their IE9 browser by installing the WebM plug-in for IE9 (found at http://tools.google.com/dlpage/webmmf). However, since we as page authors, designers, and developers can’t be sure that the WebM plug-in is installed, we have to provide support for browsers that currently don’t support WebM.
Note
People typically think that WebM is solely a video file
format. However, you can create a WebM file that consists of only
one Vorbis data stream, and it works in an audio element. The
source element’s type setting is
audio/webm. Find out more about WebM at
the project website, at http://www.webmproject.org/.
I’ve covered the popular audio and video file types, but how do browsers know if an MP4 is an audio file, or a video file? Of if this file is an Ogg, and that is a WebM? Well, they can open the file and see for themselves. Or we can provide the information directly in the media element.
Earlier, I stated that an MP4 file can be audio or video. So how does the browser or other application know which type of file it is? Or what codec is being used in the MP4 container? In fact, what codec is used in any of the containers?
One approach is to use a popular and unique file extension, such as.m4a, and then add a MIME type to your web server for the extension. You can add the MIME type directly to the mime.types file for an Apache server, or you can add a MIME type to the directory’s .htaccess file (assuming you’re running Linux):
AddType audio/mp4 m4a AddType video/ogg ogg oga AddType video/webm webm
You should also use the type
attribute in the source element. The type attribute provides information to the
browsers and other user agents about the container and codec, as well
as type of file listed in the src
attribute. The syntax for the type
attribute is the type of file, followed by the type of container. In
Example 1-4, the
container and media file type for each audio file is added to each of
four source elements.
Example 1-4. HTML5 web page with embedded audio element with four separate audio types, each with their specific MIME type provided in the source element’s type attribute
<!DOCTYPE html> <head> <title>Audio</title> <meta charset="utf-8" /> </head> <body> <audio controls> <source src="audiofile.mp3" type="audio/mpeg" /> <source src="audiofile.ogg" type="audio/ogg" /> <source src="audiofile.wav" type="audio/wav" /> </audio> </body>
Of course the last file, the WAV, never gets played, at least not with our target browsers. IE, Chrome, and Safari pick up the MP3, while Firefox and Opera pick up the Ogg file. Each browser traverses the source elements until if finds a file it can play, and then stops. Minimally, you can provide an MP3 and an Ogg or WebM audio file, which covers all five target browsers, in addition to iOS and Android.
Table 1-2
contains a summary of the different audio codecs and containers
covered, as well as modern browser support, common file extension(s),
and type setting.
Table 1-2. Audio container/codec support across popular modern browser versions
Container/Codec | Type | Extension(s) | IE9+ | Firefox | Safari 5+ | Chrome | Opera 11+ |
|---|---|---|---|---|---|---|---|
WAV | audio/wav or audio/wave | .wav | No | Yes | Yes | Yes | Yes |
MP3 | audio/mpeg | .mp3 | Yes | No | Yes | Yes | No |
Ogg Vorbis | audio/ogg | .ogg, .oga | No | Yes | No | Yes | Yes |
MPEG-4 AAC | audio/mp4 | .m4a | Yes | No | Yes | Yes | No |
WebM Vorbis | audio/webm | .webm | No | Yes | No | Yes | Yes |
As I mentioned in the last section, video files are far too large to serve up in anything other than a lossy compressed format. As with audio codecs, no one video codec works in all browsers.
One of the most popular lossy video codecs is MPEG-4 Part 10, commonly known as H.264. H.264 is a high quality, popular format that’s common on the Internet and supported in YouTube and iTunes. It is also one of the three mandatory codecs supported by Blu-Ray players. It’s a mature codec, first standardized by the MPEG group in 2003. H.264 is also a controversial choice because of the patents held on the codec by the organization, MPEG-LA. Though video files encoded in H.264 that are distributed without cost aren’t subject to royalties, tools that encode or decode H.264 do have to pay royalties. The cost for these royalties is usually passed on to the tool buyer.
The H.264 video codec is combined with either the AAC or MP3 audio codec in an MPEG-4 container. This combination is typically known as MP4, and files are usually given an .mp4 extension . You’ll also see files with .m4v extensions for H.264. Apple iTunes uses the .m4v extension with its videos, but they’re also encumbered by DRM and won’t play in HTML5 video elements.
The H.264 codec is the only video codec that Microsoft supports for IE. It’s also supported by Safari. Chrome has dropped support for H.264, and Firefox and Opera have never supported it because of the patent issues.
Firefox, Opera, and Chrome do support another codec, Theora, from the same organization (Xiph.Org) that provided the Ogg container and Vorbis audio codec described in the last section. The Ogg Theora container/codec was originally the mandatory codec and container for video elements in HTML5 until Apple and other companies objected to the restriction. Neither IE nor Safari support Ogg Theora, though there are plug-ins that can be installed to provide support in both browsers.
Note
Xiph.Org provides a plug-in that enables support for the Theora and Vorbis codecs in QuickTime, which indirectly enables support for Safari. Access the plug-in at http://www.xiph.org/quicktime/about.html. Another plug-in, OpenCodecs, provides more generalized support for Ogg Vorbis, Ogg Theora, WebM, and various other Ogg container/codec pairings, and can be accessed at http://xiph.org/dshow/downloads/.
The last video container I’ll cover is WebM, which I introduced in the section on audio codecs. Unlike many of the other containers, WebM supports only one audio codec, Vorbis, and one video codec, VP8. VP8 was created by a company named On2, which was later bought by Google—who promptly open-sourced the VP8 codec.
WebM is supported by Chrome, Firefox, and Opera. There is no built-in support for WebM in Safari and IE, but, as mentioned earlier, there is plug-in support for WebM for both browsers.
Since Chrome, Firefox, and Opera support both Ogg Theora and WebM, which should you use? The answer is: it depends.
Both should continue to be supported for the foreseeable future. The Open Source community, including Wikipedia, still primarily support Ogg Theora, but since Google open-sourced VP8, this may change in the future. VP8 is generally considered a better codec than Theora, but I’ve never seen much difference in quality when it comes to videos sized and optimized for the web. But then, I’m not a picky videophile, either.
You can’t assume your web page readers have plug-ins installed
to play Ogg or WebM videos in Safari or IE. In order to ensure that a
video is accessible by all of the target browsers, you’ll need to
provide, at minimum, two different source elements for your video element.
Example 1-5 shows an
HTML5 web page with a video element containing two different video
sources: one in H.264 (necessary for IE and Safari), and one in WebM
(for Firefox, Opera, and Chrome). In addition, if you want to ensure
that non-HTML5 compliant browsers have access to the video, you’ll
also need to provide some form of fallback. In the example below, the
fallback is a YouTube video. Another choice can be Flash or another
plug-in.
Example 1-5. HTML5 web page with video that works in all target browsers
<!DOCTYPE html>
<head>
<title>Big Buck Bunny Video</title>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
</head>
<body>
<video controls>
<source src="videofile.mp4" type="video/mp4" />
<source src="videofile.webm" type="video/webm" />
<iframe width="640" height="390"
src="http://www.youtube.com/embed/YE7VzlLtp-4">
</iframe>
</video>
</body>Figure 1-2 shows the video playing in Chrome, which supports the embedded HTML5 video. Figure 1-3 shows a YouTube video playing in IE9 with compatibility mode turned on, emulating an older version of IE that doesn’t support HTML5 video.
Note
If you’re unsure how to structure the fallback content in the video element to ensure access to the video in all user agents and browsers, I recommend reading Kroc Camen’s article, “Video for Everybody”, at http://camendesign.com/code/video_for_everybody.
Table 1-3 provides a summary of coverage of the three video codecs I covered in this section.
Table 1-3. Video container/codec support across popular modern browser versions
Container/Codec | Type | Extension(s) | IE9+ | Firefox | Safari 5+ | Chrome | Opera 11+ |
|---|---|---|---|---|---|---|---|
MP4+H.264+AAC | video/mp4 | .mp4, .m4v | Yes | No | Yes | No | No |
Ogg+Theora+Vorbis | video/ogg | .ogg, .ogv | No | Yes | No | Yes | Yes |
Before I return to the media elements, there is one more
attribute supported on the source
element, though it is rarely used: the media attribute. The
media attribute provides the intended media source
for the element. The default value is all,
which means the media file is intended for all media sources. There
are several other allowable values, but the two that make most sense
(other than all) for media elements,
especially video, are handheld and
screen. In combination with media
queries in CSS, one can have a web page serve both desktop
and handheld devices.
However, I’m not overly fond of trying to get one page to work in two completely different environments. Many sites have a mobile only version of the content, usually designated with a m subdirectory setting (such as http://m.burningbird.net). In the end, it may be simpler just to provide the separately formatted sites—especially with Content Management Systems (CMS) that serve all of the contents from a database.
Note
Drupal, the CMS I use, has a custom module named Domain Access (available at http://drupal.org/project/domain), that allows us to designate a different theme, page and content structure for pages served up with the m subdomain. Most other CMS tools offer something comparable.
Get HTML5 Media now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
