HTML5 Video Take 2

Streaming Media West LogoMany thanks to StreamingMedia.com for inviting Michael and me to speak about HTML5 and our adventures with creating the Kaltura HTML5 Media library.

Now that Streaming Media Europe and West HTML5 Media Workshops are over (and that I’m back home), as promised, here are the slides and some answers to common questions. Slides: HTML5 Streaming Media West Workshop 2010

This post is about the current state of HTML5 Media, links to resources about the Kaltura HTML5 Media Library, answers to the common questions from the workshop and some of my own personal view.

General note: “HTML5 is ready to rock!”

It is, almost all the new features are deployed across all major browsers (YES! including Microsoft IE), new devices will ship with browsers ready for HTML5 and we have almost all features defined and ready to go.

But it will take long to mature…

Yes, it is also not complete yet. Just like a newborn, the new features with version 5 of HTML will take some time to stabilize, grow and mature.
The spec & implementations need to be finalize and get fully compatible across all browsers. The authoring tools are starting to show (Cooties, HTML5 for Illustrator, Dreamweaver HTML5 Pack, Adobe Edge, Flash to HTML5…), nevertheless, it will take these a while to get to the level of Flash IDE. And unfortunately, it doesn’t seem like we’re going to have a solution for old browsers soon as adoption is sooo slooooww… (Maybe we should write a worm that upgrades old browsers… err)

There’s a great post by Craig Buckler demystifying much of the latest confusion of “HTML5 ready or not” debate: The W3C and the “HTML5 Isn’t Ready” Backlash. While I agree with most his notes, I found the 2 comments below somewhat lacking:

  • “Interoperability always has and always will be an issue…” – While it’s true that it has always been a pain, this is due to the way browsers were made. Part of what made Flash so successful was its ability to provide a “write once deploy everywhere” solution, and more so, it’s ability to update and fall-back very well. I don’t think there will be many reasons for companies, website creators to make a business decision to prefer pure HTML5 Media adoption if cross-browser experience support stays awkward. Instead, except for where HTML5 is a must (iOS), we’ll use Flash.
  • “…but there’s little to stop you writing HTML5 code today” – This is just an overlooking of what HTML5 Canvas, Video and the addition of SVG and JS control on these means. It’s true that to write HTML you don’t need more than a text editor. But you don’t expect designers to create rich-user-interfaces and animations just by writing code… The lack of good Authoring Tools slows down the ability of designers to create great content. The alternative is yet again Flash IDE, which hopefully materialize the promise of JS, Canvas and SVG export.

Streaming Media West Workshop Discussion [[Download The Slides]]

The live discussion with developers and users of online video during the workshops made it clearer – the 3 outstanding questions are:

  1. Does feature parity with Flash media exist? (Adaptive, live, drm, performance, etc.)
  2. Backward support.
  3. Performance.

So why is it such a big buzz?

This yields another question, how can HTML5 make such a big noise, while none of the above questions can positively be answered today?
Truth is, HTML5 have been fortunate enough to gain Apple’s push, excluding Flash from the iOS devices.

So considering the above general points, let’s try to answer in more details.

Who’s using HTML5 <video> today?

Note there are no Hulu or NetFlix on this list…
Part in beta part production – HTML5 video is definitely already in use by leading websites. So it is just a matter of time… let’s look at the features:

Starting with the hardest – DRM

As it faces growing interest from the media world, there still 1 main are that left open – the way to protect ones content.
According to the W3C HTML5 FAQ, current approach that seem to be taken by the standard is of a bystander waiting for the various stakeholders (members of W3C; browser makers, etc.) to decide among them on a unified solution, what seem to be unlikely to ever happen due to the nature of DRM;

Evil or not, mathematically impossible or not, DRM is a solution currently forced upon premium content networks like Hulu and NetFlix. It looks as if DRM will not be made possible across browsers, premium sites will have to rely on alternatives like Flash and Silverlight, or carefully chose their target audience based on available DRM technology for HTML5.

Interesting discussions on the topic can be found on the W3C issue queue and lists.

Live streaming and Adaptive bitrate

Possible on Apple’s HTTP Live Streaming and via VLC. The standard, doesn’t constrain upon content length, thus making live broadcasting possible, combined with chunking the stream this makes simple HTTP Adaptive Streaming. Additionally, groups like FOMS (Foundations of Open Media Software) and IETF (Internet Engineering Task Force) are developing solutions for better adaptive streaming. That said, the spec does not define any specific details for adaptive or live broadcast, so we’re about to face different solutions and different implementation details.

Hopefully, the standard for Media Fragments:

“Davy is looking for videos about allergies and would like to get previews at a lower frame rate to decide whether to download and save them in his collection. He would like to be able to specify in the URI a means of telling the media server the adaptation that he is after. For video he would like to adapt width, height, frame rate, colour depth, and temporal subpart selection.”

…there would be a standardized way to communicate between the client and the server asking for specific parts or specific playback flow of a media asset.

The Video Codec War

Browser wars? IE6 special HTML?… Now we have it the video way – The standard don’t and isn’t going to in the near future define a must have common base codec. What it does instead is providing a graceful fallback option using the <source> tag. Not the end of the world considering you only need two codecs today: h264 and webm or h264 and ogg theora. And if you’re using Flash or Silverlight as your fallback, h264 is enough (iOS will play it using HTML5 based on built-in QT and Flash on all the other devices/browsers).

Performance

With latest GPU additions to Flash Player and other plugins, video is really not a cpu hog anymore, what seem to soon be the case with all HTML based video too – As all browsers are going to have GPU acceleration soon (if not already), Safari and IE will both have it in the underlying OS video rendering, Mozilla will be utilizing the GPU soon and so is Chrome.

Let’s build an HTML5 <video> Player. What should I know?

If you don’t have to, don’t. At least till early 2012 after comments phase is over upon the May 2011 future released final specification, Flash is easier and the only true cross-browser experience. Even then, we will still have issues with backward compatibility; unfortunately, browser adoption rate is very slow.

But you want iOS playback too… So you’re left with JavaScript solutions to do the calculated fall-back/forward and somewhat unified cross-browser experience.

Here are some pointers when you choose to deploy your HTML5 player:

  1. Codecs are not that big deal – To be honest, if you choose to use Flash as the default, and HTML5 just for iOS, then h264 is all you need.
  2. Though, playback quality and bandwidth usage are bit more complicated. Whether you use HTML5 or not, different devices have different screen sizes and bandwidth varies from place to place. This requires different video flavors for best experience. You can transcode the videos and specify various <source> tags, but how does the browser choose between mp4 for iPhone, mp4 for iPad and HD mp4 for TV? -Use JS to identify the device and use the right <source> accordingly.
  3. Every browser has it’s controls, it’s ugly, and you need your own unified branded player UI. Simple: Use HTML for the UI, standard clean HTML4 elements. Use Flash as black box replacement for to cover up for non-HTML5 browsers.
  4. If you need DRM – go Flash or Silverlight.
  5. Advertising is possible, but not entirely; some devices don’t support overlays that well, especially not with full-screen, and JS based video advertising doesn’t really make sense when simple grease monkey scripts can automatically strip it out…
  6. Sharing is not straight forward. With Flash you only had to copy one <object> or <embed> tag. Sometimes our player will have many lines of code… The easy way – use iframe.

JavaScript mixes Flash and HTML5 for truly cross-browser/device experience: Kaltura’s HTML5 Media Library

HTML5 and Flash hybrids are the solution to this mess, for now. My currently preferred way: HTML for the UI, Flash as the default playback and HTML5 <video> for the iOS. Throw a Kaltura server (CE v3.0 is out! and available as free public AMI too!) on the back end, providing the encoding and management service for the various devices out there and you’re all set to go. On devices where no Flash or HTML5 exist – the library shows a direct link to the supported video format (usually 3gp).

Moreover, the library aims to solve the problems of easy embedding through the use iFrame encapsulation and API bridging, VAST integration for advertising, integrated plugins, custom skinning (either via ThemeRoller and jQuery or event binding to DOM elements) and more…

Learn more at Kaltura.org – Kaltura HTML5 Video Media JavaScript Library

More advanced use cases… Let’s explore Wikipedia

About 2 years after the early announcement about getting video on Wikipedia and a year and half after the project launched, it is now showing fruits. As we keep pushing the bar of HTML5, this project aimed from the very beginning (long before many have heard of the HTML5)  to create a fully functional collaborative online HTML5 based video editing system complete with audio and video tracks, transitions, effects and more. The Kaltura HTML5 Sequencer, developed as part of the Kaltura HTML5 Media Library is a great proof of the underlying potential of HTML5 media. Try the Sequencer here.

After blurb…

In this post I tried to focus mainly on the one specific controversial <video> tag and its aspects. HTML5 and the even wider HTML.next define a much larger set of tools that will change the online experience. We are now standing upon a new web, where semantic, media and applications are blending together with the physical world (devices), and a new mesh is to be created.

If you’ve read so far, thanks (!). Let me know what you think below.