H.264 and Hulu: Day Two, Additional Data

Over the last four months (specifically at January’s CES and the earlier-this-month NAB), I’ve had at least a half-dozen different Hulu representatives emphatically tell me that all of the content served by the company is H.264-encoded. Therefore yesterday’s two-part post. And therefore my confusion when commenter ‘plasticbloodm’ subsequently noted that he was able to watch Battlestar Galactica Season 4, Episode 3 just fine via a pre-H.264-supportive version of the Adobe Flash plugin.

So I hit Google, and quickly discovered what was really going on. My Hulu contacts, as it turns out, were only partly right. All of the standard-definition Hulu content, at minimum, is encoded using On2’s VP6 codec, and served at bitrates of between 480 kbps and 700 Kbps (with unspecified corresponding resolution) depending on a particular recipient’s downstream bandwidth capability. Since I’m fed by a 2.5 Mbps downstream DSL pipe, I likely was watching Battlestar Galactica content at the upper end of the bandwidth range. Some standard-definition Hulu material (but not the Season 4 material I was watching) is also encoded in 1 Mbps-peak H.264; you select that specific stream by clicking on the ‘480p’ button in the UI. And all of the high-def Hulu content (such as the Street Kings trailer) is encoded as 720p H.264, albeit at unspecified bitrate.

So the H.264-vs-VP7 CPU burden and quality observations I made yesterday are still valid, but only in comparing Street Kings to Lost, and since I don’t know the Street Kings bitrate, they’re not definitive (ABC’s Full Episode Player generously reports the incoming VP7 bitrate, which in my case is top-end ~1.5 Mbps). In evaluating Battlestar Galactica versus Lost, it turns out I was actually comparing standard-definition (or below) VP6 to high-definition VP7; both codecs come from On2 Technologies. And, as I indicated yesterday (reflecting back on a past color-space-conversion burden discussion) and a reader comment left earlier this morning confirmed, the up-scaling of that standard-definition (or below) content puts a tremendous processing load on the host CPU:

Adobe’s Flash plug-in doesn’t appear to use the hardware video overlay, hence all colour space conversion (YUV to RGB) and video scaling has to be done on the CPU. You can tell by rapidly dragging the browser window around- if the hardware overlay were used you would see the colour key used (usually blue or green) being "left behind". That’s probably the reason ABC’s standalone player works fine full-screen (the GPU would do the scaling instead) and Hulu’s Flash-based video doesn’t.

My revised results imply, but again can’t definitively confirm, the truth of a different assertion On2’s CEO Bill Joll lobbed at me during our NAB dinner. Video codec generational progression usually tracks Moore’s Law-delivered silicon performance improvement trends. A newer iteration of an codec (such as, for example, the transition from industry-standard MPEG-2 to MPEG-4, or from MPEG-4’s simple profile to advanced simple profile to H.264) delivers higher quality at an equivalent bitrate to its predecessor (or, said another way, equivalent quality at a lower bitrate) but requires heavier-duty processing than its predecessor both at the encode and decode stages of the delivery chain.*

Joll claimed, though, that each successive iteration of the VPx codec family delivers better quality at lower bitrates, along with reduced encode and decode processing requirements as compared to its predecessors. The company accomplishes this feat, he said, via several key focuses:

• Fundamental improvements in video content (therefore compression) understanding and algorithmic implementation
• A continuous and iterative code optimization emphasis, and
• Leveraging microprocessors’ multimedia-tuned instruction set enhancements, such as MMX, 3Dnow! and SSE.

I can’t use this case study to prove (or disprove) Joll’s boast, of course, because my testing is apples-vs-oranges in several key areas:

• The Hulu- and ABC-delivered content isn’t of the same show/trailer
• The respective bitrates, resolutions and (potentially) framerates don’t match, and I also have no way of knowing if either or both pieces of content are constant- or variable-bitrate-encoded.
• I’m also using two different playback utilities; Adobe’s Flash browser plugin, and the Move Networks-developed Full Episode Player.

This situation highlights why meaningful video codec testing is so difficult to accomplish. Even if you make all of the test conditions identical, for example, someone else with a different computing platform might achieve completely different encode and decode performance results. Video quality is similarly impermanent by virtue of its inherent subjective nature; artifacts that might not be noticed at all by one set of eyes-and-brain could be evaluated as highly objectionable by a different observer, and visa versa.

*As a quick aside, most video codecs (as is the case with all of the candidates mentioned in this writeup) are asymmetrical in nature, with much higher processing requirements for encode versus decode, therefore enabling lower-cost (aka consumer electronics price point-friendly) electronics at the playback stage. One notable exception to this rule is videoconferencing, in which both ends of the communication ‘chain’ are doing both video encoding and decoding (and in this particular case, in real time). The codecs used in such applications therefore tend to be symmetrical in nature.