Article Addendum: 'Video characterization creates hands-on headaches', July 25, 2002

This'll be a 'living' document; I'll periodically update it with additional evaluation results on both existing and new codecs, plus additional links that I come across. So occasionally check back, and drop me an email with any feedback. Thanks!

• July 12, 2002
• July 16, 2002
• July 25, 2002

July 12, 2002

• Reference clips
• Software settings
• My MPEG-2, MPEG-4 and Windows Media Video results (published in article)
• My MPEG-4 Advanced Simple Profile results (not published in article)
• Test clips and corresponding batch and report files (mine)
• Test clips (from others) and my results on those clips
• Other video compression studies
• Software and other useful links

Reference clips

Below you'll find the twenty Video Quality Experts Group clips I used in my study. The '625' (PAL) clips are 25 fps with 720x576 pixel resolution, while the '525' (NTSC) clips are 30 fps with 720x486 pixel resolution. See the print article's table 1 for file sizes. To convert them to AVIs, first run them through Stewart Worrell's VQEG2STD program (you can convert multiple files in one session with a batch file), then through his YUV2AVI utility, using the following settings:

Input YUV File = FILENAME
YUV Format = 4:2:2
Output Format =  AVI File, 25 (PAL) or 30 (NTSC) Frame Rate, Disable Compression
Number of Frames = 222 (PAL) or 262 (NTSC), 1x From Source
Frame Resolution = 720x576 (PAL, from drop-down list) or 720x486 (NTSC, type in custom resolution)

As mentioned in the print article, I was unfortunately not able to secure permission to post either the original or lossy-compressed versions of the University of Hannover clips. Rob Koenen, president of the MPEG-4 Industry Forum, reports that his organization is developing freely distributable test clips. Regularly check the consortium's website for more details.

Doug Dixon from Sarnoff Labs also reports that the SMPTE (Society of Motion Picture and Television Engineers) has D1 format digital video library clips available for purchase. They're currently offered on tape, and DVD-based versions are under development.

Software settings

Settings for all of the software I used in compressing the reference AVI clips are listed below for your reference:

Global software settings

JNDmetrix
InputReferenceFile=FILENAME
InputReferenceFrameIndex=0
InputTestFile=FILENAME
InputFullColor=1
InputFieldInterlace=0
InputFieldRate=0
InputReferenceCrop=0, 0, 0, 0
InputTestCrop=0, 0, 0, 0
InputBorder=16
ViewDistance=4
ViewRoomAmbient=3.0
ViewDisplayBlack=16.0
ViewDisplayGamma=2.25
ViewDisplayLuminance=70.0
ViewDisplayPhosphor=0
OutputLogFile=FILENAME
OutputMapType=0
OutputMapFormat=BMP
OutputMapScale=10
OutputChannelLog=0
InputReferenceFormat=800
InputTestFormat=800
InputReferenceHeader=0
InputTestHeader=0
InputReferenceResolution=0, 0
InputTestResolution=0, 0
InputReferenceContiguous=0
InputTestContiguous=0
InputReferenceColorSample=0, 0
InputTestColorSample=0, 0
InputReferenceLumaCalc=0
InputTestLumaCalc=0

Windows Media Encoder 8
-allstats
-input FILENAME
-output FILENAME

Adobe Premiere
General window:
    File Type: LSX-MPEG Encoder
    Range: Work Area
    Export Video
Video window:
    Pixel Aspect Ratio: Square Pixels (1.0)
Audio window: nothing
Keyframe and Rendering window:
    Fields: No Fields
Special Processing window (all defaults):
    Gamma: 1.0
    Noise Reduction: None
    Better Resize: Off
    Deinterlace: Off
    Cropping: Off

LSX-MPEG
Pixel aspect ratio 1:1
Video format NTSC
Assume input luminance is 16 to 235 (CCIR-601)
Audio tab (default):
    Audio bitrate = 128 kbps

MPEGable
Preprocessing:
    Crop top: 0 (default)
    Crop bottom: 0 (default)
    Crop left: 0 (default)
    Crop right: 0 (default)
    Downsampling factor: 1x (default)
    Target framerate: Same as input (default)
    In/Out Point: All Frames (default)
    Deinterlace: off
Audio:
    Channels: Same as input (default)
    Precision: Same as input (default)
    Sample Rate: Same as input (default)
Encoding:
    Keyframe Period: 100 frames (default)
    Image Quality/Motion Smoothness: Very Smooth Motion
    Compression Quality/Encoding Speed: High Quality
    Buffer Size: 5 seconds (default)
Interoperability:
    Profile: Advanced Simple Profile
Streaming:
    Add Hint Track: disabled (default)
    RTP Packet Size: 500 (default)

Hannover CIF encoding software settings

JNDmetrix
InputTestFrameIndex=0 (MPEG-4), 1 (WMV8)
InputFrameCount=(reference clip frame count) (MPEG-4), (reference clip frame count minus 1) (WMV8)

Windows Media Encoder 8
-v_mode 0 (1-pass), 1 (2-pass)
-v_bitrate 300000 -v_quality 45 (for funfair_cif)

Adobe Premiere
Video window:
    Frame Size: 352x288
    Frame Rate: 30

LSX-MPEG
Output Type = MPEG-4
Motion Estimation 10 (out of 20, frame, default)
Bit Rate tab:
    Constant bitrate, bitrate = 300 kbps
Motion Vectors tab (all defaults):
    For P frames, maximum horizontal motion vector 32, maximum vertical motion vector 32
Sequence tab (default):
    Number of P frames between nearest I frames:15
MEG-4 tab (all defaults):
    Simple profile, level 3
    Enable unrestricted motion estimation
    Enable advanced prediction mode

MPEGable
General:
    Total bitrate 300 kbps (except funfair_cif = 400 kbps)

VQEG D1 encoding software settings

JNDmetrix
InputTestFrameIndex=0 (MPEG-2), 1 (WMV8)
InputFrameCount=(reference clip frame count) (MPEG-2), (reference clip frame count minus 1) (WMV8)

Windows Media Encoder 8
-v_mode 2 (1-pass), 3 (2-pass)
-v_bitrate 800000

Adobe Premiere
Video window:
    Frame Size: 720x576
    Frame Rate: 25

LSX-MPEG
Output Type = MPEG-2
Motion Estimation 16 (out of 20, frame, default)
Bit Rate tab:
    Variable bitrate, maximum bitrate 8000 kbps, rate control mode=keep specified average bitrate of 800 kbps
Motion Vectors tab (all defaults):
    For P frames, maximum horizontal motion vector 32, maximum vertical motion vector 32
    For B frames, maximum horizontal motion vector 16, maximum vertical motion vector 16
Group of Pictures tab (all defaults):
    Sequence (versus Headers (GOP))
    Number of P frames between nearest I frames = 3
    Number of B frames between nearest P frames = 2
Detect tab (default):
    No Scene Change Support
Input tab (all defaults):
    Encoded chroma format: 4:2:0
    Filter: None
MPEG-2 tab (all defaults):
    Main Profile & Main Level
    Sequence Display Extention: Disable Display Extention

MPEGable
General:
    Total bitrate 800 kbps (except src9_ref__625 and src_17__525 = 1000 kbps)

My MPEG-2, MPEG-4 and Windows Media Video results (published in article)

Here's a sneak peak at the results that will appear in the second part of my print article, in EDN's August 8 issue.

N/R = Not reported

1. MPEG-4 average bitrates were determined by converting the encoded file size from bits to bytes then dividing by the clip play time. Encoding times were determined by subtracting each file's 'created' time stamp from its 'modified' time stamp (note possible system rounding).
2. WMV8 average bitrates were determined via the encoder output report. Encoding times were determined by capturing the system clock value just prior to, and just after, encoding.
3. I re-ran this clip through the encoding, transcoding and analysis flows because initial command line interface options inadvertently resulted in one-pass compression
4. MPEG-2 average and peak bitrates were determined using the freeware Bitrate Viewer utility. Encoding times were determined via an undocumented technical support debug mode's report output.

Below, please find the main article text that corresponds to the above table. I didn't include the entire part 2 article (ie sidebars, references, company contact info, etc) so stay tuned for the August 8 issue.

Results du jour

As you analyze the results in the table, keep in mind the original mammoth file sizes. Comparing them with the compressed file sizes in the table, you should be impressed with how well the codecs retain a semblance of the original image at such significantly slimmed bit rates. I resist drawing any definitive conclusions from the table because, depending on what clip or combinations of clips represent the kind of video media you typically encode and whether encoding speed, resulting file size, image quality, or some combination of these variables in an application-specific priority ordering is important to you, you’ll likely reach a different conclusion from other readers. As I put on various applications’ “sets of eyes,” I consequently interpret the data in unique ways.

The most important question to ask up-front is whether JNDmetrix results correlate to the quality differences I observed. I am confident that JNDmetrix is doing all that it claims to do and that its results are meaningful beyond its MPEG heritage. The codec-versus-codec numbers matched what my eyes and brain interpreted, and it was fascinating to peruse the report files and see how the frame-to-frame JNDmetrix number variances matched changes in scene complexity and transitions between key and delta frames.

The JNDmetrix reports give per-frame results for both luma and chroma; combine them, placing a higher priority on luma, as does the human visual system; and then average the per-frame results to give a clip result. I intended to put not only the clip average, but also the “best” and “worst” frame results in the table. However, Dixon convinced me that this approach would unfairly represent the codecs, because one or two “bad” frames in an otherwise-solid clip would be acceptable to the eyes. I still think that standard-deviation numbers in the reports would be useful, however. Options I didn’t enable further break down the numbers into various color channels, and you can even obtain per-frame bit maps to display the pixel-versus-pixel variances from reference to test clip. As you compare the JND numbers for the codecs and settings, the following guidelines from Sarnoff's documentation may be of benefit:

"A 1 JND rating indicates a 75% probability of seeing a difference between two images (vs. 50% guess). Two image sequences below 1 JND apart cannot be distinguished, even when the exact nature and location of differences are known in advance. Two image sequences below 3 JNDs apart are not obviously different. Differences are visible, but only when an observer knows exactly where to look. At 5 JNDs, differences are readily apparent. In probability terms, a 2 JND difference corresponds to a 93.75% probability of discrimination (0.7510.753(120.75)), and a 3 JND difference corresponds to a 98.44% probability. Although probability of discrimination asymptotes quickly as a function of JNDs, the units are useful because they correspond to roughly linear magnitudes of subjective visual difference."

One of the first things you might notice is that the codecs  only rarely achieved an encoded bit rate that was near, at, or below the 300- or 800-kbps target—a disappointing fact. Admittedly, I intentionally chose many challenging clips. But when I tell an encoder that I want a certain average bit rate, regardless of whether I’m encoding with VBR or CBR, the encoder should hit that bit rate without my having to do iterative work.

When you specify CBR encoding in WMV8, you might expect the Windows Media Encoder to deliver a bit stream with a constant number of bits per video frame. Look closely at the WMV encoding report files, though, and you discover that the Windows Media Encoder delivers a constant transport rate stream with bounded bit-rate variance on a frame-by-frame basis. Note the term “bounded.” The encoder defines an intermediate buffer, whose size the encoded bit stream communicates to the decoder. Streaming CBR clips from a server results in an initial buffer-filling delay before playback begins. However, a frame’s or a sequence of frames’ encoded bits are never large enough to exceed the available playback-buffer size.

WMV’s VBR mode follows the customary unbounded per-frame bit-rate definition the industry uses. One-pass VBR mode, according to Microsoft documentation, prioritizes the specified quality over the specified bit rate, whereas two-pass VBR flips these variables’ priorities. A comparison of the one- and two-pass numbers by and large validate these suppositions, though a few baffling contradictions existed. More generally, look at whether the longer encoding times of two-pass mode in both CBR and VBR encoding result in higher-enough JNDmetrix quality measures for the types of media you’re interested in.

The Ligos CBR encoder, like all modern codecs that support a mix of key and delta frames, is also a bounded-bit-rate VBR technology. Analysis of the MPEG-4 bit streams in Interra's M4 software verifies the presence of a mix of I and—theoretically much smaller—P frames, correlating to the encoder defaults during compression. Note that the MPEG-4 Simple Profile Level 3 decode buffers are much smaller than those that Windows Media Video requires. This memory-density difference may be important in cost-conscious consumer-electronics devices.

JNDmetrix-analysis comparisons of MPEG-4 and WMV8 bear out what my eyes saw. When the MPEG-4 files were smaller than the WMV8 files, large smeared areas appeared in the MPEG-4 images, whereas the WMV8 images contained detail. More important, with comparably sized files, MPEG-4 contained more large “blocky” artifacts and other irregularities, such as significant loss of facial details. The poor quality extends to MPEG-4 files significantly larger than those of WMV8. But keep in mind that those artifacts would be far less disagreeable on a tiny cell- phone screen display than on the large monitors I viewed them on. And, because I couldn’t measure processor load during decoding, MPEG-4 Simple Profile may—and from anecdotal information, probably does—have an advantage in this regard over WMV8.

Comparing MPEG-2 with WMV8, like MPEG-4 versus WMV8, provides an intriguing industry standard-versus-proprietary tradeoff that was one of my fundamental motivations for this study. I also hoped to infer some measure of the viability of advanced video codecs in conjunction with mature red-laser technology as an alternative to blue lasers for high-definition DVD. WMV8 generally achieved image quality moderately lower than, to comparable to, MPEG-2 with slightly to significantly smaller files than MPEG-2. But it did so with the trade-off of much slower encoding. Keep in mind that WMV8 encoding ran as a bare-bones DOS command-line utility, whereas the MPEG-2 and -4 encoders ran as a plug-in under Adobe Premiere. MPEG-encoding speeds in a simpler configuration might be even faster. And keep in mind, too, that WMV8 is months’ old technology lacking bit-stream backward-compatibility constraints that other codecs have; MPEG-2 is nearing its 10th birthday and has bit-stream-format constraints.

Onward and upward

Until recently, I thought that the reason JNDMetrix only supported the AVI and MPEG formats was because it integrated AVI and MPEG file-wrapper parsers (It also inputs sequences of still image files at one per frame.) I was half-right. Actually, JNDmetrix hands off any file with a MPG extension to the PC’s DirectShow filter stack for parsing. So, I theoretically may not need to decode a compressed file to AVI but could simply rename its file extension as MPG, although this technique doesn’t currently seem to be working with MPEG-4.

I’m concerned, too, given my quality-degrading attempt to transcode MPEG-2 to lossless AVI, that the Vegas Video WMV decoder might be doing more quality damage than simply tacking on redundant frames, adversely and unfairly affecting the WMV scores. After Sarnoff adds support in JNDmetrix for DirectShow decoders that implement seeking via not only frame-stamping (currently supported) but also time-stamping, I'll try WMV-to-MPG renaming. Microsoft has also promised me a more reliable WMV-to-AVI transcoder. I’ll communicate any difference in the results to you via the Web-site addendum.

I'd like to obtain JNDmetrix results for MPEG-4 Advanced Simple Profile, for both the CIF- and D1-resolution reference clips. The MPEG-4 encoder in Apple Computer's QuickTime Version 6 Public Preview supports both Simple and Advanced Simple profiles and saves in both MP4 and Premiere-friendly MOV file formats. Unfortunately, it significantly prioritizes per-frame quality, offers limited adjustment of quality settings, and consequently drops frames unless I encode to very high bitrates. For example, I had to encode the mobile_cif clip to 800 kbps to preserve frames, making comparisons against both Ligos' Simple Profile and Microsoft's Windows Media Video 8 meaningless. Dicas' encoder, from my limited experimentation thus far, offers more quality options and should produce more comparable bitrates. Check the Web site addendum for the data.

I’d like to test other codecs via transcoding to AVI or through the DirectShow rename-to-MPG loophole. WMV9, code-named Corona, is due out later this year and on my evaluation wish list. Other candidates include the open-source VP3 codec from On2 Technologies, which the company is offering to the Internet Streaming Media Alliance as an alternative to license-fee-burdened MPEG-4, and the company’s more advanced, proprietary VP5 codec. DivX 5 is on my list, too. Apple’s embrace of MPEG-4, coupled with its current legal quagmire with former partner Sorenson, has diminished the importance of evaluating Sorenson Video. I might also again try deinterlacing the VQEG clips with a more robust Cleaner, Adobe AfterEffects, Canopus ProCoder or VirtualDub algorithm, but this task is low-priority.

Although I'd like to look at RealVideo, RealNetworks isn't cooperating, despite numerous discussions it conducted during the past few months, both with me and with Sarnoff. Real-Networks isn't confident in JNDmetrix's analysis results, and the company's programs don't supply a DirectShow filter that I can tap into. Third-party workarounds, such as Tinra and TechSmith's Camtasia exist, but RealNetworks can easily dismiss their transcoding accuracy. As a result, I depend on obtaining software from RealNetworks that decodes RealVideo to AVI, a series of still images, or some other format that JNDmetrix accepts.

I’d also like to assess how much my choice of computing hardware and operating system influenced my results. My motherboard supports the addition of a second PIII-800 CPU, and I also plan to install the multitasking Windows 2000 OS in a separate hard-drive partition. I also have a Pentium 4-based system that will be useful in case I determine that any of the encoders support and can performance-benefit from the CPU's SSE2 instruction set.

In all cases, after I, as a mainstream user, complete my work, I’ll hand the reference AVIs to the codec vendors, as more advanced users, for optimized encoding. Numerous optimization options are possible, including processing the video before encoding it and as part of the subsequent decode; adjusting intermediate buffer sizes: altering the I, P, and B frame proportions and patterns—going so far, perhaps, as to manually insert key frames; and tweaking the quality and other encoder settings. In all cases, though, I’ll insist that what the vendors provide me in response are 300-kbps CBR and 800-kbps VBR clips, the original targets of this project.

My MPEG-4 Advanced Simple Profile results (not published in article)

Below you'll find the results of my testing on MPEG-4 Advanced Simple Profile. I encoded the source files to MP4 format in Dicas's mpegable S4, and decoded them back to uncompressed AVIs in Apple's Quicktime 6 Public Preview, prior to running them through JNDmetrix.

Notes:
1. MPEG-4 average bitrates were determined by converting the encoded file size from bits to bytes then dividing by the clip play time. Peak bitrates were determined by 'eyeballing' the statistics graph provided by the encoder. Encoding times were determined by subtracting each file's 'created' time stamp from its 'modified' time stamp (note possible system rounding).
2. One-frame offset between reference and test clips
3. Total bitrate set to 400 kbps to prevent frame dropping.
4. Total bitrate set to 1000 kbps to prevent frame dropping.

To ensure that the Dicas encoder didn't drop frames (believe me I tried many many many combinations of encoder options!), I had to do one of two things:

1. Set the encoded bitrate significantly higher than the 300 kbps and 800 kbps targets, which would have made a comparison of the JNDmetrix results against other codecs meaningless (this was the only option available to me with the Apple Quicktime 6 Public Preview encoder, so thus far I haven't included that encoder in my study and similarly did not select this option for the Dicas encoder), or
2. Enable the 'Very Smooth Motion' option for Image Quality/Motion Smoothness, versus 'Sharp Image' or the default 'Smooth Motion'.

Looking at the resultant files frame-by-frame, and at the JNDmetrix results, it's clear to me that in this mode and at these aggressive bitrates, the Dicas encoder is not necessarily compressing each and every frame of the source file but is instead sometimes repeating multiple copies of some source frames. For (hypothetical) example:

    Source file: 1-2-3-4-5-6-7-8-9-10

    Encoded file: 1-1-3-4-4-6-7-7-7-10

    JNDmetrix analysis result: good-bad-good-good-bad-good-bad-bad-bad-good

This pattern will be very clear to you when you look at the JNDmetrix report files, which give per-frame analysis results. Also, when single-stepping through the source and encoded files, I sometimes found a 1-to-1 match between frames, but sometimes there was a one-frame offset, within the same clip! As a result, I ran two sets of analyses; one with a 1-to-1 match between reference and test clips, and one with a one-frame offset (labeled alternate).

I commend Dicas on their development of a fully- and flexibly-optioned encoder that enabled me, compared to other codecs I tested, to come very close to the target bitrates. When I set it to 'Sharp Image' or 'Smooth Motion', it delivered very impressive quality, and I barely (if at all) noticed that the encoder had dropped frames. Even in the 'Very Smooth Motion' mode, I wasn't aware that frames had been discarded and other repeated in their stead, until I looked at the JNDmetrix report files.

Given this scenario, I better understand RealNetworks' reluctance to participate in the study. JNDmetrix is a useful tool, don't get me wrong. But its intolerance of dropped frames is incompatible both with the operation of modern video codecs at aggressive bitrate targets and with the interpolation capability of the human visual system. Folks who just look at the JNDmetrix numerical results, and don't take the time to also read the background information, would likely obtain an inaccurate (or at least incomplete) understanding of a codec's strengths and shortcomings. Sarnoff's planned future support for DirectShow decoders that implement seeking via time-stamping, thereby breaking its exclusive ties to frame-by-frame synchronization, may improve its capability as a technology-independent codec evaluation tool.

Test clips and corresponding batch and report files (mine)

Below you'll find the encoded video clips I've generated and have copyright permission to distribute (i.e. no Hannover clips, sorry!). Please DO NOT stream the files directly from the server. To make better use of non-infinite network bandwidth, right-click on the file links to download and save them to your computer's hard drive, then play them from there (see the Software and other useful links section of this addendum for player suggestions). The above results tables indicate file sizes.

Dicas MPEG-4 Advanced Simple Profile (300 kbps)

Dicas MPEG-4 Advanced Simple Profile (800 kbps)

Ligos MPEG-2 Main Level, Main Profile (800 kbps, VBR)

Ligos MPEG-4 Simple Profile, Level 3 (300 kbps, CBR)

Microsoft Windows Media Video (300 kbps, 1-pass CBR)

Microsoft Windows Media Video (300 kbps, 2-pass CBR)

Microsoft Windows Media Video (800 kbps, 1-pass VBR)

Microsoft Windows Media Video (800 kbps, 2-pass VBR)

Test clips (from others) and my results on those clips

As the vendors deliver to me their attempts at optimally compressing the reference clips, and as I subsequently run those test clips through JNDmetrix, you'll find the results here. Microsoft and On2 (for both the open-source VP3 and proprietary, more advanced VP5 codecs) have optimization projects underway, and Dicas's and Ligos's comments are below. The guidance I've given all of the vendors is:

You have the ability to tweak any publicly-documented setting in your encoder that you wish, as long as you DO NOT drop frames or alter frame resolution as a result (remember, JNDmetrix won't like that), and as long as you document your per-file settings for me. My suggestion is that your priorities should be:

1. Come as close as possible to the target 300 kbps and 800 kbps bitrates. I was quite blunt, frankly, in my article commenting about my disappointment in some codecs' inability to easily (or at all) allow me to do this.
2. Then maximize quality at those bitrates
3. Then minimize encoding time at those bitrate/quality combos

On that last point; I normally determined this by logging the 'created' and 'modified' timestamps in the file properties. You're doubtlessly going to be running different hardware and operating system revs than I. I'd ask you to capture the system clock values like I did, both for your chosen encode settings and for mine (for comparison purposes).

One vendor who's decided not to invest in the optimization effort is Ligos. An abridged version of an email sent to me from Robert Saint John, Ligos' Technical Marketing Manager, is below:

We've evaluated your tests and found them to be on par with what we expected. Under these constraints, I'm informed, we're not going to get any better results at this time. I really can't commit to putting more time into this project, especially if I'm not going to be able to improve the results that you're getting. The best thing we can do right now is take the feedback from the results as well as from the process itself, and see if something can be done between Ligos and Sarnoff to make the process easier, and use that to make the codecs better if they need improvement.

Dicas took a look at my results, and has the following comments, from company representative Philipp Kraetzer and edited for grammar and clarity:

Apologies for my very late reply. It took us some time to download the files and also to examine the problem with frame dropping.

Alas the answer is not quite simple:

1. The source files seem to be converted from 25 fps (PAL) to 30 fps (NTSC) by simply doubling every 5th frame. Certainly, this is not really a problem but the video looks more jerky.

2. In order to keep the bitrate and the VBV constraints, there may sometimes be a frame drop. You can avoid this by setting the value of the buffer size to a higher value, e.g. 8 sec. This yields a less accurate bitrate for short video sequences but avoids the frame dropping.

3. All other settings you have chosen are OK

4. We noticed that QT Player 6 sometimes skips frame if it is not fast enough to decode them. I am not sure but this may be a problem also for your test. I have attached a simple (but useful) command line tool to decode MP4 files and write the video data to an uncompressed AVI file or to a raw data YUV 4:2:0 file. Maybe this can be helpful for you, too.

    Usage is: MP4Coder -decode <filename>.mp4 -o <outputfile> -[avi|raw] -[yuv|bgr|rgb|bgra|rgba] -pf [0|1|2|3]
    "MP4Coder -decode" prints a help message.

For the VQEG sequences:

Set the Key Frame period to 300, all other settings are ok, except in the following 2 sequences (to avoid too much frame skipping)

-------------------------------- Settings for src13_ref_525.avi

Key frame period: 300 Bitrate: 880 Buffer Size: 8 sec

-------------------------------- Settings for src17_ref_525.avi

Key frame period 300 Bitrate 800 Buffer Size 6 sec

-------------------------------- 

For the Hannover CIF sequences:

Set buffer size to 4 sec, all other settings as before, except for

-------------------------------- Settings for funfair_cif.avi:

Key frame period: 300 Bitrate: 320 Buffer Size: 8 sec Comp. Quality vs. Speed: Medium

-------------------------------- 

The encoding time for the optimized settings is nearly the same as for your original settings. To maximize encoding speed one should disable both previews (input/output).

If you have further questions or comments please contact me. Sorry again for my late response; there are a lot of different projects underway at the moments :-)

Best,

Philipp

Other video compression studies

eTesting Labs' study from last summer was subjective in the traditional sense; instead of employing JNDmetrix as I did, it relied on test subjects. It used a subset of this project's University of Hannover clips, albeit compressed to a more aggressive 250 kbps bitrate. Since the two studies employed the same reference clips, is there any correlation between their results? See for yourself, the Adobe Acrobat document is here.

Software and other useful links

Feature-, time- and/or clip length-limited evaluation versions of most of the products that follow can be downloaded from the vendors' websites. In many cases, you can also purchase a license online.

The most straightforward way to obtain a MPEG-2 decoder is to purchase DVD playback software from companies like Cyberlink, InterVideo,  National Semiconductor (Mediamatics) and Sonic (who now maintains the former Ravisent Technologies product line). The final version of Apple's QuickTime Version 6 is also reputed to include MPEG-2 support, although it  isn't included in the Public Preview version now available. MPEG-2 encoders (usually with bundled decoders) come from companies like InterVideo (WinCoder), Ligos and MedioStream. Other MPEG-2 encoder options are TMPGEnc and the freeware AVI2MPG2 and bbMPEG. Sonic Foundry's Vegas Video includes a MPEG-2 encoder, as do CyberLink's PowerDirector and PowerVCR II, and InterVideo's WinDVR and WinProducer.

The latest version of Ligos's LSX-MPEG plugin for Adobe Premiere supports MPEG-4 encoding (Simple Profile only), and the bundled MPEG-4 decoder works fine within Windows Media Player. Dicas offers both consumer (X4) and professional (S4) versions of its MPEG-4 encoder (Simple, Advanced Simple and Core profiles), along with a SDK and an application that will embed MPEG-4 data streams within an AVI wrapper. The company also supplies standalone DirectShow filters and a free player. Apple's Quicktime 6 Public Preview, both an encoder and decoder, creates Simple and Advanced Simple MPEG-4 files, and can also embed MPEG-4 audio and video within the MOV wrapper. Similarly, DivXNetworks embeds MPEG-4-compatible data within either the industry-standard MP4 or proprietary DivX formats.

I successfully played back the MPEG-4 Simple and Advanced Simple Profile clips I created in three decoder programs; Apple's Quicktime 6 Public Preview, Dicas's mpegable Player  and the DivX Player. Other freely downloadable players, which I have not yet tried, come from companies such as Envivio (plugins for Quicktime, RealSystem, and if not now, soon, Windows Media Player), PacketVideo and Philips. Sigma Designs' website offers a freely downloadable MPEG-4 encoder, though if you read the fine print you'll discover that you're responsible for paying any license fees that may result from your use of it. See Ben Waggoner's coverage of MPEG-4 encoder-and-player compatibility in the June 2002 issue of DV Magazine.

What about Windows Media Video? For encoding purposes, you've got two options, both freely downloadable from the Microsoft website. The Windows Media Encoder 7.1 is GUI-based and will encode to Windows Media 8 audio and video formats, albeit without 2-pass and VBR support (which is also lacking in third-party programs that offer Windows Media encoding through Microsoft's SDK). To test all of the encoding options I did, you'll need the Windows Media Encoder 8 command line utility; search Google for GUIs that run on top of WME8, such as the Visual Basic program designed by Roy Lawson. Also freely downloadable from the Microsoft Windows Media site are players for Windows 9x/ ME/NT4/2K/XP/CE, Mac O/S 8.1 to X, and Sun Solaris. To transcode Windows Media Video to other formats, try Sonic Foundry's Vegas Video (which, by the time you read this, may have fixed the replicated-frames bugs I discovered); Adobe also reports that the next version of Premiere will include import support for Windows Media.

To get at the data hidden within the MPEG-2 file wrapper, such as average and peak bitrates, sequences and locations of I, P and B frames, and the like, at least two freeware utilities exist; bbTOOLS and Teco's Bitrate Viewer. A fuller-featured, albeit, not free variant called Parse MPEG is also available for download and purchase from the Teco website, and Interra's MProbe is an alternative. For MPEG-4, the only option I discovered at the present time is Interra's M4; it worked pretty well for me (although its Globetrotter content protection algorithm blocked it from running on one of my PCs even with a valid key), but it won't satisfy the fiscal needs of the extreme penny pinchers. And for AVIs, you can obtain rudimentary information through File -> Properties, but for more statistics (and a built-in viewer) search Google for the Terabits AVI Info (avinfo) utility.

I barely scratched the surface of the two full-featured, expensive video editing programs I used in this project; Adobe Premiere and Sonic Foundry Vegas Video. At $30, Apple's Quicktime Pro is a low-cost alternative which bears serious consideration, and if any amount of money is an issue for you, check out the freeware programs VirtualDub and MovieXone, the latter from AIST. Although I wouldn't necessarily recommend Microsoft's ancient VidEdit as your primary editing utility (the program hearkens back to the Windows 3.x days) it does make an excellent test of AVI compatibility; I found a copy here. And if you don't need to do editing per se, just batch-mode processing and transcoding to multiple formats, all of these programs are probably overkill. Instead, check out Canopus's ProCoder and Discreet's Cleaner.

How do you get your hands on a copy of Sarnoff's JNDmetrix? Doug Dixon admits that the product isn't yet ready for general distribution, and at $9,850 it's also pricey, but a 45-day evaluation version is available. Send Sarnoff an email for more information. Genista sells a video quality testing product called VideoQoS which also claims to factor in perceptual effects, but I don't know much about it as the company didn't respond to my reqeuests for information. If you'd like to explore the less accurate, but much less expensive, PSNR approach to image quality analysis, I've been told that imgcmp, part of the JasPer JPEG-2000 suite, works well. Teco's website indicates that they have a PSNR-based product called Frame Comparator under development.

Now for some miscellaneous utilities. To convert the VQEG files from YUV to AVI, you'll need two programs, VQEG2STD and then YUV2AVI, both from Stewart Worrall. If the YUV files you'll be working with are 4:2:0, and if you don't want to hassle with the larger file sizes and minor image degradation of component video-to-RGB transcoding in YUV2AVI (since compressors will subsequently transcode back to component video) or you want to batch-convert a group of YUV files, try Microsoft's YUVTOAVI. It's part of the company's MPEG-4 Video Reference Software toolkit; officially you need to get a license from the company to use it, but as with VidEdit it's available for download on the Internet. Search using keywords such as 'microsoft-fpdam1-1.0-000403' or (for the newer version) 'microsoft-fdam1-2.3-001213'; at the time I wrote this, you could find a copy of the microsoft-fpdam1-1.0-000403 toolkit here and the fdam1-2.3-001213 toolkit here.

ImageMagick is a full-featured set of image conversion, editing and compositing tools; the developer recently added support for 4:2:2 YUV in response to my request. I've been told that Adobe After Effects, like Discreet Cleaner, has robust video deinterlacing capabilities. If Windows Media Player can play a video file,  Avisynth (from Ben Rudiak-Gould, who also created the Huffyuv lossless video codec) will process it and let you open it in almost any video editing or converting utility. And if you'd like to analyze (and alter) the DirectShow filter mapping that occurs during video clip decoding, search Google for the Microsoft GraphEdit utility, from the company's Virtual C++ SDK.

Here are some websites that you might find useful in tracking down video information and utilities. Some I personally reference, some were recommended to me and to you readers, and others regularly appeared at the top of search results lists during my research:

• Digital Digest
• DivX Digest
• Doug Dixon's Manifest Technology site
• Doom9.net
• Leiming Qian's software site
• VCDHelp
• Videotools.net
• Vidtools
• WiredVideo
• Ben Waggoner's site
• Brandon Wirtz's Digerat and Griffin Digital Solutions sites
• Stewart Worrall's site

Finally, I'd like to thank the following companies whose products enabled me to transfer this project's large files to the codec vendors, and enabled them to send their results back to me:

• The Apache Software Foundation for the HTTP server software
• AT&T Broadband for the 256 kbps upstream, 1.5 Mbps downstream cable modem connection
• Deerfield.com for the DNS2GO service that enabled me to supply a consistent URL to the vendors in spite of the cable modem's dynamic IP address, and
• Jarle Aase for his WAR FTP Daemon server software

July 16, 2002

The just-released final version of Apple's QuickTime 6, unlike its Product Preview predecessor, will currently not play back some MPEG-4 Advanced Simple profile content such as the Dicas-encoded material I generated for this hands-on project. Alternatively, you can use Dicas's or DivX's players, although you can't transcode from MPEG-4 to AVI from them. Or you can find a copy of the Quicktime 6 Public Preview installer stub for Windows (which supposedly expires in October) here or the standalone installer (including the full installation download cache file) here QuickTime 6 supports a subset of MPEG-2 content via a separate $19.99 plug-in component, specifically program streams (which combine audio and video with a common timeline). It does not play either DSS-encrypted DVDs or MPEG-2 transport streams (with independent audio and video time bases).

July 25, 2002

See the MPEG4IP website for open-source MPEG-4 audio and video software, including encoders and a player.