Friday, February 22, 2008

The dark side (but funny)

I don't normally post clippings, but this is so me. Try to maintain an active forum presence has its dark side.

Source XKCD, found through BoingBoing.

Thursday, February 07, 2008

CineForm 444 always better than the HDCAM-SR?

Sony's HDCAM SR and CineForm 444 are very different compression technologies, one designed for tape-based existing deck control conform/master workflows and the other designed for online disk based workflows. Below I discuss how their core differences impact image quality, and why it is not so simple to state that one is better than the other for all image sources.

About this time last year, I blogged on a test we performed comparing CineForm's then new 444 mode to that of the respected de facto industry standard HDCAM SR; the results of that test are posted on CineForm's quality analysis page (and one of the graphs is repeated below.) While we never expected to exceed SR quality, we did select test material that would not favor SR, i.e. using very detailed source material, like full frame StEM footage (which is designed to trip up compression) plus live green-screen footage shot using a Thompson Viper. We informed Sony in advance that we were performing the test and that we were not intending to show up SR; rather we set out to prove that both SR and CineForm 444 are indistinguishable from the source, and this is still completely true. We also had the results independently verified.

Back then, it seems we had only slightly ruffled feathers at Sony, as Sony Broadcast CTO Hugo Gaggioni tried to find me at NAB 2007 to work out what we did, and if the test procedure was valid. I tried to follow up with Hugo but we never overlapped at that trade show. Sony and CineForm have long been good partners and we want to keep it that way; to demonstrate our long term relationship we even have the extremely rare bread-boarded Sony Z1 camera somewhere around the office. But in the months following NAB' 07 there was no more discussion on the test, well, until a few months ago. It seems the Sony had independently performed their own CineForm vs SR tests and drew mixed conclusions of "no significant difference between CineForm and SR 4:4:4 SQ in terms of PSNR." Sony went on to say that their 880/HQ mode is better still; in our tests we found HQ to be better than the SQ mode but of no significant difference when compared with CineForm highest quality mode.

How could Sony and CineForm do valid tests, yet get different results and both be correct in their findings? The fact is, you could almost draw these conclusions without doing the actual tests and here is why: Any very light compression (say 4:1) of DCT, Wavelet, long GOP or I-frame, will yield similar image quality; the real differences happen with much higher compression ratios. As compression ratios increase, I-Frame Wavelets (CineForm) will show their advantages over I-Frame DCT (SR, D5, DVCPRO), and as you compress more again the long GOP codecs like MPEG2 and H.264 start to shine over I-frame solutions. So in these tests we really weren't testing Wavelet vs DCT, but rather constant quality (CineForm Wavelet) vs constant bit-rate (SR). Tape systems like SR are always constant bit-rate, modulating the compression quality so the data rate matches that of the tape system.

(Click on image for clearer view)

So when you see a graph like this showing quality over time, there is a reverse graph that shows the CineForm data rate changing and SR's remaining constant. In this test, the average bit-rate of CineForm happens to be lower than SR, even though the quality is higher, so there was a real measured advantage to CineForm, yet there are tests that can show the reverse. When the image is simpler for compression, like a lot of sky or large regions with low noise or detail, the data rate in a constant quality codec like CineForm's will fall. In this latter example you may be comparing an 8:1 Wavelet to a 4:1 DCT, which would likely be in favor of the DCT. The quality on the Wavelet image has not changed, yet the quality of the constant bit-rate DCT has increased, making good use of the available bits.

A recent good example of this happened with a customer (who will remain nameless until we get an OK) that is scanning a classic 65mm color print for restoration that uses CineForm as the archive format. They independently did CineForm vs SR testing, and found in some frames SR had better PSNR numbers (than CineForm 444), and sent me the frames to investigate. The 65mm source produced really clean source for a 2K scan, and the images had shallow depth of field, so there was minimal foreground material in focus that would give the compressor a full work-out. So while SR was using all its estimated 440Mb/s, CineForm's data was running around 345Mb/s (Filmscan 2 444 keying mode.) For these frames SR yielded a PSNR of 57dB and CineForm measured 55dB--both pretty awesome by the way.

As CineForm is not limited to a tape transport mechanism, we decided to increase our quality for the Filmscan 2 mode in our next release (we have always called this mode "overkill" in house.) The sample images from our unnamed source now produce 58dB PSNR using CineForm 444 compression, averaging 390Mb/s (still under SR's bit-rate.) While Sony can leapfrog our quality for these images with their new decks that can do the 880Mb/s mode (we found it had about 3dB PSNR boost over its more common 440Mb/s mode), there are diminishing benefits for a disk-based workflow. Uncompressed DPX 1080p sequences are about 1520Mb/s, so 880Mb/s is less than 2:1, and at that rate uncompressed DPX is likely a better disk based workflow. CineForm intends to target 4:1 compression or more (usually averaging 6:1) where the visual quality is the same as uncompressed (even with extreme post manipulation) but the data rates are low enough to significantly reduce the cost of storage and the bandwidth required for capture and playback.

All of this is designed to continue to improve image quality in ways you can't even see. :)

Monday, February 04, 2008

Demosaicing (a.k.a De-bayering)

When shooting with a RAW single sensor camera like Arri D-20, Red One or SI-2K, the final output quality is impacted significantly by the demosaic algorithm, possibly as much as any compression applied. The demosaic algorithm is the process that converts the one value per pixel of a raw bayer sensor to the more useful three color primaries per pixel like RGB (see wikipedia article on demosaicing.) The weird thing about demosaicing, is there is no single correct way to do it, as all the algorithms must interpolate the missing values, one guess can be as good as another. Of course some look more natural than others, and that gets tricky (i.e. impossible) to mathematically prove -- there are some parallels to building a visually lossless compression. Originally CineForm hasn't really got heavily involved in choosing one demosaic algorithm, as there are now hundreds of different algorithms, all pretty much claiming some superiority over the rest. Instead we have offered a plug-in structure that allows third parties to add any demosaic filter they wish. Silicon Imaging did this and believe Weisscam looked into doing the same. But of course CineForm is judged only by the filters we offer as standard, as we saw when we demonstrated a standalone Red One file convertor (now on hold to meet agreement terms with Red.) Graeme Nattress of Red, skillfully skewered the faults in the default demosaic we were using in that beta product in a post. Of the beauty of competitive development is we were able to address the issues the next day. All this brought my attention back to having a good range of demosacic filters for the user to select from, rather then each camera vendor offering their own.

The latest builds on all our 2K+ products, there are now twice the number of demosaic filters offered, allowing you to tweak you image look as you need. See the subtle effects of each demosaic in closup on this image :
Click each of the check boxes to see the image with the various demosaicing filters applied.

5x5 Enhanced
CF Advanced Smooth
CF Advanced Detail 1
CF Advanced Detail 2
CF Advanced Detail 3

When I find the time, I will shoot something that will really push the demosaic.

While I did this test images under Premiere Pro on Windows, we now have the same control of the demosaic filters on the Mac. For more information please see our new CineForm RAW on Mac tech note.