Shot Down By My Better Half.

If you have ever read this blog, you might get a laugh at my wife's skillful skewering of the crazy language I have used over the years. Please read Riding the Wavelet over on her blog littledragonfruit.com.

Metadata Matters

In recent builds of the CineForm tools and codecs, we are focusing a lot more on the robustness and a flexibility of metadata. The CineForm codec has been carrying a special class of metadata for the last 18 months; we use the term "active metadata" to describe how it differs from classic data like edge codes and copyright info. Active metadata is the mechanism that allows camera control unit functionality, such as white balance, curves, color matrix and 3D looks, to be stored non-destructively to maintain all your source image information, yet allow the playback to turn on and off or alter the active decoding/display of the metadata -- I need to do a post on this as it is very cool (or please come and see us at HD Expo (March 6) or NAB (April 14-17.)) We had traditional passive metadata for the same period of time; we just haven't started to emphasize it until now.

The CineForm structure for metadata data is little different than other streaming formats (AVI, MOV, MXF, etc), and as the metadata is embedded within each compressed video sample no matter the wrapper type. Streaming format typically attach metadata to a channel within the file wrapper, parallel to the video stream, or as a separate XML file, which is fine if you can stick with one wrapper type (AVI,MOV,MXF) and keep your files together. However, re-wrapping the stream or breaking up streams can result in the loss of this metadata. As CineForm is wrapper agnostic, we needed to store it differently so that it can't be lost. We still can use the wrapper standard metadata streams when needed, we are simply not depending on it.

As metadata is in each compressed sample, we can achieve greater flexibility as the video decoder is now aware of the metadata (not possible in parallel streams or side car files,) allowing applications that otherwise have little to no support for metadata to act on the data in new ways -- this is how our active metadata works. An example of this use is in our latest DPX to CineForm to DPX tools will completely preserve all the DPX metadata and reconstruct it when export back to DPX at a later date. We are working toward allowing passive metadata to be selected for optional burn-in display, in addition to all the search and retrieval applications of this data. Lots of cool stuff happening.

For those developers programmatically implementing CineForm compression into their tools, (Wafain, Silicon Imaging and Iridas are good examples), it is easy to directly set and control the adding of metadata. Via our SDK you get frame accuracy for metadata insertion. Yet we have added a mechanism to allow the non-programmer to add free-form metadata to any of their CineForm encodes. For Windows users, there two registry keys allowing you to add your own metadata :
HKEY_CURRENT_USER\\Software\\Cineform\\MetadataGlobal
HKEY_CURRENT_USER\\Software\\Cineform\\MetadataLocal

Here is an example of using MetadataGlobal:

----------- cut here --------------------
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Cineform\MetadataGlobal]
"Copyright Owner"="David Newman"
"Computer Used"="M90 Laptop"
"Random info"="put whatever you want in the file"
"Add Numbers"=dword:00000050
----------- cut here --------------------

and save as MyGlobalData.reg. Of course you can change and add your own text and numerical data (hex format, that dword:00000050 = 80) after the HKEY line. Double click on the .reg file to install. Now every frame of every capture, or render, with have this data embedded. I've been using this in the office to know which PC any particular clip originated.

The second registry entry, MetadataLocal, only stores changes made since the last frame's encode. This is useful for attaching slowly changing data, like GPS data, or other changing data streams that occasionally change or change on every frame. Or you have a large amount of data you only want attached occasionally, rather than store it in every frame. I have yet to do much with this feature myself, but I sure it will have some powerful applications in the future.

Update: Use MetadataGlobal for changing data if you want that data to be present in every frame. Timecode is a changing field that is stored in every frame, and therefore should be using the global class. Accurate frame based changes require SDK level access.

Of course there are many new features in the works that I can't yet disclose, but please comment on your own ideas for metadata features your like to see.

The dark side (but funny)

I don't normally post clippings, but this is so me. Try to maintain an active forum presence has its dark side.

Source XKCD, found through BoingBoing.

CineForm 444 always better than the HDCAM-SR?

Sony's HDCAM SR and CineForm 444 are very different compression technologies, one designed for tape-based existing deck control conform/master workflows and the other designed for online disk based workflows. Below I discuss how their core differences impact image quality, and why it is not so simple to state that one is better than the other for all image sources.

About this time last year, I blogged on a test we performed comparing CineForm's then new 444 mode to that of the respected de facto industry standard HDCAM SR; the results of that test are posted on CineForm's quality analysis page (and one of the graphs is repeated below.) While we never expected to exceed SR quality, we did select test material that would not favor SR, i.e. using very detailed source material, like full frame StEM footage (which is designed to trip up compression) plus live green-screen footage shot using a Thompson Viper. We informed Sony in advance that we were performing the test and that we were not intending to show up SR; rather we set out to prove that both SR and CineForm 444 are indistinguishable from the source, and this is still completely true. We also had the results independently verified.

Back then, it seems we had only slightly ruffled feathers at Sony, as Sony Broadcast CTO Hugo Gaggioni tried to find me at NAB 2007 to work out what we did, and if the test procedure was valid. I tried to follow up with Hugo but we never overlapped at that trade show. Sony and CineForm have long been good partners and we want to keep it that way; to demonstrate our long term relationship we even have the extremely rare bread-boarded Sony Z1 camera somewhere around the office. But in the months following NAB' 07 there was no more discussion on the test, well, until a few months ago. It seems the Sony had independently performed their own CineForm vs SR tests and drew mixed conclusions of "no significant difference between CineForm and SR 4:4:4 SQ in terms of PSNR." Sony went on to say that their 880/HQ mode is better still; in our tests we found HQ to be better than the SQ mode but of no significant difference when compared with CineForm highest quality mode.

How could Sony and CineForm do valid tests, yet get different results and both be correct in their findings? The fact is, you could almost draw these conclusions without doing the actual tests and here is why: Any very light compression (say 4:1) of DCT, Wavelet, long GOP or I-frame, will yield similar image quality; the real differences happen with much higher compression ratios. As compression ratios increase, I-Frame Wavelets (CineForm) will show their advantages over I-Frame DCT (SR, D5, DVCPRO), and as you compress more again the long GOP codecs like MPEG2 and H.264 start to shine over I-frame solutions. So in these tests we really weren't testing Wavelet vs DCT, but rather constant quality (CineForm Wavelet) vs constant bit-rate (SR). Tape systems like SR are always constant bit-rate, modulating the compression quality so the data rate matches that of the tape system.

(Click on image for clearer view)

So when you see a graph like this showing quality over time, there is a reverse graph that shows the CineForm data rate changing and SR's remaining constant. In this test, the average bit-rate of CineForm happens to be lower than SR, even though the quality is higher, so there was a real measured advantage to CineForm, yet there are tests that can show the reverse. When the image is simpler for compression, like a lot of sky or large regions with low noise or detail, the data rate in a constant quality codec like CineForm's will fall. In this latter example you may be comparing an 8:1 Wavelet to a 4:1 DCT, which would likely be in favor of the DCT. The quality on the Wavelet image has not changed, yet the quality of the constant bit-rate DCT has increased, making good use of the available bits.

A recent good example of this happened with a customer (who will remain nameless until we get an OK) that is scanning a classic 65mm color print for restoration that uses CineForm as the archive format. They independently did CineForm vs SR testing, and found in some frames SR had better PSNR numbers (than CineForm 444), and sent me the frames to investigate. The 65mm source produced really clean source for a 2K scan, and the images had shallow depth of field, so there was minimal foreground material in focus that would give the compressor a full work-out. So while SR was using all its estimated 440Mb/s, CineForm's data was running around 345Mb/s (Filmscan 2 444 keying mode.) For these frames SR yielded a PSNR of 57dB and CineForm measured 55dB--both pretty awesome by the way.

As CineForm is not limited to a tape transport mechanism, we decided to increase our quality for the Filmscan 2 mode in our next release (we have always called this mode "overkill" in house.) The sample images from our unnamed source now produce 58dB PSNR using CineForm 444 compression, averaging 390Mb/s (still under SR's bit-rate.) While Sony can leapfrog our quality for these images with their new decks that can do the 880Mb/s mode (we found it had about 3dB PSNR boost over its more common 440Mb/s mode), there are diminishing benefits for a disk-based workflow. Uncompressed DPX 1080p sequences are about 1520Mb/s, so 880Mb/s is less than 2:1, and at that rate uncompressed DPX is likely a better disk based workflow. CineForm intends to target 4:1 compression or more (usually averaging 6:1) where the visual quality is the same as uncompressed (even with extreme post manipulation) but the data rates are low enough to significantly reduce the cost of storage and the bandwidth required for capture and playback.

All of this is designed to continue to improve image quality in ways you can't even see. :)

Demosaicing (a.k.a De-bayering)

When shooting with a RAW single sensor camera like Arri D-20, Red One or SI-2K, the final output quality is impacted significantly by the demosaic algorithm, possibly as much as any compression applied. The demosaic algorithm is the process that converts the one value per pixel of a raw bayer sensor to the more useful three color primaries per pixel like RGB (see wikipedia article on demosaicing.) The weird thing about demosaicing, is there is no single correct way to do it, as all the algorithms must interpolate the missing values, one guess can be as good as another. Of course some look more natural than others, and that gets tricky (i.e. impossible) to mathematically prove -- there are some parallels to building a visually lossless compression. Originally CineForm hasn't really got heavily involved in choosing one demosaic algorithm, as there are now hundreds of different algorithms, all pretty much claiming some superiority over the rest. Instead we have offered a plug-in structure that allows third parties to add any demosaic filter they wish. Silicon Imaging did this and believe Weisscam looked into doing the same. But of course CineForm is judged only by the filters we offer as standard, as we saw when we demonstrated a standalone Red One file convertor (now on hold to meet agreement terms with Red.) Graeme Nattress of Red, skillfully skewered the faults in the default demosaic we were using in that beta product in a reduser.net post. Of the beauty of competitive development is we were able to address the issues the next day. All this brought my attention back to having a good range of demosacic filters for the user to select from, rather then each camera vendor offering their own.

The latest builds on all our 2K+ products, there are now twice the number of demosaic filters offered, allowing you to tweak you image look as you need. See the subtle effects of each demosaic in closup on this image :
Click each of the check boxes to see the image with the various demosaicing filters applied.


Bi-Linear
5x5 Enhanced
CF Advanced Smooth
CF Advanced Detail 1
CF Advanced Detail 2
CF Advanced Detail 3

When I find the time, I will shoot something that will really push the demosaic.

While I did this test images under Premiere Pro on Windows, we now have the same control of the demosaic filters on the Mac. For more information please see our new CineForm RAW on Mac tech note.

CineForm's Red One Footage Convertor

This post has been revised to reflect the current status.

If you haven't heard the news, here are the direct links to various online announcements : DVinfo.net, RedUser.net and the CineForm page for this new R3D file conversion utility. So I won't need to repeat myself.

On the blog, which really needs some more posts, I will focus on why we created this new tool.

As many of you are aware, CineForm has been using Wavelets to compress RAW bayer images longer than anyone. We starting in RAW encoding over four years ago now, and have to continued to refine the compression and post production work-flow through our integration within Silicon Imaging SI-2K camera and new partners like Iridas and Weisscam. Yet there are about to be more Red cameras than any other 4:4:4 HD+ cameras on the market, so there is a lot of interest at CineForm (and from many customers) to support our work-flow with the Red camera. We waited for RedCine release just like everyone else, hoping to find good support for CineForm's work-flow. Fortunately RedCine was pretty cool, as it supported 16-bit per channel color through the QuickTime API, we feared it might be only 8-bit. The problem is that it didn't work reliably for many of are customers, likely because their PC are too old and/or the graphic cards in not sufficient. But some issues where in RedCine itself, as we weren't the only compression type impacted. We do expect these glitches to be fixed at some point, but we couldn't wait.

In addition to our impatience, there was one feature missing within RedCine that we had requested nearly a year ago (we hoped to get it while their product was under development.) That was a RAW mode out of RedCine, through the Quicktime API would have been fine. The RAW mode would decompress the camera's wavelet, but not do a demosaic to RGB, which is slow and increases the data-rate 3 times without adding anything to the image. Through our own tool we can do this direct conversion from camera wavelet RAW to CineForm RAW. This is a significant saver on disk space and conversion time.

We still hope that RedCine upgrades and/or Red's upcoming SDK will allow for this type of flexibility, but as a small company it is really hard to wait for the big guys and Red is big compared to us.

Update: CineForm has agreed with Red to withhold distribution of the R3D2DPX utility we previously released until such time as Red makes its SDK available to others in the industry. Our understanding is that these "hooks" (as referred to by Red) should be available around the NAB 2008 timeframe. CineForm intends to offer compelling workflows for Red users, so all feedback is welcome.

IRIDAS and CineForm Announce Full RAW and RGB Post Solution

I never got to see this press release before it was posted, so it was nice to read it for the first time. The guys over at IRIDAS have been so supportive, we hope to continue this type of innovation through partnership.

Read the release here at Studio Daily