Sunday, April 16, 2006

CineForm RAW™ -- this skunkworks project becomes a reality.

About two years ago I discovered an interesting group of people over in's Alternative Imagining Methods forum. This is a wacky group of individuals set on building lens adaptors out of CD player motors, and their own HD cameras out of industrial sensors. To me this was a lot of fun, so I became a regular reader. I started noticing that many of these camera "inventors" where stuck in the mindset that uncompressed direct-to-disk was the way they were to create the ultimate cinematic experience. This was the common thinking at the time, with potential projects like Kinetta giving many the idea that anyone can do it (two years later Kinetta still hasn't released their camera.) I saw this "alternative imagining" world differently; why introduce a new camera that makes the post workflow harder? Uncompressed wasn't the answer, better compression was.

It was at this point I started to modify the CineForm wavelet codec to handle raw sensor data for this large single-sensor camera design. I knew this design would be the direction that low cost 35mm replacement cameras where likely to take. Today's high-end film replacement cameras like the Dalsa Origin, Genesis and the Arriflex D20 all use large single-sensor designs. Yet these ultra expensive cameras where not the low cost camera that I imagined for this new codec - I was thinking more Indie than Hollywood. Without a proven market for a new compression technology I pretty much worked on this project on my own time. I got some great help from very early adopters like Obin Olson (thanks again) who sent me some raw sequences from his camera experiments that enabled me to test the new codec and backend workflow idea without having a working camera at home. Initially these early images were 720p24, pretty straightforward for our real-time compressor on a desktop PC; the trick was to encode higher resolutions or higher frame rates on an embedded CPU (after all a P4 is a power hog.)

The first higher-resolution images came from Silicon Imaging (developer of the new SI-1920HDVR camera.) They loaned me (well really CineForm, but I took the camera home) an early 2M-3MPixel camera that could do 24p over a bus format called CameraLink. So I purchased a couple of cheap lenses off eBay and began coding a direct real-time compressor of the raw sensor data. Unfortunately my patience didn't mix well with the frame grabber SDK, so I end up getting the best images from Silicon Imaging directly, but the concept was now proven, and the SI relationship established.

Over the next year there were camera upgrades and the switch to Gigabit from CameraLink (thank you SI -- much nicer) but no standalone camera product was forming yet. Remember this was the time Prospect HD was in beta, while on-lining Dust to Glory, so the CineForm RAW project was still very much on the back-burner. Then at NAB 2005 I meet up with one of the Silicon Imaging team (Jason) for the very first time, and also met part of the Arriflex D20 engineering team. So, for a tradeshow demonstrating our existing products - HDV acceleration and the full launch of Prospect HD - there was lot of discussion about what CineForm could do for these next generation cameras. The Arri guys were very cool and sent me some pre-release raw data from the D20 camera head -- WOW! this image is so clean. The image is so damn clean the bit-rates of the compressed output were close to a CineForm 1920x1080 HD 10-bit data rate, yet the image was 3 times larger at 2880x2160. We proved we could encode this 6MPixel image on a dual proc. workstation at 24P, but unfortunately I couldn't get much more out of Arri. This sent me back to my original market problem, how to produce a successful software product if the only camera that can utilize it costs $100K+ (i.e. a rental model.) For a good software product you need volume - that means I needed a low-cost camera, something filmmakers could actually buy.

Now, having met one the Silicon Imaging team, the project really took form after NAB 2005, this would be the camera to launch a better filmmaker workflow. The development continued on both ends (CineForm & SI) over the last year, preparing for a full functional product to be shown at NAB 2006. The software is ready - Prospect HD has just been upgraded to support the CineForm RAW workflow, and the camera is ready for orders, shipping in only a few months.
That is a little of the history as to how we got here. If you haven't looked into it already, here are the direct links that explain why the SI-1920HDVR is such a cool and unique camera, and how CineForm RAW, makes post-production of raw image data completely seamless while enhancing the final results. We hope that CineForm RAW encourages more cameras to follow, from SI and others. Come check out the camera and Prospect HD (RAW) within the Adobe booth (SL3732 ) at NAB. (Hopefully we’ll have our own booth at a future NAB.)
Relevant links:


Anonymous said...

I hope it wasn't too wacky in alternative imaging ;) Most of use were pretty forthright.

It isn't too difficult to do a uncompressed RAW camera, it is just that most lack the experience and ability, while people like yourself, Steve (SI) and Rai (the Drake camera) quickly developed full systems.

I am thankful and grateful for your efforts, and it is good to see it out.

For volume, I wish to discuss with you about a low end $1K 720p Indie camera.

Wayne Morellini
DVinfo Digital Cinema Camera projects participant.

David said...

Nothing wrong with being wacky, that is how new ideas come to exist. For your volume product wish, you will need to talk to the Silicon Imaging guys -- we can easy make compression that handles 720p to 4k and beyond.

Anonymous said...

I meant your volume wish for cineform (as well as the other).

Wacky, no, honestly, I see a lot of people there, particularly the core people that develop things or figure things out, as honestly seriously minded, maybe not as much as an engineers though.

This camera is the first step, but I think every serious camera should have something at least as good as cineform, and pure consumer grade something in between. People would still argue about the superior performance of lossless/12bit/SHD cameras and pay premium pricing for those cameras, but low end footage will look less terrible on large screen TV's. The market will still have a tier structure for existing camera makers.

Cineform Lossless:
But something I have not got around to check out, what is the actual determination of visually lossless. I take it that it reproduces the image in regard to the human visual range, so that we perceive it to be lossless, even though it doesn't completely reproduce the original information?

I'm curious, because debayering depends on the correspondence between pixels in a group being preserved.



David said...

What is visually lossless is different for every viewer, yet the average home viewer will not be able to tell the source from compressed result at about 35-40dB PSNR (see CineForm quality pages for a definition of PSNR.) I frozen image, artifacts can be view by some between 38-43dB (note these number are off the top of head from head, hopefully somewhat based on all the papers I've read on the subject.) Image professionals can add another 3-5dB -- after all we know what to look for. For the CineForm encoding we aim of over 60dB first generation and 70-80dB per generation after that (or better.) That is why we are comfortable with the visually lossless definition.

As for "debayering depends on the correspondence between pixels in a group being preserved", that is why this is a new compression scheme. :)

Anonymous said...

Would be interesting to know how you do it. Suppose it is already somewhere on the site ;) .

Anonymous said...

CineForm Raw is the most inovative product I have seen at NAB in many years. Yes the next year will be fun.