When is camera Raw, Raw?

I posted this in response to recent thread on CML, but seems it might be of interest to others, so it is reproduced here:

----

Whether RAW is from a CFA Bayer sensor, vertical striped sensor, or three chip source, there is no desirable true sensor RAW, as all camera manufacturers do (or should do) some pre-processing. Direct sensor data is not pretty; you want black balancing, fixed pattern noise (FPN) removal, non-linearity fixes, all applied before "RAW" is recorded, as these are hard to correct for in post. Dual Link HDSDI has its RAW data log encoded (lossy) and packed into 10-bit, yet it is still "RAW" to the workflow, and significantly better than if it was left in linear and truncated to 10-bit. Whether the pre-processing is FPN removal or 10-bit log encoding, these are only to make the workflow easier, to help deliver an image with the greatest creative potential into post.

Part of the thread discussed whether compressed RAW is RAW at all. My company kind of invented the field of compressed RAW, and first publicly launched this as the compression within SI-2K. I argue that compressed RAW is RAW, given that all pre-processing in camera is only to make the image more usable in the workflow; so as long as that processing doesn't reduce the creative potential downstream, it is still RAW. Compression is only another pre-processing step to help the acquisition and post workflow. We see Red One's success because of its compression, not in spite of it. Can compression be used without "reducing the creative potential?" Absolutely, and the 2009 Oscars helped prove that. On a technical level, pre-processed sensor linear to 12-bit log compressed at 4:1 (average SI-2K compression level) is potentially less damaging than linear to log 10-bit for DPX storage.

As for comparing uncompressed to compressed, that is happening everyday for many SI-2K users--shoot a detailed scene in the FS2 mode on the full body camera, and it will occasionally leave some frames uncompressed, every 4 to 8 frames or so. This is not for quality reasons; rather it helps manage computer resources for a CPU limited device (all the compression is happening in software.) The compression is so lite, it is impossible to tell whether you are decoding a compressed or uncompressed frame. Finally, when you consider the CFA Bayer style RAW, which is not a usable image without a demosaic (the process of guessing the missing chroma values, which has no true/ideal solution,) compression is even less of a factor. We have customers converting uncompressed Phantom HD to CineForm RAW, as they prefer the demosaicing options (currently 9 of them.) So compression is not impacting the quality anywhere near as much as the demosaic filter, which most post workflows simply accept as if it were nothing more than a format conversion.

When all manufacturers use the term RAW, that simply means they have made their best efforts to apply no creative image development in camera -- seems like a reasonable definition to me. We only now need to compare the results, not the in camera process.