Wednesday, April 25, 2007

CineForm 444 vs HDCAM-SR revisited

During NAB, I had an honored visitor to the CineForm booth, Hugo Gaggiono, CTO of Sony's Broadcast Production System Division. Unfortunately, I was not there to meet him, so he spent time with Jeff Youel of Wafian discussing our recent test with HDCAM-SR, and how we were able to achieve the results we did (the reason for his visit.) I later tried to follow up with Hugo at the Sony booth, but I had no luck with paging him. Pity, as I now had data as to why CineForm did so well against his format.

That same day, I answered the challenge for an outsider (Russell Branch of InnoMedia Systems Ltd.) to evaluate the test materials comparing CineForm to HDCAM-SR. Russell had been following the discussion on the Cinematographer's Mailing List (CML) regarding this experiment, and kindly offered to perform the quality measuring tests himself. It's likely Russell had access to the OmniTech picture quality anaylsis device, and along with some help from the OmniTech guys we ran a DVD full of frames through their analysis. Their conclusion was very similar to our own, and it also showed the HDCAM-SR doing worse on the StEM, just as we had found. The main difference in this test (other than the OmniTech being much faster than my own analysis), was that the OmniTech device could show us why SR's quality dropped for StEM footage.

Often for testing CineForm I would gain up the difference (between compressed and the source) to show what has changed during the encoding. This photo is the OmniTech showing that very difference, subtracting the CineForm compressed images from the uncompressed source, revealing an effectively grey frame (no significant differences.) However, I never did this test with HDCAM-SR; after all, I can't fix any issues I might find, so the test never occurred to me. The OmniTech operator had no such issues; he didn't know what compression he was testing and was therefore completely unbiased (sorry, I didn't write down his name.) When he flipped on the differencing, there was a distinct image within the frame -- he even suspected it was HDCAM-SR based on the images revealed in the differencing. The brighter image parts of the display, indicating a larger encoding error, were the black regions of the source image. It turns out the Sony deck will not encode the very lowest values of black. I must quickly note that a standard broadcast signal will not contain these black levels and the Viper footage didn't either, and that is why SR is closer to the CineForm quality for the live footage. This black level issue will only come up for content meant for theatrical presentation--a film-out master, just like the StEM footage if designed to stress test. Now the amount of black truncation is not huge, but enough to explain the 2-3 dB drop in measured quality performance. Can anyone tell me why the whole 0 to 1023 range isn't recorded when the same range is valid over HDSDI/dual link?

While the original tests are valid, if we accounted for HDCAM-SR black levels by limiting the range of the source material, the difference would be less significant. Our original goal for these tests was to prove a software wavelet compressor can be as good as the best and most widely accepting compression solution. This we have achieved. But I will also acknowledge that with the two formats so close in quality, there will be some sequences that will favor SR over CineForm. I leave it up to others to find those sequences for their own comparisons. ;)

update April 30, 2007: I was just informed that it is a dual link HDSDI, not HDCAM-SR, that limits the valid range to only 4-1019. As we were only testing a file based workflow with CineForm 444, I re-ran the PSNR analysis only using the 4-1019 data range by clamping the extreme values. Surprisingly I found the PSNR numbers for SR only moved up a very small amount, far less than the 2-3dB I had predicted. The highly detailed StEM footage does seem to favor CineForm, likely due to our VBR nature versus the SR's CBR design, and the black levels where just a red herring.

update May 12, 2007: I have recently learned that the OminTek operator was Mike Hodson, the company founder and designer of the OmniTek PQA. So the testing was in very good hands.


Anonymous said...

Hello David,
Just a quick comment regarding the HDCAM-SR, and the tests we performed with the OmniTek PQA at NAB. The maximum video signal range that is possible with SDI signals is, as you correctly state, 4 ~ 1019. The missing codes are used as part of the H & V timing syncs. [Note that the recommended range for 'real' video signals is 64 ~ 940 (Y) and 64 ~ 960 (Cb & Cr), as specified in SMPTE 274M et al]. However, in experiments that I have performed with HDCAM-SR equipment, it appears that the maximum range that the Sony equipment allows is actually 16 ~ 1019, any values outside this range are clamped. This is what we saw at NAB: Your source material contained black levels between 4 and 15, and these were all clamped to 16 by the HDCAM-SR resulting in poorer PSNR figures.
Best wishes,
Mike Hodson
Managing Director, OmniTek

David said...

Thank you Mike,

That explains a lot. While video compliant signals will be fine, data encoded for film-out can be impacted, like the StEM footage we were testing. I'll have to compare the numbers again with a 16-1019 clamp to see how much of the 2-3dB quality difference is due to this SR limitation.