When uncompressed 10-bit is not 10-bit.

In our HDCAM-SR vs CineForm 444 vs Uncompressed experiment and Green Screen Challenge (still running,) I encountered a problem in reading and writing reliable 10-bit uncompressed masters. This can be an issue for anyone working in deeper than 8-bit media on the PC; if I have your attention, please read on.

One part of the experiment used StEM footage played out uncompressed from a Blackmagic dual link card into an AJA Xena 2K (running in a Wafian DDR for CineForm compression) and simultaneously into the HDCAM-SR deck. This way we can compare a controllable uncompressed source to the two compression systems. The issue came from creating an uncompressed 10-bit reel to play out to the compressors. My source footage was the large 4096x1714 16-bit TIFF files, so they need to be resized. Within After Effects Pro 7.0 in 16-bit mode, I scaled the image to 1920x1080, then attempted to output via the "Blackmagic 10-bit 4:4:4 Codec" using the render queue. As I know a little about the Video for Windows interface used, I felt compelled to look at the data it created; my fears were realized, the output was only 8-bit data, shifted up two bits with zero in the least significant bit locations of each channel. I had only succeeded in creating 8-bit file that was 20% bigger, but still only 8-bit precision. It turns out AE can only read and write 10-bit data as 8-bit through the old VfW interface, not really a bug as that interface doesn't support deeper I/O, the problem is the name of the compressor may lead you to the wrong conclusion -- causing you to truncate your data. Watch out for this.

The solution is to use DPX or any other deep format like TIFF (or CineForm) for I/O within AE -- anything that has the trillians of color option. However, I actually needed a Blackmagic 10-bit AVI to simplify the playback (with some effort I may have worked out how to play DPXs out the Blackmagic card, but my field is compression, this uncompressed stuff is a pain.) So created a little tool for "pushing" a series of DPX Files into a single Blackmagic AVI without using the official Blackmagic VfW codec, so I could have 10-bit rather than 8-bit data in my source AVI. This created a second problem, I had source data that was truely 10-bit and now captured data that was truely 10-bit, but AE still uses the old Blackmagic codec which truncates everything to 8-bit (without warning sirens telling you this is happening.) When I each the point when things that really annoy, it paves the way for new features for our customers. To simplify my testing I have now built in all the common uncompressed AVI formats I could find into the 16/32-bit float CineForm AVI importer. Now even those blackmagic 10-bit AVIs are actually 10-bits within AE, for those already using CineForm Prospect HD or 2K (this feature will be out in the next build.) If you're not using CineForm, Mike Kanfer of Adobe mentioned there is an external tool for converting uncompressed 10-bit AVIs into DPX files (if you know what that tool is, please put it in the comments -- thanks.)

update 2007-03-21 : The AJA guys assure me that their plugins address this issue by bypassing the Video for Windows interface. I don't use their retail software (I prefer our own CineForm implementation of AJA support,) but it is nice to know.

Green Screen Challenge

Yes, I'm taking this post's title from The Colbert Report, but I have a different challenge in mind here. As mentioned in my last post, we have just conducted a Viper shoot to test out various 4:4:4 compression solutions in comparison to uncompressed capture. I'm in the middle of conducting some very clinical PSNR tests to measure image distortion, which is not always the best way to measure the impact of compression. So I'm seeking your feedback in this "green screen challenge" to determine how much compression is a factor in creating a nice key. There are four images created from a single instant captured live from the Viper in Filmstream mode, yet each DPX file has undergone different compression processing or it is the pure uncompressed signal.

Here are the four 1920x1080 4:4:4 DPX files (8MB each.)
viper1.dpx
viper2.dpx
viper3.dpx
viper4.dpx

As we were short on "talent," I'm the one in front of the screen. To make the key more interesting, the Viper was shot in 2.37:1 mode in its RAW Filmstream format for enhanced dynamic range. The image is untouched, which is why it has a green cast, typical of the Filmstream output. I'm also wearing a greenish shirt--Scott Billups said that was a good thing (what do I know about keying?)

The fun part of the challenge is that it is a blind test. I'm not going to tell you which has had no compression, HDCAM-SR compression, stock CineForm compression or a compressor optimized for keying. I'm not looking to see if you can tell compressed from uncompressed, rather which is the easiest to key, or if the compression is impacting the key at all. Please send feedback to me at cineform.com (my email is "dnewman" at that domain) in one of the following forms :

a) Compression sucks, and image X is obviously the uncompressed source and easiest to key.
b) Keying image A was easiest, followed by B, C then D.
c) I can't tell the difference; are you sure these aren't all uncompressed?
d) I tried one and here is the funny scene I made.

I will post results with the rest of our findings.

---

Addition March 16-2007 : it seems that many of you do want to guess which image is which, so email me your guess on that too. So far no one has it correct. Still interested in your keying experiments.

Addition March 18-2007 : So far only 25% of contestants have selected the uncompressed file correctly, with four choices, that is as good as random guessing. ;) Please, I need more tries emailed to me.

Addition March 24-2007 : The statistical deadlock remains. Only a quarter of entrants have selected the uncompressed source from the four images. Still a few more days to submit your analysis.

CineForm vs HDCAM-SR

On Tuesday, March 13 we had an interesting shoot-out: CineForm compression vs the highest-end tape-based acquisition and mastering format, Sony's HDCAM-SR. CineForm is always striving to improve our image quality, adding 4:4:4 encoding to existing wavelet codec design, but frankly I didn't know where we stood compared with Sony SQ 440Mb/s format (and the new HQ at the crazy 880Mb/s too.) Did we even have the right to be in the same room with this gear? Fortunately, others seemed to think we might be worthy. So a few weeks back Mark Chiolis of Thompson Grass Valley arranged for some time with a Viper FilmStream™ camera at Plus 8 Digital with the skills of George Palmer (digital imaging technician or DIT) to operate this fancy gear with the SRW1. Jeff Youel at Wafian and myself worked against the clock to get the new CineForm 4:4:4 codec running on a tweaked out Wafain HR-1 (guess this will become the HR-2.) With very little time for thorough testing, the pressure was on.

Jump ahead two weeks, and I find myself eating pizza while having great discussions on the color science of digital cinema cameras, really geeking out with Scott Billups, Mike Kanfer of Adobe, Mark Chiolis, Jeff Youel, David Taylor (CineForm) and George Palmer. Mark, thank you again for arranging the shoot (and the pizza.) The relaxed atmosphere was helped by the shoot going so well; we only had some minor setup glitches getting dual link HDSDI to pass through a Decklink card. I will be documenting more on this shoot--how the tests were performed, and of course the results -- but before that happens there is a lot of data to collate and analyze (which should be very interesting.)

CineForm vs MJPEG

I don't have time for complete this comparison myself, but Richard Leadbetter of Digital Foundry did a great write up on his findings when using the Blackmagic MJPEG codec and CineForm codec for producing HD video game promo reels (Richard's work is very cool.) I wasn't surprised that our wavelet compression would outperform MJPEG in general, but I was pleased to hear it does well even in the low quality setting. Please read his finding over at DVInfo.net.

HDV via HDMI

On the drive into the office this morning I was listening to the Pixelcorps's This Week in Media podcast (this is an excellent podcast to get you fill for many things production related.) The first segment of Episode 42 (March 4th, 2007) discusses the advantages of HDMI out from inexpensive HDV cameras (like the excellent Sony V1U and the upcoming Canon HV20.) They were discussing their recent experiments with Sony V1, although they quote nearly every spec. wrong (such as HDV compression is 4:2:0 not 4:1:1 and the V1 has three 960x1080 sensors not 1440x1080 like the Canon XL-H1), but none of the specs matter for their findings. These guys were judging the results on the HDMI output versus the tape compressed HDV signal, and discovered how much better the live HDMI (4:2:2) feed is for color correction and keying etc. Take a listen.

Now for the plug : for those with a Blackmagic Intensity HDMI card, what this podcast describes is much easier to do with all of the CineForm tools (which added HDMI support,) capturing live 4:2:2 directly in a CineForm AVI, without have to deal with the bandwidth of uncompressed. The other thing the podcast got wrong was that only Sony Vegas can handle the 24p data from the V1U, again all the CineForm tools supported that the camera 24p format the day the camera shipped. So whether you shoot to tape / disk based HDV or live HDMI capture, we've been ready for a while (see the recent press release for more details.) If only those Pixelcorps guys would add some more PCs to their productions tools.

CineForm at Sundance

Just a quick note that I will at Sundance this year (my first time) for the opening weekend. I will be doing a couple of presentations with Jacob Rodenberg titled "Hollywood Workflows on an Indie Budget: High def & 2k Workflows." This is part of Adobe and HP's presentation series at the New Frontier Theater on Main (333 Main Street, Park City, UT.) The seasons will be at Friday 1/19: 3pm-5pm and Monday 1/22: 4pm-6pm. Jacob will be showing work he has done with CineForm Intermediate for feature film work, on HD and 2K projects. I will be demostrating some next generation workflows that handle RAW image data from large sensor for camera like SI-2K and Dalsa Origin. If you are in Park City, please drop by.

CineForm RAW Metadata

A month later, time for another update. Most of my time has been focused preparing for the full release for the SI-2K digital cinema camera (now finally in production, with the first of the next generation units shipping January -- yes later than we had hoped.) The software support RAW cameras in general has been evolving, but has remained mostly under wraps (first public showing will be at Sundance and then at Hollywood Post Alliance). But a short sneak peak to what is coming I encoding this demo showing some of the CineForm RAW metadata* controls.

* - Note: Metadata is really a wrong word. Traditional metadata like timecode, logging text, userbit, shot notes extra is passive in nature. This is active metadata that works as a non-destructive operator on the moving image in real-time.

So this little demo show the tip of the iceberg for some of the "look" controls that the media can handle. Through post controls for standard camera settings like white-balance, color matrix, and defect pixel coordinates, to full 3D LUT selection to model film stocks and look style -- all with 32-bit float precision.
Short clip here : DemoCineFormRAW-WIP.wmv

The really cool stuff is under global controls. ;)