Tuesday, March 20, 2007

When uncompressed 10-bit is not 10-bit.

In our HDCAM-SR vs CineForm 444 vs Uncompressed experiment and Green Screen Challenge (still running,) I encountered a problem in reading and writing reliable 10-bit uncompressed masters. This can be an issue for anyone working in deeper than 8-bit media on the PC; if I have your attention, please read on.

One part of the experiment used StEM footage played out uncompressed from a Blackmagic dual link card into an AJA Xena 2K (running in a Wafian DDR for CineForm compression) and simultaneously into the HDCAM-SR deck. This way we can compare a controllable uncompressed source to the two compression systems. The issue came from creating an uncompressed 10-bit reel to play out to the compressors. My source footage was the large 4096x1714 16-bit TIFF files, so they need to be resized. Within After Effects Pro 7.0 in 16-bit mode, I scaled the image to 1920x1080, then attempted to output via the "Blackmagic 10-bit 4:4:4 Codec" using the render queue. As I know a little about the Video for Windows interface used, I felt compelled to look at the data it created; my fears were realized, the output was only 8-bit data, shifted up two bits with zero in the least significant bit locations of each channel. I had only succeeded in creating 8-bit file that was 20% bigger, but still only 8-bit precision. It turns out AE can only read and write 10-bit data as 8-bit through the old VfW interface, not really a bug as that interface doesn't support deeper I/O, the problem is the name of the compressor may lead you to the wrong conclusion -- causing you to truncate your data. Watch out for this.

The solution is to use DPX or any other deep format like TIFF (or CineForm) for I/O within AE -- anything that has the trillians of color option. However, I actually needed a Blackmagic 10-bit AVI to simplify the playback (with some effort I may have worked out how to play DPXs out the Blackmagic card, but my field is compression, this uncompressed stuff is a pain.) So created a little tool for "pushing" a series of DPX Files into a single Blackmagic AVI without using the official Blackmagic VfW codec, so I could have 10-bit rather than 8-bit data in my source AVI. This created a second problem, I had source data that was truely 10-bit and now captured data that was truely 10-bit, but AE still uses the old Blackmagic codec which truncates everything to 8-bit (without warning sirens telling you this is happening.) When I each the point when things that really annoy, it paves the way for new features for our customers. To simplify my testing I have now built in all the common uncompressed AVI formats I could find into the 16/32-bit float CineForm AVI importer. Now even those blackmagic 10-bit AVIs are actually 10-bits within AE, for those already using CineForm Prospect HD or 2K (this feature will be out in the next build.) If you're not using CineForm, Mike Kanfer of Adobe mentioned there is an external tool for converting uncompressed 10-bit AVIs into DPX files (if you know what that tool is, please put it in the comments -- thanks.)

update 2007-03-21 : The AJA guys assure me that their plugins address this issue by bypassing the Video for Windows interface. I don't use their retail software (I prefer our own CineForm implementation of AJA support,) but it is nice to know.

6 comments:

Jason Rodriguez said...

BTW, David, I just ran some tests, and the 10-bit Blackmagic RGB QT codec version does not have this problem on the Mac . . . I set AE to 32-bit float, and generated a linear ramp. I then rendered out a 10-bit Blackmagic QT RGB file using "Millions of Colors" and "Trillions of Colors" to simulate the 8-bit and then deeper pixel output modes in AE. I brought the files back in, overlaid them over the ramp (again, in 32-bit float mode), and then set the layer styles to difference. I overlaid an Adjustment layer and used a levels filter to create a white-count. The "Trillions of colors" version had quite a bit more precision than the "Millions of colors" version, so it seems that 10-bits is making it into the 10-bit RGB quicktime codec version.

David said...

Hi Jason,

Yes, this is an AVI issue on Windows AE, not a QuickTime issue on Mac. Although now that we have been working with deep pixels format on the Mac, QuickTime does not make it much easier, trucation can still happen if you're not careful.

Joseph said...

Thanks for the update David, this answers a question I had about outputting >8bpc from fusion and bringing it into AE. I can at least put it into Cineform wavelet for final editing.

Joe C.

Mike Curtis said...

Hey David!

Some potentially helpful info for dealing with 10 bit image output in AE, I ran into similar problems:

http://www.hdforindies.com/2006/06/after-effects-stuff-workflow-tips-and.html

Also in that genre for FCP:

http://www.hdforindies.com/2007/02/ah-hell-hell-hellfcp-workflow.html

-mike

Sergio Sánchez said...

Hi David:
I was asking Jake, a Cineform support guy, about the output to 10bits in AE he told me to ask you in this blog. The thing is that im using Prospect 2k, my source material was captured from a prototype SI2K in Cineform RAW. I color corrected all the material natively in cineform using the fimscan 2 RGB444 presets for the final master.

Now, I have to do a data to film, and I have to deliver all the data as a Cineon sequence, but i know that AE only exports 8bpc.

I have Combustion as another option for the render, but i dont know how the program deals with the avi files.

What do you recommend, David?

David said...

Sergio,

I doubt Jake ment to send you here, as there is no way have a discussion. Try DVInfo.net instead. After Effects Pro is greater than 8-bit and has no issue creating 10-bit log Cineon exports for film out.