New Canon 5D Mk-II support with Neo Scene 1.1.2

Another Canon 5D Mk-II post, you would think I owned one, I don't -- Canon can you help me solve that? It is the camera's behavior that has caught my interest, plus the large video post market it is generated.

In the previous two posts I determined that the Canon 5D is using the full YUV range all 0 to 255 values. This can cause headaches throughout post as cameras typically use a luma range of 16 to 235, so it requires the operator know there is image data in the blacks and clipped highlights, and that the NLE can handle that information to restore it (many can't.) However, using the the full range offers the best tonal range for a limited 8-bit compression, and we don't want to lose that by crushing the range into the regular 16-235 as the CoreAVC decoder does with a user option (see the resulting spikey histogram.)

Neo Scene has a 10-bit solution. Reducing the 8-bit data from 255 levels to only 220 levels (with 16-219) something will always be lost, so we bump the data to 10-bit first, a 0 to 1023 range, then reduce that data to the 10-bit YUV standard range of 64-940, which has the precision to maintain all the source levels without introducing image banding. It is this 10-bit range-corrected data that we compress to CineForm from the Canon 5D Mk-II. By normalizing the Canon 5D data, the full dynamic range is now preserved in all editing and viewing environments.

We will be adding this feature to all our products over the next few weeks.

Correction: Canon 5D is fine, tools are wrong.

In my last post I discovered that using different decoders produce different results on Canon 5D footage, some more favorable than others, but I was wrong in my conclusion that something was being misinterpreted in the Canon 5D bit-stream. The Canon 5D video bit stream is fine, in fact is a little better that, as it making the best use of its heavily compressed 8-bit MOV (of course a less compressed 10-bit would have been nice.)

I concluded this by directly examining the luma outputs of two popular AVC/H.264 decoders: MainConcept and CoreAVC (both tested by snooping the YUV output via DirectShow.) Those users with CoreAVC were finding the output data easy to color correct, seeing the full dynamic range, whereas many MainConcept users (standard within Adobe and Sony NLEs) much less so. See these two histograms below explain this issue.

First CoreAVC. The luma data ranged between the broadcast standards of 16 to 235, not a single value existed out of this range (which is odd for most YUV source.) However, this is why CoreAVC users are having no big issues with dynamic range, even when using 8-bit RGB decodes, which will map the 16-235 luma to 0-255 in RGB -- they get everything. However, the spikes in the histogram are a tell-tale sign that something else is wrong with this picture.

Compare this to the Mainconcept decoder's output (same output as QT and other common AVC decoders), the completely smooth histogram over the full range from 0 to 255, shows that CoreAVC is in fact post compressing the data into the broadcast 16-235 range, and that is not how the Canon 5D compressed the image. This is range compression is reducing the tonal range, and the remapping of 8-bit values into a smaller 8-bit range can introduce banding. Think of a sky gradient with values 10,11,12,13,14,15 being compressed to 10,11,12,(12),13,14 -- the flat spot can add to visible contouring for which 8-bit signal are already prone.

So if the default decoders are correct, but dynamic range is truncated in the NLE, what is the solution? Stop using 8-bit RGB tools and codecs helps. The problem only arises when remapping the 8-bit YUV data into 8-bit limited computer graphics RGB space (Studio RGB would be fine *.) If your application and intermediate codec can decode to YUV, you are in fine shape, the problem is none of the common NLEs are doing this with native AVC/H.264 decoding. Intermediate codecs like ProRES and CineForm, support full range YUV (float) precising within FCP and Premiere Pro, so those are good routes to go. So you need to convert from the Canon MOV outside of NLE, using tools that do all processing in YUV (image remains unclipped.) Stu Maschwitz has posted that Apple Color will support the full range, and I can happily report that all CineForm products are handling this correctly (even for ProRes output if you are on a Mac with NEO HD/4K.) But even a ProRes file will clip if 8-bit RGB tools are used.

CineForm a cool solution for this using NEO HD/4K for the Mac and Prospect HD/4K on the PC, even if you have to use an 8-bit RGB clipping tools like Compressor, QT Player or VirtualDub, etc. The full range 0-255 YUV is preserve in the CineForm MOV or AVI, but using Active Metadata you can choose to decode the smaller range when needed. As this is all happening with high precision math in the decoder, we can safely mapping the full range YUV to full ragne RGB, without truncation or introducing banding. Normally we using Active Metadata for attaching look to the decode stream, but it can also use it to reformat the data to fix the needs of your tools.

The first RGB parade is the untouched CineForm encode from the MainConcept AVC decoder output, it appears that the shadows and highlight have been crushed. While I could place a 32-bit Levels filter to correct it on the timeline when work with the CineForm encoded file, correcting it within Active Metadata will allow this clip to also be corrected in 8-bit tools.
The second RGB parade is corrected through Active Metadata alone. I simply bumped up the black level and reduced the gain inside the Active Metadata control panel (Mac or PC) to reveal all the missing data. Active Metadata uses the clip's GUID (Global Unique IDentifier) to set new decoding rules, for any application use that clip with the computer system. Expect there to be more problems solved using Active Metadata with our next major software release. So much cool stuff coming -- need more software engineers.

* - At the time of the post Apple released a patch to QuickTime (7.6) that addresses some of the this. Now under QT 7.6 the 5D YUV data is decoded using Studio RGB, which places luma 0 (not 16) at RGB 0,0,0, allowing RGB to have the same gamut of the YUV source. I imagine the 5D is flags the data to do this and various AVC decoders are either ignoring or misusing the flag.

----

01/22/2009 It turns out the CoreAVC control has an auto mode, trying to guess the correct decode behavior (16-235 vs 0-255). Turn this off, make it behave just like all other. So all these decoders work if you manage the outputs correctly.

Full dynamic range video from Canon 5D MkII.

I been seeing many posts on how crushed the black levels and highlights are with the Canon 5D video mode. Stu Maschwitz has a great post on how to tweak the camera to improve the situation, but also points out the default presentation FCP is also wrong. It turns out that this presentation fault is in all popular editing packages on Mac and PC. They could possibly share the same AVC decoder, but I have a different theory below.

Using a 5D MOV video originally posted DPReview.com, here you can see the clipping that occurs within tools like Vegas, Premiere Pro and FCP when directly opening the MOV files. This zoomed in section shows much of the tree details are crushed into the blacks, when compared to the same sequence converted to an AVI using HDLink with CineForm NEO HD.

After some image enhancement, the black truncation becomes more apparent. These images where manipulated with a 32-bit levels filter (it is not truncating the data) so we see how much data is being lost, a failure to see "the trees from the forest" type of problem.

One common reason for all this error, is the camera H.264 I-frame compression is using 4:2:0 YUV, where display black is 16 and white is 235 out of a 0-255 luma range. If the NLE took this YUV data directly, everything should be fine, but instead is likely going through series YUV to RGB transforms that are truncating the data. Most NLEs extracting RGB from YUV sources map the 16-235 YUV to 0 to 255 RGB, truncating data in the supers, yet it turns out the Canon 5D doesn't even use much in super range (blacks between 0-15 and whites up to 255) as this non-truncated luma histogram shows: It seems that the 5D is limiting the output to broadcast standards (yet another thing the 5D does wrong for video heading for post.) Whereas most video cameras do use the extra range, they assume the broadcast limiting will be handled in post (as they should.) So the NLE's mapping of the 16-235 to 0-255 RGB is actually not the issue, somehow the H.264/AVC decoder used is actually mapping a range like 30-220 only into the NLE space. I have not seem such a big error before, which makes me think there an incorrect flag in the 5D bit-stream that is causing the use of the wrong output math.

Luckly within CineForm NEO HD we use direct YUV to YUV conversion wherever possible, this eliminates many trunction errors, and preserves highlight for mulitple camera sources. In current version of NEO/Prospect HD (v3.4) we don't ship an AVC decoder, so NEO HD searches out for registered decoders already installed, resulting in some users reporting this error and others not so. So the failure is AVC decoder dependent, although this may not be a bug in the decoders, as they all handle regular AVCHD sources without issue (again leading me to believe there is something wrong/different in the 5D bit-stream.) The one AVC decoder that produces the full range of YUV range is CoreAVC. We have been recommending it to AVCHD users as it is both fast and inexpensive (only $15), but it seems it might have extra value for those trying to get the most out of their Canon 5Ds.

In our next major release of NEO HD (and up) we intend to direct support Canon 5D, yet in the meantime for those using CineForm products I recommend you get CoreAVC, the latest version of NEO or Prospect which now favors CoreAVC over any other AVC decoder it might find. This version is currently in public beta and will likely become official in a day or so.

------
1/21/09
Update 1: The new software is now released and available for upgrade or trial directly from cineform.com.

Update 2: I have discover more on AVC Decoders issues for the Canon 5D in the next post.

CineForm's Winning Streak

Not only did Slumdog Millionaire clean up at the Golden Globes this year, Slumdog being a mostly CineForm RAW acquired feature shot with the Silicon Imaging SI-2K Mini, one of CineForm's engineers just won a Audi RS4 (Audi Record the Rush Challenge) from a CineForm posted video he self produced also shot with an SI-2K and a Sony V1U.  Looking to be a good 2009 for those at CineForm.

Mastering 24p DVDs from HD using Premiere Pro.

24p DVD mastering.

First some basics, there is no true 24p mode on standard DVD, there are is only 60i encoding (and 50i for PAL.)  So all those film source are encode to DVD by adding 3:2 pulldown (or 2:2 for PAL with a 4% speed up for 24p sources.)  How this pulldown is added impacts how well your DVD presents on today's common progressive displays. 

Adding 3:2 pulldown for NTSC DVD creation has been has been tricky, seemingly there are lots of dead ends (like using Encore to encode directly to 24p -- this should work, I have made several blurry DVDs trying to use it.)  This is one of the most common support questions.  McCarthyTech has a good blog post on using AE to manually insert pulldown before encoding as 60i, and this will work everytime.  It was his post that has prompted this one, as there is an even simpler way using Premiere CS3+ directly, and if you are careful it can be even better for final presentation.  McCarthyTech's technique can be improved if we trust the MPEG2 encoder can to add the pulldown using repeat flags.  Fortunately this works correctly using the 23.976p encode mode within Premiere's Adobe Media Encoder (MPEG2-DVD preset), now we just have to watch out for other Premiere limitations.  

The advantage of pulldown that uses the MPEG2's repeat flags, this can help with quality as only fields used to construct the 24p signal are compressed, the repeat flag pad the data out to 60i. This flags also help progressive scan DVD players reconstruct the correct 24p signal more reliably. The manually created pulldown in AE works for most situations as many DVD players can use the data-rate pattern to guess the pulldown, but it is not always extracted correctly (seen as a weave pattern during motion.)  Of course there is no issue for non-progressive outputs where the display is reasonable for pulldown detection (if needed.)

The way I produced several 24p DVDs in the last weeks is to export out of Premiere as a 1920×1080 24p (23.976) master CineForm AVI, then I used VirtualDub and scale using Lanzcos 3 filter to 720×480 and export out to CineForm 444 SD.  Load the SD clip back into Premiere SD 24p preset, interpret footage back to 16×9, and export with Adobe Media Encoder to MPEG2-DVD 24p. Encore will take this file without further transcoding.  Now that seems to be an odd path and it is, plus using VirtualDub is unnecessary, but the export to 1920x1080 master first is a very important step. 

I first made the mistake of exporting my 1080p timeline directly to MPEG2-DVD, and it looks horrible.  I've made this mistake before, as you simply expect it to work (never has in Premiere up to the tested PPro v3.2), but here why it doesn't always do what you want.  When you add any spatial distorting filter (motion, blurs and sharpens, etc), you see the results previewed at 1080p, any scaling for your preview display is applied after the filter operations -- so you adjust you filter so they look correct on a 1080p source.   We you use the Adobe Media Encoder, the scaling is applied first, before any of your filters -- as a result your output doesn't look like you previewed -- spatial filter are around 2.5 times stronger than you intended.   In one of my recent projects, I was using the additional resolution of the 1080p source to reframe for a nicer DVD output, see below how much it matters in what order of operations that scale occurs (see below.)

The image on the left is soft and badly aliased, and it looks far worse in motion.  Simply exporting the timeline to 1080p first, then using that new file to export to DVD solves the problem, without ever leaving Premiere.  The VirtualDub step in my above technique can be as simply skipped, as loading the exported 1080p AVI into Premiere will use the CineForm importer's own Lanzcos 3 scaler for exactly the same results, much faster and more convient.  

What, no more Aspect HD?

After 5 years we are now retiring Aspect HD, and I say Yah!!!! David Taylor, CineForm's CEO, has post as to why on DVInfo.net, but I like this change as we no longer need to remove key features from Prospect HD to make Aspect HD (that is how it is built.) This will streamline development and testing, plus slicing off all the really good stuff always bothered me. We will still have entry level products in the NEO line (more news to come,) but today's customers are increasingly purchasing their first CineForm product at the Prospect HD level (some going directly to P4K as Red users really need a workflow.) Recently Aspect was falling into no man's land, too pricey for the hobbiest and too feature limited for many professionals. The Prospect product line now starts at $749, lowering the entry point for professionals and existing Aspect users can product jump for $199 (only around $50 more than typical upgrade between AHD versions.) This upgrade will also include the upcoming CS4 version of PHD.

Aspect and Prospect a short History.

5+ years ago when Aspect HD was first released it retailed $1199, it was the only HDV professional product of its day for post. The only HDV camera at the time was the barely HD single chip 720p JVC HD10U, so Aspect HD took in only one camera format, 720p HDV and converted into an 8-bit AVI for much easier post within Premiere 6.5 (wow, we come so far since then.) Aspect HD was upgraded to Premiere Pro 1.0 and we soon added 1080i support as Sony FX1/Z1U was coming (we have a bread-boarded prototype of the Z1 in box at the office -- somewhere.) Then HDV exploded, many great cameras, and CineForm implemented the first HDV support for Adobe into Premiere 1.5.1, and Aspect HD prices fell and the sales volumes grew. Around the same time a Prospect HD beta version was used in the post of Dust to Glory, the first film sourced (and HDCAM, DV, etc.) theatrical release ever to have had an compressed online DI workflow. Support for more and more cameras where added, XDCAM-HD, P2, AVCHD, etc. While Aspect was growing beyond it HDV roots (bumping up against it 1440x1080 8-bit limits,) and Prospect was turning in the swiss army knife for all input types with the addition of DPX sequences, XDCAM-EX, Grass Valley Infinity, SI-2K, Red, Dalsa, Phantom all with HDSDI I/O. Over the years of Prospect, feature additions includes 10-bit, 12-bit, 4:4:4, alpha channels, RAW compression, Active Metadata and much more.  So today the Prospect line fits the role for today's indie professional, with so many input formats and many possible output types, it is nice to have an intermediate that handle yours resolution, bit-depth needs in real-time, without breaking the budget, your disk system, or the PC your are try run on. Aspect HD was being left behind when compared to Prospect, so I hope many Aspect users will make the jump to Prospect, as it is a big step up.

Bits of Fun

Here are a couple of videos internally created by CineForm engineers (plus friends) just for fun. The first is this year's entry to the 48 Hour Film Project. We drew the tough genre of "Holiday Film", so it is now seasonally appropriate (we created this one back in August.)



Competition requirements: a "Holiday Film" to be written, shoot, edited and delivered within 48 hours with these elements:
1) Character: Joe or Josie Beeble -- Construction Worker
2) Prop: Tweezers
3) Line of Dialog: Hey, have you heard the news?

The second is currently a finalist on www.recordtherush.com, which is a competition entry to help one of our engineers win an Audi S4. The entry is called Visceral Thirst, so please vote for Tim (he said I can have a drive if he wins.)

Both films where shot using an SI-2K Mini, although Tim used some Sony V1U footage (all the beautiful car shoots in Visceral Thirst are SI-2K,) and both where finished in tight time-frames using Prospect 4K and Active Metadata.