Thursday, March 15, 2007

Green Screen Challenge

Yes, I'm taking this post's title from The Colbert Report, but I have a different challenge in mind here. As mentioned in my last post, we have just conducted a Viper shoot to test out various 4:4:4 compression solutions in comparison to uncompressed capture. I'm in the middle of conducting some very clinical PSNR tests to measure image distortion, which is not always the best way to measure the impact of compression. So I'm seeking your feedback in this "green screen challenge" to determine how much compression is a factor in creating a nice key. There are four images created from a single instant captured live from the Viper in Filmstream mode, yet each DPX file has undergone different compression processing or it is the pure uncompressed signal.

Here are the four 1920x1080 4:4:4 DPX files (8MB each.)
viper1.dpx
viper2.dpx
viper3.dpx
viper4.dpx

As we were short on "talent," I'm the one in front of the screen. To make the key more interesting, the Viper was shot in 2.37:1 mode in its RAW Filmstream format for enhanced dynamic range. The image is untouched, which is why it has a green cast, typical of the Filmstream output. I'm also wearing a greenish shirt--Scott Billups said that was a good thing (what do I know about keying?)

The fun part of the challenge is that it is a blind test. I'm not going to tell you which has had no compression, HDCAM-SR compression, stock CineForm compression or a compressor optimized for keying. I'm not looking to see if you can tell compressed from uncompressed, rather which is the easiest to key, or if the compression is impacting the key at all. Please send feedback to me at cineform.com (my email is "dnewman" at that domain) in one of the following forms :

a) Compression sucks, and image X is obviously the uncompressed source and easiest to key.
b) Keying image A was easiest, followed by B, C then D.
c) I can't tell the difference; are you sure these aren't all uncompressed?
d) I tried one and here is the funny scene I made.

I will post results with the rest of our findings.

---

Addition March 16-2007 : it seems that many of you do want to guess which image is which, so email me your guess on that too. So far no one has it correct. Still interested in your keying experiments.

Addition March 18-2007 : So far only 25% of contestants have selected the uncompressed file correctly, with four choices, that is as good as random guessing. ;) Please, I need more tries emailed to me.

Addition March 24-2007 : The statistical deadlock remains. Only a quarter of entrants have selected the uncompressed source from the four images. Still a few more days to submit your analysis.

15 comments:

Anonymous said...

http://www.thomsongrassvalley.com/docs/Miscellaneous/cameras/viper/logcurves.xls
in case you need it

Salah Baker said...
This comment has been removed by a blog administrator.
David said...

Salah Baker,

thanks for posting your guesses, I try and restore it when the answers are revealed. However I would prefer emails so that others aren't influenced.

Salah Baker said...

David, I tried to drum up some interest by posting on vftalk and xlcinema also. Lets hope they guess too.

David said...

Thanks, particularly for two forums I haven't spend time on before.

Hugh Macdonald said...

What's your e-mail address? I can't find it.... Although I may be being a muppet....

Cheers!

--
Hugh Macdonald
VFXTalk.com
Rainmaker

David said...

hugh,
my email is in the post, sorry it was hard to read (trying not to add more spam to my account.)

"Please send feedback to me at cineform.com (my email is "dnewman" at that domain)"

Geoff Boyle said...

Great idea, maybe I can cross post the test on CML?

Anyway I'll download the images and guess when I get back to the UK

Geoff

David said...

Geoff,

Yes please. You are welcome to post these or any of the images we gathered. Let me know what you need.

Hugh Macdonald said...

David: Thanks - I missed that - I just skimmed the post looking for a link!

paddy eason said...

I can see some difference between the images, but nothing that significant. None seem particularly cleaner or to contain more information than any other. The difference I see is mainly in the grain/noise - as one would expect. Does more noise equal less compression? Probably. Compression analysis tests I have done previously with HD images (from the Arri D20 - straight to disk versus 2 different types of compressed HD on tape) showed that the differences are mainly in the least significant bit - a bit of noise/grain is smoothed off. Nothing that relates to true image detail.

Anyway - with regard to keying with this stuff, my initial reaction is "YUK".

A straight key, using Keylight in Shake, sampling the green and then just running a key will usually show you everything you need to know. With a really well-lit, good quality green or bluescreen, that is literally all you need to do - plug in your fg and bg, sample the green, and you have a comp. With a poorly lit screen, on a poor choice of filmstock/camera, there's an additional 3 weeks of screwing around to get any kind of acceptable key.

So where on that continuum does this Viper-originated material lie? I'd say somewhere in the middle. The grain is pretty nasty-looking - leaving the comp with a peppered look. Sure - it's garbage matteable, but you will be fighting grainy boiling edges a lot with this material. The low saturation of the greenscreen isn't helping - it's not lit with filtered lights, I would guess. But in that sense, it's a fair real-world test, more or less what you might get with an exterior shot lit by daylight.

So - to sum up... the differences between the kinds of compression don't seem to be all that significant, though a wider range of test material might start to reveal something more. The camera, on the other hand, creates noise that may well be quite problematic for keying with not-very-saturated greenscreens. And, judging from the very nasty blue channel (where compression seems to be showing more of an effect) I'd be very worried about keying bluescreens.

David said...

Paddy,

Keying directly from the Viper Filmstream curved data my not be the way to go -- I would like to here from the pros about this, what pre-processing step are due to simplify a Viper Filmstream key?

Regarding "...very nasty blue channel (where compression seems to be showing more of an effect)..." if you look at the analysis a few posts later, there is no additional distortion on the blue channel -- compression is not an issue. The filmstream curve will gain up the shadow detail/noise, this is what you are seeing. Reserving the FS curve is a likely step needed.

paddy eason said...

"Keying directly from the Viper Filmstream curved data my not be the way to go"

.. absolutely. I would not generally want to do any image processing in log. I lived all through the log wars, and have no wish to go back.

So I did a homebaked removal-of-the-green-offset-and convert-to-linear, just making things look decent by eye (skin tones, greys, picture of the babes looking clean etc). I can't find any clear explanation of the "right" way to do this. Interestingly, it seemed I had to increase the blue quite a bit to make things look in the ballpark. Was this material shot with tunsten light? How does the Viper deal with different white balances?

Not sure what you are saying regarding the blue channel. If I just view a single colour channel, and flick between any of the 4 images, I see a slight difference at the grain level. In the blue channel, which is most noisy anyway, I see a much greater difference than in the red or green. Maybe it's because it's noisier anyway...

John said...

I played around with viper1.dpx a bit in photoshop. I expanded to 2560x1080 for correct 2.37:1 aspect ratio with squre pixels, color-corrected assuming the surf in the picture you're holding is supposed to be white, and put in an overall contrast curve to give subjectively reasonable skintones and overall contrast range.

Having done that, the noise in the red and especially blue channel is pretty bad IMHO. Selecting a small uniform area in the green background (selected area shown here:
http://www.bealecorner.com/D30/misc/viper1-CFtest-crop.jpg ) the Photoshop histogram tool reports a mean & standard deviation of
Red: 89.65 mean 5.16 std.dev
Grn: 151.05 mean 3.09 std.dev
Blu: 82.21 mean 10.27 std.dev

Is this typical of the Viper output? I was also interested to see the specular highlight on your wedding ring goes to 255,255,255 but a few pixels to the left there is a small region which goes almost completely black. I assume this is a sharpening artifact in the camera (?)

David said...

The Viper Filmstream curve will amplify shadow noise and detail, so you may need to take that curve into account. This was my first Viper shoot so I can't comment if the imagery is typical, however we did have George Palmer operating the camera -- he know's what he is doing. The green channel was twice the amplitude of red and blue, which means that after white balance your noise levels on those channels will increase. Many are choosing to shoot the Viper with a magenta filter to normalize the channel sensitivity. The black dot is in the uncompress capture (Viper3.dpx) also, so it must be from the camera.