Silicon Imaging in 48 hours a Success.

News Update -- We won Best Cinematography for San Diego out of 44 entrants!
-----
Last night was our premiere screening for the 48 Hour Film Project here in San Diego. We got a very pleasing reaction for our comedic short, laughs in the correct places, and it may have even won the night if it weren't for a very funny parody mashup of the PC vs. Mac commercials with the Matrix by Genesis Films (excellent work by writer/ director J.P. Pierce.) In this post I want to talk about the making of this film as it involved some totally new aquistion and post-production workflows, all in a high stress situation -- namely using a prototype camera (Silicon Imaging SI1920) with brand new editing software (the new HDR CineForm RAW within Prospect HD.) What better way to find bugs then using your own tools on a very tight deadline (although I'll have you know I do this competition for fun, not to find bugs.)

This was the third year I have entered the competition, so I should know never to go into a shot with untested equipment, but this year it couldn't be helped. Ever since signing up for the project, I was pleading to get my hands on the latest camera from Silicon Imaging. Due to situations out of my control, the new camera arrived only a few hours before the 48 hour clock started ticking. And the camera needed to be assembled from parts shipped from all over the country.

The actual SI camera is tiny, about 1.5"x 2"x3" in size, and weighs next to nothing, so setting up our rig involved adding a lot of components to make shooting on a tight schedule more practical (you can just add a lens + power and run the tiny camera in a huge range of ultra-tight spots -- anywhere you can run Ethernet cable.) SI had also provided two C-mount Super-16 primes, standard mounting rails, handles for the rail, a shoulder mount, a 7" touch-screen display with Noga arm, two Anton Bauer batteries plus charger to run the screen and camera (for many, many hours.) In addition I picked up two 100' CAT-6 cables (next time we won't choose cables of the same color, as 200' of cable is a lot to untangle), a VGA to Ethernet extender, ND filters (in addition to those loaned to me -- thanks Jason), and an old 16mm Canon Zoom (off eBay) that came in very handy.

We used all these cables and parts to build a highly flexible tethered camera that could be up to 100 feet from the ingest station. The ingest station was a killer Sony Viao laptop with 2.133Ghz Core Duo CPU and a 1920x1200 display (in a laptop!) The laptop runs all the camera's operations; it sets the frame rates and sizes (from 2k at 24p to 1280x720 at 72p) and controls white balance, gain, compression quality and file management (all in simple software that I had not spent much time learning.)

This image is of Jake (the Director, and CineForm support manager) composing the next shot (or reviewing the last) from our "video village" using the Sony VIAO.

The laptop performed flawlessly, even though we shot for hours in direct sunlight (on the hottest day this summer in San Diego.) The Intel CPU has no issues with real-time encoding of 1080p24, and did an excellent job with 720p72 for slow motion, although the frame buffer did start to fill at this rate. One of the beauties of the SI camera software is it has a huge buffer of frames before compression, so even if XP gets distracted, or the compressor has a lot of work to do (like 72p), the buffer allows you to get the shot. We found that at 720p72 the buffer would fill after about 30 seconds of shooting (that is 90 seconds of 24p slow motion -- so not an issue.) A 2.33GHz Core Duo would likely have no issues with the buffer level.

So basically we had no problems with the camera and its picture. The biggest hassle was the audio. SI camera doesn't have sound, so I purchased an M-Audio USB sampler (another eBay item) so the SI camera software could capture synced sound in laptop (into a standard AVI.) For some reason, the M-Audio sampler didn't work on the shot day. This was my fault; I never did any thorough testing of this audio hardware. So all the audio had to be shot with a Canon XL-H1, and we slated everything and sync'd the picture to sound in post (something we really shouldn't have had time for in 48 hours.) Now the Canon was there to be our second and back (often collecting the same scene as the SI camera), plus for pick up shoots, basically just in case we broke the SI camera. :) But during the shoot we fell in love with the images from the SI, so the Canon was hardly used for its (still beautiful) image output. One reason is that matching the look from Canon to SI would have taken more time than we had for post. Canon was used in one scene of the final movie (the first person to email me which scene can have a DVD copy for our entry.)

During the shoot we used a USB/FireWire drive to move data from laptop to our editing workstations. As the CineForm RAW data is so damn small (averaging around 9MBytes/s for the new FilmScan 1 quality at 1080p24), this operation was fast, only a couple minutes for all the shots of each scene. Note: At NAB we showed an early version of CineForm RAW, which was not quite as high quality and averaged 12Mb/s+; this brand new encoder produces 20-30% lower bit-rates with higher quality. This new codec with the bit-rate drop had not yet been given to SI, so not only were we running a camera we had not used before, we were running brand new, never tested (in camera) compression, all in the spirit of this competition.

You may have noticed in some of the pictures we didn't have the 7" screen attached. On the morning of the shoot, we found the screen would turn on and off at random, frustating us for shot setup (and therefore it was removed.) So we mostly used the laptop screen for shot composition. In the afternoon we worked out we just had a flaky cable connection, so we used the 7" screen for the rest of the shoot (not documented with photos as we were a little behind schedule.)

Our editing workstations were the best of AMD and Intel. We had a dual Opteron 280 system for capturing data from the Canon (our audio), and a blazing (pre-release) Intel dual proc Woodcrest Xeon system for real-time editing. Both machines scream, but the Intel box was a prefered editing system for the project.

We began editing during the day of the shoot, better than the last two years when the editing didn't begin until the Sunday (with the submission at 7.00pm.) We had more all-rounders this year, so when Jake was editing, I become second unit director and did the interior non-dialog shots, Tim (another engineer) set up the next set and Brian (codec engineer) was script supervisor. We had a great team. In addition to the CineForm guys, we had Jake's brother and sister, Kevin and Kanda (both are far more production- savvy than myself) who provided the lighting setup and sound. For music production and sound design, Renee Coulombe, a professor of music at UCR, set the mood for the entire film, plus acted in the film. Also doing a voice over, Samantha (my wife) wrote the dialog (in 48 hour terms, that means telling the actors their lines; there really is no time for writing things down.)

On the Sunday we had things mostly under control (normally it is totally out of control.) The one export concern we had was we needed to master a 1920x1080p24 timeline out to 720x480p23.976 as the required deliverable (sadly, no HD projection for the theatre.) We should have shot at 23.976 which is supported by the camera, but due to the lack of setup time, we had shot the first scene exactly at 24 before anyone noticed, so we had to keep the whole production at 24. So early in the day I planned how the export would happen and how much extra time would be needed to master out (and fix the 24 to 23.976 issue.) If we exported from the HD directly to 23.976 DVD or DV (both supported submission formats), several issues would occur. Neither SD exporter used the beautiful 32-bit float that Premiere Pro 2.0 now offers, and the 24 to 23.976 conversion causes frame blending (a nasty ghost in the image.) Instead we exported to 720x480p24 as a CineForm Intermediate AVI; this exporter uses the 32-bit float, so we got the high dynamic range advantages of shooting CineForm RAW, plus it uses a much nicer scaler. Then I did my 24 to 23.976 trick using Virtual Dub (I probably could have used Interpret Footage), instead VirtualDub allows me to change the frame rate using direct stream copy so that there is no time wasted in recompression (i.e. fast.) This resulted in a SD 23.976 CineForm AVI, which was placed into an SD project for encoding M2V (for DVD via Encore.) However, as I had tweaked the video from 24 to 23.976, I need to slow the audio to match; this is done by setting the audio to speed 99.9% (quick and dirty -- works great.) The additional VirtualDub step only added a couple of minutes to our limited post time, and the use of a CineForm SD export improved the quality, so that was worth the time.

Now do you want to see the results?

First let me tell you about the required elements. We drew the genre of Comedy on Friday night, and were required to have these three elements in our movie to be eligible for a prize:

Prop : Beach Towel
Character : Sally or Sam Brown, Event Planner
Line of Dialog : "What would Elvis do?"

So you will see these items pop-up in the movie.

The various on-line HD versions have moved to www.cineform.com/48hour

For those interested in the story, not the image quality, there is a low bit-rate version on YouTube.

I hope you enjoy it.

Silicon Imaging mini used on 48 Hour Film Project

This is a brief announcement for tonight's screening for the 48 Hour Film Project. Every year some of the guys at CineForm put a team into the competition, this year we shot with a prototype Silicon Imaging Mini, capturing to a Sony Viao laptop. The results are stunning.

We always shoot with the name "Cane Toad Productions", this year we drew comedy and made a film titled "Burnin' Love."

Come and see the premiere screening at San Deigo's Landmark La Jolla Village Theater, 8879 Villa La Jolla Drive La Jolla. Here is the Map link. Our film will screen with nine others at 9am tonight. Buy tickets online here as they will likely sell out.

Some recent events, CineForm related.

Last Friday David Taylor (CEO, CineForm) and myself presented at the LA Film Festival (thanks to Intel to sponsoring this event.) We had a really nice projector (worth $30k apparently) to show the latest in real-time post-production with Prospect 2k and CineForm RAW -- all running off a single dual core Intel processor, the next generation -- Conroe. Now if only we had time to promote this last minute event (last minute to us), some more of you could have been there. Here is an interview with D.T. at the LA Film Festival of on the topic of The Revolution Will Be Digitized by Jason Lopez of PodTech.net.

Also please check out the latest over on indiefilmlive.blogspot.com, the recent updates from the set of Spoon are engrossing.

Four Cores -- the More the Merrier

7/28/06 updated with even newer Intel performance numbers
---
We are now seeing a large number of users with dual proc dual core system like AMD Opteron 275s to 285s, and dual core HyperThreaded processors like Intel 965 Extreme Edition -- each of these systems present four processing paths for us to play with. Plus with quad-core single processors on the horizon, we have good reasons to do a little more with our threading architecture. The main reason I've haven't been blogging so much is that improving thread can be a real headache -- particularly as we are already pretty well threaded. Sometimes we might spend a week on a particular algorithm to realize we only got another 5% more speed -- and I want much more than that. For encoding we have N-way threading - that is really nice - and we've had that for sometime. It is made easier by the fact that we can encode many separate frames simultaneously on ingest -- we launch an encoding thread for every CPU available (real or logical core like HT.) Threading the decoder is not so simple, after all that has to be compatible with DirectShow, Premiere Pro and MediaPlayer, etc. Often threading tricks applied to the encoder don’t work as well with the decoder. We do now have an N-way threaded decoder for extreme playback, 4k etc, but not yet for editing – we’re still working on that.) Previously our products included an efficient 2-way threaded DirectShow decoding component which was ideal for single dual-core processors, dual Opterons 248-254 and HT enabled P4s. Yet on quad systems we were seeing the encoder is faster than the decoder (because of N-way threading) -- rather odd behavior for a symmetric compression technology (encoding and decoding should be similar in performance.) – Sorry, that was perhaps lots of boring background about the new threaded decoder that just shipped in all our products -- it will be up to 30% faster on quad system over the previous decoder.

On a related subject, it has been suggested that we make a standard encoder/decoder test suite to characterize real-time video processing performance on various CPU/memory/drive configurations. What do you think? Here is a sample of recent characterization we measured for the decode speed using a very demanding sequence captured from an XBOX 360 at 720p (note: consoles produce more demanding data then anything acquired by a lens -- a lens adds natural anti-aliasing which is easier to compress and playback, whereas gaming material typically has an infinite depth of field and harder edges. So HD camera playback rates are higher than these shown.)

Machine		Decoder 2	Decoder 3
Yonah 2.0GHz, 667 FSB		60.17
Merom 2.0GHz, 667 FSB Merom 2.33GHz, 667 FSB (new data)		84.42 -	122.66
Pentium D 840 EE 3.2GHz, 800 FSB		93.65
Pentium D 965 EE 3.73GHz, 1066 FSB		110.83	146.61
Conroe 2.66GHz, 1066 FSB (next generation desktop)		122.81	137.59
Glidewell 3.2 GHz, 1066 FSB Woodcrest 2.66Ghz, 1333 FSB (new)		117.74 -	223.75 269.92

All numbers are in frames per second for a full-resolution, full-quality decode. Decoder2 is used in our Premiere Pro editing solution (Aspect HD and Prospect HD); its preview mode literally doubles and triples these numbers. Decoder3 is our N-way threaded presentation codec (think Digital Cinema.) The Glidewell's (new Xeon) 223.75 fps was due to the 8 threads (!) that can run on HT dual-core dual-proc Xeons (4 real cores.) 720p at 223.75fps equates to 100fps 1080p and 30p for 4k at 2.35:1 (note: higher resolutions are more efficient on compression, so the frame rate numbers are very conservative.) And we have more optimization coming. :)

New Data 7/28/06 : Same tests on the new Woodcrest Xeons hits 270 fps, with a lower clock speed than Glidewell and no HT. Amazing!

Note: For those who want AMD numbers, these processors are also very good (and have been for a while.) Yet we have older AMD systems in-house than these newer Intel boxes, so I can't show the best AMD vs the best Intel. Intel is certainly bringing-it-on and is currently the performance leader in our office; previously we only recommended AMD for Prospect HD Ingest - not so any more.

Creative Cow Interview with Jacob Rosenberg

Here is the direct link to the Creative Cow pod cast, where Jacob Rosenberg discusses all things Premiere Pro with some focus on past, present and future HD/Film projects using CineForm. Plus he drops my name so I have to link it. :)

Interview with Jacob Rosenberg

Not that I like to be on TV,...

Studio Daily did a series of interviews on the NAB show floor regarding the new Silicon Imaging digital cinema camera. The final production features Ari Presler of Silicon Imaging demonstrating how the camera works and myself showing the advantages in post production using footage shot with CineForm RAW. This is a nice concise news story covering why this camera is so cool (I why I had such a blast demoing it.) See the video story here under Some Like It Raw.

Shooting begins with the SI1920HDVR camera

Please check out the latest IndieFilmLive blog entry . In that blog you can read about the first beta customer for the new Silicon Imaging digital cinema camera, and their experience with shooting directly to CineForm RAW. We get to watch with our figures crossed and hope that the first feature shot this way will be a huge success.