This is not an official news source for CineForm or GoPro product releases, just some bits and pieces of stuff I happen to be working on. My work and hobbies are pretty much the same thing. -- David Newman
This post showcases one of the many applications for the CineForm codec. BrainSalt Media uses CineForm decoders to drive 16 seamlessly tiled projectors on a 250 square meter curve screen for an awesome virtual aquarium installation in China.
To top the aquarium, here is BrainSalt's 30 Megapixel at 60 Frames dome projection also using CineForm (15 titled 2K projectors.) Here is a photo for the dome during setup
P.S. I have just heard that domed presentation above won an award -- Themed Entertainment Association Award 2009. Remember that is 1.8GPixels per second running from a CineForm fully software/CPU based decoder.
Just before the Thanksgiving break, CineForm released new betas for the Neo and Prospect product lines (http://tr.im/FNSb.) These have a new feature that we have been planning for ages (years it seems,) the ability for the decoder to render its own passive metadata. The decoder has been applying Active Metadata for many years, developing the image through color parameters and 3D LUTs for creative looks, yet the classic metadata has remained dormant within each compressed frame -- we left it up to vendors using the SDK to extract as needed (and so few do this.) As metadata is so often lost and misplaced, you are lucky if you are left with just the timecode in many workflows, so we long ago moved metadata from side-car files or within the file wrapper (AVI/MOV/MXF) and placed it within the compressed sample itself. This enables the decoder to read its own metadata (not possible with 99% of video types), all that was missing was the font engine to render the results in the display. The decoder now has that font engine. Offline workflows typically have a range of burn-ins top of the video image, returning to burnin free media for online/finishing. The CineForm burnins are non-destructive allowing the operator to enable the overlay display, choose which elements to display, switch from offline to online with a single click. Any tools that uses the CineForm decoder will gain this feature.
The First Light control panel:
The placement and font controls are primitive today, but the engine already works with transparency, color and outline stroke controls (vendors using SDK can select these today.) Sample images from the overlay engine tests: http://twitpic.com/obs9l http://twitpic.com/ob1d9 http://twitpic.com/oay77
For those who want get started with the metadata burn-ins, here are some simple steps:
1) Start the new First Light (within version 4.1.3 of Neo or Prospect.) 2) Import a CineForm clip that has the type of metadata you wish to display. 3) Select the Passive Metadata tab to reveal all the metadata types with. 4) Select an item from the list. 5) optionally -- click the 9-way justification control to determine where you want to place the burn-in. 6) Click "Add / Remove" to apply the burn (and again to remove.) 7) Repeat steps 4-6 to add more metadata to the output.
8) The Overlay checkbox (near histogram control,) globally enables these burn-ins for all clips in the system.
If you want custom formatting for your metadata we are using the C language printf formatting. Instead of the recode data "2009-11-26", in the customer formatting add "Date: %s" for a display of "Date: 2009-11-26". You can use this to add freeform burnins like "property of me" by select a random metadata line and not putting a "%s" (string) or "%d" (for decimals) in the formatting. Font name and sizing also are active, setting Arial and size 70 will render the next added burn-in with those characteristics.
Users of Red and SI cameras will have lots of metadata to display, unfortunately there is not much for HDV/AVCHD users -- yet. The reason we implemented this feature, is not just to display today's metadata, but to encourage more metadata to be stored at acquisition time (something our own tools have a reason to do more of.) Also to store changing metadata during capture -- good examples would be lens data and GPS/orientation coordinates that are coming to more cameras. Even Red One metadata out from the SDK is only per clip, not per frame (we understand this with be addressed with a future R3D SDK release.) There is an opportunity for those doing live HDSDI/HDMI captures into CineForm, to generate their own metadata streams (see how at techblog.cineform.com.)
We aren't stopping at display of new metadata, next we will use this metadata to trigger external applications and tools to act in new and programmable ways -- think of third party apps for your decoder. We want metadata to approach the power of the image data that it is stored with.
I'm am enlisting your help (testers needed) -- details at the end of this post *.
For many years now, fans of CineForm compression have been wanting a single frame version in addition to streaming AVIs/MOVs, as this would be particularly handy for frame based render-farms. Yet there is a catch-22 for supporting any new image format, as too many tools need to support the format before it is viable, prevented the new format's creation. This is very different for streaming formats like AVI and MOV, as most tools use a registered codec infrastructure, like those provided by QuickTime, DirectShow or VideoForWindows, enabling new compressors to be created without modifying the calling application -- we added CineForm compression to FCP without Apple's need to change FCP (even though we wish they would.) For still formats we can't just add our compression to TIFF or PNG, and expect existing tools to support it. Unfortunately still image formats don't use the same codec model like that of QuickTime, slowing down the implementation of new image formats (consider how digital cameras still mostly use JPEG, even though there have been better formats for years.)
CineForm has an idea to help bridge some of that catch-22, to enable support for a CineForm compressed frame format more quickly -- the idea is compressed DPX, I'm call them DPX-C. By using existing DPX structures, some level of backward compatibility can be obtained. Of course we are not expecting existing tools to magically open the compressed data of a DPX-C file, so we add a thumbnail of image that visually reports that the image is compressed (a text overlay within the thumbnail image.) The compressed data remains within the proprietary data fields of the DPX file. While the thumbnail greatly helps for image browsing, we still need a way to full resolution decoding. The first basic step is to provide free tools that convert DPX-C back to DPX, or DPX-C back into a streaming format like AVI or MOV as needed. But we also thinking about file system virtualization, so that a directory of DPX-C files is seen a standard DPX files that load within existing tools without tool modification (** more on this to follow.)
CineForm file virtualization has been done before such that 2K DPX files can be played in real-time (24p) off a bandwidth limited device like a thumb-drive (this was done at a facility outside of CineForm -- there are some very smart users out there.) As the CineForm compression (and decompression) is so fast, the increase in CPU load is often less that percentage of bandwidth load that uncompressed DPX consumes. e.g. DPX playback might take 70% of you RAID bandwidth, yet DPX-C would use 30% of modern CPU for the same playback rate and only 10-15% of RAID. This means DPX-C is will produce greater throughput even as is moves some of the disk load to CPU load -- it is a good trade-off.
Encoding can be virtualized in the same way, enabling a render farm to export standard 10-bit RGB DPX files to a volume that automatically stores DPX-C files, greatly improving network performance at the same time. This DPX to DPX-C virtualization solves the problem of trying to render farm frame directly in a stream media file -- frames don't arrive in order, and CineForm is a variable length format, meaning some frames are bigger than others. DPX to MOV virtualization would require file padding or a constant bit-rate designs, both are undesirable -- not such issue with DPX-C files. While a wide range of virtualization configurations are possible, DPX to DPX-C is the simpliest, particularly as file naming would be identical, allowing switching between virtualized and extracted DPX files as needed.
The ultimate goal is to virtualize DPX to those tools that need it, while encouraging native support in existing tools. DPX-C files will contain any the CineForm compression types from 422, 444, 4444 and RAW (anyone for a DPX-C CineForm RAW still camera?) The first tools to add DPX-C will be CineDDR and Drastic's DDR recorders and virtual decks. The DPX-C was needed for better VTR emulation, allowing telecine sessions to operate with normal inserts without creating overlapping AVI/MOV files, that may need to be re-ordered into the virtual tape. With these products the DPX-C file sequences can be simply and losslessly parsed into MOV or AVI streams for archive or downstream processing.
* While all this sounds great, it hasn't completely happened yet. I need your help in determining the best way to extend DPX with the widest range of compatibility with your tools. This download: www.miscdata.com/DPX/dpxc.zip contains five DPX-C files with subtle variations on how the data is stored, like thumbnail first following by the compressed data or vise-versa, and how the extra data is flagged. The files labeled A through E, are 1920x1080 frames compressed from 8MB to around 715k (422), some or all will load in you existing video tools -- I would like to know which work and which don't in your existing DPX applications. So far I have only tested PC based tools; these work for all DPX-C files: AE CS4, Combusion 2008, Vegas 9 and XnView. C, D and E files work in AE CS3 and Photoshop. So something works in all tools tested so far. So please report back you own findings with these tools or others, and with different versions (note the behavior change between CS3 and CS4.)
** We looking for a developer who has implemented virtual file systems based of FUSE or similar for all platforms (Windows, OS-X and Linux -- we will need Win/OS-X versions first.) We believe a FUSE/MacFUSE implementation would be straight forward, we just have too many projects internally at the moment, and will consider contracting this part out. Of course if we can't find the right person or team, links to good samples would be helpful, as we want this to happen as soon as we can.
Thanks.
---- Update March 15, 2010 ----
Several developers are now using this format, thinks to all the testing users performed for us. We end up using a variation on type-E, being the most compatible and extra useful in image browsing systems as the thumbnail was stored before the compressed image data. For the more complete technical specification on DPX-C, please visit the CineForm Techblog. Within v4.3 or (greater) of Neo and Prospect PC tools, the DPX2CF and CF2DPX utilities now support the DPX-C foramt.
This will likely be the first Canon still camera I ever own. Had plenty of Canon video cameras, but I've been a Pentax guy for years (my lens collection has kept me there,) but the video features of the Canon 7D will change that. While the 5D Mk-II produces some awesome looking motion images, the 30.0fps has been a complete pain. We competed against several 5D movies for San Diego's last 48 Hour Film Project, and some of those teams didn't follows the rules and submitted 30.0 masters (we had to convert them all to 29.97 for presentation.) For the 50 teams that submitted, everything was presented at 1080p23.976 except the 5DmkII films -- I'm sure none of those filmmakers wanted to shoot 30p. So the big fix for the 7D, the camera also supports 29.97 and 23.976 -- these frame rates make much more sense for the filmmaker.
Now the 7D is still a heavily compressed I-frame H.264 4:2:0 8-bit, so like the 5D, we will see a huge attach-rate to our Neo product lines (Mac and PC.) The decoder speed of H.264 is poor, so many convert to CineForm just for the improved editing speed.
Now for some speculation, we're hoping the Live View is now full 1080i/p over HDMI, and that the burn-ins can be turned off, as then you would able to capture directly, bypassing the H.264 compression. Fingers crossed! If this came true, the 23.976p mode will likely go out as 60i, with pulldown (for wider display compatibility), fortunately we can remove that on the fly. In addition to removing compression issues, the HDMI preview may be 4:2:2, rather than just 4:2:0, so it will be nicer for keying. It will still be 8-bit -- but for a $1700 camera, with tethering recording with much lighter compression (via BM Instensity or AJA Xena/IOexpress) -- you can't beat it.
We look forward to testing and optimizing our workflow for this new camera.
I've been missing from much of my usual online forum and twitter activities over the last 10 days, I've been flat out with the 48 Hour Film Project, my annual step in the creative side of the business. Lots of new things to report this year. First non-tech thing is CineForm staff members Jake, Tim, Craig and myself, formed two teams rather than just one this year. While this was saddened me at first, the results do not show anything negative from our divided efforts, both resulting films received more awards than the sum of our previous 5 year history with this competition. My team film draw the genre Detective/Cop to make "The Case of the Wild Hare" in a comic Film Noir style, winning an audience award a two jury prizes. Tim and Jake's team draw genre Film de Femme, shot a thriller, and got a jury prize and runner up for best of San Diego (out of 47 submitted films.)
Check both films out on Vimeo in HD, or just watch embedded.
Now for the technical: one film used an SI-2K (running beta 2.0 software -- nice), a Kenyon gyro stabler (interesting), and range of 30+ year-old C-mount primes and zooms (classic), the other used an HV30 with a lens adapter with Canon still lenses. Can you tell which is which? Shooting technology is clearly only a small factor in making an enjoyable film, as both films won against very polished projects shooting from 5Ds to Red Ones. While the shooting technology could hardly me more different, the post workflows where very alike. Editing was done is Premiere Pro CS3 (Windows) on i7s or dual quad Core2 Intel systems (OSes range form XP64 to Win7.) CS4 wasn't used only for HDSDI monitoring reasons, as we haven't got that working yet. CS4 would have helped as we always used the beta features of our own tools, and one of the new features only worked in CS4 (we didn't realize until mid-post.) Both films where shoot for 2.35:1 presentation, as 16:9 has become so TV like these days. The new feature that neither team got to use was an addition to First Light, enabling 2.35:1 crop and centering as Active Metadata -- would have saved some time in post for positioning and rendering. First Light was used extensively on both projects, all grading was done as a real-time operation in First Light, particularly aided with the new auto sync. feature which keeps First Light connected with the NLE's timeline. No color correction was needed within Premiere itself.
More on First Light. One thing that helped us was a range of 3D LUTs (look files) that we have been preparing for sometime. These LUTs are now available for download to use with your Neo HD/4K and Prospect under Windows (Mac versions soon.) In such compressed schedule, you get very little time to work on your look, I think I put about 30-40 minutes into color correcting Wild Hare, about twice the time I had for color work for last year's project, but it is not much so prepared LUTs helped greatly. The Active Metadata LUT system, works on the final output for the image, with all the linear light processing for white balance, color matrixing (saturation) and channel gains, lift, and gamma being applied to the input of the LUT. This makes it pretty easy to mix and match a range of sources to produce on common look (stylized or not.) As I was working with two co-directors, one of whom had never working in film before (only stage work), I prepared different look profiles as switchable color databases, so the entire timeline could have is look/style switched dynamically. This helped showcase possible finishing styles without impacting the editing session, which went into 47th hour.
Here is an example of a before and after First Light processing.
Source:
Final:
After our two teams had submitted, the remainder of the week was spent preparing the other 45 films of presentation in CineForm format out to the Christie 2K projector to the local UltraStar theater. As CineForm is San Diego 48 Hour sponsor we requested that 1080p CineForm AVI/MOV be the default HD submission format (we gave all teams a 30-day Neo HD license,) fortunately more than half of films where submitting this way, making our life easier. It is was the remaining SD submissions, with its many pixel aspect ratios, letter-boxing, pillar-boxing, cropping, and pulldown variations, that was a time consuming headache (upres'ing it to look decent.)
For theatrical presentation we were prepared to use a Wafian F-1 as a playback server, which has worked flawlessly in previous years, and is not so rarely used for this purpose (as works really well -- F-1(s) where use to present HD at Comicon.) Unfortunately an hour before the first theatrical presentation the drive sled that goes into the Wafian crashed. This piece of bad luck put as in a panic, as it was going to take more than the remaining hour to prep a new drive with the 23 films that were screening that evening. Fortunately Tim had been experimenting with scheduled playback using, CineForm AVIs, a Blackmagic card and the open source Media Player Classic, and his system than all the films on it. Basically a cheap PC with a $300 HDSDI card hooked to a theater projector, become an awesome 10-bit 1080p presentation system. This was not flawless, it stopped about 7 films into the second screening group (resulting in 45 seconds of black as Tim run up into the projection booth to give MPC a kick.) While a Wafian would never have done that, the cheapness of this presentation solution made us pursue the same setup the next night and for the best of San Diego screening. There is still work to tweak this solution, but something like this is needed, as it is crazy that San Diego is still the only city to project in HD for this international festival, this being San Diego's third HD year.
Extending our experience from the local festival,we want every festival to stop presenting DVDs or Beta (still so common,) when most sources are film or now HD or originated. The solution has got to be cheap and simple, to allow for last minute playlist changes for pre-rolls, differing frame rates (24p, 30p and 60i,) audio level adjustments and skipping bars and tone, etc, i.e. BluRay will not do. We had one beautifully master film this year, that had the wrong black levels, put it through First Light, fixed it in seconds without any rendering. Duane Trammel, San Diego 48 Hour producer, used the playlist flexibility, to inter-cut interviews with several of the filmmakers in this short problem -- it was a pretty cool touch. More of this style of flexible is what is needed, and we are hoping to help.
This year will be the first time I'll will make to IBC, thanks to the persistence of Phil Streather to have me present the editing workflow section of his "3D at the Movies" super panel. The full panel will cover many elements such as pre-visualization, shooting and post, all presented in a large Real-D equipped movie theater. It will be a first for me to demonstrate 3D convergence manipulation live onto a screen of that size (seating over 800 people apparently.) So if you're in Amsterdam the morning of September 14th, might be worth dropping by early to seek good seating (for the better 3D experience.)
Something closer to home. This weekend I will be presenting part of the editing panel, covering post for the 48 Hour Film Project. If you have read here at all, you know I'm a big fan of this competition. I will be at the SaVille Theater at San Diego City College from around 1pm through 5pm, this Saturday. Primarily I will be showing local 48 Hour teams how to produce HD masters that are ready for projection. If you are putting a team in this year, and many already are (almost at capacity for teams now), and if you can't come to this session, please read up on the new HD submission guidelines -- And practice before the competition weekend. The a seminars on film making for this competition (and others) starting at 10am, you can see them all for only $15 -- more info here.
For last 5 years CineForm as put a team in this touring film making competition. In the last two we have sponsored the event by providing HD playback and HD up-res services for those submission that need it (everything is converted to CineForm AVI/MOVs for playback.) San Diego's 48 Hour is still the only city that is allows HD submissions for 1080p projection. This year we expanding, we are helping in Filmmaker Seminars in the weeks before, and we are putting in two teams this year to the competition -- this is not part of the sponsorship we pay like everyone else, and we don't even use the company name for these teams, never have -- always just for fun.
