Slow at the full blogging, trying microblogging

I will be doing more updates here, just been too busy since NAB to create any well thought out content. In the meantime I'm trying out Twitter, you can follow me at http://twitter.com/David_Newman. Yet me know if there is any particular content you want be to try and post there.

Recorded at NAB

I didn't completely avoid being captured on tape or disk this NAB, but I tried. There where more bloggers snapping photos this year, and people stopping me in the aisles, as it seems that having an avatar over on reduser.net has killed my anonymity. Also I think I was captured in 4K at the reduser workflow event, I don't think anyone needs to see me in that much detail. :) On the show floor David Taylor did most of our on-camera interviews, including this cool one showcasing the Mac workflow using active metadata here at freshdv.com (see the video link titled "NAB 2008 -- CineForm 2K and 4K workflow".) I did one short audio interview for Bob Diaz's podcast. No new information here, but for anyone wanting an answer to the question, "what is CineForm?", that five minute effectively covers it.

NAB -- The Results

I'm now back in Solana Beach, CineForm HQ, returning a day early from the show. The CineForm reception at NAB this years was simply awesome, with more and more high-end companies seeing the value of compression in high resolution workflows. While most of the staff are still in Vegas, I had to get back to working on our 3.3 release, adding great features suggested by customers and fixing the bugs we always find at show time. Trade shows always have the habit of showcasing bugs, so it is good to have engineers doing the demos with pen and paper jotting down the list of things they will need fix back at the office -- we never hold back demoing work in progress features, but showing the cutting-edge of our development has it risks. This year as no different, but fortunately most the bugs showed themselves during Sunday setup and were fixed by Monday morning when the show opened. It turned out we never tried playing 3K on 4K timeline at full res. before (in preview it worked fine on our dinky 24" displays,) but the at the office we don't have the wonderful 30" monitors Intel loaded us, so we cranked up the resolution to make use of the display area. At show start everything looked great and presented smoothly from HD to 4K. We were playing 125 frame per second over cranked 4K Phantom 65 footage, thanks to guys at Vision Research, AbleCine Tech, and the Dual Quad HP xw8600 workstations that power our booth -- the most demanding footage I have worked with to date (clean, but amazingly detailed, and played real-time even while using Iridas's 3D LUTs.) It fun dealing with the extremes like discussing the detail of over cranked 4K footage with one customer, and the next minute discussing pulldown removal from Canon HV20s.

"All sources welcome" could have be a slogan at the CineForm booth, but fortunately I'm not in marketing (anyone seeing our website will know we don't have marketing department.) On the booth we where showing footage and editing Dalsa Origin, Grass Valley Infinity, Vision Research Phantom 65, Red One, Silicon Imaging SI-2K, and Sony EX1 (likely many more.) Grass Valley kindly loaned us an Infinity camera (pretty awesome BTW) to help us showcase the Infinity support we will be offering shortly, and someone loaned us the tripod to put it on (thank to that unknown company.)

In addition to the regular CineForm crew, I would like to thank those friends of the company that helped out at the booth: Isaac Anderson, Jim Hays, Mike McCarthy and Jason Salonen, providing the much valued outside prospective for CineForm and our customers. Thank you to Steve Sherrick who invited me to present at the Red User Workflow panel, and the warm reception from the engaging audience. For the Reduser presentation, thanks to Terry Cullen of 1Beyond who loaned CineForm a QuadCore laptop, and amazing machine that made the demo fly. And finally to all the companies that made up a "end-to-end digital workflow" booth: Wafian, Silicon Imaging, Iridas and Intel, each one helping with a sizable piece of the digital film making puzzle.

NAB preparations.

Sorry there have been no blog updates. I have been working like crazy getting Prospect v3.3 ready for NAB along with several Mac upgrades. There will be a lot of cool new features to show this year, which helps as CineForm has a bigger presence then ever before (in a booth 3 times bigger -- partnering again with Wafian and Silicion Imaging and adding Iridas to the mix.) Booth SL10609 (Lower South Hall.)

I will be at the show from Sunday through Wednesday, I have normally lost my voice by the Thursday, so don't wait if you want to come by the booth.

Shot Down By My Better Half.

If you have ever read this blog, you might get a laugh at my wife's skillful skewering of the crazy language I have used over the years. Please read Riding the Wavelet over on her blog littledragonfruit.com.

Metadata Matters

In recent builds of the CineForm tools and codecs, we are focusing a lot more on the robustness and a flexibility of metadata. The CineForm codec has been carrying a special class of metadata for the last 18 months; we use the term "active metadata" to describe how it differs from classic data like edge codes and copyright info. Active metadata is the mechanism that allows camera control unit functionality, such as white balance, curves, color matrix and 3D looks, to be stored non-destructively to maintain all your source image information, yet allow the playback to turn on and off or alter the active decoding/display of the metadata -- I need to do a post on this as it is very cool (or please come and see us at HD Expo (March 6) or NAB (April 14-17.)) We had traditional passive metadata for the same period of time; we just haven't started to emphasize it until now.

The CineForm structure for metadata data is little different than other streaming formats (AVI, MOV, MXF, etc), and as the metadata is embedded within each compressed video sample no matter the wrapper type. Streaming format typically attach metadata to a channel within the file wrapper, parallel to the video stream, or as a separate XML file, which is fine if you can stick with one wrapper type (AVI,MOV,MXF) and keep your files together. However, re-wrapping the stream or breaking up streams can result in the loss of this metadata. As CineForm is wrapper agnostic, we needed to store it differently so that it can't be lost. We still can use the wrapper standard metadata streams when needed, we are simply not depending on it.

As metadata is in each compressed sample, we can achieve greater flexibility as the video decoder is now aware of the metadata (not possible in parallel streams or side car files,) allowing applications that otherwise have little to no support for metadata to act on the data in new ways -- this is how our active metadata works. An example of this use is in our latest DPX to CineForm to DPX tools will completely preserve all the DPX metadata and reconstruct it when export back to DPX at a later date. We are working toward allowing passive metadata to be selected for optional burn-in display, in addition to all the search and retrieval applications of this data. Lots of cool stuff happening.

For those developers programmatically implementing CineForm compression into their tools, (Wafain, Silicon Imaging and Iridas are good examples), it is easy to directly set and control the adding of metadata. Via our SDK you get frame accuracy for metadata insertion. Yet we have added a mechanism to allow the non-programmer to add free-form metadata to any of their CineForm encodes. For Windows users, there two registry keys allowing you to add your own metadata :
HKEY_CURRENT_USER\\Software\\Cineform\\MetadataGlobal
HKEY_CURRENT_USER\\Software\\Cineform\\MetadataLocal

Here is an example of using MetadataGlobal:

----------- cut here --------------------
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\Cineform\MetadataGlobal]
"Copyright Owner"="David Newman"
"Computer Used"="M90 Laptop"
"Random info"="put whatever you want in the file"
"Add Numbers"=dword:00000050
----------- cut here --------------------

and save as MyGlobalData.reg. Of course you can change and add your own text and numerical data (hex format, that dword:00000050 = 80) after the HKEY line. Double click on the .reg file to install. Now every frame of every capture, or render, with have this data embedded. I've been using this in the office to know which PC any particular clip originated.

The second registry entry, MetadataLocal, only stores changes made since the last frame's encode. This is useful for attaching slowly changing data, like GPS data, or other changing data streams that occasionally change or change on every frame. Or you have a large amount of data you only want attached occasionally, rather than store it in every frame. I have yet to do much with this feature myself, but I sure it will have some powerful applications in the future.

Update: Use MetadataGlobal for changing data if you want that data to be present in every frame. Timecode is a changing field that is stored in every frame, and therefore should be using the global class. Accurate frame based changes require SDK level access.

Of course there are many new features in the works that I can't yet disclose, but please comment on your own ideas for metadata features your like to see.

The dark side (but funny)

I don't normally post clippings, but this is so me. Try to maintain an active forum presence has its dark side.

Source XKCD, found through BoingBoing.