CineForm Goes Open Source

16 years is a long time for a piece of software to remain useful. The CineForm codec, initially developed in 2001, was designed to enable real-time consumer video editing. CineForm, Inc. was a team of engineers that knew image and video processing, but very little about codec design (which likely helped.) Back then, DV (remember mini-DV tape?) was popular but too slow on the consumer's Pentium III and Pentium 4 desktops for software based video editing. We had worked out that the Intel processors had plenty of speed for editing without dedicated hardware, but the camera's compression, DV or MPEG based, were too difficult to decode. So in 2001, the CineForm codec became the first "visually lossless" intermediate codec (not a proxy) that replaced the source with a new file with much more speed. We didn't know this was a somewhat new idea. While there were other performance optimized codecs, like HuffYUV, CineForm was the first to offer significant compression, balancing quality, speed and performance like no other before it. Since then, Avid DNxHD and Apple ProRES have followed similar strategies. As CineForm Inc (the start up), the codec became a core differenciator, open source was never felt to be a viable option (we were probably wrong.)

Consumers weren't the only one's with speed and compression issues. So in 2003, the company pivoted from the consumer market to the professional when we created the CineForm HD codec.  This version shares the same DNA as today's compression, with 10 and 12-bit support and the resolution increase needed for film and television production. Video producers aren't interested in codecs, workflow is what was sold. CineForm compression was bundled with products like Aspect HD, Prospect HD, NEO 4K and Neo 3D, selling a workflow that depended on a codec. We knew there was little value in compression modules alone. Typically codecs sold for only a few dollars a unit, so the workflow obfuscated the codecs value. While the decoder was free, the encoder was not sold separately. The little start-up was not brave enough to make the encoder free, let alone its source.

One reason for keeping it closed, you might be surprised to hear, is that the codec's core tech was very simple. The codec idea was sketched out on a single piece of paper and the performance was determined first by counting the number of Intel instructions needed using an Excel spreadsheet -- even that fit on a single page. The codec was primarily written and maintained by two engineers, Brian Schunck and myself, with some great early help from Yan Ye. The simplicity meant that the magic of the compression was best shrouded in secrecy.

In 2011, CineForm was aquired by GoPro, which it used it to drive its HD and 3D editing utility, GoPro Studio and resulted in producing millions of consumer-created edits. Our initial play for real-time consumer video editing finally happened…10 years later. Now the CineForm codec could be released, and it was licensed to many, including Adobe, for free distribution within Creative Cloud. But it was still not open source.

The CineForm codec has shined most brightly when there is a market change, changes like SD to HD, HD to 2K+ compressed RAW, 2K to 4K, 2D to 3D and HD to 4K. At these times of transition, CPU and GPU vendors didn't yet have hardware optimized solutions -- try software HEVC decoding of 4K60 so see this issue. For the last few years, our computers and mobile devices have had hardware accelerated video decoding for most types of media we are producing. Even GoPro switched from the CineForm powered GoPro Studio to the hardware accelerated Quik for Desktop. So for a couple of years, CineForm didn't get much attention. It was used internally for prototyping, but it received very few public updates.

In 2017, the production market changed again, with new opportunities for the CineForm codec to provide solutions… particularly for greater than 4K sources from premium 360° products, like GoPro's own Fusion and Omni cameras. The new Fusion Studio desktop software defaults to CineForm for all the greater than 4K exports. But unlike GoPro Studio, where CineForm was primarily an internal format, Fusion Studio is not an editor. The CineForm files are meant to be edited in external tools, like Adobe Premiere Pro, After Effects, DaVinci Resolve and HitFilm. While many video tools do have native CineForm support, not all do… and some can’t. This gives GoPro more compelling reasons to make CineForm open source to help support our high resolution cameras.

The CineForm SDK is now open sourced, dual licensed under Apache 2.0 or the MIT license (integrator's choice.) You are now welcome to build new applications around the CineForm SDK, build it into existing applications, and extend its abilities with new performance optimizations and explore new image formats. I would love to see it go into many new places and new markets, but in particular, into popular open source players and transcoders. While the CineForm wavelet engine might be simple, 16 years of coding and optimization have complicated the source. So, to help with new upcoming compression-ists, the open source includes a simplified wavelet compression modeling tool -- simplified CineForm -- for education purposes and some codec design and tuning.

This was another great step for GoPro as it contributes more original projects to the open source community. Learn more about the CineForm SDK open source project on GitHub at https://gopro.github.io/cineform-sdk/