GPMF - GoPro's Metadata Format

Have you ever noticed how little metadata you get from a video file?  Take any JPEG or RAW photo from an iPhone, a GoPro or a DSLR, and you will find extensive metadata stored in Exif, a photographic metadata standard that stores hundreds of properties such as exposure, focal length and camera model etc.  For your average MP4, MOV or AVI, not so much, particularly without the various sidecar files which are so easily lost. Some of the earliest GoPro HD cameras didn't even say "GoPro" anywhere in their video bit-streams.  I feel a manufacturer ID is the most fundamental of metadata: answering, what made this file?  Even some of the professional cameras I worked with at CineForm weren't much better.  Metadata should also answer, how was this file made? In what environment? At what location? Was the camera moving?  Etc. So why is photo metadata ubiquitous, and video metadata spotty at best.  The lack of useful video metadata relates to standards, or the lack of them. We have no Exif equivalent for video files, particularly for consumer video within MP4, so GoPro created one.

For GoPro cameras we couldn't just place an Exif within an MP4, as the exposure is changing per frame, so that would be a lot of Exif data, and Exif had no clear model for time varying signals. This photo standard also assumes that the metadata applies to a particular frame, that doesn't work so well for gyro and accelerometer data. GoPro needed to store both high frequency sensor data and Exif-like frame-based data to describe the image exposure.

A multiple-year effort begins.

The software ecosystem would love JSON or some XML variant, but embedded camera developers will have none of that, complaining of excessive memory usage, system load, blah blah blah.  The camera developers would love nothing more than storing raw I²C packets (actually proposed once), very low overhead, but completely opaque to software developers.  There needed to be a compromise.

How I accidentally became a Firmware engineer.

To investigate a possible low overhead metadata format, I started with the internal format used by CineForm's Active Metadata, it needed some work and extensions, but it was efficient, easy to read and write, and should appeal to both embedded firmware and software camps.  So I wrote a sample application to demo the reading and writing of metadata stored in any camera-native format.  I though it was only a model, I was thinking if the format was accepted the camera team would write something better. Firmware liked it, used it as is, then filed bug reports against it which I had to fix. That was about two years ago, and I've been part-timing as a firmware engineer ever since, focusing on metadata.

Fast-forward to today.

Here is a taste of GoPro metadata and telemetry:

Property	Freq. (Hz)	Comment
3-axis accelerometer	200	All HERO5 cameras
3-axis gyroscope	400	All HERO5 cameras
lattitude, longitute, altitude, 2D ground speed, and 3D speed	18	HERO5 Black with GPS enabled
UTC time and data from GPS	1	Within the GPS stream
GPS Fix	1	Within the GPS stream: 0 - no lock, 2 or 3 - 2D or 3D Lock
GPS precision (PDOP)	1	Within the GPS stream, under 300 is good
Image sensor gain	24, 25 or 30	HERO5 v2.0 or greater firmware
Exposure time	24, 25 or 30	HERO5 v2.0 or greater firmware

You can use some of this data today within GoPro's Quik Desktop tool, generating cool overlays, or do even more using the GoPro acquired Dashware tools.   A random video a created 8 months ago using GPMF extracted within Dashware, gauges rendered and composited in HitFilm:

Dashware+Telemtry from David Newman on Vimeo.

However, for this to be truly interoperable, its support needs to go beyond GoPro.  In a big step in that direction, GoPro has open-sourced the GPMF reader, as a way for third parties to collect, process and display video metadata from MP4s.

Developers please check it out https://github.com/gopro/gpmf-parser  There is extensive GPMF documentation and simple code samples.