GPMF Introduction

The GPMF structured storage format was originally proposed to store high-frequency periodic sensor data within a video file like an MP4. Action cameras, like that from GoPro, have limited computing resources beyond that needed to store video and audio, so any telemetry storage needed to be lightweight in computation, memory usage and storage bandwidth. While JSON and XML systems where initially considered, the burden on the embedded camera system was too great, so something simpler was needed. While the proposed GPMF structure could be used stand-alone, our intended implementation uses an additional time-indexed track with an MP4, and with an application marker within JPEG images. GPMF share a Key, Length, Value structure (KLV), similar to QuickTime atoms or Interchange File Format (IFF), but the new KLV system is better for describing sensor data. Problems solved:

• The contents of new Keys can be parsed without prior knowledge.
• Nested structures can be defined without 'Key' dictionary.
• Structure prevents naming collisions between multiple sources.
• Nested structures allows for the communication of metadata for telemetry, such as scale, units, and data ranges etc.
• Somewhat human (engineer) readable (i.e. hex-editor friendly.)
• Timing and indexing for use existing methods stored within the wrapping MP4 of similar container format.

GPMF -- GoPro Metadata Format or General Purpose Metadata Format -- is a modified Key, Length, Value solution, with a 32-bit aligned payload, that is both compact, full extensible and somewhat human readable in a hex editor. GPMF allows for dependent creation of new FourCC tags, without requiring central registration to define the contents and whether the data is in a nested structure. GPMF is optimized as a time of capture storage format for the collection of sensor data as it happens.

GPMF-parser

Included Within This Repository

• The complete source to an GPMF parser library
• Demo code for using the GPMF parser with extraction of GPMF from MP4 or MOV files.
• CMake support for building the demo project.
• Tested on:
  • macOS High Sierra with XCode v8 & v9
  • Windows 10 with Visual Studio 2015 & 2017
  • Ubuntu 16.04 with gcc v5.4

License Terms

GPMF-parser is licensed under either:

• Apache License, Version 2.0, LICENSE-APACHE
• MIT license LICENSE-MIT

at your option.

Contribution

Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.

GPMF Writing

A new open source repository has been created for the purpose GPMF-Write

Quick Start for Developers using Docker

Run the Docker container

docker run runsascoded/gpmf-parser samples/hero8.mp4

Run on your own video(s) by mounting an /input directory (or individual files to paths matching /input.*):

docker run -v "/path/to/my/movie.mp4:/input.mp4" runsascoded/gpmf-parser

Setup

Clone the project from Github (git clone https://github.com/gopro/gpmf-parser).

Now you can run a quick demo on the sample data by compiling the demo:

cd gpmf-parser/demo/
make
./gpmfdemo ../samples/Fusion.mp4

This will return a brief overview of what metadata is stored in the given video file (Fusion.mp4). The following command will show the first few samples of the recorded payload (i.e. accelerometer, gyroscope, magnetometer, GPS, etc.):

./gpmfdemo ../samples/Fusion.mp4 -g

Sample Code

GPMF-parser.c and .h provide a payload decoder for any raw stream stored in compliant GPMF. Extraction of the RAW GPMF from a video or image file is not covered by this tool.

#include <GPMF-parser.h>
GPMF_stream gs_stream;
if(GPMF_OK == GPMF_Init(&gs_stream, buffer_with_GPMF_data, size_of_the_buffer))
{
	do
	{
		switch(GPMF_Key(&gs_stream))
		{
		case STR2FOURCC(“ACCL”):
		  // Found accelerometer
		  samples = GPMF_Repeat(&gs_stream);
		  if(GPMF_OK == GPMF_ScaledData(&gs, temp_buffer, temp_buffersize, 0, samples, GPMF_TYPE_FLOAT)) 
		    {  /* Process scaled values */ }
		  break;
	  
		case STR2FOURCC(“cust”): 
		  // Found my custom data
		  samples = GPMF_Repeat(&gs_stream);
		  if(GPMF_OK == GPMF_FormattedData(&gs, temp_buffer, temp_buffersize, 0, samples)) 
		    { /* Process raw formatted data -- unscaled */ }
		  break;
	  
		default: // if you don’t know the Key you can skip to the next
		  break;
	}
} while (GPMF_OK == GPMF_Next(&gs_stream, GPMF_RECURSE_LEVELS)); // Scan through all GPMF data

If you only want particular a piece of data

#include <GPMF-parser.h>
GPMF_stream gs_stream;
if(GPMF_OK == GPMF_Init(&gs_stream, buffer_with_GPMF_data, size_of_the_buffer))
{
  if(GPMF_OK == GPMF_FindNext(&gs_stream, STR2FOURCC("ACCL"), GPMF_RECURSE_LEVELS))) 
   {
   	uint32_t key = GPMF_Key(&gs_stream);
   	char type = GPMF_Type(&gs_stream);
   	uint32_t samples = GPMF_Repeat(&gs_stream);
   	void *data = GPMF_RawData(&gs_stream);
	
	/* do your own byte-swapping and handling */	
   }
}

GPMF Deeper Dive

Definitions

| Word | Definition | | --- | --- | | Track | An MP4 container element used for storing video, audio or sub-titling over time | | FourCC | The four character Key that marks a segment of data | | Key | FourCC is the first element in Key-Length-Value 3-tuple | | Length | This field describes the data that follows, including basic type, structural hints, sample counts | | Value | Raw data to be stored in Big Endian | | Device | Metadata sources below to devices, a camera my device and a connected BlueTooth sensor would be a separate device | | Stream | Each metadata Device can have multiple streams on sensor data, e.g. Camera Device could have GPS, Accelerometer and Gyro streams |

KLV Design

All data is Big Endian.

FourCC

7-bit ASCII Key created for readability. All uppercase fourCCs are reserved Keys, mixed case is for any third party ingested data.

A few Keys reserved for communicating structure, although only DEVC is required at the beginning of any GPMF payload.

| FourCC | Definition | Comment | | --- | --- | --- | | DEVC | unique device source for metadata | Each connected device starts with DEVC. A GoPro camera or Karma drone would have their own DEVC for nested metadata to follow. | | DVID | device/track ID | Auto generated unique-ID for managing a large number of connect devices, camera, karma and external BLE devices | | DVNM | device name | Display name of the device like "Karma 1.0", this is for communicating to the user the data recorded, so it should be informative. | | STRM | Nested signal stream of metadata/telemetry | Metadata streams are each nested with STRM | | STNM | Stream name | Display name for a stream like "GPS RAW", this is for communicating to the user the data recorded, so it should be informative. | | RMRK | Comments for any stream | Add more human readable information about the stream | | SCAL | Scaling factor (divisor) | Sensor data often needs to be scaled to be presented with the correct units. SCAL is a divisor. | | SIUN | Standard Units (like SI) | If the data can be formatted in GPMF's standard units, this is best. E.g. acceleration as "m/s²". SIUN allows for simple format conversions. | | UNIT | Display units | While SIUN is preferred, not everything communicates well via standard units. E.g. engine speed as "RPM" is more user friendly than "rad/s". | | TYPE | Typedefs for complex structures | Not everything has a simple repeating type. For complex structure TYPE is used to describe the data packed within each sample. | | TSMP | Total Samples delivered | Internal field that counts all the sample delivered since record start, and is automatically computed. | | TIMO | Time Offset | Rare. An internal field that indicates the data is delayed by 'x' seconds. | | EMPT | Empty payload count | Internal field that reports the number of payloads that contain no new data. TSMP and EMPT simplify the extraction of clock. |

Length (type-size-repeat structure)

The tradition 32-bit "Length" is broken up to help describe the data that follows.

Structure Size

8-bits is used for a sample size, each sample is limited to 255 bytes or less.

Repeat

16-bits is used to indicate the number of samples in a GPMF payload, this is the Repeat field. Struct Size and the Repeat allow for up to 16.7MB of data in a single KLV GPMF payload.

Type

The final 8-bits, Type, is used to describe the data format within the sample. Just as FOURCC Keys are human readable, the TYPE is using a ASCII character to describe the data stored. A character 'f' would describe float data, 'd' for double precision, etc. All types are reserved, and are not end user definable.

Current types:

| Type Char | Definition | typedef | Comment | | --- | --- | --- | --- | | b | single byte signed integer | int8_t | -128 to 127 | | B | single byte unsigned integer | uint8_t | 0 to 255 | | c | single byte 'c' style ASCII character string | char | Optionally NULL terminated - size/repeat sets the length | | d | 64-bit double precision (IEEE 754) | double | | | f | 32-bit float (IEEE 754) | float | | | F | 32-bit four character key -- FourCC | char fourcc[4] | | | G | 128-bit ID (like UUID) | uint8_t guid[16] | | | j | 64-bit signed unsigned number | int64_t | | | J | 64-bit unsigned unsigned number | uint64_t | | | l | 32-bit signed integer | int32_t | | | L | 32-bit unsigned integer | uint32_t | | | q | 32-bit Q Number Q15.16 | uint32_t | 16-bit integer (A) with 1