Extracting GPS Information from .MP4?

[johnichan]

HI Phil,

Can you guide to make the user defined tag to get the 'JUNK' inside the RIFF?  I have no idea on how to extract a tag from a video file.  Google guides to RIFF.pm written by you at git hub https://github.com/alchemy-fr/exiftool/blob/master/lib/Image/ExifTool/RIFF.pm. I hope there will be some guides on how to use it.

[Phil Harvey]

Here is the config file:

Code Select Expand

%Image::ExifTool::UserDefined = (
  'Image::ExifTool::RIFF::Main' => {
    JUNK => { Binary => 1 },
  },
);

and here is the command:

exiftool -junk -b FILE > junk.dat

- Phil

[johnichan]

Hi Phil,

Here goes my solution. I am not a professional in perl coding. So making things some how working is my only aim.

https://drive.google.com/open?id=192NaREqmfK6smein9tqjH2Qlx2NUogkO

Thanks again for helping out without which I would have dropped this part.

[diplonics]

Hi Phil,
     I've just lately started using a new dashcam, Roadhawk, and want to get the GPS data from it.
I've used Exiftool many times before on other cameras like a GoPro to get the GPS data but since changing to the Roadhawk the GPS data, I'm pretty sure, exports but is garbled in a text field.
I know the GPS is in the standard NMEA format so wondering if there is an option I missing to properly decode these strings?

Using the basic method: exiftool -g -ee Roadhawk_Sample.mp4 > out.csv
I get output like this:
*************************************************************************

Code Select Expand

Sample Time                     : 0 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=-;;6+;;;D;NL;4;;;=D;P?;O;;;=D=PJ?HO71G>F;-?==BF:FNJJ;DPBN?F3F;;PL?D;-B;F0F;=-DPF=;BDRLF;N;?=JF;;?D=F:*5D~
Sample Time                     : 0.20 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=-;;6;;;;D;R?;4;;;;DBNL;O;;;;DBLRRHO71G>F;-?==BF:FNJJ;DPBL-F3F;;PL?D;-RLF0F;=-DNF=;BDRJF;N;?=JF;;?D=F:*58~
Sample Time                     : 0.40 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D?=P;6;;;;D=?P;4;;;=D;;-;O;;;=D;R-PHO71G>F;-?==BF:FNJJ;DPBLNF3F;;PL?D;-N-F0F;=-DLF=;BD-NF;N;?=JF;;?D=F:*53~
Sample Time                     : 0.60 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=;-;6;;;;D;B;;4;;;;DBJP;O;;;;D-BNBHO71G>F;-?=?;F:FNJJ;DPBL=F3F;;PL?D;-L=F0F;=-DLF==;D;PF;N;?=JF;;?D=F:*56~
Sample Time                     : 0.80 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=LL;6;;;;D;NL;4;;;;DBR?;O;;;;DBP-LHO71G>F;-?=?;F:FNJJ;DPBJRF3F;;PL?D;-?PF0F;=-DJF==;D??F;N;?=JF;;?D=F:*57~
Sample Time                     : 1.00 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=LL;6;;;;D;NL;4;;;;DBR?;O;;;;DBP-LHO71G>F;-?=?;F:FNJJ;DPBJLF3F;;PL?D;-=;F0F;=-D?F==;DR=F;N;?=JF;;?D=F:*56~
Sample Time                     : 1.20 s
Sample Duration                 : 0.20 s
Text                            : .;;;;D=B-;6+;;;D;JP;4;;;=D;;-;O;;;=D;NPPHO71G>F;-?=?;F:FNJJ;DPBJ;F3F;;PL?D;RBLF0F;=-D=F==;D-RF;N;?=JF;;?D=F:*5B~
Sample Time                     : 1.40 s
............
*************************************************************************

Is there a format mask I can provide to decode the Text field, which I'm pretty sure holds an NMEA message with the GPS.
Or is the decode more fundamental and needs deeper decoding within ExifTool?

Thanks

[StarGeek]

I edited you post to add the Code formatting to make it easier to read.

Can you make a small sample file available with movement, like going to the end of the block?  Phil might need to actually look at the file to see if he can support it.

[diplonics]

Thanks StarGeek for the format fix.
This sample file link should work:

https://ufile.io/24orw4k2

[Phil Harvey]

I've had a quick look at this.  I haven't seen anything like this before, and can't find any NMEA in this text.  The checksum at the end is the only thing that looks vaguely like NMEA.  I've tried the standard decryption technique, using various keys, but no luck.

Do you know the coordinates at the start of the video?  I'll analyze this when I get a chance, and knowing this may help.

- Phil

Edit: Ah, I found the answer for how to decrypt this on stackexchange:

Code Select Expand

#include <stdio.h>

const char *decode_table = "#I8XQWRVNZOYPUTA0B1C2SJ9K.L,M$D3E4F5G6H7";

void decode_in_place(char *s) {
    while (*s) {
        *s = decode_table[*s-43];
        s++;
    }
}

int main(void) {
    char data[] = ".+;;;D=;-;6;;;;D;JP;4;;;=D;P?;O;;;=D=L;-HO71G>F=;;;JJF:FNJNBDL=R?F3F;;=;PDLR;;F0F;;=DRFJJ?DRJF??;J=LF;;;D;F:*59~";
    decode_in_place(data);
    printf("%s", data);
    return 0;
}
// Output: X#000.1080Y0000.0360Z0001.0620G0001.1408$GPRMC,100033,A,5359.4172,N,00106.4700,W,001.7,332.73,220314,000.0,A
//X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,UTS_Position,Status,Latitude,N/S,Longitude,E/W,Speed,Course_Over,Ground,Date,Magnetic_variation,Checksum~


I should be able to add support for this in the next release.

[diplonics]

Hi Phil,
       Brilliant in loads of ways, spent ages on google looking for Roadhawk specs etc... to solve this, finding nothing.
Think I need a lesson of effective searching!!! either way thanks a mill with this and even better, till next release, I can use this mask to decode as is currently output by ExifTool.

Also looks like the end of the string doesn't need decoding, the *59~ has no substitution for * or ~ and GPRMC strings end with this *59 type checksum anyway..
As all the 'text' field strings seem the same length I'm assuming then I can just end the substitution at the *..

[Phil Harvey]

I'm pretty sure the first character in the decode_table should be "-" instead of "#".  This makes sense for the signs in the accelerometer data.  I'll decode it this way.

Also, I am not going to decode the magnetic variation.  This should be two fields in NMEA RMC, but the second field ("E" or "W", indicating the direction of the variation) is missing.

- Phil

(That will make 42 different types of streaming GPS that ExifTool extracts from video files.)

[diplonics]

Yea that seems sensible changing # for a -.
The magnetic variation field is probably a bug on their side as its always a : (decodes to A) in all the examples that I have that I've run through ExifTool.

Also, re your 42 different types, if there a definitive list?
Through my University work I've access to a number of different cameras, old and new, with loads of data to test, some legacy.
Testing ExifTool on some of it reveals the following:

Roadhawk (File Type .mp4) - ExifTool works great, just need to handle the mask operation separate until next ExifTool release

Parrott Anafi (File Type .mp4) - ExifTool works fine, seems to dump all GPS Date/Time data to the end of the output in 1 block, rest seems sequential

GoPro Hero 8 (File Type .mp4) - ExifTool works fine, very high GPS hrz in these files but seems reasonable to parse

Nextbase 312GW, 412GW and 512G (File Type .mov) - ExifTool works perfect..
   
360Fly (File Type .mp4) - Exiftool doesn't find GPS but I have PHP code to rip it.   

Have an old .ts MPEG-2 format - ExifTool doesn't find GPS it but I have c code to rip it.
Have an old .mts AVCHD format - ExifTool seems to find lots of detailed frame level data but no GPS, maybe no GPS in it..

[Phil Harvey]

Also, re your 42 different types, if there a definitive list?

No.  I don't even know the make/model of some of my samples.

360Fly (File Type .mp4) - Exiftool doesn't find GPS but I have PHP code to rip it.

Have an old .ts MPEG-2 format - ExifTool doesn't find GPS it but I have c code to rip it.

Could you send me samples of these?  (philharvey66 at gmail.com)

- Phil

[Phil Harvey]

I got the samples, thanks!

I should be able to add the ability to extract GPS from the 360fly video, hopefully for the next release.  Your code is very helpful.  Just a few questions: 1. Do you know what the mode values for each structure type signify?  2. Why 3 sets of yaw/pitch/roll (ie. what is "Fusion"?  I guess the other 2 are derived separately from the accelerometers and gyros).

- Phil

[diplonics]

Seen as the 360fly can be uploaded to Google Streetview I reckon their specs relate to these:
https://developers.google.com/streetview/publish/camm-spec#data-format

This seems to give a reasonably detailed explanation of all the sensor fields, some real some derived it seems.
The relationship between the ID3 outputs and Google specs is probably:
fusion_data => position
accel_data => acceleration
gyro_data => gyro
magno_data => magnetic-field
gps_data => gps long form
rotation => angle-axis

[Phil Harvey]

Great, thanks.  I'll look into this.

- Phil

[osakahk2009]

Hi Phil,
I have a mp4 with GPS info. shot by Sony A7R IV and I use your exiftoolwrapper to extract them but it doesn't show up.
Please help to check it out.
I can send you a small sample and please let me know how.

Thanks.