ExifTool Forum

ExifTool => Newbies => Topic started by: SM26 on February 13, 2019, 09:41:18 AM

Title: Extracting GPS data from INSV file
Post by: SM26 on February 13, 2019, 09:41:18 AM
Hello everyone

I have a problem, and I hope you can direct me.

I have a 360 camera, Insta360 One X.
It produce an unstitched file in INSV (proprietary) file format.
I can change it to MP4 and it works just fine
or I can use a dedicated programme to produce a stitched version in MP4 format.

The camera records GPS data, and when I use the Android app with the INSV files, I can see all of the GPS data

I was wondering if it was possible to extract all of the GPS data (preferably in GPX format) out of it.

My system:
Windows 10, 64 Bit.
ExifTool V11.26
ExifToolGUI V5.16

In the Gui, I've clicked "Export data to XMP"
but I get "LargeFileSupport not enabled" warning.
I've made a .ExifTool_config file in the same directory as the exiftool itself, with an additional 2 lines (LargeFileSupport => 1 & print "LOADED!\n"; ):

%Image::ExifTool::UserDefined::Options = (
    CoordFormat =&gt; '%.6f',  <span class=com># change default GPS coordinate format</span>
    Duplicates =&gt; 1,        <span class=com># make -a default for the exiftool app</span>
    GeoMaxHDOP =&gt; 4,        <span class=com># ignore GPS fixes with HDOP &gt; 4</span>
    RequestAll =&gt; 3,        <span class=com># request additional tags not normally generated</span>
    LargeFileSupport => 1,

);
print "LOADED!\n";
<span class=com>#------------------------------------------------------------------------------</span>
1;  <span class=com>#end</span>
</pre>
<hr>
<p class='lf'># <a href="index.html">&lt;-- Back to ExifTool home page</a></p>
</body>
</html>

I still get the warning, and I've read FAQ 11 and followed it to the letter, still happening.

Edit:
If it matters, I've also tried to use MetaData++, which is based on ExifTool. still no luck.

I would really appreciate any direction you could throw at me.
I'm also currently upload the INSV file to Google Drive so you can take a look for yourself (it's gonna take some time, it's a 5.8GB file).

TIA
-SM26
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 14, 2019, 08:11:18 AM
Here is the INSV file if someone is willing to take a look:
https://drive.google.com/open?id=1bsEVAGx4Co1DqINziaXF2hy-HinD54kl
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 14, 2019, 09:55:32 AM
From the command line I get the following information from your file with -api largefilesupport added to the command:

> exiftool tmp/VID_20190126_125708_00_046.insv -api largefilesupport
ExifTool Version Number         : 11.26
File Name                       : VID_20190126_125708_00_046.insv
Directory                       : tmp
File Size                       : 5.9 GB
File Modification Date/Time     : 2019:02:14 09:44:40-05:00
File Access Date/Time           : 2019:02:14 09:45:58-05:00
File Inode Change Date/Time     : 2019:02:14 09:44:40-05:00
File Permissions                : rw-r--r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 Base w/ AVC ext [ISO 14496-12:2005]
Minor Version                   : 0.0.0
Compatible Brands               : avc1, isom
Movie Data Size                 : 6271586288
Movie Data Offset               : 48
Movie Header Version            : 0
Create Date                     : 2019:01:26 12:56:58
Modify Date                     : 2019:01:26 12:56:58
Time Scale                      : 1000
Duration                        : 0:09:49
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 4
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 1
Track Duration                  : 0:09:49
Track Layer                     : 0
Track Volume                    : 0.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Image Width                     : 3840
Image Height                    : 1920
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 50000
Media Duration                  : 0:09:49
Handler Type                    : Video Track
Handler Description             : Ambarella AVC
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 3840
Source Image Height             : 1920
X Resolution                    : 72
Y Resolution                    : 72
Compressor Name                 : Ambarella AVC encoder
Bit Depth                       : 24
Pixel Aspect Ratio              : 1:1
Stereoscopic 3D                 : Monoscopic
Metadata Source                 : .AMBA Spherical Video
Pose Yaw Degrees                : 0
Pose Pitch Degrees              : 0
Pose Roll Degrees               : 0
Projection Bounds Top           : 0
Projection Bounds Bottom        : 0
Projection Bounds Left          : 0
Projection Bounds Right         : 0
Video Frame Rate                : 50
Spherical                       : true
Stitched                        : true
Stitching Software              : AMBA Spherical Video
Projection Type                 : equirectangular
Stereo Mode                     : mono
Source Count                    : 2
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 2
Track Duration                  : 0:09:49
Track Layer                     : 0
Track Volume                    : 100.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 48000
Media Duration                  : 0:09:49
Handler Type                    : Audio Track
Handler Description             : Ambarella AAC
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 2
Audio Bits Per Sample           : 0
Audio Sample Rate               : 48000
Track Header Version            : 0
Track Create Date               : 2019:01:26 12:56:58
Track Modify Date               : 2019:01:26 12:56:58
Track ID                        : 3
Track Duration                  : 0:09:49
Track Layer                     : 65535
Track Volume                    : 0.00%
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2019:01:26 12:56:58
Media Modify Date               : 2019:01:26 12:56:58
Media Time Scale                : 1000
Media Duration                  : 0:09:49
Handler Type                    : NRT Metadata
Handler Description             : Ambarella EXT
Meta Format                     : camm
Warning                         : [minor] The ExtractEmbedded option may find more tags in the movie data
Avg Bitrate                     : 85.1 Mbps
Base Name                       : vid
File Extension                  : insv
File Type Description           : MPEG-4 video
Image Size                      : 3840x1920
Megapixels                      : 7.4
Rotation                        : 0


So this should work for you if you can get LargeFileSupport enabled.

The config file you posted contains some HTML markup which would stop it from working.  Try this config file named ".ExifTool_config" in the exiftool.exe directory and/or your home directory:

%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);
1; #end


- Phil

Edit:  But wait.  You wanted to extract GPS data.  There is no valid GPS that I can find in this file.  It does contain some GPS information, but it is all void in this video (ie. the receiver didn't acquire a GPS fix).
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 18, 2019, 04:10:53 AM
Thank you Phil,

I've tried this short config file you posted and it works!

and now to the main event, GPS Data.
I can try to create some new testing materials, this time short in duration and small in size for ease of processing.
Do you think that this might help?

I know for a fact that via the app, I can post pictures to Google Street View, so I'm guessing there must be some GPS data hidden somewhere....
I've also dug in their app, nothing useful that I can see.

I'll post here a link to Google Drive when I'll finish uploading all of the new test data.

Thanks Phil!

- Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 18, 2019, 07:38:12 AM
I'll take a look at the files that you upload.  Please confirm that the app can obtain GPS from this file alone (often cameras write sidecar metadata files.  Be sure it isn't getting the information from here.  Also, let me know the GPS coordinates it obtains, and whether it gives a single GPS point or a full track from the video.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 19, 2019, 10:28:23 AM
Ok, so I've done 3 short test videos.

But before that, here is how I set it up:

Camera is Insta360 One X, FW V1.17.16.
4K 30 FPS, with "stats" indicator turns on in the app, with "Google Street View" indicator turns on in the app.
Insta360 say that the GPS data is recorded from the phone's GPS, but from what I can tell, it's not inside the app data.

1. Shot first video (vid_050.insv) , saw that I can see GPS data in the app.
2. Backup the INSV file to the PC
3. Format phone, switched to android 9.
4. Load back the INSV file into the phone, GPS data is in the app.
5. Shot while the phone is stationary, and I move around with the camera (Vid_051.insv).
6. Shot while the camera is stationary, and I move around with the phone (Vid_052.insv).

I've also tried to use:
ffprobe vid_050.insv output:
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'vid_050.insv':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1isom
    creation_time   : 2019-02-19T16:26:32.000000Z
  Duration: 00:00:58.49, start: 0.000000, bitrate: 46167 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, smpte170m), 3840x1920 [SAR 1:1 DAR 2:1], 44678 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella AVC
      encoder         : Ambarella AVC encoder
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella AAC
    Stream #0:2(eng): Data: none (camm / 0x6D6D6163), 7 kb/s (default)
    Metadata:
      creation_time   : 2019-02-19T16:26:32.000000Z
      handler_name    :  Ambarella EXT
Unsupported codec with id 0 for input stream 2

Followed by:
ffmpeg -y -i vid_050.insv -codec copy -map 0:2 -f rawvideo out-050.bin
which basically gave me a whole lot of:
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50650 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50750 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50850 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 50950 
X0000.0000Y0000.0000Z0000.0000G0000.0000$GPRMC,000125,V,,,,,000.0,,280908,002.1,N*71~, 51050 


XMP output is:

<?xpacket begin='' id='W5M0MpCehiHzreSzNTczkc9d'?>
<x:xmpmeta xmlns:x='adobe:ns:meta/' x:xmptk='Image::ExifTool 11.26'>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

<rdf:Description rdf:about=''
  xmlns:GPano='http://ns.google.com/photos/1.0/panorama/'>
  <GPano:ProjectionType>equirectangular</GPano:ProjectionType>
  <GPano:StitchingSoftware>AMBA Spherical Video</GPano:StitchingSoftware>
</rdf:Description>

<rdf:Description rdf:about=''
  xmlns:GSpherical='http://ns.google.com/videos/1.0/spherical/'>
  <GSpherical:SourceCount>2</GSpherical:SourceCount>
  <GSpherical:Spherical>True</GSpherical:Spherical>
  <GSpherical:StereoMode>mono</GSpherical:StereoMode>
  <GSpherical:Stitched>True</GSpherical:Stitched>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
<?xpacket end='w'?>


Did this for 051 and 052 with no change.
Files can be seen here:
https://drive.google.com/open?id=1NbzpGAF-mSrpNAzMDuyDGfF0_SV9_92S

Thanks in advance,
- Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 19, 2019, 10:49:09 AM
OK, I think I can see where the GPS is stored.  Can you tell me ALL of the information displayed by "the app" for the first fix of VID_050?  I need the GPS coordinates, speed, time and whatever else is extracted to be able to decode this from the binary data.

- Phil

Edit:  What is "the app" BTW?  I could run it myself it there is a Mac version available.
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 19, 2019, 11:03:05 AM
from vid_050.insv, the data in the app is:

Time: 00:28 (I can't see frame number in the app)
location: 32° 5′ 16″ N 34° 51′ 22″ E
direction: 288° W
Route +  a dot where I'm on the route + number of Km from the start.
Slope: 47.7%
Elevation: 61.63M
Speed: 2.2 Km/h

I've added a screen shot from the app inside the google drive folder.
(I know that the picture is all black, it's fine, the camera was in my pocket)
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 19, 2019, 12:31:51 PM
Thanks.  Decoding all this will take some time.  I have found 4 different types of unknown records.  One of these records contains the following information, which gives time/lat/lon plus some other numbers that I don't yet understand.  Here is the dump of the start of this record as decoded from VID_050:

Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:34 2019 0 0 A 32.08812115 N 34.85583211 E 0 0 0 0 0 3.3813259601593
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:35 2019 0 0 A 32.08812112 N 34.85583217 E 0 0 0 0 0 3.3813681602478
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:36 2019 0 0 A 32.08812111 N 34.85583224 E 0 0 0 0 0 3.38141536712646
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:39 2019 0 0 A 32.08812049 N 34.85583874 E 0 0 0 0 0 3.38130974769592
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:40 2019 0 0 A 32.08811732 N 34.85583865 E 0 0 0 0 0 3.37976598739624
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:41 2019 0 0 A 32.0881204 N 34.8558457 E 192 1.77749991416931 192 3.23906230926514 0 3.37661838531494
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:42 2019 0 0 A 32.08812321 N 34.85584637 E 160 1.70999991893768 160 3.23984360694885 0 3.37342476844788
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:43 2019 0 0 A 32.08812381 N 34.85584867 E 32 1.76499998569489 96 3.25234365463257 0 3.36754560470581
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937
...


- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 20, 2019, 09:17:18 AM
More information:  Here is a dump of the first ten lines of each of the 5 different unknown record types I can find in VID_050.  Please let me know if you know the significance of any of the floating-point numbers in records 0x300, 0x400 or 0x700 (other than the lat/lon in record 0x700, which is obvious).  The first number in records 0x300 and 0x400 is a time code in milliseconds, but I don't know how to relate this to either clock time or a frame in the video.  Records 0x101 and 0x200 are presented as a hex dump, while the others are decoded into variable types that seemed appropriate.

Also, is "IXE3518NYGDB3N" the serial number of your Insta360 One2?

============ Record 0x101
    0000: 0a 0e 49 58 45 33 35 31 38 4e 59 47 44 42 33 4e [..IXE3518NYGDB3N]
    0010: 12 0d 49 6e 73 74 61 33 36 30 20 4f 6e 65 32 1a [..Insta360 One2.]
    0020: 0f 76 31 2e 31 37 2e 31 36 5f 62 75 69 6c 64 31 [.v1.17.16_build1]
    0030: 2a 6e 32 5f 31 34 37 37 2e 39 38 38 5f 31 35 32 [*n2_1477.988_152]
    0040: 37 2e 39 31 35 5f 31 35 32 33 2e 37 34 36 5f 30 [7.915_1523.746_0]
    0050: 2e 30 30 30 5f 30 2e 30 30 30 5f 30 2e 30 30 30 [.000_0.000_0.000]
    0060: 5f 31 34 38 31 2e 30 37 38 5f 34 35 34 34 2e 33 [_1481.078_4544.3]
    0070: 31 32 5f 31 35 32 34 2e 35 38 33 5f 30 2e 31 31 [12_1524.583_0.11]
    0080: 39 5f 2d 30 2e 31 36 35 5f 2d 31 37 39 2e 37 30 [9_-0.165_-179.70]
    0090: 31 5f 36 30 38 30 5f 33 30 34 30 5f 32 33 32 33 [1_6080_3040_2323]
[...]
============ Record 0x200
    0000: 00 00 00 01 7a 01 01 00 00 23 e9 10 00 3d 4c 04 [....z....#...=L.]
    0010: 00 02 00 01 20 10 00 00 00 01 67 4d 40 33 9a 64 [.... .....gM@3.d]
    0020: 01 e0 07 8d 80 b7 06 06 06 40 00 00 fa 40 00 3a [.........@...@.:]
    0030: 98 3a 18 00 0b eb c0 00 02 fa f0 ae f2 e3 43 00 [.:............C.]
    0040: 01 7d 78 00 00 5f 5e 15 de 5c 3e 11 08 b7 00 00 [.}x.._^..\>.....]
    0050: 00 01 68 ee 38 80 00 00 00 01 65 b8 01 00 00 8f [..h.8.....e.....]
    0060: f5 2a 11 14 72 19 cc 78 2f e9 1f 26 ea fc 1c 79 [.*..r..x/..&...y]
    0070: 44 64 6a 67 a3 7a 94 d1 3c 50 52 93 8a ab 2c ec [Ddjg.z..<PR...,.]
    0080: d1 10 19 c3 9d 13 44 d8 08 2f 6c a8 30 12 60 9c [......D../l.0.`.]
    0090: b3 47 2e 04 21 22 0c 32 5b b3 4f 77 37 69 91 1f [.G..!".2[.Ow7i..]
[...]
============ Record 0x300
50.394 0.555908203125 -0.826416015625 0.10595703125 -0.00426105774412678 0.00319579330809508 0.00319579330809509
50.396 0.558349609375 -0.828369140625 0.10205078125 -0.00213052887206339 0.00213052887206339 0.00213052887206339
50.398 0.55712890625 -0.830322265625 0.103759765625 -0.00426105774412678 0.00106526443603169 0.00213052887206339
50.400 0.55908203125 -0.8291015625 0.105224609375 -0.00213052887206339 0 0.00106526443603169
50.402 0.560546875 -0.8291015625 0.10205078125 -0.00213052887206339 0 0.00106526443603169
50.404 0.55859375 -0.830322265625 0.101318359375 -0.00213052887206339 0.00106526443603169 0
50.406 0.559814453125 -0.82958984375 0.103271484375 -0.00213052887206339 0.00213052887206339 0
50.408 0.5595703125 -0.828857421875 0.10595703125 -0.00213052887206339 0.00426105774412678 -0.00106526443603169
50.410 0.561279296875 -0.827880859375 0.10791015625 -0.00213052887206339 0.00426105774412678 -0.00319579330809509
50.412 0.558349609375 -0.830322265625 0.107177734375 -0.00213052887206339 0.00319579330809508 -0.00213052887206339
[...]
============ Record 0x400
50.391 0 0 0 96 1.25778567790985
50.425 0 0 0 96 1.25778567790985
50.458 0 0 0 96 1.25778567790985
50.492 0 0 0 96 1.25778567790985
50.525 0 0 0 96 1.25778567790985
50.558 0 0 0 96 1.25778567790985
50.592 0 0 0 96 1.25778567790985
50.625 0 0 0 96 1.25778567790985
50.658 0 0 0 96 1.25778567790985
50.692 0 0 0 96 1.25778567790985
[...]
============ Record 0x700
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:32 2019 0 0 A 32.08812118 N 34.85583192 E 0 0 0 0 0 3.38119339942932
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
Tue Feb 19 09:26:33 2019 0 0 A 32.08812117 N 34.85583205 E 0 0 0 0 0 3.38129377365112
[...]


It seems that Record 0x300 may contain accelerometer information from the phone because it contains more interesting numbers for VID_052:

============ Record 0x300
120.810 0.697021484375 0.349609375 -0.662353515625 0.181094954125388 -0.385625725843473 -0.0372842552611093
120.814 0.701171875 0.355224609375 -0.663330078125 0.196008656229832 -0.382429932535378 -0.0362189908250776
120.816 0.70068359375 0.35986328125 -0.662353515625 0.208791829462212 -0.380299403663315 -0.0351537263890459
120.818 0.69970703125 0.367919921875 -0.664306640625 0.219444473822529 -0.376038345919188 -0.0340884619530142
120.820 0.696533203125 0.368408203125 -0.659912109375 0.227966589310783 -0.371777288175061 -0.0319579330809508
120.822 0.69580078125 0.3740234375 -0.664794921875 0.232227647054909 -0.365385701558871 -0.0276968753368241
120.824 0.69482421875 0.375 -0.6572265625 0.234358175926973 -0.360059379378713 -0.0223705531566656
120.826 0.693359375 0.3720703125 -0.654541015625 0.234358175926973 -0.356863586070618 -0.0191747598485705
120.828 0.692626953125 0.372314453125 -0.654052734375 0.234358175926973 -0.352602528326491 -0.013848437668412
120.830 0.69580078125 0.3671875 -0.655029296875 0.234358175926973 -0.350471999454427 -0.00852211548825355
[...]


- Phil

Edit: I think it is likely that after the time code, the next 3 numbers are XYZ acceleration in units of g, and the last 3 numbers are angular velocities about each axis (probably rad/s).
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 21, 2019, 07:48:06 AM
I've figured out a bit more, and can now decode GPSSpeed, GPSTrack and GPSAltitude, but I need to know what units the GPSSpeed is measured in.  For VID_051, the GPSSpeed starts with a value of 0.83.  I'm thinking this may be in m/s, which would be 2.99 km/h.  What does your app show for the speed at the start of this video?

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 21, 2019, 12:08:54 PM
Hello Phil,

First of all, I want to say a huge thank you for taking the time and handling it.
I'll try to answer everything as best of my knowledge, let's hope I don't forget anything.

1. From the dump of Vid_050:

Tue Feb 19 09:26:37 2019 0 0 A 32.08812121 N 34.85583249 E 0 0 0 0 0 3.38151264190674
Tue Feb 19 09:26:38 2019 0 0 A 32.08811989 N 34.85583259 E 0 1.62999999523163 64 3.67499995231628 0 3.37774848937988

in the first line, I'm guessing that the info we have here is: longitude, latitude, elevation.
on the second line, I have no idea what are the rest of this stuff: 1.62999999523163 64 3.67499995231628

2. As far as I know, accuracy of longitude and latitude data is measured up to 7 numbers after the floating point.
here I can count 8, which means that it's accurate to about 1 meter, 7 is fine by me (accurate to about 10 meters)
for me accuracy of elevation data of 4 numbers after the floating point is also sufficient.
I don't mind the extra accuracy, but just to let you know what I consider "accurate data measurement"

3. Regarding time code: I know that the video is 30 FPS, is this helping?
I don't think that the phone is sampling the GPS data every second, but I might be mistaking.

4. Regarding 0x101:
Yes, IXE3518NYGDB3N is the camera S/N.
Insta360 is the name of the company
One2 is the technical name of the One X (just like phones, where Samsung Galaxy S5 is SM-G900F)
V1.17.16 is my firmware version.
lines 30 and onward - I have no idea.

5. Record 0x200 also makes no sense to me.

6. Just like you said, 0x300 seems like 6 axis data to me.
I have no idea how to interpret it (what axis is what number) but I'm assuming it's in the following order:
Roll: rotation around the front-to-back axis, Pitch: rotation around the side-to-side axis, Yaw: rotation around the vertical axis
And then the next 3 numbers are from the accelerometer.
I have a friend who might be able to help a bit, I'll run it by him.

7. Regarding recorded 0x400, can it be that 96 is Fahrenheit of the phone? does it fluctuate much?

8. When looking at Vid_050 dump:

Tue Feb 19 09:26:44 2019 0 0 A 32.08812709 N 34.85585249 E 224 1.80249989032745 0 3.23828125 0 3.35669207572937

Can it be that the two 0 before the "A" is altitude? and that's why it's 0? (don't know why you got 0 twice though)
And after the "E", 224 looks like a direction to me, like SW on a compass.

9. Vid_051 is when the phone is stationary and the camera moves.
So, logicly, the speed should be 0
(because the base assumption is that the camera don't get a GPS signal, only the phone)
I've checked the video, the data I see in the app for Vid_052 at 00:00 is:
speed: 0.58 km/h
elevation: 58.33
slope: 0.0%
routh: 0.00km
direction: 293° NW
location: 32°5'16"N , 34°51'22"E

And at 00:03 (approx when I put my phone down and start walking around with the camera):
speed: 0.00 km/h
elevation: 58.33
slope: 29.7%
routh: 0.00km
direction: 0° N
location: 32°5'16"N , 34°51'22"E

I found out in the app that for Vid_050 at 00:15 I get to speed of 2.99 km/h, here is the full data:
speed: 2.99 km/h
elevation: 89.96M
slope: 100.0%
routh: 0.00km
direction: 86° E
location: 32°5'17"N , 34°51'21"E



Sorry this post got so long,
for me all of this is really an uncharted territory and this is the first time I'm dealing with this kind of a challenge
Thanks again
-Or
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 21, 2019, 12:37:26 PM
Great, thanks.  I'm pretty confident now that I'm extracting GPSLatitude/Longitude/Speed/Altitude/Track correctly.  Note that you must use the -ee option to extract this information.

I've just released ExifTool 11.28 with this update.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 21, 2019, 12:49:16 PM
Great!

Thanks,
I'll download the new version
and create some more test samples, we will see how it goes from there.
(so far I've only used the GUI version, so I'll probably need a day or two to read the documentation about the CMD version.)
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 06:45:00 AM
Quote from: Phil Harvey on February 21, 2019, 12:37:26 PM
Great, thanks.  I'm pretty confident now that I'm extracting GPSLatitude/Longitude/Speed/Altitude/Track correctly.  Note that you must use the -ee option to extract this information.

I've just released ExifTool 11.28 with this update.

- Phil

Hello Phil,

I think I got it,
one thing I'm not 100% sure about,
I use the command: exiftool -ee -p gpx.fmt Vid_052.insv > Vid052.gpx

And I've used the GPX.FMT file from here:
https://www.exiftool.org/geotag.html#GPX

currently I see: longitude, latitude, elevation.
Should I modify the FMT file somewhat?
just want to make sure I'm not missing out on any data available.

Just found this little gem:
https://exiftool.org/forum/index.php/topic,5095.msg46514.html#msg46514
do we have some gyro data to add to the FMT file like so; "ACC $framenumber $accelerometer" ?

Thanks in advance.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 07:03:27 AM
Yes.  Modify the gps.fmt file to write anything you want.  You may have to consult the GPX specification to see how to format this properly.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 07:45:26 AM
Sorry, But I don't quite follow

Like we said earlier, record 0x300 seems like accelerometer information.

When I look at the GoPro and Ricoh tag pages, I see they have accelerometer data.
What and how should I alter the FMT to show all of the data?

I've made small changes to the FMT file, but now it shows up blank...

#------------------------------------------------------------------------------
# File:         gpx.fmt
#
# Description:  Example ExifTool print format file for generating GPX track log
#
# Usage:        exiftool -p gpx.fmt FILE [...] > out.gpx
#
# Requires:     ExifTool version 10.49 or later
#
# Revisions:    2010/02/05 - P. Harvey created
#               2018/01/03 - PH Added IF to be sure position exists
#               2018/01/06 - PH Use DateFmt function instead of -d option
#
# Notes:     1) Input files must contain GPSLatitude and GPSLongitude.
#            2) Add the -ee option to extract the full track from video files.
#            3) The -fileOrder option may be used to control the order of the
#               generated track points when processing multiple files.
#------------------------------------------------------------------------------
#[HEAD]<?xml version="1.0" encoding="utf-8"?>
#[HEAD]<gpx version="1.0"
#[HEAD] creator="ExifTool $ExifToolVersion"
#[HEAD] xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#[HEAD] xmlns="http://www.topografix.com/GPX/1/0"
#[HEAD] xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
#[HEAD]<trk>
#[HEAD]<number>1</number>
#[HEAD]<trkseg>
#[IF]  $gpslatitude $gpslongitude
#[IF]  $accelerometer
#[BODY]<trkpt lat="$gpslatitude#" lon="$gpslongitude#">
#[BODY]  <ele>$gpsaltitude#</ele>
#[BODY]  <acc>$accelerometer#</acc>
#[BODY]  <time>${gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")}</time>
#[BODY]</trkpt>
#[TAIL]</trkseg>
#[TAIL]</trk>
#[TAIL]</gpx>

From the GoPro page, I think that the following tags are interesting:
ACCL, GLPI, GPRI, GPS5, GYRO, MAGN
And from the Ricoh page, I think that the following tags are interesting:
Ricoh ThetaSubdir Tags ->0x0003 Accelerometer, 0x0004 Compass

Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 07:57:06 AM
The main problem with what you are trying to do is that the accelerometer readings aren't timed the same as the GPS, so they can't be combined like this.  They are based on a TimeCode, while the GPS readings are based on GPSDateTime.  You can add the GPSSpeed and GPSTrack to the GPS records in the GPX file, but it would take some work to be able to relate the TimeCode values to a clock time so you could add in the accelerometer information.  Frankly, I don't know how to do this.  Use this command to see all of the available information:

exiftool -ee -G3 FILE

In this output, each document number represents the information extracted for a specific SampleTime, TimeCode or GPSDateTime.  Note that GPS is sampled at a much different rate than the accelerometer readings.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 08:53:06 AM
Ok, I think I have an idea,

Based on Garmin data that I found (and let's hope that Insta360 log data the same way as Garmin)
each GPS record correspond with 10 Accelerometer and Angular Velocity docs. (for now let's leave the offset out of this)
https://www8.garmin.com/xmlschemas/AccelerationExtensionv1.xsd
Let's hope that I what the XYZ value match correspondingly

So, for file 052, the EOF should look like this:
<trkpt lat="32.08802777777778" lon="34.85615555555556">
        <ele>61.5087</ele>
        <time>2019-02-19T14:29:02Z</time>
        <extensions>
          <acc:AccelerationExtension xmlns:acc="http://www.garmin.com/xmlschemas/AccelerationExtension/v1">
            <acc:accel x="0.065185546875" y="-0.977294921875" z="0.24462890625" />
            <acc:accel x="0.06591796875" y="-0.97607421875" z="0.242431640625" />
            <acc:accel x="0.064453125" y="-0.975830078125" z="0.23974609375" />
            <acc:accel x="0.064453125" y="-0.975341796875" z="0.24267578125" />
            <acc:accel x="0.0654296875" y="-0.979248046875" z="0.2421875" />
            <acc:accel x="0.061279296875" y="-0.977294921875" z="0.239501953125" />
            <acc:accel x="0.065185546875" y="-0.977294921875" z="0.24462890625" />
            <acc:accel x="0.0625" y="-0.9765625" z="0.24267578125" />
            <acc:accel x="0.0625" y="-0.97265625" z="0.239501953125" />
            <acc:accel x="0.066162109375" y="-0.974609375" z="0.240478515625" />
          </acc:AccelerationExtension>
          <Gyro:Angular Velocity>
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00852211548825355" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00852211548825355" y="0.0149137021044437" z="0.00958737992428525" />
            <acc:accel x="0.00852211548825355" y="0.0159789665404754" z="0.0106526443603169" />
            <acc:accel x="0.00852211548825355" y="0.013848437668412" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.013848437668412" z="0.00745685105222186" />
            <acc:accel x="0.00639158661619017" y="0.013848437668412" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0149137021044437" z="0.00852211548825355" />
            <acc:accel x="0.00639158661619017" y="0.0127831732323803" z="0.00958737992428525" />
          </Gyro:Angular Velocity>
        </extensions>
      </trkpt>


Any change to make something like this?
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 09:03:36 AM
I still don't know how to relate the accelerometer TimeCode values to the GPSDateTime.  You can't just assume that the first TimeCode corresponds with the first GPSDateTime because.

1. It is quite likely that the GPS system takes a while to acquire a lock, and hence the first reading may be a while after the start of the video.

2. You would likely be wrong by half of the GPS sampling period on average even if they did start at the same time.

But even if you can relate these times, constructing the combined GPX output file is currently beyond the capabilities of ExifTool, and would require some dedicated scripting to achieve.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 09:30:10 AM
Ok,
I'll spend some more time with it, maybe I'll manage to learn something new.

Last question (hopefully)
I have a file that is 18 GB (19,220,410,368 bytes to be exact)

When I run Exiftool -ee -p gpx2.fmt Vid_long.insv > Vid_Long.gpx
I get:
Out of memory!

for the record, my config file has "LargeFileSupport => 1" in it.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 25, 2019, 09:34:07 AM
Unfortunately there is a limit on the amount of memory that a Windows command-line process may allocate.  I don't know why this limit is imposed, but it is around 300 MB or so.  Mac and Linux have no such limitation.  I don't know any way around this on Windows.  Processing very large video files with the -ee option does use a fair bit of memory.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 25, 2019, 09:37:09 AM
oh, Ok.
Thanks!

good think I have a linux laptop laying around
Title: Re: Extracting GPS data from INSV file
Post by: SM26 on February 28, 2019, 04:28:18 AM
So, just to sum it up
Phil, you have helped me a lot.

Insta360 replied to my email, and they said that they are not going to help understand the Gyro data.

But nevertheless, for your time and effort
as a token of my appreciation, let me buy you a pint!

PayPal's Transaction ID: 3RE879167D940642R
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on February 28, 2019, 07:00:32 AM
Thanks!

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 11:33:00 AM
Hello Everyone,

I am trying to use Exiftool 11.66 for extracting GPS data from .insv file taken using (Insta 360 ONEX camera).

The command I used:
exiftool -ee -p "GPS $gpslatitude $gpslongitude" -n "C:\VID_003.insv" > VID_003.txt

I am getting results. Latitude looks right however Longitude shows incorrect value.

GPS 40.74103544 -5450.57354413288

Appreciated your help.

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on September 30, 2019, 11:38:42 AM
Hi Yamuna,

It could be that the format of this metadata is somehow different than my current INSV sample.

Could you possibly send me a (hopefully small) INSV file with GPS information?  I'll see if I can figure out what is going on.  My email is philharvey66 at gmail.com

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 01:00:12 PM
Hi Phil,

Thanks for the swift response. sent you the sample insv file.

Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on September 30, 2019, 01:20:01 PM
Hi Yamuna,

I got the sample, thanks.

This was a bug in ExifTool that I have fixed in version 11.67 (just released).

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on September 30, 2019, 01:51:04 PM
Thanks Phil, It works.
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 12:30:19 PM
Hi Phil,

When I try to extract larger file (9GB) I am getting "out of Memory" error.

exiftool -ee -p "$gpsdatetime $gpslatitude $gpslongitude" -n "test.insv"> test.csv

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 04, 2019, 12:32:43 PM
Could you attach the output of the -v3 command for this file?
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 01:56:25 PM
Attached the image file.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 04, 2019, 01:57:32 PM
That doesn't help as I only see the start of the output.  Also, you will need to add the LargeFileSupport option:

So try running this command and attaching the "out.txt" file (or email it to me if you want: philharvey66 at gmail.com)

exiftool -api largefilesupport -v3 FILE > out.txt

Thanks.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 04, 2019, 02:07:09 PM
Attached. Thanks.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 07:21:39 AM
Unfortunately I can't figure out the problem from the verbose output.  I will need the sample file to be able to reproduce the problem.  Is there any way you could upload this to a file sharing service somewhere?  Then email me a link (philharvey66 at gmail.com)

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 07, 2019, 11:30:52 AM
Hi Phil,

I sent you the video link to download 9gb insv file.
Let me know once you download the file, i will remove the link.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 12:12:05 PM
Hi Yamuna,

I got the file, thanks.

The problem is simply that the file contains too much metadata to read into the limited memory space available to the Windows exe version of ExifTool (about 200 MB or so).  The file contains about 800000 accelerometer records, which is the main problem.  I'll see about limiting the number of these that ExifTool will read to work around this limitation.  It contains about 16500 GPS records, but ExifTool should be able to read all of these.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 07, 2019, 02:07:04 PM
Hi Phil,

I tried from Linux Virtual machine and I got into the same issue.

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 07, 2019, 08:57:01 PM
I'm surprised.  At any rate, ExifTool 11.70 will fix this by not attempting to load all of the accelerometer data unless you use the -m option.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 09:42:40 AM
Thanks Phil.
When are you planning to release version 11.70 ?

Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 09:47:10 AM
Probably within a week.  I'm attaching the updated code which you can use to replace lib/Image/ExifTool/QuickTimeStream.pl in the Linux version to fix this in your copy before the new version comes out.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 11:29:02 AM
Hi Phil,

I tried using that file, no error and no data.

./exiftool -api largefilesupport -ee -p "$gpstimestamp,$gpslatitude,$gpslongitude"  "rawvideos/VID_002.insv" > VID_002.csv

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 11:31:58 AM
1. Use single quotes on Linux.

2. Use GPSDateTime, not GPSTimeStamp

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 11:40:24 AM
How to get all param names?
I tried "$latitude,$longitude"
"$gpslatitude,$gpslongitude"

all null. I think the command works but no param with the name "$gpslatitude,$gpslongitude". And the same command works from windows.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 11:51:39 AM
This command should work for you:

./exiftool -api largefilesupport -ee -p '$gpsdatetime,$gpslatitude,$gpslongitude'  'rawvideos/VID_002.insv' > VID_002.csv

I tested it with a smaller INSV file and it works fine.

If you want to list all available tags, you can use this command:

./exiftool -s -ee rawvideos/VID_002.insv

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 12:05:59 PM
Hi Phil,

Thanks a lot for all your support.
I am getting results now.
However the result from Linux is different than the windows.
Linux: 2019:10:02 14:09:38Z,40 deg 38' 24.62" N,73 deg 58' 1.64" W
Windows: 2019:10:02 13:06:07Z 40.69177316 -73.99179094

Is it possible to get the same windows format from linux?

Thanks,
Yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 12:16:06 PM
You are using a config file on windows to change the default coordinate format.
Title: Re: Extracting GPS data from INSV file
Post by: yamuna on October 08, 2019, 03:02:14 PM
Hi Phil,

No I am not using any config in windows. Just downloaded the exe and executed.

Thanks,
yamuna.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 08, 2019, 05:36:43 PM
You will only get that result with the Windows command if you use a config file.  There must be a default config file named ".ExifTool_config" in either your home directory or the exiftool app directory.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Kajuna on July 30, 2020, 04:12:20 AM
Hello everyone, fantastic job, Phil!

I've been extracting the Insta360 data successfully. It's my first time using Exiftool and I find the interface very user friendly. I have a couple of questions.

- I am using the -m option in order to extract all the accelerometer data. That seems to work well, but am I going to hit the Windows output size limitation with large files? Is there a way to prevent that, like requesting the data chunk by chunk, with an increasing offset or something similar?

- The accel data is "timecode" based. The timecode is in seconds but it does not start at zero. I have tried assuming the first samples correspond to the first frame, but there seems to be a slight offset (the data seems a bit delayed, so maybe some data samples before the first video frame are retained). Does anyone know what that timecode is relative to? The video track timecode does start at zero, and I cannot find any tag that resembles that value, to use as a base offset

Really appreciated
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on July 30, 2020, 06:19:45 AM
Quote from: Kajuna on July 30, 2020, 04:12:20 AM
- I am using the -m option in order to extract all the accelerometer data. That seems to work well, but am I going to hit the Windows output size limitation with large files? Is there a way to prevent that, like requesting the data chunk by chunk, with an increasing offset or something similar?

Currently it is all or nothing.  If you run into memory limitations, you can switch to the 64-bit version (https://oliverbetz.de/pages/Artikel/ExifTool-for-Windows).

Quote- The accel data is "timecode" based. The timecode is in seconds but it does not start at zero. I have tried assuming the first samples correspond to the first frame, but there seems to be a slight offset (the data seems a bit delayed, so maybe some data samples before the first video frame are retained). Does anyone know what that timecode is relative to? The video track timecode does start at zero, and I cannot find any tag that resembles that value, to use as a base offset

I'll take a look at my Insta360 samples when I get a chance to see if I can see anything, but I think the chances are small that I'll see anything obvious.  Do you have any software that shows the accelerometer data at the correct time?

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Kajuna on July 30, 2020, 10:36:50 AM
Quote from: Phil Harvey on July 30, 2020, 06:19:45 AM
If you run into memory limitations, you can switch to the 64-bit version (https://oliverbetz.de/pages/Artikel/ExifTool-for-Windows).

That will work, thanks!

Quote from: Phil Harvey on July 30, 2020, 06:19:45 AM
I'll take a look at my Insta360 samples when I get a chance to see if I can see anything, but I think the chances are small that I'll see anything obvious.  Do you have any software that shows the accelerometer data at the correct time?

Thanks for having a look. I'm not aware of any software that displays the accelerometer data. Insta's own software uses it for stabilisation, but nothing more, as fas as I know. I created this visualization to show the offset: https://youtu.be/z6aFhLs3wHo

And in case it's useful, here's the original file https://we.tl/t-DJjjrpeV2Y
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on July 30, 2020, 01:02:50 PM
Run this command to see everything that is available, and how it is grouped:

exiftool -ee -G3 FILE

You can modify the gpx.fmt file to write anything you want.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Kajuna on July 30, 2020, 01:55:12 PM
The only tag that I can imagine having that sort of data would be

[Main]          Parameters                      : 1 2576.934 2014.997 1506.038 0.291 -0.115 89.496 4056 3040 1058

Are those parameters known?
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on July 30, 2020, 04:05:43 PM
Not by me.
Title: Re: Extracting GPS data from INSV file
Post by: martincarlin87 on August 25, 2020, 06:43:02 AM
Hi,

I have been trying to extract the GPS data from an insv file and found ExifTool and this forum by extension (thank you btw), so I have been trying to follow the posts in this thread but I can't quite get it to work, but I'm not sure if it's because the data isn't actually present in the video, and not user error.

(On OS X) I ran:


exiftool -api largefilesupport -ee -p '$gpsdatetime,$gpslatitude,$gpslongitude'  'VID_20200511_143314_10_002.insv' > test.csv


and I get


Warning: [Minor] Tag 'gpsdatetime' not defined - VID_20200511_143314_10_002.insv


The resulting csv file is also empty.

If I run:


exiftool -s -ee VID_20200511_143314_10_002.insv


I get


ExifToolVersion                 : 12.03
FileName                        : VID_20200511_143314_10_002.insv
Directory                       : .
FileSize                        : 2.9 GB
FileModifyDate                  : 2020:08:24 17:36:34+01:00
FileAccessDate                  : 2020:08:24 17:36:34+01:00
FileInodeChangeDate             : 2020:08:24 17:37:15+01:00
FilePermissions                 : rw-r--r--
FileType                        : MP4
FileTypeExtension               : mp4
MIMEType                        : video/mp4
MajorBrand                      : MP4 Base w/ AVC ext [ISO 14496-12:2005]
MinorVersion                    : 0.0.0
CompatibleBrands                : avc1, isom
Warning                         : End of processing at large atom (LargeFileSupport not enabled)


Is there something I'm doing wrong or is it just that the GPS data isn't actually present?

Cheers,
Martin
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on August 25, 2020, 06:44:14 AM
Hi Martin,

Try adding -api largefilesupport to that last command (and maybe add "-gps*" to avoid getting all the other tags).

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: martincarlin87 on August 25, 2020, 08:02:41 AM
Hi Phil,

Thanks for taking the time to reply - much appreciated.

That seems to work:


exiftool -s -ee VID_20200511_143314_10_002.insv -api largefilesupport



ExifToolVersion                 : 12.03
FileName                        : VID_20200511_143314_10_002.insv
Directory                       : .
FileSize                        : 2.9 GB
FileModifyDate                  : 2020:08:24 17:36:34+01:00
FileAccessDate                  : 2020:08:24 17:36:34+01:00
FileInodeChangeDate             : 2020:08:24 17:37:15+01:00
FilePermissions                 : rw-r--r--
FileType                        : MP4
FileTypeExtension               : mp4
MIMEType                        : video/mp4
MajorBrand                      : MP4 Base w/ AVC ext [ISO 14496-12:2005]
MinorVersion                    : 0.0.0
CompatibleBrands                : avc1, isom
MediaDataSize                   : 3119186880
MediaDataOffset                 : 48
MovieHeaderVersion              : 0
CreateDate                      : 2020:05:11 21:33:04
ModifyDate                      : 2020:05:11 21:33:04
TimeScale                       : 48000
Duration                        : 0:08:19
PreferredRate                   : 1
PreferredVolume                 : 100.00%
MatrixStructure                 : 1 0 0 0 1 0 0 0 1
PreviewTime                     : 0 s
PreviewDuration                 : 0 s
PosterTime                      : 0 s
SelectionTime                   : 0 s
SelectionDuration               : 0 s
CurrentTime                     : 0 s
NextTrackID                     : 3
TrackHeaderVersion              : 0
TrackCreateDate                 : 2020:05:11 21:33:04
TrackModifyDate                 : 2020:05:11 21:33:04
TrackID                         : 1
TrackDuration                   : 0:08:19
TrackLayer                      : 0
TrackVolume                     : 0.00%
MatrixStructure                 : 1 0 0 0 1 0 0 0 1
ImageWidth                      : 2880
ImageHeight                     : 2880
MediaHeaderVersion              : 0
MediaCreateDate                 : 2020:05:11 21:33:04
MediaModifyDate                 : 2020:05:11 21:33:04
MediaTimeScale                  : 30000
MediaDuration                   : 0:08:19
HandlerType                     : Video Track
HandlerDescription              : Ambarella AVC
GraphicsMode                    : srcCopy
OpColor                         : 0 0 0
CompressorID                    : avc1
SourceImageWidth                : 2880
SourceImageHeight               : 2880
XResolution                     : 72
YResolution                     : 72
CompressorName                  : Ambarella AVC encoder
BitDepth                        : 24
PixelAspectRatio                : 1:1
Stereoscopic3D                  : Monoscopic
MetadataSource                  : .AMBA Spherical Video
PoseYawDegrees                  : 0
PosePitchDegrees                : 0
PoseRollDegrees                 : 0
ProjectionBoundsTop             : 0
ProjectionBoundsBottom          : 0
ProjectionBoundsLeft            : 0
ProjectionBoundsRight           : 0
VideoFrameRate                  : 29.97
Spherical                       : true
Stitched                        : true
StitchingSoftware               : AMBA Spherical Video
ProjectionType                  : equirectangular
StereoMode                      : mono
SourceCount                     : 2
TrackHeaderVersion              : 0
TrackCreateDate                 : 2020:05:11 21:33:04
TrackModifyDate                 : 2020:05:11 21:33:04
TrackID                         : 2
TrackDuration                   : 0:08:19
TrackLayer                      : 0
TrackVolume                     : 100.00%
MatrixStructure                 : 1 0 0 0 1 0 0 0 1
MediaHeaderVersion              : 0
MediaCreateDate                 : 2020:05:11 21:33:04
MediaModifyDate                 : 2020:05:11 21:33:04
MediaTimeScale                  : 48000
MediaDuration                   : 0:08:19
HandlerType                     : Audio Track
HandlerDescription              : Ambarella AAC
Balance                         : 0
AudioFormat                     : mp4a
AudioChannels                   : 2
AudioBitsPerSample              : 0
AudioSampleRate                 : 48000
ImageSize                       : 2880x2880
Megapixels                      : 8.3
AvgBitrate                      : 50 Mbps
Rotation                        : 0



to filter the noise:


exiftool -s -ee VID_20200511_143314_10_002.insv -api largefilesupport -gps*



no matches found: -gps*


so, no GPS data? At least it explains why - thanks for the help!
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on August 25, 2020, 08:15:32 AM
The "no matches found" error is from the shell, not ExifTool:  The -gps* requires quotes: "-gps*"

But in the full output, I don't see any GPS tags.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: martincarlin87 on August 25, 2020, 08:47:41 AM
Ah, so it does, I missed that from your original answer - apologies.

Great, makes sense now - thanks again for your help!

Cheers,
Martin
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on September 21, 2020, 01:15:50 AM
Hi Phil:
Thank you for providing this useful tool! I was trying to extract GPS data from INSV file, and I successfully did it. I use this command to extract it:

exiftool -api largefilesupport -ee -p '${GPSDateTime;DateFmt("%s")} ${gpslatitude#} ${gpslongitude#}' 'VID_20200427_163409_00_044.insv' > gps.txt

the result is something like below:

1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976450 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607
1587976451 31.2103383080857 121.593486648607

I find that the uint of gps timestamp is second, that means the frequency is 1Hz, but actually insta360 provide gps data at about 10Hz, so there is precision lose. And so does to the latitude and longitude precision, the other format of latitude data is 31 deg 12' 35.56" N, this means the accuracy is 0.01", but I gauss that insta360 provide more accurate data, so the output has many duplicated data. Can you fix this or can you give some advice on how I can fix it myself?
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on September 24, 2020, 06:27:37 AM
It will take me a while before I can answer this question.  I need to dig up my INSV sample and do some analysis, but the weather has been nice recently so I've been spending time outside instead of on the computer.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on October 11, 2020, 09:07:29 PM
Thank you for your reply, Phil! And I can provide you my insv file if you need. Have a good vacation!
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 13, 2020, 08:11:57 AM
Sure.  A sample would be helpful.  My email is philharvey66 at gmail.com

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on October 16, 2020, 07:22:10 AM
I got the sample, thanks.

Wow, I really need a bigger disk.  This video sample is > 2 GB.  :(

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on October 19, 2020, 05:10:19 AM
Sorry to send you a so big sample, I was going to record a smaller one, but I think that maybe from a bigger video you can see the trajectory and the accuracy problem will be more obvious.
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on November 24, 2020, 10:21:23 PM
Hi Phil, is there any progress? :)
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on November 25, 2020, 08:03:07 AM
Thanks for the reminder.

Maybe I'm missing something, but I don't see any GPS in the file you sent (VID_20200909_144502_00_002.insv).  ie) this command produces no output (only a warning "Truncated '\x00\x00\x00\x00' data"):

exiftool -ee -api largefilesupport "-gps*" FILE

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on November 30, 2020, 12:00:22 AM
Hi Phil, I use the same command exiftool -ee -api largefilesupport "-gps*" FILE to extract GPS data and I succeed. I will email you the output.

How big is the insv file you received, I suspect that the file is incomplete. The file is 2,297,323,736 bytes on my desktop with ubuntu16.04.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on November 30, 2020, 08:25:58 AM
That new file worked.  I can see what you are talking about.  The GPS is stored at 10 Hz, but the GPS position is only updated every 6-7 seconds, which results in sets of 60-70 identical GPS records in the file.  I've looked at the GPS data in detail (Insta360 record 0x700), and there is no more information to be extracted from this -- the records are truly identical.  It looks like the camera is only updating the GPS position every 6-7 seconds even though it stores the information at 10x per second.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on December 02, 2020, 03:38:39 AM
Thank you for your help! How confusing they do this, but may be there is something wrong with my camera, I don't know.
Title: Re: Extracting GPS data from INSV file
Post by: zengcancan on December 08, 2020, 08:33:32 PM
Hi Phil,  the insta360 One X support record GPS from mobile phone, or from a remote controller. The file I send you last time is recoreded by mobile phone, and yesterday I bought a remote controller, I find that the data is changing in one second, so you may still promote the time accuracy and gps accuracy(to 0.001'').
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on December 09, 2020, 06:52:33 AM
Quote from: zengcancan on December 08, 2020, 08:33:32 PM
you may still promote the time accuracy and gps accuracy(to 0.001'').

I don't understand.  Is there something you want me to do?

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on January 15, 2021, 03:48:23 AM
Hello.

First of all, I would like to express my thanks to Phil Harvey for this great tool. It has helped me many times. :) Especially in terms of extracting insta360 data.

Because of that, maybe I can add some additional information...

Quote from: zengcancan on December 08, 2020, 08:33:32 PM
Hi Phil,  the insta360 One X support record GPS from mobile phone, or from a remote controller. The file I send you last time is recoreded by mobile phone, and yesterday I bought a remote controller, I find that the data is changing in one second, so you may still promote the time accuracy and gps accuracy(to 0.001'').

The data quality in the insv file of an OneX/R/X2 depends on the GPS signal quality provided by phone / GPS smart remote / Apple or Garmin watch. All three cameras can add GPS data at 10Hz which includes to have 10 different GPS coords per second. But that strongly depends on the device used to get the data and its GPS lock / signal quality. I've done some tests with different devices => https://www.facebook.com/groups/Insta360OneCommunity/permalink/3246276772128895/

Unfortunately for most mobile devices you have to live with having doubled data points in the 10/s.
The best way to deal with it is to use additional tools to to even out the data / to optimize the data to an average...
Like GPSbabel (https://software.frankingermann.de/gpxtrackmap/gtm-documentation/9-joomla-plugins/gpxtrackmap/22-gtm-filtering-gpx-tracks-with-gpsbabel) ...
That's nothing I would see ExifTool in charge.

Coming to "gps accuracy", GPX does not provide a tag for that kind of "accuracy".
There is an element for "extensions" where you could place some own format.
(http://www.topografix.com/GPX/1/1/#type_extensionsType)

There are tags for "Dilution Of Precision" ( https://en.wikipedia.org/wiki/Dilution_of_precision_(navigation)  )  for every data point => http://www.topografix.com/GPX/1/1/#type_wptType
but that's not what you have in mind. Those only specify how reliable the data is. So the usage would be, the greater the number of doubles in the 10 data points /s, the greater the DUP value would be.

Quote from: Kajuna on July 30, 2020, 01:55:12 PM
The only tag that I can imagine having that sort of data would be

[Main]          Parameters                      : 1 2576.934 2014.997 1506.038 0.291 -0.115 89.496 4056 3040 1058

Are those parameters known?

Those are a kind of helper data for the stitcher, calibration / lens related data.

Quote from: Kajuna on July 30, 2020, 04:12:20 AM
- The accel data is "timecode" based. The timecode is in seconds but it does not start at zero. I have tried assuming the first samples correspond to the first frame, but there seems to be a slight offset (the data seems a bit delayed, so maybe some data samples before the first video frame are retained). Does anyone know what that timecode is relative to? The video track timecode does start at zero, and I cannot find any tag that resembles that value, to use as a base offset

That time code, as well as the timecode for exposure data, is uptime based.
You wont find an anchor point in the video data to hook up. You have to assume that the first appearance of an accelerator time code is at / near the starting point of the video. Guess you already connected the video tracks time code zero with the first accelerator time code.

Unfortunately due to the way the data is acquired and saved you will face a delay.

Phil.
Title: Re: Extracting GPS data from INSV file
Post by: g_gantic on January 19, 2021, 12:13:39 AM
Quote from: Phil_K on January 15, 2021, 03:48:23 AM
Hello.

First of all, I would like to express my thanks to Phil Harvey for this great tool. It has helped me many times. :) Especially in terms of extracting insta360 data.

Because of that, maybe I can add some additional information...

Quote from: zengcancan on December 08, 2020, 08:33:32 PM
Hi Phil,  the insta360 One X support record GPS from mobile phone, or from a remote controller. The file I send you last time is recoreded by mobile phone, and yesterday I bought a remote controller, I find that the data is changing in one second, so you may still promote the time accuracy and gps accuracy(to 0.001'').

The data quality in the insv file of an OneX/R/X2 depends on the GPS signal quality provided by phone / GPS smart remote / Apple or Garmin watch. All three cameras can add GPS data at 10Hz which includes to have 10 different GPS coords per second. But that strongly depends on the device used to get the data and its GPS lock / signal quality. I've done some tests with different devices => https://www.facebook.com/groups/Insta360OneCommunity/permalink/3246276772128895/

Unfortunately for most mobile devices you have to live with having doubled data points in the 10/s.
The best way to deal with it is to use additional tools to to even out the data / to optimize the data to an average...
Like GPSbabel (https://software.frankingermann.de/gpxtrackmap/gtm-documentation/9-joomla-plugins/gpxtrackmap/22-gtm-filtering-gpx-tracks-with-gpsbabel) ...
That's nothing I would see ExifTool in charge.

Coming to "gps accuracy", GPX does not provide a tag for that kind of "accuracy".
There is an element for "extensions" where you could place some own format.
(http://www.topografix.com/GPX/1/1/#type_extensionsType)

There are tags for "Dilution Of Precision" ( https://en.wikipedia.org/wiki/Dilution_of_precision_(navigation)  )  for every data point => http://www.topografix.com/GPX/1/1/#type_wptType
but that's not what you have in mind. Those only specify how reliable the data is. So the usage would be, the greater the number of doubles in the 10 data points /s, the greater the DUP value would be.

Quote from: Kajuna on July 30, 2020, 01:55:12 PM
The only tag that I can imagine having that sort of data would be

[Main]          Parameters                      : 1 2576.934 2014.997 1506.038 0.291 -0.115 89.496 4056 3040 1058

Are those parameters known?

Those are a kind of helper data for the stitcher, calibration / lens related data.

Quote from: Kajuna on July 30, 2020, 04:12:20 AM
- The accel data is "timecode" based. The timecode is in seconds but it does not start at zero. I have tried assuming the first samples correspond to the first frame, but there seems to be a slight offset (the data seems a bit delayed, so maybe some data samples before the first video frame are retained). Does anyone know what that timecode is relative to? The video track timecode does start at zero, and I cannot find any tag that resembles that value, to use as a base offset

That time code, as well as the timecode for exposure data, is uptime based.
You wont find an anchor point in the video data to hook up. You have to assume that the first appearance of an accelerator time code is at / near the starting point of the video. Guess you already connected the video tracks time code zero with the first accelerator time code.

Unfortunately due to the way the data is acquired and saved you will face a delay.

Phil.

I have been looking at the gyroscope data to provide a different way to stabilize the video other than insta360 own flowstate. When opening insta360 Studio, it seems to be able to sync the video and gyroscope data right away despite the offset varying in the multiple the sample videos I looked at . Looking at the data, we seem to have a longer track of gyroscope/exposure data than video itself (1-2s more of gyroscope data than video). While the metadata are timestamped with uptime, the video/audio tracks are not and they start at timestamp 0 as you were saying so we can't sync them.

I am convinced there must be a way to map this side data to the video store somewhere in the file but I can't find where. It could be hidden in some of the documents that exiftool can't decode yet: 0x200, 0x400, 0x900 or 0xa00.

I'd be curious to understand Phil Harvey's approach on how he was able to decode this data from scratch. And I do join you, other Phil, in thanking Phil Harvey for this incredible tool!
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on January 19, 2021, 12:32:51 AM
Hello.

Quote from: g_gantic on January 19, 2021, 12:13:39 AM
I have been looking at the gyroscope data to provide a different way to stabilize the video other than insta360 own flowstate. When opening insta360 Studio, it seems to be able to sync the video and gyroscope data right away despite the offset varying in the multiple the sample videos I looked at . Looking at the data, we seem to have a longer track of gyroscope/exposure data than video itself (1-2s more of gyroscope data than video). While the metadata are timestamped with uptime, the video/audio tracks are not and they start at timestamp 0 as you were saying so we can't sync them.

There is, but currently not in plain sight for us. ;)
And you are right, that gyro data starts before the actual first frame of the video is saved. If you look at the time code of the Accelerator and the time code for the ExposureTime - coming between gyro and GPS data - they have a small offset, too.

For example in one of my videos the first time codes for both read like
TimeCode                        : 562.873
Accelerometer                   : -0.993896484375 0.120849609375 -0.077392578125

TimeCode                        : 562.881
ExposureTime                    : 1/353

But I haven't found a reference in the rest for that .008 difference yet.

Quote from: g_gantic on January 19, 2021, 12:13:39 AM
I am convinced there must be a way to map this side data to the video store somewhere in the file but I can't find where. It could be hidden in some of the documents that exiftool can't decode yet: 0x200, 0x400, 0x900 or 0xa00.

I'd be curious to understand Phil Harvey's approach on how he was able to decode this data from scratch. And I do join you, other Phil, in thanking Phil Harvey for this incredible tool!

I'm sure it's somewhere, and I'm also curious.
Well, what has escaped my attention a bit is the fact that Insta360 has been releasing some material on github over the last few months.... So far I haven't had the time to look into it in detail => https://github.com/Insta360Develop

The part around the "Player" might be interesting...

Öhm... I'm hopping that's not too offtopic for this forum at all...

Phil.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on January 19, 2021, 08:08:58 AM
Quote from: g_gantic on January 19, 2021, 12:13:39 AM
I am convinced there must be a way to map this side data to the video store somewhere in the file but I can't find where. It could be hidden in some of the documents that exiftool can't decode yet: 0x200, 0x400, 0x900 or 0xa00.

0x200 contains a lot of unknown information.  I think ExifTool decodes most useful information from 0x400.  I don't have any samples with 0x900 or 0xa00.

QuoteI'd be curious to understand Phil Harvey's approach on how he was able to decode this data from scratch.

After staring at binary data for long enough, sometimes you can see patterns that look familiar.  "You get used to it, I don't even see the code, All I see is blond, brunette, redhead"

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on January 19, 2021, 12:07:47 PM
Hi all.

I've looked again on all the data we already have and I compared a few of my recordings, in 5.7k and 4k. Every time I could match the time code of the first ExposureTime to a time code of the Accelerometer or it was between two time codes.

The number of ExposureTime values fit the number of calculated frames for those video.
By that the first ExposureTime can be taken to represent the very first frame in the video.

Now connect that with the comparing time code of the Accelerometer or the next one if that Acc-TimeCode is between two.

Btw, the "Unknown2" in the 0x700 GPS part looks like a millisecond counter, which also represents the n-th parameter per second. It does not always start with "0 0" and not all videos have GPS data coverage right from the start. But the GPS start, can be synced with the "TrackCreateDate" down to the second.

For one of my videos the first 7 seconds are without GPS data, that would also mean that GPS starts at frame number 210. Now get the 210th ExposureTime time code and you can sync GPS to Accelerometer.
And based on the Unkown2 counter you can sync that down to millisecond related to the time code.

It needs a little data management and calculations, but it's worth a try I guess.... :)
... if you want to do that based on the extracted data, I wouldn't see that task by exifTool.

Phil.
Title: Re: Extracting GPS data from INSV file
Post by: Kajuna on January 21, 2021, 01:37:54 PM
Quote from: Phil_K on January 19, 2021, 12:07:47 PM
I've looked again on all the data we already have and I compared a few of my recordings, in 5.7k and 4k. Every time I could match the time code of the first ExposureTime to a time code of the Accelerometer or it was between two time codes.

Great job, Phil. I wanted to try exactly that, but haven't been able to read the ExposureTime values you are seeing. What command are you using?

Thanks.
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on January 22, 2021, 12:35:29 AM
Hi Kajuna.

Quote from: Kajuna on January 21, 2021, 01:37:54 PM
Great job, Phil. I wanted to try exactly that, but haven't been able to read the ExposureTime values you are seeing. What command are you using?

I'm using version 12.12 of the 64Bit ExifTool package for Windows.
The command options I use are
-api largefilesupport -ee -G
with a pipe to a file for a quick look which also gets me Exif, File, XMP & QuickTime tags.

At the later I used
-api largefilesupport -m -ee -u -QuickTime:all -Trailer:Insta360:all
-m makes sure to get all Accelerometer data
-u shows all unknown tags but without binary ones as they are not needed

The rest are filter for the Tags I need, which is QuickTime and all Insta360 related ones.
With that you get all data to connect the different streams and define the virtual zero for
the time code and the virtual millisecond of Unknown2 allows to connect the GPS data to that
two, even if the GPS data starts a few seconds later then the video.

Title: Re: Extracting GPS data from INSV file
Post by: Kajuna on January 22, 2021, 12:55:47 PM
Quote from: Phil_K on January 22, 2021, 12:35:29 AM
I'm using version 12.12 of the 64Bit ExifTool package for Windows.

Thanks Phil_k. I was using an older version of Exiftool that did not label the Exposure Times. Some thoughts and additional information on your findings:

- The Track Create Date does not seem to be based on the same source as the GPS Date/Time, and therefore does not seem viable for determining when the GPS started recording samples (relative to the video start). I suppose the former is based on an internal camera clock and the latter comes from the GPS signal. For example, I just shot a file where the GPS was acquired about 125 seconds after the recording started. The Track Create Date is 2021:01:22 16:41:54 but the first GPS sample has a GPS Date/Time of 2021:01:22 15:44:31Z. So the gap is about 157 seconds (after subtracting the time zone). Close-ish but not enough for a visually correct sync.

- As you suggested, the unknown GPS values seem like the missing milliseconds of the GPS Date/Time (although they do not have the precision one would expect from GPS timestamps. They're more like deciseconds). Maybe this could be parsed by ExifTool, as otherwise multiple GPS samples have the exact same date. I know this ALSO happens when you record the data with a phone and there are 10 exact copies of each sample (1Hz multiplied to mimic 10Hz), but when using the GPS remote the 10 samples are actually different (in position, speed). For example (summarised for visibility):

GPS Date/Time                   : 2020:12:10 17:33:12Z
GPS Latitude                    : 41 deg 35' 3.67" N
Unknown 02                      : 0 700
GPS Date/Time                   : 2020:12:10 17:33:12Z
GPS Latitude                    : 41 deg 35' 3.60" N
Unknown 02                      : 0 800
GPS Date/Time                   : 2020:12:10 17:33:12Z
GPS Latitude                    : 41 deg 35' 3.54" N
Unknown 02                      : 0 900
GPS Date/Time                   : 2020:12:10 17:33:13Z
GPS Latitude                    : 41 deg 35' 3.47" N
Unknown 02                      : 0 0
GPS Date/Time                   : 2020:12:10 17:33:13Z
GPS Latitude                    : 41 deg 35' 3.41" N
Unknown 02                      : 0 100
GPS Date/Time                   : 2020:12:10 17:33:13Z
GPS Latitude                    : 41 deg 35' 3.34" N
Unknown 02                      : 0 200


I think it's safe to assume, as both the Unknown values and the number of samples suggest, that the correct Date/Times are

: 2020:12:10 17:33:12.700Z
: 2020:12:10 17:33:12.800Z
: 2020:12:10 17:33:12.900Z
: 2020:12:10 17:33:13.000Z
: 2020:12:10 17:33:13.100Z
: 2020:12:10 17:33:13.200Z


But it's just a suggestion. I can parse this on my end.

Edit: I just checked with a video that has iPhone GPS data and, indeed, the Unkown value is the same between samples, which would make the previous assumptions correct. It also shows more decimals than the GPS remote value (Unknown 02 : 0 998), so that would indicate that the GPS remote records times with a precision of deciseconds at 10Hz, while the phone has a precision of milliseconds, but at just 1Hz.

- Ok. Now to the initial time code of the accelerometer. The Exposure times look promising, but here are my results in practice. I shot videos where I could clearly check the accel sync. The results are consistent between videos, for example for one video:

Visually checked offset: 1.150 seconds
First exposure time: 15.811
First accelerometer: 15.288
Difference: 0.523

The offset between accel time code and exposure time code is generally much smaller than the visually checked offset. So let's see if for any reason exposures are aligned to the end of the video:

Last exposure time: 33.171
Last accelerometer: 33.138
Difference: -0.033 (Accel ends before exposures)

Seems not. So let's see if the exposure samples match the number of frames. It's generally close, but not exact (there are more exposure samples than frames):

Frames in video 412
Exposure samples 432
Difference: 20 frames

This was at 25fps, so the excess of 20 frames would span 0.8 seconds.

I've tested this with other videos, including long ones, and the excess of samples seems to be always there and around that number (have not tried other frame rates or formats).

This is pure speculation, but we could try assuming these excessive samples are at the beginning of the video, and then add the time to the "exposure vs. accel" offset (that would leave us with an offset of 1.323) or we could split them between the beginning and the end of the video, predicting that that's what happens on average (the final offset would be of 0.923). Both results look slightly off visually, but if all the assumptions are correct (which is a long shot), all videos should look close to sync.

I'll try to keep testing.
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on February 09, 2021, 10:44:54 PM
Hello.

Sorry for the delay...

Let me start, from my point of view this is all speculation and the best approach at this time to merge the data.

Quote from: Kajuna on January 22, 2021, 12:55:47 PM
- The Track Create Date does not seem to be based on the same source as the GPS Date/Time, and therefore does not seem viable for determining when the GPS started recording samples (relative to the video start). I suppose the former is based on an internal camera clock and the latter comes from the GPS signal. For example, I just shot a file where the GPS was acquired about 125 seconds after the recording started. The Track Create Date is 2021:01:22 16:41:54 but the first GPS sample has a GPS Date/Time of 2021:01:22 15:44:31Z. So the gap is about 157 seconds (after subtracting the time zone). Close-ish but not enough for a visually correct sync.

Agree. I would say it (TrackCreateDate) comes from the cameras internal clock.

Quote from: Kajuna on January 22, 2021, 12:55:47 PM
- As you suggested, the unknown GPS values seem like the missing milliseconds of the GPS Date/Time (although they do not have the precision one would expect from GPS timestamps. They're more like deciseconds).

Jip, deciseconds.. I git distracted by its number of digits.

But it is not accurate or usable in all instances.
I did some reviews with my recordings across different versions of apps (OneX, OneR and Insta360app)
and also with different devices providing the GPS data.

What I see is that the GPS smart remote is the most reliable source to provide those deziseconds.
On an OneR app recorded video with GPS provided by Garmin Glo2 on an iPad ini5 wifi, there are values doubled and repeated even the GPS coords are individual. I have no idea why that is so.

Quote from: Kajuna on January 22, 2021, 12:55:47 PM
This is pure speculation, but we could try assuming these excessive samples are at the beginning of the video, and then add the time to the "exposure vs. accel" offset (that would leave us with an offset of 1.323) or we could split them between the beginning and the end of the video, predicting that that's what happens on average (the final offset would be of 0.923). Both results look slightly off visually, but if all the assumptions are correct (which is a long shot), all videos should look close to sync.

I'll try to keep testing.

Good work.
In summary it sounds like a good approach to get near a minimized off-sync and also minimize manual interaction to shift data in sync with the video.... by the way... depending on the tool you are creating...
Wouldn't it be an additional feature to give the user an adjustment slider to adjust that "sync" / offset to his favor? Like "manual/auto" mode. :)

Greetings
Phil.
Title: Re: Extracting GPS data from INSV file
Post by: paluan01 on May 07, 2021, 12:59:55 PM
     
I have just read all the threads but I was not able to identify a clear procedure to extract a GPX trace from.INSV file I own an insta360 one R with its remote GPS control.
May some of you so kind to summarize step by step the procedure to be followed?
Many thanks
l
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on May 08, 2021, 07:37:43 AM
This is the command:

exiftool -p gpx.fmt -ee3 FILE > out.gpx

Where you need this gpx.fmt file (https://raw.githubusercontent.com/exiftool/exiftool/master/fmt_files/gpx.fmt) in the current directory when running the command.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: paluan01 on May 09, 2021, 03:35:24 AM
Thanks first of all for the answer.

I tried with "prova.insv"
and this was the result was:

Warning: [Minor] Tag 'Doc33211:ss' not defined - prova.INSV

Opening .GPX file this is the syntax of one section

<?xml version="1.0" encoding="utf-8"?>\
<gpx version="1.0"\
creator="ExifTool 12.25"\
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"\
xmlns="http://www.topografix.com/GPX/1/0"\
xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">\
<trk>\
<number>1</number>\
<trkseg>\
{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fmodern\fcharset0 Courier;}
{\colortbl;\red255\green255\blue255;\red0\green0\blue0;}
{\*\expandedcolortbl;;\cssrgb\c0\c0\c0;}
\paperw11900\paperh16840\margl1440\margr1440\vieww21140\viewh17780\viewkind0
\deftab720
\pard\pardeftab720\sl280\partightenfactor0

\f0\fs24 \cf2 \expnd0\expndtw0\kerning0
\outl0\strokewidth0 \strokec2 #------------------------------------------------------------------------------\
<trkpt lat="45.0924981666667" lon="7.63498883333333">\
  <ele>260.1</ele>\
</trkpt>\
\pard\pardeftab720\sl280\partightenfactor0
\cf2 \outl0\strokewidth0 \
\
}{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fmodern\fcharset0 Courier;}
{\colortbl;\red255\green255\blue255;\red0\green0\blue0;}
{\*\expandedcolortbl;;\cssrgb\c0\c0\c0;}
\paperw11900\paperh16840\margl1440\margr1440\vieww21140\viewh17780\viewkind0
\deftab720
\pard\pardeftab720\sl280\partightenfactor0

.
.
.
.

How to fix it?
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on May 10, 2021, 12:40:08 AM
Good morning.

Quote from: paluan01 on May 07, 2021, 12:59:55 PM
I have just read all the threads but I was not able to identify a clear procedure to extract a GPX trace from.INSV file I own an insta360 one R with its remote GPS control.
May some of you so kind to summarize step by step the procedure to be followed?
Many thanks

Besides the info an fmt file Phil has already given to you, allow me to direct your attention also to
https://www.facebook.com/groups/insta360onexusers/permalink/454627628467347/
as a kind of cross reference.

Even we go a bit offtopic, the steps are equal for OneX/R/X2 regarding 360° video.
For the OneR it might depend which lens module you used.
Personally I can only speed about videos done in 360° because I hadn't the chance to look at videos done with the 4k or 1" module in regards to GPS data.

Also - because of the size of the data - i found it a good idea to use the 64bit version of exifTool provided via the "alternate ExifTool Windows installer" linked on the download page.

For your video files, if you did a 5.7k 360° recording make sure to let exifTool work on the file with _00_ in the name. The second lens video (_10_ part) does not contain GPS data.

In your special case it seems the line
#[BODY]  <time>${gpsdatetime#;my ($ss)=/\.\d+/g;DateFmt("%Y-%m-%dT%H:%M:%SZ");s/Z/${ss}Z/ if $ss}</time>
in the gpx.fmt file seems to make problems because the regex substitution can not be done.
You may replace it with
#[BODY]   <time>${gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")}</time>
which is simpler but works with OneX/R/X2 GPS data in the files.
Title: Re: Extracting GPS data from INSV file
Post by: paluan01 on May 10, 2021, 03:23:49 AM
Good morning to you!
Let me thank you again for the support.

Some info:
- I am working with MacBook
- I am using Insta360 one R and _00_file
- The GPS data are from Its Remote Controller

I modified the .fmt file according to your indication but, unfortunately, was not good (yet :-))

these the warnings
Warning: Truncated '\x00\x02\x00\x00' data - prova.insv
Warning: [Minor] Insta360 accelerometer data is huge. Processing only the first 20000 records - prova.insv

______________________________
this the content of out.GPX file
______________________________

<?xml version="1.0" encoding="utf-8"?>\
<gpx version="1.0"\
creator="ExifTool 12.25"\
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"\
xmlns="http://www.topografix.com/GPX/1/0"\
xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">\
<trk>\
<number>1</number>\
<trkseg>\
{\rtf1\ansi\ansicpg1252\cocoartf1671\cocoasubrtf600
{\fonttbl\f0\fmodern\fcharset0 Courier;}
{\colortbl;\red255\green255\blue255;\red0\green0\blue0;\red234\green233\blue255;\red115\green0\blue2;
}
{\*\expandedcolortbl;;\cssrgb\c0\c0\c0;\cssrgb\c93333\c93333\c100000;\cssrgb\c53333\c0\c0;
}
\paperw11900\paperh16840\margl1440\margr1440\vieww21140\viewh17780\viewkind0
\deftab720
\pard\pardeftab720\sl280\partightenfactor0

\f0\fs24 \cf2 \expnd0\expndtw0\kerning0
<trkpt lat="45.0924981666667" lon="7.63498883333333">\
  <ele>260.1</ele>\
\pard\pardeftab720\sl240\partightenfactor0
\cf2 \cb3 \outl0\strokewidth0 \strokec4 #[BODY]\'a0 \'a0<time>$\{gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")\}</time>\cf2 \cb1 \outl0\strokewidth0 \
\pard\pardeftab720\sl280\partightenfactor0
\cf2 #[BODY]</trkpt>\
\
\
_____________________________________

My goal is to have a GPX file to be managed with Google Earth


I appreciate if you can follow me to fix all the problems!
Many thanks again

Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on May 10, 2021, 03:51:33 AM
Quote from: paluan01 on May 10, 2021, 03:23:49 AM
Good morning to you!
Let me thank you again for the support.

Some info:
- I am working with MacBook
- I am using Insta360 one R and _00_file
- The GPS data are from Its Remote Controller

I modified the .fmt file according to your indication but, unfortunately, was not good (yet :-))

these the warnings
Warning: Truncated '\x00\x02\x00\x00' data - prova.insv
Warning: [Minor] Insta360 accelerometer data is huge. Processing only the first 20000 records - prova.insv

The first warning - I guess - is related to how the data is saved / finalized in the file but can be ignored.
The second warning is because that data is really huge and you don't need that, but if you want to see it all "-m" option is your friend.

Phil may correct me, but that your command does not give you any further gps to gpx output seems to me because the command line has a typo.

Besides the fmt file option I use "-ee -G3" so you would use something like

exiftool -p gpx.fmt -ee -G3 FILE > out.gpx

But you don't really need the "-G3" option for only extracting, it's more helpful for debugging.

Oh btw, please, if you quote the output use the code tags around it. Or select and hit the button with # over the editor window. That helps reading it. :)

Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on May 10, 2021, 07:33:02 AM
There is something wrong with the fmt.gpx file that you are using.  Maybe you cut and pasted it into a file and the linefeeds got messed up?

The "Truncated" warning probably indicates that there is some type of non-standard trailer at the end of the file that ExifTool doesn't recognize.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: paluan01 on May 11, 2021, 01:39:34 AM
Hu,
this si my .fmt with the last indication

#[BODY]   <time>${gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")}</time>


hereby all the .fmt file


#------------------------------------------------------------------------------
# File:         gpx.fmt
#
# Description:  Example ExifTool print format file to generate a GPX track log
#
# Usage:        exiftool -p gpx.fmt -ee3 FILE [...] > out.gpx
#
# Requires:     ExifTool version 10.49 or later
#
# Revisions:    2010/02/05 - P. Harvey created
#               2018/01/04 - PH Added IF to be sure position exists
#               2018/01/06 - PH Use DateFmt function instead of -d option
#               2019/10/24 - PH Preserve sub-seconds in GPSDateTime value
#
# Notes:     1) Input file(s) must contain GPSLatitude and GPSLongitude.
#            2) The -ee option is to extract the full track from video files.
#            3) The -fileOrder option may be used to control the order of the
#               generated track points when processing multiple files.
#------------------------------------------------------------------------------
#[HEAD]<?xml version="1.0" encoding="utf-8"?>
#[HEAD]<gpx version="1.0"
#[HEAD] creator="ExifTool $ExifToolVersion"
#[HEAD] xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
#[HEAD] xmlns="http://www.topografix.com/GPX/1/0"
#[HEAD] xsi:schemaLocation="http://www.topografix.com/GPX/1/0 http://www.topografix.com/GPX/1/0/gpx.xsd">
#[HEAD]<trk>
#[HEAD]<number>1</number>
#[HEAD]<trkseg>
#[IF]  $gpslatitude $gpslongitude
#[BODY]<trkpt lat="$gpslatitude#" lon="$gpslongitude#">
#[BODY]  <ele>$gpsaltitude#</ele>
#[BODY]   <time>${gpsdatetime#;DateFmt("%Y-%m-%dT%H:%M:%SZ")}</time>
#[BODY]</trkpt>
#[TAIL]</trkseg>
#[TAIL]</trk>
#[TAIL]</gpx>
Title: Re: Extracting GPS data from INSV file
Post by: paluan01 on May 18, 2021, 02:38:53 AM
any solutions? ;)
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on May 18, 2021, 03:48:28 PM
Unfortunately you pasted your fmt file instead of attaching it, so I can't reproduce what you are seeing.  But I'm sure the problem is that your .fmt file is not plain ASCII text.  From the corrupted output my guess is that you saved it as RTF.  Try saving it as plain ASCII text.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: martinr36 on August 20, 2021, 11:21:16 AM
Hi, I'm trying to do this but  aren't having a lot o9f success.
Ive downloaded and installed the 64 bit version from here https://oliverbetz.de/pages/Artikel/ExifTool-for-Windows and I've tried using the command exiftool -p gpx.fmt -ee3 H:\onex2\VID_20210813_155708_00_017.insv > out.gpx which produced the following results

Warning: End of processing at large atom (LargeFileSupport not enabled) - H:/onex2/VID_20210813_155708_00_017.insv
Warning: [Minor] Insta360 accelerometer data is huge. Processing only the first 20000 records - H:/onex2/VID_20210813_155708_00_017.insv

So after a bit of reading through here i tried exiftool -api LargeFileSupport -p gpx.fmt -ee3 H:\onex2\VID_20210813_155708_00_017.insv > out.gpx which produced this

Warning: Truncated '\x00\x00\x00\x00' data - H:/onex2/VID_20210813_155708_00_017.insv
Warning: [Minor] Insta360 accelerometer data is huge. Processing only the first 20000 records - H:/onex2/VID_20210813_155708_00_017.insv

I did find a file called gpx.out, and when i looked at it in notepad this is what i found  (see Attachment)

Can someone please point me in the right direction



Title: Re: Extracting GPS data from INSV file
Post by: StarGeek on August 20, 2021, 11:36:15 AM
As you figured out, the first error indicated the need to add the -api LargeFileSupport option (https://exiftool.org/ExifTool.html#LargeFileSupport).

Looking at your GPX file shows a second problem.  The output is nothing but "gpx.fmt" over and over.  This indicates that you do not have the GPX.fmt file.  Download the GPX.fmt (https://raw.githubusercontent.com/exiftool/exiftool/master/fmt_files/gpx.fmt) file from Github and place it in current directory that you're running exiftool from or give the full path the the fmt file after the -p (-printFormat) option (https://exiftool.org/exiftool_pod.html#p-FMTFILE-or-STR--printFormat), e.g. -p C:\path\to\GPX.fmt

Finally, because of the "accelerometer data is huge" warning, the output may not show all the GPS coordinates.  I'm not sure.  But I think you can add the -m (-ignoreMinorErrors) option (https://exiftool.org/exiftool_pod.html#m--ignoreMinorErrors) to tell exiftool to process all the entries, though because of the size, it might take a long time.
Title: Re: Extracting GPS data from INSV file
Post by: martinr36 on August 20, 2021, 12:29:37 PM
Quote from: StarGeek on August 20, 2021, 11:36:15 AM
As you figured out, the first error indicated the need to add the -api LargeFileSupport option (https://exiftool.org/ExifTool.html#LargeFileSupport).

Looking at your GPX file shows a second problem.  The output is nothing but "gpx.fmt" over and over.  This indicates that you do not have the GPX.fmt file.  Download the GPX.fmt (https://raw.githubusercontent.com/exiftool/exiftool/master/fmt_files/gpx.fmt) file from Github and place it in current directory that you're running exiftool from or give the full path the the fmt file after the -p (-printFormat) option (https://exiftool.org/exiftool_pod.html#p-FMTFILE-or-STR--printFormat), e.g. -p C:\path\to\GPX.fmt

Finally, because of the "accelerometer data is huge" warning, the output may not show all the GPS coordinates.  I'm not sure.  But I think you can add the -m (-ignoreMinorErrors) option (https://exiftool.org/exiftool_pod.html#m--ignoreMinorErrors) to tell exiftool to process all the entries, though because of the size, it might take a long time.

Many thanks for this, that did the trick
Title: Re: Extracting GPS data from INSV file
Post by: st_lupo on July 27, 2022, 04:04:03 PM
Sorry to resurrect a zombie but this thread has helped me immensly in extracting the GPS data from an insv file from an Insta 360 One RS.  Just one question though, does anybody have any ideas on synchronizing the GPS data (that is sequenced by gps date-time) with a corresponding frame in the video stream?   Thanks!
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on July 27, 2022, 09:01:38 PM
Knowing the time at the start of the video, and the video frame rate, you should be able to determine the time for each frame within a second, and use this to match up with the GPS track.  You will probably have to get creative to get sub-second accuracy.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: st_lupo on July 28, 2022, 03:00:19 AM
Hi Phil, thanks for the reply!  I think I finally have this figured out (I just needed to sleep on it), and have been approaching this the wrong way.  I was originally thinking to use the GPS data embedded in the Insta360 video (which depending on the source for the GPS data, has varying update rates).  Now I guess the easiest way is to record the GPS data on one of my GPS devices (and export that to gpx).  Then in my video I make sure I initially record the GPS time displayed on the GPS device (and capture a change to the seconds) so I can get a hack to synchronize the video sample-time to the gps date-time.  Should be good to under a second?
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on July 28, 2022, 07:23:07 AM
Should be.
Title: Re: Extracting GPS data from INSV file
Post by: kitarolivier on April 05, 2023, 09:22:24 AM
As perfect neewbie (first post  ;) ), i'm also resurrecting a zombie.

I'm posting here, because I could not find another place on the web where solutions are so close.

After reading, I still can't figure out how to sync GPS data and video.

I tried a different approach by using last GPS Date/Time minus movie duration.
I tried with one example and have strange results :
Last GPS Date/Time : 2023:04:05 07:17:52Z
Duration : 0:01:31
With the formula : 07:17:52 - 0:01:31 = 07:16:21

First GPS entry : 2023:04:05 07:15:42Z  (WTF ?!?!?)

Substracting firt GPS time from last GpsTime give a (gps) movie length of 0:02:10 !!

And all dates in main section are 2023:04:05 07:16:12

When opening file in Studio, it says no gps data from 00:00 to 00:03.

Can anyone help me ?

Thank you.
Title: Re: Extracting GPS data from INSV file
Post by: Phil Harvey on April 05, 2023, 10:04:46 AM
I think you need to shoot a few test videos with the camera pointed at a GPS-synced clock.

- Phil
Title: Re: Extracting GPS data from INSV file
Post by: StarGeek on April 05, 2023, 10:07:37 AM
Quote from: kitarolivier on April 05, 2023, 09:22:24 AMWhen opening file in Studio, it says no gps data from 00:00 to 00:03.

Can anyone help me ?

If there's no data in the first few seconds, and exiftool agrees with that, then whatever recorded the file had a delay of some sort in recording the GPS data.  Not much can be done about that.
Title: Re: Extracting GPS data from INSV file
Post by: kitarolivier on April 06, 2023, 09:42:40 AM
Quote from: Phil Harvey on April 05, 2023, 10:04:46 AMI think you need to shoot a few test videos with the camera pointed at a GPS-synced clock.

- Phil

Thank you for your answer.

That's a good idea. But it doesn't tell me why the GPS duration is longer than the video file.
Title: Re: Extracting GPS data from INSV file
Post by: kitarolivier on April 06, 2023, 09:44:14 AM
Quote from: StarGeek on April 05, 2023, 10:07:37 AM
Quote from: kitarolivier on April 05, 2023, 09:22:24 AMWhen opening file in Studio, it says no gps data from 00:00 to 00:03.

Can anyone help me ?

If there's no data in the first few seconds, and exiftool agrees with that, then whatever recorded the file had a delay of some sort in recording the GPS data.  Not much can be done about that.

Of course, non recorded GPS are lost. But how Insta360 studio computes the time between the video start time and the gps start time ?
Title: Re: Extracting GPS data from INSV file
Post by: Phil_K on November 12, 2023, 12:27:40 PM
Hi.

Quote from: kitarolivier on April 05, 2023, 09:22:24 AMSubstracting firt GPS time from last GpsTime give a (gps) movie length of 0:02:10 !!

And all dates in main section are 2023:04:05 07:16:12

When opening file in Studio, it says no gps data from 00:00 to 00:03.

Can anyone help me ?

Well, which recording mode did you use?

There are a few modes, like Timelapse or Timeshift, which playback time is not equal to the recording session time. Timelapse is pretty straight because the interval defines a kind of internal fps but creates a 30fps video. So with interval of 1s recording for 1 minutes gives you 2s playback time.

TimeShift video uses a dynamic scheme to speed up the final video which again is saved with 30fps.

GPS data is recorded and stored for the complete recording session time.
Staying with the Timelapse video example, you get 60s of GPS coverage while the video playback time is only 2s.

If you want to sync GPS and video playback you have to speed up the GPS playback by 30x, or slow down the video playback... for example by re-timing the video to 1fps.

On TimeShift that is much harder to do because of the dynamic scheme used.
Even in tools like Telemetry overlay you would need to speed-up parts of the metadata playback differently.

Now timestamps... it's important to remember that the timestamps in the GPS timestamps the Insta360 cameras up to now use GPST without any timezone data (indicated by the "Z"), which is nearly UTC by a few seconds. Currently GPST equals UTC+18s.

The file dates and media/track times saved in the insv/mp4 are related to date/time of the camera, mostly local time. The cameras get their time by the app. So make sure to connect the camera to the app now and then. The app transfers the time of the mobile device it running on to the camera.

There is a chance that the internal clock of the camera went off a bit.

Depending which way you used for recording GPS data with Insta360 consumer cameras you will face a different delay at the start of a video. Most times there is a gap of ~3s, especially if you use the app to record GPS.

Another thing to note, up to version v4.8.10 Studio2023 has a bug related to GPS data in those time-changed videos like Timelapse / TimeShift. For those Studio2023 v4.8.10 does not take all GPS data stored in the files into account, but only those for the video playback time beginning at the start of the file/video.

Now this is speculation, but I guess data sync is done from end to start. This would allow to determine the starting gap, but creates additional errors for those time-changed video.

That error also reduces data for the GPX export in Studio2023 v4.8.10.
If you need the GPS data in other tools make sure they can read directly from insv files or use exiftool to extract to a GPX file.