Parsing a gpx file generated by dnrgps

[frankchicken]

Hello Phil,

Very nice and useful tool you have created in exiftool.

A few days ago, I was using a Garmin Foretrex 301 and a Kodak Playsport Zx5 to make some geotagged photos.  I record a gps track point every second on the Garmin Foretrex 301.

I use dnrgps (a very nice and useful tool) to retrieve gpx tracks from the Garmin units.

The problem is that exiftool seems to have trouble parsing the gpx track file generated by dnrgps.

I captured the output from the exiftool run:

Code Select Expand



C:\openstreetmap\tophil\07072012>exiftool -ver
8.97

C:\openstreetmap\tophil\07072012>exiftool -geotag 07072012_foretrex301a_01.gpx -geosync=+5 .
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0459.JPG
Warning: No writable tags set from ./100_0459.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0460.JPG
Warning: No writable tags set from ./100_0460.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0461.JPG
Warning: No writable tags set from ./100_0461.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0462.JPG
Warning: No writable tags set from ./100_0462.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0463.JPG
Warning: No writable tags set from ./100_0463.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0464.JPG
Warning: No writable tags set from ./100_0464.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0465.JPG
Warning: No writable tags set from ./100_0465.JPG
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0466.JPG
...
...
same warning for rest of photos in directory...
...
...



Wonder how difficult it would be to get exiftool to parse the gpx file generated by dnrpgs?

Don't spend a lot of time on this.  exiftool will parse the gpx file generated by the Garmin Foretrex 301.  Just access the Garmin Foretrex 301 like a USB device and copy the current.gpx file.

The real problem is that other Garmin units do not behave like the Garmin Fortrex 301.  For example, my Garmin Forerunner 205 does not behave like a USB device and I have to use dnrgps to retrieve the gpx tracks.

If you think it is worthwhile, I can send more info.

Cheers,
Frank Davis

[Phil Harvey]

Hi Frank,

Sorry for the delay in responding.  I was on vacation... So now I've got some geotagging of my own to do.

Could the problem be that your images don't contain DateTimeOriginal?

If it is a problem with the GPX file, I would be surprised, but please post a sample log file and I'll take a look.

- Phil

[frankchicken]

Good morning Phil,

Thank you very much for the reply.

When I go out mapping for openstreetmap, I use a Garmin Foretrex 301 on my left wrist and a Garmin Forerunner 205/305 on my right wrist.  Since I ride about 15-20 miles on a bicycle, I want to make sure to capture my tracks.  I don't want to ride 20 miles and find out later I forgot to turn on GPS tracks or that the battery died...

It just by coincidence, I noticed exiftool would parse the Foretrex 301 gpx file copied as a USB device and insert the GPS coord.  However, exiftool would not insert the GPS coord from the same exact gpx path from the Foretrex 301 if it was retrieved by dnrgps.

I validated the gpx files using the SaxCount.exe program.  Google for "gpx file validator" to find the utility.  As far as I can tell, both the original gpx file on the Foretrex 301 as a USB device and the gpx file retrieved by dnrgps are valid gpx files.

I am attaching 1) the very first photo from the Kodak Playsport Zx5 of the sequence with the time on the Garmin Forerunner 305, 2) the gpx file copied from the Garmin Foretrex 301, and 3) the gpx file retrieved from the Foretrex 301 using dnrgps.

No prob if you decide it is not worthwhile exercise.

Cheers,
Frank Davis

[Phil Harvey]

Thanks.

I only find one file attached, but ExifTool won't read the track points from this file because the information is stored in "ogr" (http://osgeo.org/gdal) extensions, and not in the standard GPX locations.  I will have to do some research to see if I can find documentation on this.

- Phil

[frankchicken]

Hello Phil,

You are the wizard.

I suppose everyone knows by now why I try redundant paths...I have trouble with a simple forum attachment.

I attached (just in case you needed to see the gpx) the garmin USB file copy/paste from the Foretrex 301.

Now that you pointed it out, it is obvious that dnrgps added extensions.

Cheers,
Frank Davis

[Phil Harvey]

I am on a faster internet connection now, so I am able to download the entire first GPX file and test it out.  I was wrong.  ExifTool does read the track points from this file.

The standard GPX information actually exists in the file, along with the "ogr" extensions (which are ignored).

So maybe this wasn't the file that ExifTool couldn't read.  Could you please upload a GPX file that ExifTool has problems with?  Thanks.

- Phil

[frankchicken]

Hello Phil,

Thank you for checking this out.

1) test 1

The filename from using the Foretrex as a USB device is

07072012_foretrex301_01.gpx

When I ran exiftool with the file above, the gps positions were written to image files.


2) test 2

The filename retrieved from the Foretrex using dnrgps is

07072012_foretrex301a_01.gpx

When I ran exiftool with the file retrieved by dnrgps, the gps positions were not written to the image files.

The output from 07072012_foretrex301a_01.gpx is

C:\openstreetmap\tophil\07072012>exiftool -geotag 07072012_foretrex301a_01.gpx -geosync=+5 .
Warning: Duplicate FPXR stream data at offset 62029 - ./100_0459.JPG
Warning: No writable tags set from ./100_0459.JPG


I used a copy of the original image files for each test.  I made a test directory for testing exiftool with the  original images and each gpx file.

The original pictures are still on my camera.  After work, I will repeat my test.   I am subject, but I'll just repeat my test just to make sure...

Cheers,
Frank Davis

I attached the file retrieved by dnrgps.  It has a 301a after the Foretrex.

[Phil Harvey]

There could be some other problem.  Could you try running your command with the -v3 option?  (or even -v4 to show  details about all track points read from the GPX file)

Thanks.

- Phil

[frankchicken]

Hello Phil,


I got home from work and got the same results.....test1 worked and test2 failed.


Just to make sure, I copied the original pics from the Kodak Playsport Zx5.  They are just as I shot them on July 7.


Oooh, I see what you mean by -v4.  There is very much output.



I created a test1 and a test2 subdirectory. 

I made a batch file in each directory to make sure to capture the inputs to and output from exiftool.exe



test1)

It has the files:
photos
07072012_foretrex301_01.gpx
dotest1.bat

This is dotest1.bat:

Code Select Expand


exiftool -ver

exiftool -v4 -geotag 07072012_foretrex301_01.gpx -geosync=+5 .

rem exiftool -r -if "$gpsdatetime" -fileOrder gpsdatetime -p gpx.fmt -d %Y-%m-%dT%H:%M:%SZ . > out.gpx



On the command line in subdirectory test1, type

dotest1 2>&1 | more > test1out.txt




test2)

It has the files:
photos
07072012_foretrex301a_01.gpx
dotest2.bat

This is dotest2.bat:

Code Select Expand


exiftool -ver

exiftool -v4 -geotag 07072012_foretrex301a_01.gpx -geosync=+5 .

rem exiftool -r -if "$gpsdatetime" -fileOrder gpsdatetime -p gpx.fmt -d %Y-%m-%dT%H:%M:%SZ . > out.gpx




On the command line in subdirectory test2, type

dotest2 2>&1 | more > test2out.txt




Thank you very much for tracking this down.

I have attached the very first photo that shows the time on the Forerunner 305 on my right wrist...just in case you want to duplicate the test on your machine.  I can send more photos if you need them.  They are just pics of street signs when I was mapping for openstreetmap.

Cheers,
Frank Davis


I will attach the output from test1 and test2 in separate replies.  I'm too dumb to attach multiple files....

[frankchicken]

Hello Phil,

I have attached the ouput from test1.

It was many MB in size, so I only kept the output from the first pic.  Everything seemed to be working.

Cheers,
Frank Davis

[frankchicken]

Hello Phil,

So to such a dumb-dumb on these attachments.

I have attached the output from test2.  It is not edited.  It has the full output from the batch file with -v4.

Cheers,
Frank Davis

[Phil Harvey]

Hi Frank,

I get this:

Code Select Expand

> exiftool 100_0459.JPG -geotag ~/Desktop/07072012_foretrex301_01.gpx -v3
Argument "-Geotime<DateTimeOriginal" is assumed
Loaded 4466 points from GPS track log file '/Users/phil/Desktop/07072012_foretrex301_01.gpx'
  GPS track start: 2012:06:30 20:17:56 UTC
  GPS track end:   2012:07:07 12:46:56 UTC
Writing File:Geotag
======== 100_0459.JPG
Setting new values from 100_0459.JPG
  Geotime value:   2012:07:07 11:32:55 UTC (local timezone is -04:00)
  Interpolating pos between fixes (f=0):
    2012:07:07 11:32:55 UTC - alt=123.19 lat=31.462572 lon=-83.503824
    2012:07:07 11:32:56 UTC - alt=123.19 lat=31.462572 lon=-83.503824
[...]

> exiftool 100_0459.JPG -geotag ~/Desktop/07072012_foretrex301a_01.gpx -v3
Argument "-Geotime<DateTimeOriginal" is assumed
Loaded 4463 points from GPS track log file '/Users/phil/Desktop/07072012_foretrex301a_01.gpx'
  GPS track start: 2012:07:07 07:32:32 UTC
  GPS track end:   2012:07:07 08:46:56 UTC
Writing File:Geotag
======== 100_0459.JPG
Setting new values from 100_0459.JPG
  Geotime value:   2012:07:07 11:32:55 UTC (local timezone is -04:00)
Time is too far beyond track in File:Geotime (ValueConvInv)
[...]


The problem is that the second track log doesn't contain track points at the time of the image (DateTimeOriginal).  Take a look at the GPS track start/end times in the -v3 output.

- Phil

[frankchicken]

Hello Phil,

YES, YES, I see what you are saying.

The two files contain the exact same gps track; the lat/lon/time must be
the same for each track point.  They cannot be different.

I made the Garmin gpx track a little more readable.

Comparing the dnrgps gpx to the Garmin gpx, it is easy to see that there
is something wrong with the dnrgps time.  They are mixing Z time and localtime.
dnrgps is using localtime for both Z time and localtime (but, that is OK if one
is in the Zulu time zone).


EDT is -4.


I was wearing the Garmin Foretrex 301 on my left wrist.


---  Garmin gpx ---

Code Select Expand


<trk>
  <name>ACTIVE LOG: 07 JUL 2012 07:32</name>
  <trkseg>
    <trkpt lat="31.462572" lon="-83.503823">
      <ele>123.67</ele>
      <time>2012-07-07T11:32:32Z</time>           <=== correct Z time
    </trkpt>
    <trkpt lat="31.462572" lon="-83.503823">
      <ele>123.67</ele>
      <time>2012-07-07T11:32:35Z</time>           <=== correct Z time
    </trkpt>
    ....
    ....




Looking at the trkpt below, dnrgps is using localtime for Z time.


---  dnrgps gpx file  ---

Code Select Expand


  <trk>
    <name>ACTIVE LOG: 07 JUL 2012</name>
    <trkseg>
      <trkpt lat="31.462572" lon="-83.503823">   <=== OK
        <ele>123.67</ele>
        <time>2012-07-07T07:32:32Z</time>        <=== wrong, Z time is 11:32:32Z
        <name>T1</name>
        <type>TRACK</type>
        <extensions>
          <ogr:lat>31.462572</ogr:lat>
          <ogr:lon>-83.503823</ogr:lon>
          <ogr:y_proj>31.462572</ogr:y_proj>
          <ogr:x_proj>-83.503823</ogr:x_proj>
          <ogr:new_trk>True</ogr:new_trk>
          <ogr:new_seg>True</ogr:new_seg>
          <ogr:model>garmin Foretrex 301</ogr:model>
          <ogr:ltime>2012/07/07 07:32:32</ogr:ltime>   <=== correct local time
        </extensions>
      </trkpt>
      <trkpt lat="31.462572" lon="-83.503823">   <=== OK
        <ele>123.67</ele>
        <time>2012-07-07T07:32:35Z</time>        <=== wrong, Z time is 11:32:35Z
        <name>T1</name>
        <type>TRACK</type>
        <extensions>
          <ogr:lat>31.462572</ogr:lat>
          <ogr:lon>-83.503823</ogr:lon>
          <ogr:y_proj>31.462572</ogr:y_proj>
          <ogr:x_proj>-83.503823</ogr:x_proj>
          <ogr:new_trk>False</ogr:new_trk>
          <ogr:new_seg>False</ogr:new_seg>
          <ogr:model>garmin Foretrex 301</ogr:model>
          <ogr:ltime>2012/07/07 07:32:35</ogr:ltime>   <=== correct local time
        </extensions>
      </trkpt>
      ....
      ....






Thank you very much for tracking this down.

I will contact the dnrpgs maintainer and reference this thread.




Incidentally, I checked the gpx filed retrieved by dnrgps from the
Garmin Forerunner 305.  Z time is wrong in that gpx file, too.
It took a few seconds to turn on tracks on the 305.

I was wearing the Garmin Forerunner 305 on my right wrist.

---  dnrgps gpx file from the Forerunner 305 ---

Code Select Expand


  <trk>
    <name>1</name>
    <trkseg>
      <trkpt lat="31.462572925" lon="-83.503854387">
        <ele>111.650269</ele>
        <time>2012-07-07T07:32:44Z</time>        <=== wrong, Z time is 11:32:44Z
        <name>T6882</name>
        <type>TRACK</type>
        <extensions>
          <ogr:lat>31.462572925</ogr:lat>
          <ogr:lon>-83.503854387</ogr:lon>
          <ogr:y_proj>31.462572925</ogr:y_proj>
          <ogr:x_proj>-83.503854387</ogr:x_proj>
          <ogr:new_trk>True</ogr:new_trk>
          <ogr:new_seg>True</ogr:new_seg>
          <ogr:model>garmin Forerunner30</ogr:model>
          <ogr:ltime>2012/07/07 07:32:44</ogr:ltime>   <=== correct local time
        </extensions>
      </trkpt>
      <trkpt lat="31.462574434" lon="-83.50385447">
        <ele>111.650269</ele>
        <time>2012-07-07T07:32:45Z</time>        <=== wrong, Z time is 11:32:45Z
        <name>T6882</name>
        <type>TRACK</type>
        <extensions>
          <ogr:lat>31.462574434</ogr:lat>
          <ogr:lon>-83.50385447</ogr:lon>
          <ogr:y_proj>31.462574434</ogr:y_proj>
          <ogr:x_proj>-83.50385447</ogr:x_proj>
          <ogr:new_trk>False</ogr:new_trk>
          <ogr:new_seg>False</ogr:new_seg>
          <ogr:model>garmin Forerunner30</ogr:model>
          <ogr:ltime>2012/07/07 07:32:45</ogr:ltime>   <=== correct local time
        </extensions>
      </trkpt>
      ...
      ...






My bad.  I first thought exiftool was not correctly parsing the dnrgps gpx.

exiftool is correct.


Solved.


Cheers,
Frank Davis

[Phil Harvey]

Hi Frank,

Glad that this problem is now understood.

Until you get a fix at your end, you can fix it with exiftool either by using the -geosync feature, or by specifying a bogus timezone for -geotime.  See the Geotag documentation for details.

- Phil