ExifTool Forum

ExifTool => Bug Reports / Feature Requests => Topic started by: ktarasenko on August 29, 2016, 02:45:28 PM

Title: Request to support TimeSpan field for KML files
Post by: ktarasenko on August 29, 2016, 02:45:28 PM
Hi,
Thanks for a tool, I really love it.
I'm usually use it with google maps timeline KML files (track that google creates when stalks on user). Previously it worked flawlessly, but as far as I understood, recently they changed format a bit and now use TimeSpan fields instead of TimeStamp.

      <TimeSpan>
        <begin>2016-08-27T16:20:02.861Z</begin>
        <end>2016-08-27T17:04:15.217Z</end>
      </TimeSpan>


Could I expect this to be implemented in nearest future or I should rather automate the conversion to supported format?

Thanks
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on August 29, 2016, 10:02:44 PM
Interesting.  Thanks for the heads up.

Could you post a full KML file with a track so I can see how it is structured?

Thanks.
Title: Re: Request to support TimeSpan field for KML files
Post by: ktarasenko on August 30, 2016, 02:42:39 AM
Of course
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on August 30, 2016, 07:28:53 AM
Thanks for the sample.

Hmmm.  This is unfortunate:

    <Placemark>
      ....
      <gx:Track>
        <altitudeMode>clampToGround</altitudeMode>
        <gx:coord>10.9886 45.45117 0</gx:coord>
        <gx:coord>10.9886578 45.4512008 0</gx:coord>
        <gx:coord>10.9899935 45.4500292 0</gx:coord>
        <gx:coord>10.989993500000002 45.45002920000001 0</gx:coord>
      </gx:Track>
      <TimeSpan>
        <begin>2016-08-27T08:28:54.679Z</begin>
        <end>2016-08-27T08:32:24.480Z</end>
      </TimeSpan>
    </Placemark>


The coordinates aren't each associated with a time.  Instead, there are a bunch of coordinates and a time span.  There is not enough information in the file to geotag properly. :(

Maybe the best thing to do would be to average all of the coord values and assume you were at the average position for the entire TimeSpan, but this isn't very satisfying, and would require a significant change to the ExifTool code, so I'm not sure it is worthwhile.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: ktarasenko on August 30, 2016, 05:06:52 PM
I see now. That makes sense.  Thanks for your answer.

For anybody who have similar problem, there's workaround: "download a copy of all your data" in kml format (it will take a while, but as an output, the good old format will be delivered), probably it will stop working this way sooner or later.
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on October 04, 2016, 11:10:32 AM
Hi,

I have a similar problem.
The only way I have to acquire a KML file for past photo trips, where I did not register  my track is to go to Google Location History.
There, I often have my placemarks in the form:

-<Placemark>
<name>Belmonte</name>
<address>Portugal</address>
-<ExtendedData>
-<Data name="Category">
<value/>
</Data>
-<Data name="Distance">
<value>0</value>
</Data>
</ExtendedData>
<description> from 2016-09-03T13:49:21.850Z to 2016-09-03T14:00:25.540Z. Distance 0m </description>
-<Point>
<coordinates/>
</Point>
-<gx:Track>
<altitudeMode>clampToGround</altitudeMode>
<gx:coord>-7.348178600000001 40.3598503 0</gx:coord>
</gx:Track>
-<TimeSpan>
<begin>2016-09-03T13:49:21.850Z</begin>
<end>2016-09-03T14:00:25.540Z</end>
</TimeSpan>
</Placemark>

It is perfectly clear that I was at this location from 13:49 to 14:00. So, I would love to have my photos all geotagged to this location if the fell within this timespan. Yet, ExifTool ignores all TimeSpan tags, and it only uses TimeStamp (I understand it is less ambigous).
So, is there any way to convert a TimeSpan KML to a TimeStamp KML? If not, is there any way for ExifTool to consume it?

Thanks,

Jose Luis
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on October 04, 2016, 11:12:56 AM
Here's the full KML file
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 11:22:55 AM
Hi Jose,

I have the same problem with the KML you posted.  What should ExifTool do with Placemark Track that contains multiple coordinates?  For example:

<Placemark>
<name>Driving</name>
<address>
</address>
<ExtendedData>
<Data name="Category">
<value>Driving</value>
</Data>
<Data name="Distance">
<value>1604</value>
</Data>
</ExtendedData>
<description> Driving from 2016-09-03T07:14:16.078Z to 2016-09-03T07:23:56.000Z. Distance 1604m </description>
<Point>
<coordinates>
</coordinates>
</Point>
<gx:Track>
<altitudeMode>clampToGround</altitudeMode>
<gx:coord>-8.8667576 40.159571 0</gx:coord>
<gx:coord>-8.866517 40.1600814 0</gx:coord>
<gx:coord>-8.86656 40.1600385 0</gx:coord>
<gx:coord>-8.8657012 40.1609289 0</gx:coord>
<gx:coord>-8.8642696 40.1618189 0</gx:coord>
<gx:coord>-8.8637476 40.1629397 0</gx:coord>
<gx:coord>-8.8619408 40.1631159 0</gx:coord>
<gx:coord>-8.8610847 40.163662 0</gx:coord>
<gx:coord>-8.8606244 40.1636782 0</gx:coord>
<gx:coord>-8.858654 40.1638653 0</gx:coord>
<gx:coord>-8.8563227 40.1641873 0</gx:coord>
<gx:coord>-8.8530766 40.1645021 0</gx:coord>
<gx:coord>-8.8513966 40.1644282 0</gx:coord>
<gx:coord>-8.8516482 40.165216699999995 0</gx:coord>
</gx:Track>
<TimeSpan>
<begin>2016-09-03T07:14:16.078Z</begin>
<end>2016-09-03T07:23:56.000Z</end>
</TimeSpan>
</Placemark>


- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on October 04, 2016, 12:46:01 PM
Actually, you only get lists of coordinates when you are on the move, traveling from one place to another.
In those periods, you are unlikely to have photographs. In the places where I have photographs, I stayed longer in the same spot, and I only have one coordinate, just like the example I quoted.


So, i would be happy if exiftool simply considered those timespans where there's a single GPS coordinate.


That would solve 95%  of my needs.

As for the traveling timespans, you could interpolate the coordinates along the time span, and you would have a perfectly reasonable estimate of the locations during that time.


Of course, nome of tais is ideal, but when you're trying to geotag old photos for which Google location history is your only record, it would be a life changer
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 01:06:13 PM
Quote from: malaquias on October 04, 2016, 12:46:01 PM
As for the traveling timespans, you could interpolate the coordinates along the time span, and you would have a perfectly reasonable estimate of the locations during that time.

You're suggesting assuming the coordinates to be equally spaced in time within the specified TimeSpan?  So if there were two coordinates, the first would be at the TimeSpan start and the second at the end?  Hmm.  Yes, maybe that would be the best thing to do.

As you mention, this is not ideal since it isn't very accurate.  Also, it will require a LOT of new code in the ExifTool Geotag module to handle the very different logic needed to geotag from tracks like this.

But I will look into it and post back here after I've tried to work through this in more detail.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on October 04, 2016, 01:15:16 PM
Thank you so much for being open to that.
Like you said, it's not ideal, but it is way better than no information at all.
I am also a coder (but mostly C). Please let me know if I can help with the coding.

Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 01:21:21 PM
Hi Bill,

OK, as a zeroth order approximation you can add this line to the %xmlTag definition in lib/Image/ExifTool/Geotag.pm to load a single track point at the end of each TimeSpan:

    end         => 'time',      # KML

You can try this yourself if you are using the Perl version.

Implementing your full suggestion will of course be more work.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 02:12:55 PM
OK.  New problem.  What happens if the end of one TimeSpan is exactly the same as the start of another.  Using the algorithm you suggested then you could have two different fixes at the same time. :(

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 03:19:25 PM
I've got a working solution.  Here is what I've done:

1. If the end of one TimeSpan is the same as the start of the next, then I add 0.0001 seconds to the start time of the next TimeSpan to avoid the ambiguity.

2. I assume that the Track points are equally spaced in time within the time span, with the first point at the start of the TimeSpan and the last point at the end.

3. If there is only one Track point in the TimeSpan, then I generate two Track points each with the same coordinates, on for the start of the TimeSpan and one for the end.

4. When geotagging within a TimeSpan, the interpolation is done no matter how far apart in time the points are. (ie. GeoMaxIntSecs is ignored).

5. When geotagging outside a TimeSpan, the nearest track points must be within GeoMaxIntSecs for the interpolation to be performed.  If outside this but the time is within GeoMaxExtSecs of one of the TimeSpans, then the position is taken as the closest end of the nearest TimeSpan.

This seems reasonable to me, but it will be interesting to see if it produces good results.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on October 04, 2016, 05:55:48 PM
OK. I'm pretty excited.
When can we begin testing it?
Will you release a new versio with those changes?

Thanks

Jose Luis
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 04, 2016, 09:56:41 PM
Hi Jose,

I'll see about doing a release tomorrow.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: Phil Harvey on October 05, 2016, 08:47:26 AM
Exiftool 10.29 is now available.

- Phil
Title: Re: Request to support TimeSpan field for KML files
Post by: malaquias on December 20, 2016, 06:58:38 PM
 :D
Dear Phil,

I am sorry that I only noticed your reply today, when I ran into the same problem.
I downloaded a new version of ExifTool, and PRESTO. It works like a charm.
You are a true programming genius.

Nowadays, I carry an app on my cell phone continously recording a gpx file when I'm taking pictures. Unfortunately, today, my phone rebooted, and so I lost the location info for most of my day. I could only avail myself of the Google Location History (since Big Google Brother is always tracking me anyway), with its awful TimeSpan fields.

So, I tried your new version, and it works perfectly. You made a great solution out of a very bad data format. Congrats.

Have yourself a Merry Christmas, since you already provided me with a wonderful Christmas Gift.