News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Date arithmetic revisited

Started by e2b, September 07, 2019, 10:34:48 PM

Previous topic - Next topic

e2b

I have been attempting something similar to what was raised by Peterzz some time ago in https://exiftool.org/forum/index.php/topic,8513.msg43754.html#msg43754 (Date Arithmetic)

I want to perform inverse geotagging on some images that were recorded together with GPS data (on a Samsung S8 phone). However, the GPS reception was intermittent so in many cases the CreateDate differs considerably from the GPSDateTime. My objective is to write an -if condition that will only select images for which the two times differ by less than 30 seconds (before or after); I have not yet succeeded.

To help understand what is happening, I have used -p option to output the two times in both formatted and UNIX seconds, by using a .fmt file containing the specifications:
GPS=$GPSDateTime  ; GPS(sec)=${GPSDateTime;DateFmt("%s")}
Create=$CreateDate ; Create(sec)=${CreateDate;DateFmt("%s")

For example, with an image recorded in Italy (UTC +2):
File=20170526_155611.jpg
GPS=2017:05:26 13:56:05Z ; GPS(sec)=1495806965
Create=2017:05:26 15:56:11 ; Create(sec)=1495785371

As shown by the formatted dates, the GPS record was made 6 seconds prior to the image creation. However, working in UNIX seconds, the times differ by 21594 seconds, which indicates that the time zones as interpreted in the exif data differ by 6 hours (21600 seconds), rather than the actual 2 hours. This seems odd, but manageable.

I tried to create -if conditions following the pattern that Peterzz described in the previous thread, e.g.:

-if "${GPSDateTime;DateFmt("%s")}-${CreateDate;DateFmt("%s")}<21570" (i.e. less than 30 sec difference)

However, I have run into problems. When I run this in a command (for files in i:/Fix), I get the warning:
Warning: Global symbol "%s" requires explicit package name (did you forget to declare "my %s"?) for 'GPSDateTime' - i:/Fix/20170526_155611.jpg

1.   What does this mean (I am using Exiftool in Windows 10)?

Moreover, the condition is evaluated true, regardless of the number I specify. On the other hand, when I tried to write a condition for the opposite time difference (GPS time later that image time), e.g. difference > 21630, testing showed that the condition is always false, regardless of the actual number.

2.   Is there any way of seeing the numerical value of the difference between the times, as used in the evaluation of the condition?

Help would be greatly appreciated.

StarGeek

At least one problem is the quotes.  Your example:
-if "${GPSDateTime;DateFmt("%s")}-${CreateDate;DateFmt("%s")}<21570"
is using fancy quotes "" which should not be used on the command line.  CMD does not recognize them as quotes.

Additionally, under Windows CMD, you want to use regular double quotes as the exterior quotes and single quotes on the interior (Mac/Linux is the reverse).
-if "${GPSDateTime;DateFmt('%s')}-${CreateDate;DateFmt('%s')}<21570"
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

e2b

Thanks StarGeek!

Your corrections solved the problem; I should compose commands in Notepad, rather than Word.

Any idea as to why the difference in apparent time zones is 6 hours rather than 2? I wouldn't have been able to get the condition right if I hadn't used -p to see the values in '%s' format.

And is there any way to see the result of date arithmetic? Could the result of such a calculation be assigned to a tag?

StarGeek

Quote from: e2b on September 07, 2019, 11:44:28 PMI should compose commands in Notepad, rather than Word.

Yep.  Many word processors (including Google Docs) will "help" you in this way. 

QuoteAny idea as to why the difference in apparent time zones is 6 hours rather than 2? I wouldn't have been able to get the condition right if I hadn't used -p to see the values in '%s' format.

Even though the original time was in Italy at +02:00, The CreateDate tag doesn't include the timezone.  The %s format is Epoch time, which is UTC.  In order to calculate that time properly, exiftool has to use the timezone of the computer it's running on.  For me, when I test your data, I have 9 hours different (-07:00 TZ instead of +02:00).  So I'm guessing you're currently East Coast time zone.

Two work arounds.  One, add the proper time zone to EXIF:OffsetTimeDigitized
exiftool -EXIF:OffsetTimeDigitized=+02:00 DIR
and use SubSecCreateDate instead of CreateDate
${SubSecCreateDate;DateFmt('%s')}
SubSecCreateDate is a composite tag which compiles the create date, the subseconds, and the time zone into a nice easy package. 

The other option would be to copy EXIF:CreateDate to XMP:CreateDate, include the timezone in the copy, and then use the XMP tag instead.
"-XMP:CreateDate<${EXIF:CreateDate}+02:00"
${XMP:CreateDate;DateFmt('%s')}

QuoteAnd is there any way to see the result of date arithmetic? Could the result of such a calculation be assigned to a tag?

It's doable, but very tricky.  I can't do it off the top of my head and it's getting late for me.  I'll give you some examples tomorrow.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

e2b

Thanks again,

Actually, I am in Western Australia (UTC+8) :)

I (and am sure others, too) look forward to seeing your solution, but there's no rush.

StarGeek

#5
Ah, 6 hours the other way.  I wasn't paying that close attention to if the difference was positive/negative, just the overall number difference.

Now that my brain isn't too fuzzy, I'm realizing it's a bit trickier than I thought, but still doable.  It would require using the -d (dateFormat) option to format all dates as Epoch seconds, as well as needing either one of my previous suggestions for an alternative tag or adjusting the time zone manually.

In order to do the math, you have to get the value of the second tag from within the first tag.  That's done with the $self->GetValue call.

To output the difference while manually adjusting for the time zone (6 hours*60 minutes*60 seconds = 21,600 seconds, correct me if my math below is wrong)
exiftool -d "%s" -p "${GPSDateTime;$_=$_-$self->GetValue('CreateDate')-21600}" FILE.jpg
and to copy the difference to another tag
exiftool "-Description<${GPSDateTime;$_=$_-$self->GetValue('CreateDate')-21600}" FILE.jpg

To use a different tag, such as SubSecCreateDate, just change the CreateDate part to the new tag and remove the -21600 if the new tag contains the time zone.  Take note that this part is case-sensitive, unlike most exiftool tag name operations.  The group name isn't, so if you wanted to use the XMP tag, you could use xmp:CreateDate or XMP:CreateDate.

The output is going to be in seconds.  If you wanted a different format, than that may be possible, but I can't think of a way off hand without resorting to creating a user defined tag, which is often the better and safer option anyway when doing more complex processing on a regular basis, as there are more basic sanity checks built in to such a process.

Edit: Forgot to mention removing the manual time zone adjustment when using a tag that contains the time zone.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

e2b

Wow! Your solution reminds a bit of being given the solution of a crossword puzzle; it's all logical in retrospect, but how on earth did you get to that point? This has been a virtuoso performance in exiftool mastery!

I have learnt a great deal about exiftool from your explanations, as well as achieving my original objective of extracting good GPS tracks from my phone pictures.

Thanks, once again.

StarGeek

Quote from: e2b on September 09, 2019, 10:38:38 AM
Wow! Your solution reminds a bit of being given the solution of a crossword puzzle; it's all logical in retrospect, but how on earth did you get to that point?

Actual years of experience and occasionally giving the wrong answer in which Phil corrects me on.  And taking a lot of notes.  And then pushing the boundaries of exiftool to the point of driving Phil crazy with the extremely obscure bugs I occasionally find :D

QuoteI have learnt a great deal about exiftool from your explanations, as well as achieving my original objective of extracting good GPS tracks from my phone pictures.

Glad to hear it.  I try to hopefully explain my examples in a way that's not to confusing, though in the more advance examples confusion can become unavoidable.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Phil Harvey

Quote from: StarGeek on September 09, 2019, 12:15:15 PM
pushing the boundaries of exiftool to the point of driving Phil crazy with the extremely obscure bugs I occasionally find :D

Actually, I really, really like it when someone helps me find a bug in ExifTool.  This especially includes obscure bugs which could otherwise linger for a long time before they are fixed.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Alan Clifford

QuoteMy objective is to write an -if condition that will only select images for which the two times differ by less than 30 seconds (before or after); I have not yet succeeded.

What happens if you stand still for 30 minutes?  The gps app I was running on my old iphone would stop recording the position as it hadn't changed.

e2b

The situation was somewhat different from the one you have in mind.

I wasn't running a tracking app on my phone. Instead, I was using the GPS data that the phone (Samsung S8) writes into the exif data for the images. However, this phone appears to use its last (stored) GPS reading to write the GPS data into the exif data. The result was that where I was walking in city streets with intermittent GPS coverage, some photos clearly had incorrect (out-of-date) GPS data. My aim was to produce a GPS track (by exiftool inverse geotagging) using only images taken at more-or-less the same time as the recorded GPS reading, on the basis that these GPS data were reasonably accurate. When I checked the tracks (with GPS Track Editor) I found a few ambiguous points that I resolved manually. I then used the GPS track record to geotag all the images by interpolation, and also to geotag other photos taken on another camera I was using at the same time.

Alan Clifford

Quote from: e2b on October 12, 2019, 04:38:08 AM
However, this phone appears to use its last (stored) GPS reading to write the GPS data into the exif data. ing at the same time.

That would be OK as the gps time stamp associated with the gps location would be different from the datetimeoriginal of the photograph.  But the best you could do would be to interpolate.

e2b

Yes, it's basically getting what I can out of limited information.

However, your note that "the gps time stamp associated with the gps location would be different from the datetimeoriginal of the photograph", raises an important point that I'd quite overlooked. This has prompted me to ask a question about this in another post https://exiftool.org/forum/index.php/topic,10515.0.html