ExifTool Forum

Hi all:

I'll end of ask for your help on this forum since I saw many good and accurate replies. Thanks a lot for your time.

Really, I spent a HUGE amount of hours trying to understand the way to go with my puzzle.. which is a REALLY common one. Let me elaborate:

1) I'm an amateur photographer (video+photo).
2) I travel often.
3) I have a set of devices without GPS (video, camera mainly), and only one with GPS (android Mobile).
4) I'm looking for convert my RAW images to DNG, set correctly GPS position + correct time over all my devices via scripting.
5) I'll import all to Lightroom.

My goal: Have everything with more or less correctly geotagged and with a unified criteria about creation time, for having a true sequence of videos & Pictures of my travels.

Well.. after many tries.. reading ..etc..  This is my approach:

1) Put all my no GPS devices statically to UTC timing and NEVER change them.
2) I downloaded a great open source GPS tool on my android device GPSlogger. it creates a .gpx file 24h per day.
3) Put all media content together.
3a) If it's a GPS device (my phone) leave the file untouched.
3b) if it's a non GPS device (camera or video) run the script:
3b1) If it's a RAW image, convert it to DNG (I need it since Lightroom doesn't support ARW).
3b2) add geo info with exiftool -geotag "*.gpx" file.ARW
3b3) Fix the timestamp.. and now is when everything crash!!!

In orded to follow I need to have clear some concepts:

1) Should I change creation time for files or should add offset that belongs to that time (based on the offset of my location respect UTC and saving time changes.. therefore what my phone has on the picture moment from the local GSM network / GPS gave to GPSlogger)?

2) I want to have coherence with Android. Seems like Lightroom is ignoring offset?. Any clue here?

3) Based on #1... it's same if I add some offset than change the create date? I.E ¿ 2018:12:08 17:42:26+02:00 = 2018:12:08 19:42:26+00:00 = 2018:12:08 19:42:26?

4) GPS is giving me a ZULU time (UTC) that is different what UTC time was on time that the picture has been taken (see following info). Why?

5) Which is the best way to determine the real (means the time that the android has)?. Now.. I'm using automate app that stores offset each time my mobile network changes... and I'm using it on my script.. but probably there are better ways to do it.

Sharing some examples:

Based on one ARW info... BEFORE geotagging exiftool gives this info:

ExifTool Version Number         : 11.30
File Name                       : t.ARW
..
..
Date/Time Original              : 2018:12:08 17:42:26
Create Date                     : 2018:12:08 17:42:26
Offset Time                     : +00:00
Offset Time Original            : +00:00
Offset Time Digitized           : +00:00
..
..
Create Date                     : 2018:12:08 17:42:26+00:00
Date/Time Original              : 2018:12:08 17:42:26+00:00
Modify Date                     : 2018:12:08 17:42:26+00:00
..
..

After exiftool -geotag "*.gpx" t.ARW this is what I have:

ExifTool Version Number         : 11.30
File Name                       : t.ARW
..
..
GPS Version ID                  : 2.3.0.0
GPS Latitude Ref                : North
GPS Longitude Ref               : West
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 16:42:26
GPS Status                      : Measurement Void
GPS Map Datum                   : WGS-84
GPS Date Stamp                  : 2018:12:08
GPS Differential                : No Correction
..
..
GPS Altitude                    : 343.8 m Above Sea Level
GPS Date/Time                   : 2018:12:08 16:42:26Z
GPS Latitude                    : XX deg XX' XX.XX" N
GPS Longitude                   : Y deg YY' YY.YY" W
GPS Position                    : XX deg XX' XX.XX" N, Y deg YY' YY.YY" W
(I've changed the real output of the GPS location).
..
..

Thanks!!.. I'm really frustrated with this.

Quote from: b193709 on May 02, 2019, 04:06:04 PM
1) Should I change creation time for files or should add offset that belongs to that time (based on the offset of my location respect UTC and saving time changes.. therefore what my phone has on the picture moment from the local GSM network / GPS gave to GPSlogger)?

This is where you need to check the software you use and what it does with time data.  I've read that some people do as you mention, keep everything UTC time and just deal with it.  But I believe that most programs will deal with image data as if it were local time to the location it was shot at.  I'm honestly not sure if there is a standard that actually states this exactly, though.  The Metadata Working Group pdf (unfortunately the site is still down) implies that it should be local to the image and explicitly says the time zone should not be assumed to be the same as the computer that is processing it.  I can't find anything offhand in the IPTC standard (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata) about it.

This can be in direct conflict with video time stamps, which, by the standard, are supposed to be UTC.

Quote2) I want to have coherence with Android. Seems like Lightroom is ignoring offset?. Any clue here?

My copy of Lightroom is old (ver 4.4) and I haven't tested it with regards to time zones (though I probably should do so with Adobe Bridge).  But most programs don't respect the EXIF time zone, as those tags are relatively new.  I'd would think that LR might respect the XMP time stamps, that do allow for time zone data.  But again, it probably will treat it as local time for the image.

I can't really comment on the rest.  You lose me a bit and I'd say you have to just check you programs with regards to time differences.

Except for times in the QuickTime group (mostly video), times are supposed to be local times (that is, the time at the location at which the image/video was taken), always. So setting your camera to UTC is not what I'd advise. I'm sure this would also throw off loading the gps data from the track info since it might take the wrong time to synchronize the two.

With regards to times in the QuickTime group, many cameras actually also store local time there. That's why Phil actually also treats those as local time by default, use -api QuickTimeUTC=1 to override this.

So in your example, the question is: what was the local time at which you took the image, what timezone (offset) was applicable there at that time? With those questions answered, it should be possible to tell you what things should be.

Posting a sample image and tracking info would help too.

Cheers,
Hayo

Thanks.

The main idea is get close to the reality that I experience when I'm traveling (or not). Let's suppose that I take a selfie everyday at 9:00 with my Android phone. All pictures will reflect the correct GPS and the 9:00 local time. So.. no differences if I'm at home, or in the other side of the planet. GPS is stamped correctly (it knows where is), and time is automatically switched based on the GMS network. Therefore.. there is a coherence.

If I repeat same with my DSLR, it forces me to switch local time of my camera each time I travel (belive me .. will not work.. I can remind it maybe 50% of the time.. but it's a huge mess when I forget it). What about GPS?.. GPX is stored based on UTC (afaik) + the correct offset. If I forgot change the offset of my camera.. GPS tag will not work either.

Then multiply by x times (one for each non GPS device) >> This approach didn't work to me.

Moving all to UTC.. eliminates the need of change for daylight saving.. local offset..etc.. you can after correct this.. (this is what I'm trying to do)..

The point is.. what to do correctly?:

1) Set the correct offset and leave the original time?    So for 2018:12:08 17:42:26 (UTC) and +02:00 offset to 2018:12:08 17:42:26+02:00.

2) Change the creation time to  2018:12:08 19:42:26?

From your responses I saw that probably LR and others will ignore offset (not sure about). I can try with LR.. but in general I don't have a clear statement should be done. Also for video?

Thanks

Right, I understand the standardisation of your cameras to UTC.

Problem is though that all software will interpret the times in the camera metadata as local time so if you view your image in e.g. Lightroom, it will show it with the wrong time (UTC instead of local).

So since you are already running a "script" on the non-phone files, I suggest you extend it a little to correct the times to local time, once the location is known?
The issue with this though is that you likely can't do this automatically since the timezone offset won't be known; it is not recorded in the camera file. It might be recorded in the GPX track info, but I don't know if that's the case (probably not, actually).

One thing that strikes me as odd in the example you showed is that the camera recorded the image at 2018:12:08 17:42:26, which according to your story is UTC. However, the GPS DateTime is set to 2018:12:08 16:42:26. This shouldn't be the case: they are both in UTC so they should be the same! Unless the GPS track is synchronised wrongly, that is. But to be honest, this is what I suspect anyway since exiftool converts to the DateTimeOriginal times recorded in the file to UTC, assuming the local timezone (you didn't tell exiftool to do otherwise in your -geotag command). What is your local time zone? It was +1:00 at the moment you performed the -geotag, right?

You need to correct that first before we can start thinking about getting the times in your files to local time (as expected by all software), and with the correct timzeone offset.

For my own purposes I have written a script which, amongst others, corrects the timezone (and daylight savings) info based on the location. Since, apart from my iPhone, I don't have GPS info, I do this based on the country information I set in the files (manually). For countries where there are multiple timezones, I also use the state (this still doesn't work everywhere, e.g. some states in the US have multiple timezones, but I haven't been there so this has not been a problem yet). This works quite well and does take into account the date at which an image was taken (to e.g. determine daylight savings was to be applied or not).

I have been thinking of determining the TimeZone from GPS info (that would solve my problem with timezone borders crossing states), but this isn't straightforward and requires e.g. a reverse geocode service that gives the timezone as part of the location info. I think I've found such a service (free) but haven't yet started to work this out (I don't really have a need for it at the moment). You'll need something similar to be able to automate setting the timezone though (unless, which I don't think, it is already part of the GPS track info).

Some comments:

(Hayo) Problem is though that all software will interpret the times in the camera metadata as local time so if you view your image in e.g. Lightroom, it will show it with the wrong time (UTC instead of local).

(Alvaro) How can LR or exiftool knows where I am now?. Even in the case I give it some language info.. how can it knows?

(Hayo) So since you are already running a "script" on the non-phone files, I suggest you extend it a little to correct the times to local time, once the location is known?
The issue with this though is that you likely can't do this automatically since the timezone offset won't be known; it is not recorded in the camera file. It might be recorded in the GPX track info, but I don't know if that's the case (probably not, actually).

(Alvaro) Actually .. on purpose, I record a text file each time my phone changes its offset, based on the GSM info. I store on this file the diff between UTC and it's local time. Therefore.. if I read the file and I apply I.E +02:00... If I add +02:00 I'll obtain the sync time with the photos I have in my phone (that is my principal goal).

(Hayo) One thing that strikes me as odd in the example you showed is that the camera recorded the image at 2018:12:08 17:42:26, which according to your story is UTC. However, the GPS DateTime is set to 2018:12:08 16:42:26. This shouldn't be the case: they are both in UTC so they should be the same! Unless the GPS track is synchronised wrongly, that is. But to be honest, this is what I suspect anyway since exiftool converts to the DateTimeOriginal times recorded in the file to UTC, assuming the local timezone (you didn't tell exiftool to do otherwise in your -geotag command). What is your local time zone? It was +1:00 at the moment you performed the -geotag, right?

(Alvaro) I notice the same.. and yes.. offset was +01:00 the local offset when I did the test was +01:00. But.. again.. how exiftool knows that I was on +01:00?. Is it inside of the GPX?.. probably. All this is really messy. Anyway.. if I determine the offset my phone has on the moment that I took the photo.. i can add this to creation time and all fixed?..

I notice that GPS match with photo.. so my photo on UTC creation time was correctly GPS located. I can stamp GPS and after correct the creation time..?

Something that is not completely clear is..should I add an offset or should I change creation time..

About your workflow.. for me this seems to be so demanding in terms of time and dedication. I think I can solve the offset stuff at least with my correlating point (Android) well.. but things like the GPS stuff and localisation of the software.. makes me crazy..

Quote from: b193709 on May 03, 2019, 09:42:53 AM
How can LR or exiftool knows where I am now?.

Your (presumably Windows) system has a time zone setting that ExifTool can access.

- Phil

Quote from: b193709 on May 03, 2019, 02:24:20 AM
1) Set the correct offset and leave the original time?    So for 2018:12:08 17:42:26 (UTC) and +02:00 offset to 2018:12:08 17:42:26+02:00.
2) Change the creation time to  2018:12:08 19:42:26?

Quote from: b193709 on May 03, 2019, 09:42:53 AM
Something that is not completely clear is..should I add an offset or should I change creation time..

Reading this now, I think you are misunderstanding time zone data.  Time zones in the time stamp metadata is not something added or subtracted to the time stamp. It is just there to let the computers know what the time zone was where the image was taken.  For example, the time as I'm writing this is about 7:45 am on the west coast of the US.  The time stamp for this would be 2019:05:03 07:45:00-07:00.  The time zone is included in the time stamp but is not added or subtracted from the timestamp.   The correct output from exiftool would be something like the following.  This output shows the EXIF, GPS, IPTC, and XMP tags that correspond to each other.  Note that the GPS time stamp would be 2019:05:03 14:45:00 (UTC time) while everything else is 2019:05:03 07:45:00 (local time).

c:\>exiftool -time:all --system:all -g1 -a -s y:\!temp\Test4.jpg
---- ExifIFD ----
DateTimeOriginal                : 2019:05:03 07:45:00
OffsetTimeOriginal              : -07:00
---- GPS ----
GPSTimeStamp                    : 14:45:00
GPSDateStamp                    : 2019:05:03
---- IPTC ----
DateCreated                     : 2019:05:03
TimeCreated                     : 07:45:00-07:00
---- XMP-photoshop ----
DateCreated                     : 2019:05:03 07:45:00-07:00


This is how I would recommend setting your time data, as it would be the most compatible with programs such as Lightroom.  Based upon your previous posts, I would suggest correcting the DateTimeOriginal, most likely using the GlobalTimeShift option (https://exiftool.org/exiftool_pod.html#globalTimeShift-SHIFT).  Then add the correct OffsetTimeOriginal.  Then you could use the Composite tag (https://exiftool.org/TagNames/Composite.html) SubSecDateTimeOriginal to properly set any other tag you'd like, though I'd suggest setting XMP:DateCreated at the very least, as Lightroom will most likely be able to get the timezone data from that tag over the OffsetTimeOriginal tag.

If you keep your camera set on utc, you loose information.  Your camera might record 9 am but it is really 4 pm so, months or years later, you might think you took the photo in the morning which would be misleading.

I suggest that everytime you move country or timezone, take a photo of your watch or 'phone screen with the dslr. Then, when processing the photos, you can see and adjust the time stamps if the dslr is incorrectly set.

I create a xmp:datetimeoriginal in all my photos as I can put a timezone in it and it then becomes my reference tag.

Quote from: Alan Clifford on May 03, 2019, 02:51:49 PM
I suggest that everytime you move country or timezone, take a photo of your watch or 'phone screen with the dslr. Then, when processing the photos, you can see and adjust the time stamps if the dslr is incorrectly set.

I second this and should have thought to mention it.  I actually have a widget on my phone that shows the seconds so that way I can adjust the time exactly, as my camera tends to drift a few seconds every month. 

You can say I'm a bit overly obsessive. 

Well big thanks for all your responses.. I think now I understand better what I have to do.. and yes.. I misunderstand time zone data purpose completely.

Summarising.. this is what I have:

1) Media content (photo+video) with a creation time over UTC
2) GPX data for the content.
3) TimeZone for the content (based on the diff between the local time of my phone where I am and the UTC value). My small script on my Android does the job.

So.. my steps should be:

1) add GPS info to DNG file

exiftool -geotag "*.gpx" file.DNG

2) Change creation time to the correct local time for each photo/video. I should add the timezone offset I got from my script on each.

exiftool -globalTimeShift -offset_value -time:all file.DNG

3) Add the local time zone to the creation date:

exiftool '-datetimeoriginal<${datetimeoriginal}-offset_value' file.DNG


At this point I'll have my photos/videos sync with the local time and with the correct timezone. Therefore I can mix photos taken from my phone and from my DSLR and will be sync in time... right?

Couple of questions?..

1) Why should I be take in consideration my local timezone when I'm running exiftool?.. I'm OS X user.. It's aware about my timezone.. but.. does it care for my process?

2) Why should I add the XMP:DateCreated reference tag?. I was working on leaving the correct info on the creation date tag which seems to be the standart.. also with the timezone info.. why add this?.

Thanks

Quote from: b193709 on May 03, 2019, 08:24:02 PM
Well big thanks for all your responses.. I think now I understand better what I have to do.. and yes.. I misunderstand time zone data purpose completely.

Summarising.. this is what I have:

1) Media content (photo+video) with a creation time over UTC
2) GPX data for the content.
3) TimeZone for the content (based on the diff between the local time of my phone where I am and the UTC value). My small script on my Android does the job.

So.. my steps should be:

1) add GPS info to DNG file

exiftool -geotag "*.gpx" file.DNG

2) Change creation time to the correct local time for each photo/video. I should add the timezone offset I got from my script on each.

exiftool -globalTimeShift -offset_value -time:all file.DNG

3) Add the local time zone to the creation date:

exiftool '-datetimeoriginal<${datetimeoriginal}-offset_value' file.DNG


At this point I'll have my photos/videos sync with the local time and with the correct timezone. Therefore I can mix photos taken from my phone and from my DSLR and will be sync in time... right?

Couple of questions?..

1) Why should I be take in consideration my local timezone when I'm running exiftool?.. I'm OS X user.. It's aware about my timezone.. but.. does it care for my process?

2) Why should I add the XMP:DateCreated reference tag?. I was working on leaving the correct info on the creation date tag which seems to be the standart.. also with the timezone info.. why add this?.

Thanks

Here's what I think your steps should be (if you keep your cameras always on UTC), please note that you'll probably have to experiment here and there to see if the results are as expected; I don't have the necessary test files to check everything myself. Below, offset_value should be the offset in hours of the place in which the images where taken. Switch + and - appropriately.

1) Add GPS info, and tell exiftool it should interpret the times in the file as UTC, not as times for where you are now.
exiftool  '-Geotime<$DateTimeOriginal#Z' -geotag "*.gpx" file.DNG

2) Correct the times in the files to become local times at the place you took 'm:
exiftool -globalTimeShift -offset_value:00 -time:all file.DNG

3) Add/replace timezone info in tags that have timezone (check your own files to see which tags are really there):
exiftool '-XMP:createDate<${XMP:createDate; s/([+-Z].*|)$/+offset_value:00/;}' file.DNG
exiftool '-XMP:DateCreated<${XMP:DateCreated; s/([+-Z].*|)$/+offset_value:00/;}' file.DNG
exiftool '-IPTC:TimeCreated<$IPTC:TimeCreated;  s/([+-Z].*|)$/+offset_value:00/;}' file.DNG

Optional 4) Specify timezone offset for datetime original:
exiftool -offsetTimeOriginal="+offset_value:00" file.DNG

For video, things need to be slightly different in step 2 since you probably want to keep the times UTC (which is the spec anyway). I think if you simply add --QuickTime:all, exiftool will leave alone the times that are supposed to be UTC. In future you'll be wanting to use -api QuickTimeUTC=1 with exiftool too, to tell it to interpret movie times as UTC. Modified step 2 (which you could use above as well)
exiftool -globalTimeShift -offset_value:00 -time:all --QuickTime:all file.MOV

Hope this works as I think it should :)

Quote from: Hayo Baan on May 04, 2019, 04:40:26 AM
2) Correct the times in the files to become local times at the place you took 'm:
exiftool -globalTimeShift -offset_value:00 -time:all file.DNG

I believe you need a -TagsFromFile in there, otherwise all this does is list the time stamps with the shift:
exiftool -globalTimeShift -offset_value:00 -TagsFromFile @ -time:all file.DNG

At first, I thought that this would shift the GPS time data.  But I just tested it and it didn't change GPSTimeStamp (though it did change the composite GPSDateTime).  Today I learned.

Quote from: b193709 on May 03, 2019, 08:24:02 PM
3) Add the local time zone to the creation date:

exiftool '-datetimeoriginal<${datetimeoriginal}-offset_value' file.DNG

Quote2) Why should I add the XMP:DateCreated reference tag?. I was working on leaving the correct info on the creation date tag which seems to be the standart.. also with the timezone info.. why add this?.

Because DateTimeOriginal is an EXIF group tag and the EXIF date stamps do not allow for the addition of a time zone.  The command example you give here will set the date, but the time zone will be dropped.  XMP:DateCreated is much more flexible in how dates can be formatted and allows for the addition of the timezone.  The XMP standard was created by Adobe, so Lightroom is more likely to be able to read the time zone from that tag.  EXIF does have a place to put the time zone, OffsetTimeOriginal (and related Offset tags) but it is still fairly recent (well, almost 3 years ago) and not many programs will read it.

Quote1) Why should I be take in consideration my local timezone when I'm running exiftool?.. I'm OS X user.. It's aware about my timezone.. but.. does it care for my process?

For the most part, you don't need to as long as you set the time zone.  But in cases where the time zone is required but not set, exiftool (and many other programs) will assume the local time zone of the computer, because there isn't really any other option.