QuickTime:DateTimeOriginal written automatically, without timezone conversion

[torarnv]

Hi there, and thanks for this excellent tool!!

I've stumbled upon a behavior that I suspect is a bug:


Delete all DateTimeOriginal

Code Select Expand

❯  exiftool '-DateTimeOriginal=' foo.mp4 && exiftool -DateTimeOriginal -ee -G -s foo.mp4
    1 image files updated


Set DateTimeOriginal:

Code Select Expand

❯  exiftool '-DateTimeOriginal=2020:01:01 01:02:03' foo.mp4 && exiftool -DateTimeOriginal -ee -G -s foo.mp4
    1 image files updated
[QuickTime]     DateTimeOriginal                : 2020:01:01 01:02:03
[XMP]           DateTimeOriginal                : 2020:01:01 01:02:03

According to the documentation:

If no group name is specified, the tag is created in the preferred group, and updated in any other location where a same-named tag already exists. The preferred group is the first group in the following list where TAG is valid: 1) EXIF, 2) IPTC, 3) XMP.

But the result is that the tag is set both for XMP and for Quicktime.

This is a bit surprising, but not in itself critical. The problem is that QuickTime tags are supposed to be in UTC, so now there's a mismatch between what the two tags report.

Normally that would be fixable by setting the QuickTimeUTC API option, but according to https://exiftool.org/TagNames/QuickTime.html the DateTimeOriginal is not affected by that option, nor does it seem to be in practice:

Code Select Expand

❯  exiftool  -api QuickTimeUTC=1 '-DateTimeOriginal=2020:01:01 01:02:03' foo.mp4 && exiftool -DateTimeOriginal -ee -G -s foo.mp4
    1 image files updated
[QuickTime]     DateTimeOriginal                : 2020:01:01 01:02:03
[XMP]           DateTimeOriginal                : 2020:01:01 01:02:03

Shouldn't it? Especially if the tag can be set so easily by being a preferred group?

This particular behavior is also a bit 'scary' since the QuickTime:DateTimeOriginal tag is hidden from output unless you specify -ee. I spent some time scratching my head where the "misbehaving" tag was coming from. It was read from another application (PhotoSweeper), and showing the "wrong" time, and I couldn't figure out where the problem was since I wasn't seeing the tag. I'd expect -time:all eg to list it, but it doesn't.

[torarnv]

One way to fix this for already 'broken' files is :

Code Select Expand

❯  exiftool -d '%Y:%m:%d %H:%M:%S%z' '-QuickTime:DateTimeOriginal<XMP:DateTimeOriginal' foo.mp4 && exiftool -DateTimeOriginal -G -s -ee foo.mp4
    1 image files updated
[QuickTime]     DateTimeOriginal                : 2020:01:01 01:02:03+01:00
[XMP]           DateTimeOriginal                : 2020:01:01 01:02:03

Dunno if there's a better way?

[torarnv]

Or to completely sync the QuickTime dates:

Code Select Expand

❯ exiftool -overwrite_original -api QuickTimeUTC=1 \
-d '%Y:%m:%d %H:%M:%S%z' \
'-QuickTime:DateTimeOriginal<XMP:DateTimeOriginal' \
'-QuickTime:CreateDate<XMP:DateTimeOriginal' \
'-QuickTime:ModifyDate<XMP:DateTimeOriginal' \
'-QuickTime:TrackCreateDate<XMP:DateTimeOriginal' \
'-QuickTime:TrackModifyDate<XMP:DateTimeOriginal' \
'-QuickTime:MediaCreateDate<XMP:DateTimeOriginal' \
'-QuickTime:MediaModifyDate<XMP:DateTimeOriginal' \
foo.mp4

❯ exiftool -time:all -ee -G -s foo.mp4
[File]          FileModifyDate                  : 2020:01:11 14:37:08+01:00
[File]          FileAccessDate                  : 2020:01:11 14:37:08+01:00
[File]          FileInodeChangeDate             : 2020:01:11 14:37:08+01:00
[QuickTime]     CreateDate                      : 2020:01:01 00:02:03
[QuickTime]     ModifyDate                      : 2020:01:01 00:02:03
[QuickTime]     DateTimeOriginal                : 2020:01:01 01:02:03+01:00
[QuickTime]     TrackCreateDate                 : 2020:01:01 00:02:03
[QuickTime]     TrackModifyDate                 : 2020:01:01 00:02:03
[QuickTime]     MediaCreateDate                 : 2020:01:01 00:02:03
[QuickTime]     MediaModifyDate                 : 2020:01:01 00:02:03
[QuickTime]     TrackCreateDate                 : 2020:01:01 00:02:03
[QuickTime]     TrackModifyDate                 : 2020:01:01 00:02:03
[QuickTime]     MediaCreateDate                 : 2020:01:01 00:02:03
[QuickTime]     MediaModifyDate                 : 2020:01:01 00:02:03
[XMP]           DateTimeOriginal                : 2020:01:01 01:02:03
[XMP]           CreateDate                      : 2020:01:01 01:02:03
[XMP]           ModifyDate                      : 2020:01:01 01:02:03



[Phil Harvey]

But the result is that the tag is set both for XMP and for Quicktime.

Yes.  This is because XMP was previously the preferred location for writing this, and it was kept as a preferred location for backward compatibility after the ability to create QuickTime tags was added.  This is mentioned in the QuickTime tags documentation:

When writing, ExifTool creates both QuickTime and XMP tags by default, but the group may be specified to write one or the other separately.

The problem is that QuickTime tags are supposed to be in UTC, so now there's a mismatch between what the two tags report.

actually, the documentation states:

According to the specification, many QuickTime date/time tags should be stored as UTC

(emphasis added)

I will clarify this in the documentation.  It applies only to date/time tags that are stored as integers.  String-valued tags are not affected.

The main problem here is that you should be using CreateDate and not DateTimeOriginal for the primary timestamp of a QuickTime file.  And when writing, you should specify the time zone and use the QuickTimeUTC option.  (Ideally, the QuickTimeUTC option should be the default, but it isn't because a majority of cameras originally writing QuickTime videos didn't store it as UTC).

This particular behavior is also a bit 'scary' since the QuickTime:DateTimeOriginal tag is hidden from output unless you specify -ee.

Actually, it is the -a option that allows duplicate tags to be extracted.  -ee implies the -a option.  And yes, sorry, this is a common source of confusion.

One way to fix this for already 'broken' files is :

Code Select Expand

❯  exiftool -d '%Y:%m:%d %H:%M:%S%z' '-QuickTime:DateTimeOriginal<XMP:DateTimeOriginal' foo.mp4 && exiftool -DateTimeOriginal -G -s -ee foo.mp4
    1 image files updated
[QuickTime]     DateTimeOriginal                : 2020:01:01 01:02:03+01:00
[XMP]           DateTimeOriginal                : 2020:01:01 01:02:03

Dunno if there's a better way?

This looks good, but it would have been better if the XMP date/time had a time zone to begin with.

- Phil

[torarnv]

Thank you Phil for the clarifications about the behavior of the XMP and QuickTime groups, and the -a option! 

The main problem here is that you should be using CreateDate and not DateTimeOriginal for the primary timestamp of a QuickTime file.

My process has been that I have a bunch of files (photos and videos) where the filename indicates the capture date, e.g. 20130516_160940.jpeg, so I've pulled the dates out of that using -AllDates<Filename. That writes both CreateDate and DateTimeOriginal I think? Should I be writing CreateDate exclusively, skipping DateTimeOriginal, even for photos?

I will clarify this in the documentation.  It applies only to date/time tags that are stored as integers.  String-valued tags are not affected.

if the QuickTime:DateTimeOriginal tag is a string, and the spec allows for the string to have a time zone, would it be a problem to write the string with time zone offset included if QuickTimeUTC is enabled?

And when writing, you should specify the time zone and use the QuickTimeUTC option.

The ability to pass in the date as local time (from the file name) has been very convenient, since I don't need to manually calculate whether that particular date was summer time or not (+01:00 or +02:00). But using QuickTimeUTC in combination with a -d format that included the UTC offset, and copying the date from a localtime date-tag allows me to let exiftool do the work, even if it's a bit of a workaround.

[torarnv]

To summarize, I'd like to write all typical date tags for any given file format (photo, video), giving as input a time in localtime without UTC offset, and have exiftool:

• Write the tag in localtime with UTC offset if the tag supports it
• Convert the localtime to UTC and write the tag in UTC if that's what the tag is specified as (e.g. Quicktime tags)
• Write the tag in localtime without UTC offset if the tag can't support offsets, and isn't specced to be in UTC

Is there an easy way to do that?

[Phil Harvey]

I've pulled the dates out of that using -AllDates<Filename.

The AllDates tag is designed for common EXIF date/time tags (CreateDate, ModifyDate and DateTimeOriginal), but MOV/MP4 doesn't support EXIF.  Having said this, it does sort of work for videos as well, but you may want to target specific video tags.  You should take a look at what tags already exist in your videos and write the ones you think make sense:

exiftool -a -G1 -time:all VIDEOFILE

if the QuickTime:DateTimeOriginal tag is a string, and the spec allows for the string to have a time zone, would it be a problem to write the string with time zone offset included if QuickTimeUTC is enabled?

When using the QuickTimeUTC option ExifTool converts from the specified time zone to UTC when writing.  If no time zone is specified, then the local time zone is assumed.

To summarize, I'd like to write all typical date tags for any given file format (photo, video), giving as input a time in localtime without UTC offset

.

This will work for EXIF and QuickTime integer-format tags if the QuickTimeUTC option is used.  For string-format tags (some QuickTime tags, plus XMP tags), you should specify time zone.

• Write the tag in localtime with UTC offset if the tag supports it
• Convert the localtime to UTC and write the tag in UTC if that's what the tag is specified as (e.g. Quicktime tags)
• Write the tag in localtime without UTC offset if the tag can't support offsets, and isn't specced to be in UTC

This can be done in one step:  I think that writing all tags in local time with time zone and using QuickTimeUTC should work for everything.  If a string tag doesn't support a time zone (ie EXIF), then it will be stripped when writing.

- Phil

[torarnv]

When using the QuickTimeUTC option ExifTool converts from the specified time zone to UTC when writing.  If no time zone is specified, then the local time zone is assumed.

But this is not the case for QuickTime:DateTimeOriginal right? You said earlier (and also what I saw during testing) that:

I will clarify this in the documentation.  It applies only to date/time tags that are stored as integers.  String-valued tags are not affected.

I'm suggesting it should be the case. Is there any reason not to? You could store the string as UTC, with Z or +0000 to signify that it has been converted to UTC, as long as the spec doesn't mandate a hard-coded time zone for that spec, but allows time zone offset in the string.

This can be done in one step:  I think that writing all tags in local time with time zone and using QuickTimeUTC should work for everything.  If a string tag doesn't support a time zone (ie EXIF), then it will be stripped when writing.

They key here was that I want the input to be local time without time zone (UTC offset), since passing the offset requires me to manually calculate the correct UTC offset for a given date based on the daylight saving e.g.

I would rather want to pass inn local time, without time zone offset, and have exiftool calculate the right offset, and store that correctly in the tag if possible, or convert to UTC if that's what the tag expects. Without the QuickTimeUTC behavior for QuickTime:DateTimeOriginal that doesn't seem possible today, since the local time is directly written to the tag, without any UTC conversion or UTC offset incorporated.

[Phil Harvey]

When using the QuickTimeUTC option ExifTool converts from the specified time zone to UTC when writing.  If no time zone is specified, then the local time zone is assumed.

But this is not the case for QuickTime:DateTimeOriginal right?

...when writing integer-format QuickTime tags.

For string-format QuickTime tags you can write whatever you want, but including a time zone specification makes sense.

I would rather want to pass inn local time, without time zone offset, and have exiftool calculate the right offset, and store that correctly in the tag if possible,

This will work for integer-format QuickTime tags.  But then you won't have a time zone for your string-format tags.

- Phil

[StarGeek]

I would rather want to pass inn local time, without time zone offset, and have exiftool calculate the right offset, and store that correctly in the tag if possible, or convert to UTC if that's what the tag expects.

You could write to Quicktime:CreateDate with the -api QuickTimeUTC option, then on a separate command, use -AllDates<Quicktime:CreateDate -api QuickTimeUTC.

But this only will work as long as the computer time is set to the same time zone as the picture was taking (not counting Daylight Savings).  If the time needs to be set to a different time zone, then you're going to have to figure out the time zone anyway.

[torarnv]

You could write to Quicktime:CreateDate with the -api QuickTimeUTC option, then on a separate command, use -AllDates<Quicktime:CreateDate -api QuickTimeUTC.

But this only will work as long as the computer time is set to the same time zone as the picture was taking (not counting Daylight Savings).  If the time needs to be set to a different time zone, then you're going to have to figure out the time zone anyway.

Right, that's probably the best workaround. As you say it will fail for photos I shot in a different time zone then the one I'm processing in (say when processing vacation photos back home), but for most of them it's fine.

[torarnv]

For string-format QuickTime tags you can write whatever you want, but including a time zone specification makes sense.

Hmm, so just to clarify, the fundamental issue is that the QuickTime:DateTimeOriginal tag is stored as a string, so when the user runs -QuickTime:DateTimeOriginal=2020:01:01 01:02:03, you don't want to do any tweaking of the input (even when QuickTimeUTC is enabled), in risk of misrepresenting what the user asked for?

While for integer tags, -QuickTime:CreateDate=2020:01:01 01:02:03, you interpret the lack of a UTC offset as the date being in local time, apply the time zone (and the corresponding UTC offset for that date), and write the resulting tag shifted to UTC?

Does the same apply when doing -QuickTime:DateTimeOriginal<SomeDateTagThatDoesntEncodeUTCOffsetOrIsExplcitlyInUTC?

I guess from my perspective I don't really care what format the tag is in -- integer or string -- I just want the result to be as close a representation to the date I give. Perhaps it would be useful with an option to always write string dates with an UTC offset? That would leave those strings well-formed in terms of what exact date and time we're talking about, while at the same time letting exiftool do the heaving lifting of figuring out what the offset should be, the same way it does for integer UTC-defined tags.

Cheers, and thanks for all your help! 

[Hayo Baan]

There is no QuickTime:DateTimeOriginal tag, I guess you mean QuickTime:CreateDate instead?

The fact the CreateDate is stored as string does not influence the behaviour of exiftool, actually. The -api QuickTimeUTC flag, however still does; with the QuickTimeUTC option enabled, the input time is calculated back to UTC and then stored as the tag value (string). With it disabled, it is stored exactly as it was specified (discarding any timezone info if it was specified). Quite a big difference!

Note: since the QuickTime:CreateDate is actually supposed to be stored as UTC, I have permanently enabled the api option in my .ExifTool_config file. I do have cameras that write it as local time but those I correct to UTC almost immediately after ingest (I have written a program to do all sorts of metadata checks, and one of the things it does it correct is the Quicktime:CreateDate for those files).

Cheers,
Hayo

[StarGeek]

There is no QuickTime:DateTimeOriginal tag, I guess you mean QuickTime:CreateDate instead?l

*cough*first entry*cough*also*cough*

Sorry, anyone have a cough drop? 

The only reason I knew that it existed is because some program I've used recently set that tag.  I still can't figure out which one, though, and it's bugging the heck out of me.

[Phil Harvey]

the fundamental issue is that the QuickTime:DateTimeOriginal tag is stored as a string, so when the user runs -QuickTime:DateTimeOriginal=2020:01:01 01:02:03, you don't want to do any tweaking of the input (even when QuickTimeUTC is enabled), in risk of misrepresenting what the user asked for?

The string is tweaked to be sure it is written in a valid date/time format.  But it is valid to write a date/time without a time zone.

While for integer tags, -QuickTime:CreateDate=2020:01:01 01:02:03, you interpret the lack of a UTC offset as the date being in local time, apply the time zone (and the corresponding UTC offset for that date), and write the resulting tag shifted to UTC?

Yes.  And shifted back again to local time when reading

Does the same apply when doing -QuickTime:DateTimeOriginal<SomeDateTagThatDoesntEncodeUTCOffsetOrIsExplcitlyInUTC?

At this level, the value of the source tag is a string.  It doesn't matter where it comes from.

I guess from my perspective I don't really care what format the tag is in -- integer or string -- I just want the result to be as close a representation to the date I give. Perhaps it would be useful with an option to always write string dates with an UTC offset?

This is a good idea.

- Phil

[Phil Harvey]

BTW, there was a report here that using QuickTime:DateTimeOriginal may confuse Apple Photos, although I couldn't reproduce this myself.

- Phil

[torarnv]

The only reason I knew that it existed is because some program I've used recently set that tag.  I still can't figure out which one, though, and it's bugging the heck out of me.

That could have been exiftool itself? That's what happened to me at least, as I was setting AllDates which will write both XMP:DateTimeOriginal and QuickTime:DateTimeOriginal.

[torarnv]

Does the same apply when doing -QuickTime:DateTimeOriginal<SomeDateTagThatDoesntEncodeUTCOffsetOrIsExplcitlyInUTC?

At this level, the value of the source tag is a string.  It doesn't matter where it comes from.

Ah, I see. Does it go via the -d format in that case? So with the default format that doesn't include a time zone offset it would lose the (possible) UTC offset in the original tag?

Perhaps it would be useful with an option to always write string dates with an UTC offset?

This is a good idea.

Hurray

Another thing that crossed my mind was that the geolocation info could potentially be used to decide which timezone input dates without timezone should be interpreted as, instead of always assuming the local timezone (as I understand it does now?).

Cheers,
Tor Arne

[Phil Harvey]

Hi Tor,

Ah, I see. Does it go via the -d format in that case?

Yes.

So with the default format that doesn't include a time zone offset it would lose the (possible) UTC offset in the original tag?

Again, I think you are confused.  If the date/time tag has a time zone then the "default format" will include that time zone.

Perhaps it would be useful with an option to always write string dates with an UTC offset?

This is a good idea.

Hurray

I think we miscommunicated again here.  I meant that it is a good idea to always write the time zone.  But ExifTool is not going to do that for you.  That's something you need to do yourself.

Another thing that crossed my mind was that the geolocation info could potentially be used to decide which timezone input dates without timezone should be interpreted as, instead of always assuming the local timezone (as I understand it does now?).

There is a config file include in the full distribution that attempts to determine the time zone of an image like this.

- Phil

[StarGeek]

That could have been exiftool itself? That's what happened to me at least, as I was setting AllDates which will write both XMP:DateTimeOriginal and QuickTime:DateTimeOriginal.

No, I don't use AllDates with video files.  I rarely use it at all, since I have my own shortcuts which are more targeted and explicit as to which tags they set.

It was a program more video specific.  I thought it might have been Youtube-DL, but that sets ContentCreateDate, which shows just the date without formatting, like 20190516.  Unfortunately, it doesn't appear to set any other time stamps while using the --add-metadata switch.