Changing the original date of an mp4 file to a date between 1904 and 1970

Started by harrisos, December 18, 2016, 09:34:17 PM

Previous topic - Next topic

harrisos

Hi Phil

I'm been successfully running the latest version of your great ExifTool on macOS 10.12.1 (Sierra) to change the original dates of some .mp4 video files using the following command:

exiftool -AllDates="1955:07:01 12:01:01" ~/Pictures/etc

I have then added these files to Dropbox and the files are listed under the correct original date in the Photos/Timeline.

But if I try to do this using a date before 1970, ExifTool reports this [minor] warning: "Patched incorrect time zero for QuickTime date/time tag" and the file is not recognized correctly by Dropbox.

I've read your response to the topic "Error with date underflow" dated December 10th last year which seems to address this issue but am confused.

Regardless of ExifTool's capabilities, is it actually possible for an .mp4 file to have an original date before 1970?
If so, can ExifTool make such a change?
If so, can you advise me how to modify the above command to achieve this?

Many thanks

Simon

Hayo Baan

The problem lies in the specification of the QuickTime date fields. These are 32 bit integers measuring the time in seconds since 1904, however many brain dead programs (as Phil rightly calls them) used the "unix" standard of 1970 as starting point. To cater for this (common) error, Phil decided to assume 1970 as starting point for dates before 1970 (when counting from 1904). Effectively this means exiftool, by default, does not show you the correct date for dates < 1970. On a positive note, when writing dates, the correct value (counting from 1904) is stored! So if Dropbox doesn't show the correct date, it's probably Dropbox fault...

The work-around is to use the -api QuickTimeUTC option, but to be honest this may have unwanted side effects as all dates are then assumed UTC, which again is actually the standard but many cameras don't observe it (for instance my Nikons don't, but an iPhone does).

@Phil I made a couple of observations while checking this.
1. I don't like the abuse of the QuickTimeUTC option as that gives unwanted side effects, better have a separate option e.g. QuickTimeZeroDate1904 or whatever.
2. How common is the issue anyway? Perhaps the default behaviour of ExifTool should be to honour the standard of 1904 and have an option change that to 1970 (e.g. QuickTimeZeroDate1970). Hmm, perhaps the same should hold true for QuickTimeUTC, though there the non-conformity seems much larger.
3. Currently no warning is given when you specify a date before 1904 (nor for dates > 2040), I think you should.
Hayo Baan – Photography
Web: www.hayobaan.nl

harrisos

Hi Hays

Many thanks for your quick reply. I'll give the QuicktimeUTC api a go and see what happens. The dates I want to use are around 1955 plus or minus several months so a small UTC error is insignificant for me.

For what its worth, it seems to me that ExifTool should really work for people and systems that are correct and that common errors and mistakes should be provided for by exception. That's how I've built systems in the past.

Many thanks

Simon

Phil Harvey

Hi Hayo,

I've finally found a bit of time to think about your question/comments:

Quote from: Hayo Baan on December 19, 2016, 06:30:53 AM
1. I don't like the abuse of the QuickTimeUTC option as that gives unwanted side effects, better have a separate option e.g. QuickTimeZeroDate1904 or whatever.

I don't really see this as an abuse.  The QuickTimeUTC option enforces the standard QuickTime definition for date/time values (seconds from Jan 1 1904 UTC).  Perhaps the option name could be better (StandardQuickTimeDates?), but I don't think it is worthwhile changing this.  Also, I resist adding a new option like QuickTime1904 for two reasons: 1. It is yet another option that most people won't use, and 2. For backward compatibilty I don't want to change the behaviour of the QuickTimeUTC option.

Quote2. How common is the issue anyway? Perhaps the default behaviour of ExifTool should be to honour the standard of 1904 and have an option change that to 1970 (e.g. QuickTimeZeroDate1970). Hmm, perhaps the same should hold true for QuickTimeUTC, though there the non-conformity seems much larger.

I think that using the wrong time zero is more common than dates that are actually before 1970.  I think the ExifTool's current behaviour is correct for the most common case.

Quote3. Currently no warning is given when you specify a date before 1904 (nor for dates > 2040), I think you should.

Right.  I wasn't range-checking the values written to QuickTime.  I will fix this for the next release.  Thanks.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Quote from: Phil Harvey on December 20, 2016, 09:37:26 AM
Hi Hayo,

I've finally found a bit of time to think about your question/comments:

Quote from: Hayo Baan on December 19, 2016, 06:30:53 AM
1. I don't like the abuse of the QuickTimeUTC option as that gives unwanted side effects, better have a separate option e.g. QuickTimeZeroDate1904 or whatever.

I don't really see this as an abuse.  The QuickTimeUTC option enforces the standard QuickTime definition for date/time values (seconds from Jan 1 1904 UTC).  Perhaps the option name could be better (StandardQuickTimeDates?), but I don't think it is worthwhile changing this.  Also, I resist adding a new option like QuickTime1904 for two reasons: 1. It is yet another option that most people won't use, and 2. For backward compatibilty I don't want to change the behaviour of the QuickTimeUTC option.

OK, I can see the reasoning, and I think you're right about QuickTimeUTC having the (implicit) meaning of "following the standard". So yes, you should keep it as-is :D

Quote from: Phil Harvey on December 20, 2016, 09:37:26 AM
Quote2. How common is the issue anyway? Perhaps the default behaviour of ExifTool should be to honour the standard of 1904 and have an option change that to 1970 (e.g. QuickTimeZeroDate1970). Hmm, perhaps the same should hold true for QuickTimeUTC, though there the non-conformity seems much larger.

I think that using the wrong time zero is more common than dates that are actually before 1970.  I think the ExifTool's current behaviour is correct for the most common case.

OK, together with above item, we can conclude that current behaviour "is best for most people/situations" so should also be the default.

So if one wants to be able to properly use QuickTime dates before 1970, he/she should use the -QuickTimeUTC option, both when reading and when writing (and also provide a timezone if not the local time).

Quote from: Phil Harvey on December 20, 2016, 09:37:26 AM
Quote3. Currently no warning is given when you specify a date before 1904 (nor for dates > 2040), I think you should.

Right.  I wasn't range-checking the values written to QuickTime.  I will fix this for the next release.  Thanks.

Excellent!

Thinking about the whole debacle where (contrary to spec) cameras do not record the time in UTC, it would be nice if it where possible (at least for my own perl code using the exiftool library) to be able to have the QuickTimeUTC flag dependent on the make of the camera (so I would enable it automatically for e.g. video from an iPhone and disable it for videos from Nikon cameras). I guess, though, that when you call ExtractInfo, the options are baked in? Or is it when you call GetInfo (then I could perform the automatic switch there :) )?
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

Quote from: Hayo Baan on December 20, 2016, 11:05:56 AM
it would be nice if it where possible (at least for my own perl code using the exiftool library) to be able to have the QuickTimeUTC flag dependent on the make of the camera (so I would enable it automatically for e.g. video from an iPhone and disable it for videos from Nikon cameras). I guess, though, that when you call ExtractInfo, the options are baked in? Or is it when you call GetInfo (then I could perform the automatic switch there :) )?

Different options apply at different times.  I try to list which options apply where in the API documentation, but QuickTimeUTC has been neglected.  I will add it -- it applies to ExtractInfo.

- Phil

Edit: Interestingly, my above answer is not entirely correct:  The patch for the 1970 difference is at ExtractInto time, but the adjustment for UTC is at GetInfo time.  I have updated the API documentation accordingly.
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Excellent! This (at least partially) enables me to make an automated switch based on the camera make. I reckon the same applies to GetValue (though QuickTimeUTC isn't listed there)?

Hmm, I just tried it with a simple  exiftool -p '${CreateDate; $_=$self->GetValue("CreateDate (1)", { QuickTimeUTC => 1 });}' test.mov but I still get the non-UTC version so I reckon the option is not used? (note: CreateDate (1) is used as that is the QuickTime CreateDate tag, the other is the MakerNote version).
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

Quote from: Hayo Baan on December 20, 2016, 01:03:23 PM
Excellent! This (at least partially) enables me to make an automated switch based on the camera make. I reckon the same applies to GetValue (though QuickTimeUTC isn't listed there)?

Right.  Added.

QuoteHmm, I just tried it with a simple  exiftool -p '${CreateDate; $_=$self->GetValue("CreateDate (1)", { QuickTimeUTC => 1 });}' test.mov but I still get the non-UTC version so I reckon the option is not used? (note: CreateDate (1) is used as that is the QuickTime CreateDate tag, the other is the MakerNote version).

There are actually two problems here:

1. GetValue doesn't accept an options hash as a parameter (like GetInfo does).

2. The option only applies on the first call to GetValue (because subsequent calls will use the cached value) -- I know this isn't documented.  And unfortunately GetValue is called implicitly by the exiftool application for all tags before the -p expression is evaluated, so it will be too late if you change the option in the advanced formatting expression. 

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

OK, all clear! I already figured GetValue didn't take options, but setting the options using $self->Options didn't work either, I now know that's because of value caching. Not a big issue; I solely make use of GetInfo in my perl program.
Hayo Baan – Photography
Web: www.hayobaan.nl