Missing CreationDate tag from video files

[moondog]

I made a script to rename photos and videos to the capture date and time (Ex: 2016-03-30 [PM] 01.32.36.jpg).
For the photos, I'm using the 'DateTimeOriginal' tag, and that seems to be working reliably. However, for the videos, the 'CreationDate' tag was accurate but not always present. In such cases, every other date tag available was way off, sometimes by a day or two, except for the 'CreateDate' but that one is missing the time portion which I need.

Below are some examples of a video and photo pair that was taken around the same time as the video (within the hour). I'm using the following command to list all available dates: exiftool -a -G1 -s -*date* <file>

VIDEO SAMPLE 1:
[System]        FileModifyDate                  : 2016:12:24 12:13:37-05:00
[System]        FileAccessDate                  : 2016:12:24 12:13:37-05:00
[System]        FileInodeChangeDate             : 2016:12:24 12:13:37-05:00
[QuickTime]     CreateDate                      : 2016:03:25 00:35:39
[QuickTime]     ModifyDate                      : 2016:03:25 00:35:39
[Track1]        TrackCreateDate                 : 0000:00:00 00:00:00
[Track1]        TrackModifyDate                 : 2016:03:25 00:35:40
[Track1]        MediaCreateDate                 : 0000:00:00 00:00:00
[Track1]        MediaModifyDate                 : 2016:03:25 00:35:40
[Track2]        TrackCreateDate                 : 2016:03:25 00:35:40
[Track2]        TrackModifyDate                 : 2016:03:25 00:35:40
[Track2]        MediaCreateDate                 : 2016:03:25 00:35:40
[Track2]        MediaModifyDate                 : 2016:03:25 00:35:40
[XMP-xmp]       CreateDate                      : 2016:03:23

PHOTO SAMPLE 1:
[System]        FileModifyDate                  : 2016:10:29 05:18:34-04:00
[System]        FileAccessDate                  : 2016:10:29 05:18:34-04:00
[System]        FileInodeChangeDate             : 2016:10:29 05:18:34-04:00
[IFD0]          ModifyDate                      : 2016:03:23 20:40:44
[ExifIFD]       DateTimeOriginal                : 2016:03:23 20:40:44
[ExifIFD]       CreateDate                      : 2016:03:23 20:40:44
[GPS]           GPSDateStamp                    : 2016:03:24
[Composite]     SubSecCreateDate                : 2016:03:23 20:40:44.777
[Composite]     SubSecDateTimeOriginal          : 2016:03:23 20:40:44.777
[Composite]     GPSDateTime                     : 2016:03:24 00:40:41Z

VIDEO SAMPLE 2:
[System]        FileModifyDate                  : 2016:12:24 12:01:52-05:00
[System]        FileAccessDate                  : 2016:12:24 12:01:52-05:00
[System]        FileInodeChangeDate             : 2016:12:24 12:01:52-05:00
[QuickTime]     CreateDate                      : 2015:12:06 15:12:21
[QuickTime]     ModifyDate                      : 2015:12:06 15:12:21
[Track1]        TrackCreateDate                 : 0000:00:00 00:00:00
[Track1]        TrackModifyDate                 : 2015:12:06 15:12:21
[Track1]        MediaCreateDate                 : 0000:00:00 00:00:00
[Track1]        MediaModifyDate                 : 2015:12:06 15:12:21
[Track2]        TrackCreateDate                 : 2015:12:06 15:12:21
[Track2]        TrackModifyDate                 : 2015:12:06 15:12:21
[Track2]        MediaCreateDate                 : 2015:12:06 15:12:21
[Track2]        MediaModifyDate                 : 2015:12:06 15:12:21
[XMP-xmp]       CreateDate                      : 2015:12:05

PHOTO SAMPLE 2:
[System]        FileModifyDate                  : 2016:12:12 14:16:54-05:00
[System]        FileAccessDate                  : 2016:12:12 14:16:54-05:00
[System]        FileInodeChangeDate             : 2016:12:12 14:16:54-05:00
[IFD0]          ModifyDate                      : 2015:12:05 14:18:15
[ExifIFD]       DateTimeOriginal                : 2015:12:05 14:18:15
[ExifIFD]       CreateDate                      : 2015:12:05 14:18:15
[GPS]           GPSDateStamp                    : 2015:12:05
[Composite]     SubSecCreateDate                : 2015:12:05 14:18:15.202
[Composite]     SubSecDateTimeOriginal          : 2015:12:05 14:18:15.202
[Composite]     GPSDateTime                     : 2015:12:05 19:18:11.79Z

VIDEO SAMPLE 3:
[System]        FileModifyDate                  : 2016:12:24 12:04:08-05:00
[System]        FileAccessDate                  : 2016:12:24 12:04:08-05:00
[System]        FileInodeChangeDate             : 2016:12:24 12:04:08-05:00
[QuickTime]     CreateDate                      : 2016:03:27 19:06:05
[QuickTime]     ModifyDate                      : 2016:03:27 19:06:05
[Track1]        TrackCreateDate                 : 0000:00:00 00:00:00
[Track1]        TrackModifyDate                 : 2016:03:27 19:06:06
[Track1]        MediaCreateDate                 : 0000:00:00 00:00:00
[Track1]        MediaModifyDate                 : 2016:03:27 19:06:06
[Track2]        TrackCreateDate                 : 2016:03:27 19:06:06
[Track2]        TrackModifyDate                 : 2016:03:27 19:06:06
[Track2]        MediaCreateDate                 : 2016:03:27 19:06:06
[Track2]        MediaModifyDate                 : 2016:03:27 19:06:06
[XMP-xmp]       CreateDate                      : 2016:03:26

PHOTO SAMPLE 3:
[System]        FileModifyDate                  : 2016:10:29 05:18:34-04:00
[System]        FileAccessDate                  : 2016:10:29 05:18:34-04:00
[System]        FileInodeChangeDate             : 2016:10:29 05:18:34-04:00
[IFD0]          ModifyDate                      : 2016:03:26 13:19:52
[ExifIFD]       DateTimeOriginal                : 2016:03:26 13:19:52
[ExifIFD]       CreateDate                      : 2016:03:26 13:19:52
[GPS]           GPSDateStamp                    : 2016:03:26
[Composite]     SubSecCreateDate                : 2016:03:26 13:19:52.706
[Composite]     SubSecDateTimeOriginal          : 2016:03:26 13:19:52.706
[Composite]     GPSDateTime                     : 2016:03:26 17:19:52Z


What am I missing here? Is there another way to figure out the video capture time?

[StarGeek]

The command I suggest to view all the time related tags in a file would be
exiftool -time:all -G1 -a -s /path/to/files/
But bonus points to you for figuring out that you can use wildcards to list tags

Time data in video files is complicated.  The CreateDate/ModifyDate and the Track related timestamps are all supposed to be in UTC (see forth paragraph, staring "According to the specification", on the Quicktime tags page).  These timestamps are the Integer Based ones mentioned in that link. Programs that read that data are supposed to then display the correct time adjusted by the timezone of the computer reading it.   Older cameras did not always set these timestamps correctly, as they were not always aware of timezones.

Video files can also hold other time related tags, such as CreationDate and DateTimeOriginal, but these are supposed to be set to local time, just as with an image file. These tags are String Based.  It is advisable to include the timezone with this, as some Apple programs will display wildly incorrect times if there isn't a timezone included.

So, for a video taken at 2016:03:23 20:40:44, EDT, the timestamps should be

Code Select Expand

C:\>exiftool -time:all --system:all -G1 -a -s "Y:\!temp\Test1.mp4"
[QuickTime]     CreateDate                      : 2016:03:24 00:40:44
[QuickTime]     ModifyDate                      : 2016:03:24 00:40:44
[Track1]        TrackCreateDate                 : 2016:03:24 00:40:44
[Track1]        TrackModifyDate                 : 2016:03:24 00:40:44
[Track1]        MediaCreateDate                 : 2016:03:24 00:40:44
[Track1]        MediaModifyDate                 : 2016:03:24 00:40:44
[Track2]        TrackCreateDate                 : 2016:03:24 00:40:44
[Track2]        TrackModifyDate                 : 2016:03:24 00:40:44
[Track2]        MediaCreateDate                 : 2016:03:24 00:40:44
[Track2]        MediaModifyDate                 : 2016:03:24 00:40:44
[UserData]      DateTimeOriginal                : 2016:03:23 20:40:44-04:00
[Keys]          CreationDate                    : 2016:03:23 20:40:44-04:00

As you can see, the the Integer Based ones are "off" by the timezone.  This is how it should appear.

Looking at your samples, video one is off by the 4 hours timezone difference from the image.  So that one appears to be correctly set.

Video 2 is off from the image by 25 hours and Video 3 is of by about 30 hours, so something appears to be wrong there.

If you want to see the timestamps adjusted to local time, you can add the -api QuickTimeUTC option to any command.  It will adjust the Integer Based video tags to the local time for that computer and will not affect other time based tags.

The same file with the -api QuicktimeUTC added will display the following.  Note that in my case, the timezone is -07:00, as I'm on the West coast.  Adding 3 hours shows that it lines up with the bottom two tags, which are the string based ones.

Code Select Expand

C:\>exiftool -api QuickTimeUTC -time:all --system:all -G1 -a -s "Y:\!temp\Test1.mp4"
[QuickTime]     CreateDate                      : 2016:03:23 17:40:44-07:00
[QuickTime]     ModifyDate                      : 2016:03:23 17:40:44-07:00
[Track1]        TrackCreateDate                 : 2016:03:23 17:40:44-07:00
[Track1]        TrackModifyDate                 : 2016:03:23 17:40:44-07:00
[Track1]        MediaCreateDate                 : 2016:03:23 17:40:44-07:00
[Track1]        MediaModifyDate                 : 2016:03:23 17:40:44-07:00
[Track2]        TrackCreateDate                 : 2016:03:23 17:40:44-07:00
[Track2]        TrackModifyDate                 : 2016:03:23 17:40:44-07:00
[Track2]        MediaCreateDate                 : 2016:03:23 17:40:44-07:00
[Track2]        MediaModifyDate                 : 2016:03:23 17:40:44-07:00
[UserData]      DateTimeOriginal                : 2016:03:23 20:40:44-04:00
[Keys]          CreationDate                    : 2016:03:23 20:40:44-04:00

[moondog]

Thanks, @StarGeek for the detailed explanation and the "time:all" tip, will use it going forward.
I adjusted the time as recommended using the following command on my last video example (#3)

~ % exiftool -time:all -G1 -a -s -api QuicktimeUTC /Volumes/Prometheus/Camera\ Roll/ZZ\ Not\ date\ tag\ REVIEW/IMG_5628.MOV
[System]        FileModifyDate                  : 2016:12:24 12:04:08-05:00
[System]        FileAccessDate                  : 2016:12:24 12:04:08-05:00
[System]        FileInodeChangeDate             : 2016:12:24 12:04:08-05:00
[QuickTime]     CreateDate                      : 2016:03:27 15:06:05-04:00
[QuickTime]     ModifyDate                      : 2016:03:27 15:06:05-04:00
[Track1]        TrackCreateDate                 : 1903:12:31 19:00:00-05:00
[Track1]        TrackModifyDate                 : 2016:03:27 15:06:06-04:00
[Track1]        MediaCreateDate                 : 1903:12:31 19:00:00-05:00
[Track1]        MediaModifyDate                 : 2016:03:27 15:06:06-04:00
[Track2]        TrackCreateDate                 : 2016:03:27 15:06:06-04:00
[Track2]        TrackModifyDate                 : 2016:03:27 15:06:06-04:00
[Track2]        MediaCreateDate                 : 2016:03:27 15:06:06-04:00
[Track2]        MediaModifyDate                 : 2016:03:27 15:06:06-04:00
[XMP-xmp]       CreateDate                      : 2016:03:26


But still, it is not getting me close to the correct timestamp "2016:03:26 13:19:52"
I'm on a Mac and the clock settings show the correct time zone (-5) for the East coast.

Diggin more into this matter, I found that all the "problematic" videos have this data in them:
Handler Description             : IsoMedia File Produced by Google, 5-11-2011

Which led me to believe that these are Google Photos recompressed videos. I later confirmed it by downloading a recent video from my Google Photos library.
They are obviously not migrating all the metadata after compression This is not the case for the photos though, Google-recompressed-photos do have the "Date/Time Original" tag.

So far it looks like I might have to use the 'CreateDate' tag for the date and then guess/add the time manually for 1000+ video files...?   

[StarGeek]

But still, it is not getting me close to the correct timestamp "2016:03:26 13:19:52"
I'm on a Mac and the clock settings show the correct time zone (-5) for the East coast.

Daylight savings started March 13 in 2016, so March 26 should be -04:00

Diggin more into this matter, I found that all the "problematic" videos have this data in them:
Handler Description             : IsoMedia File Produced by Google, 5-11-2011

Which led me to believe that these are Google Photos recompressed videos. I later confirmed it by downloading a recent video from my Google Photos library.

Ahh, good to know.

So far it looks like I might have to use the 'CreateDate' tag for the date and then guess/add the time manually for 1000+ video files...?   

I can't think of any easier way if they're all off by an unknown time.  If this was from Google Takeout, you might check the json files that accompany the vids.  When you run exiftool on one of the json files, you'll see something like

Code Select Expand

---- JSON ----
CreationTimeFormatted           : Mar 24, 2020, 12:07:28 AM UTC
CreationTimeTimestamp           : 1585008448
Description                     : I have changed the description.Pray I don't change it further.
Favorited                       : true
GeoDataAltitude                 : 0.0
GeoDataLatitude                 : 36.1225498
GeoDataLatitudeSpan             : 0.001349000000003341
GeoDataLongitude                : -115.1660839
GeoDataLongitudeSpan            : 0.0013489500000044155
GeoDataExifAltitude             : 0.0
GeoDataExifLatitude             : 0.0
GeoDataExifLatitudeSpan         : 0.0
GeoDataExifLongitude            : 0.0
GeoDataExifLongitudeSpan        : 0.0
ImageViews                      : 0
ModificationTimeFormatted       : Nov 6, 2020, 8:41:08 AM UTC
ModificationTimeTimestamp       : 1604652068
PeopleName                      : Jane Smith
PhotoTakenTimeFormatted         : Jan 9, 2011, 9:19:26 PM UTC
PhotoTakenTimeTimestamp         : 1294607966
Title                           : 2011-01-09_13.19.26.Jpg

If one of the "formatted" timestamps is correct, then you would pick the corresponding "Timestamp" entry and do something like
exiftool -d "%s" -tagsfromfile %d/%F.json '-CreateDate<PhotoTakenTimeTimestamp' /path/to/files/

[moondog]

Unfortunately, I don't have the JSONs for that batch of videos but your idea does cast some light onto the problem. It involves requesting take-outs for two separate Google Photos accounts, discarding the downloaded photos, and deduping a massive amount of videos of which I already have the originals for in my server.

A bit of scripting and lots of download/compute time but should solve the issue of having to perform the tedious manual work.

Thanks for the help! I'll post back in about... 3 months...?? 

[StarGeek]

One helpful hint if exiftool fits into any of the scripts.  Try to use exiftool's batch ability and not use loops calling exiftool once for each file. See Common Mistake #3.  Exiftool's biggest performance hit is the startup time and looping can take a very long time.

If you can put all the files paths that need the same command into a text file, you can use the -@ (Argfile) option to past that list to exiftool.  Simple example, this will process all the files listed in "temp.txt" and copy the DateTimeOriginal to the FileModifyDate.  The temp file can have thousands of files listed.
exiftool "-FileModifyDate<DateTimeOriginal" -@ temp.txt

[moondog]

So I started running tests on some of the first files downloaded from the takeout. I encounter an odd issue.

On my Mac laptop running MacOS Big Sur this command works as expected:
exiftool -d "%s" -tagsfromfile %d/%F.json '-CreateDate<PhotoTakenTimeTimestamp' /path/to/files/

On my Mac mini server running macOS Mojave I get the following error for the same video and json file combo:
Error: Unknown file type - /Volumes/Prometheus/TEST/IMG_1319.MOV.json

Any idea why would that be?

Both machines have exiftool stable 12.26

[StarGeek]

Open up IMG_1319.MOV.json in a text editor and take a look. 

The error is saying that exiftool can't figure out what kind of file it is.  It may be that the file isn't a json file, has been corrupted, or has a permissions error. 

[moondog]

The json file opens fine on both computers using Visual Studio Code and also validates correctly as json. I even changed permission on the files and parent folder to 777  The same file works on one machine but not the other, really puzzling. I think my mini server is due for a fresh install.

Parsing the json file via script and passing along the timestamp works so I might have to settle for a loop through files. Sample command:
exiftool -d '%s' '-CreateDate=1550440432' <movie file>

An update on the takeout: Google provided links for 461 zip files of 2GB each. On a 30Mbps down connection, it takes about 10 min per file. It also prompts for re-authentication every couple of files so scripting this process might not be a trivial thing.

 

[StarGeek]

Parsing the json file via script and passing along the timestamp works so I might have to settle for a loop through files. Sample command:
exiftool -d '%s' '-CreateDate=1550440432' <movie file>

Unfortunately, I have no clue when it comes to Macs.

If you have a lot of files, things will be very slow if you loop.  If you can, I'd suggest extracting the data into an text file (using sed or similar programs) which could then be called with the -@ (Argfile) option.  The text file would have each parameter on individual lines (no quotes or extra spaces).  Options common to all commands would be passed with the -Common_Args option

Code Select Expand

-CreateDate=1550440432
file1.mp4
-execute
-CreateDate=1550442462
file2.mp4
-execute
-CreateDate=15504956712
file3.mp4
-execute

And then you just use
exiftool -@ tempfile.txt -Common_Args -d %s

[Phil Harvey]

On my Mac mini server running macOS Mojave I get the following error for the same video and json file combo:

Can you attach the JSON file that you used on the mini so I can take a look at it?

But the likely problem is that you are not running the version of ExifTool that you think you are.

- Phil