Date-stamping Movies in Apple Photos

[Alan Clifford]

I don't know how Phil retains his sanity dealing with this stuff.   Googling around I found https://developer.apple.com/library/content/documentation/QuickTime/QTFF/Metadata/Metadata.html which has this sentence, "Similarly if the metadata writer does not want to limit the user's language (this is not recommended) it uses the value 'und' (undetermined) from the ISO 639-2/T specification."

So the tag appears to be for Americans who speak any language and it is not a recommended use.  Just curious, did "Photos" create that tag?

[Hayo Baan]

Interesting. I'm on Mac too (Photos is a Mac only application so I figured you'd be on Mac too ). Yesterday when I tested things for you, I did everything on a 10.11 system, I'm now repeating some of the tests on a 10.12 system and the results are slightly different.

Note: don't use stat -x as it will NOT give you the birth time. My guess is that to keep things compatible with older versions, the output does not show the birth date (birth date is a relative new thing and not all file systems support it). Use either stat without options or e.g. stat -f "%SB %N" to get it.

On 10.12 it looks as if Photos behaves different and seems to be using the (und-US) date indeed. Over the weekend I will have a look at tweaking my matchdatetime script to cater for adjusting this tag.

Note: I played with Photos a little bit (man, what a piece of junk...) and I found out you can actually change the date of a movie. The bad news, however, is that that change is not reflected in the file, only in the Photos database . Not really useful, I think.

[reggoboy]

Interesting. I'm on Mac too (Photos is a Mac only application so I figured you'd be on Mac too ). Yesterday when I tested things for you, I did everything on a 10.11 system, I'm now repeating some of the tests on a 10.12 system and the results are slightly different.

Yea, I figured at least your Photos use was on the Mac, but I thought maybe you were "Unixing around" on some other system ;-)

Note: don't use stat -x as it will NOT give you the birth time. My guess is that to keep things compatible with older versions, the output does not show the birth date (birth date is a relative new thing and not all file systems support it). Use either stat without options or e.g. stat -f "%SB %N" to get it.

Aha, good catch. I just assumed what looked like the long form would have everything. Also, I did reading up on Birth date the other day when you first mentioned it since I didn't remember it at all in spite of being around unix for decades. I feel like I remember "creation date" support forever in Mac OS X and Unix (BSD, System V), and what I'm reading may be suggesting that it's just Linux that chose not to implement it because it wasn't required by POSIX?  Anyway, it seems basic, and I'm a bit appalled that it was ever not standard everywhere even though, for movie tagging and this thread, it's not really something we want it using anyway ;-)

On 10.12 it looks as if Photos behaves different and seems to be using the (und-US) date indeed. Over the weekend I will have a look at tweaking my matchdatetime script to cater for adjusting this tag.

Aha, now we're on the same page! Thanks. I found the web page and I'm going to read up on your script:

https://github.com/HayoBaan/matchDateTime

Note: I played with Photos a little bit (man, what a piece of junk...) and I found out you can actually change the date of a movie. The bad news, however, is that that change is not reflected in the file, only in the Photos database . Not really useful, I think.

Yes, you can change it from the GUI. I had tried early on to then EXPORT the modified movie OUT of Photos, in an effort to find out what tag Photos was really using. I was appalled to find that the exported movie didn't reflect the newly edited date value! Amazing. And so I figured that Photos is storing that metadata in its library somewhere, and so I didn't even bother to try to disassemble the library and find the underlying video and check it's stamp. My feeling is that once it's in that Apple Photos Library, all that data is "managed" and altering it there is too risky. Hence our current goal of scrubbing the file PRIOR to import. iTunes works a similar way for songs: some of the data is tagged within the .mp3 or otherwise files themselves, but other metadata is managed in "the library". A blessing and a curse.

Looking forward to Hayo's modified script!

[Hayo Baan]

I've updated my matchDateTime script and it now also updates the QuickTime CreationDate tags with the value of the QuickTime CreateDate tag. By default it keeps the TimeZone as was set in the CreationDate tag, but you can override this with the -tz option.

Hope it works well for you!

[Hayo Baan]

I just updated the script again and added an option for interpreting QuickTime times as UTC (which is how e.g. iPhone movies are recorded).

[reggoboy]

I've updated my matchDateTime script and it now also updates the QuickTime CreationDate tags with the value of the QuickTime CreateDate tag. By default it keeps the TimeZone as was set in the CreationDate tag, but you can override this with the -tz option.

Hope it works well for you!

So!

I ran it and it seems to not have some attributes that it's expecting.

Daves-MBP:DV dkitabji$ ~/matchDateTime
Usage:
      matchDateTime [-recurse] [-overwrite] [utc] [-tz TZ [-dst]] <Files>

Daves-MBP:DV dkitabji$ exiftool -a -G -s small-MODIFIED.mov | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2016:12:15 20:06:29-05:00
[File]          FileInodeChangeDate             : 2016:12:15 19:06:20-05:00
[File]          FileModifyDate                  : 2006:11:30 13:42:00-05:00
[QuickTime]     CreateDate                      : 2016:12:11 22:49:02
[QuickTime]     CreationDate-und-US             : 2016:12:11 17:37:56-05:00
[QuickTime]     MediaCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     MediaModifyDate                 : 2016:12:11 22:49:08
[QuickTime]     ModifyDate                      : 2016:12:11 22:49:08
[QuickTime]     TrackCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     TrackModifyDate                 : 2016:12:11 22:49:08
[XMP]           CreationDate                    : 2006:11:30 13:42:00-05:00
[XMP]           DateTimeOriginal                : 2006:11:30 13:42:00-05:00
Daves-MBP:DV dkitabji$ ~/matchDateTime small-MODIFIED.mov
Skipped: Unable to find MakerNotes DateTimeOriginal or Creation Date in small-MODIFIED.mov


Do I need to use ExifTool to copy File:FileModifyDate -> something intermediate before matchDateTime will be able to put it to where I need it, QuickTime:CreationDate-und-US?

[Hayo Baan]

I see, this is a modified file where you tried to change the CreationDate yourself already. I noticed that if you do that with exiftool, it removes/hides it from the QuickTime group and places it in the XMP group. The format of the XMP tag is different from the QuickTime version so I can't use it. I will adapt my script so it will also use the language variants of the tag (I initially thought about doing that already but decided against it as it would be somewhat more error prone).

Also note that as explained in the documentation, matchDateTime will take the QuickTime createdate as date. So in your case you'd first need to use exiftool to set the QuickTime createdate to the file modification date. Then, when you run matchDateTime, it will set everything as you want. If you need/want to change the timezone as well, you can specify it with the -tz option. Also, if you know the QuickTime times to be in UTC (not all cameras do, but e.g. an iPhone does), make sure you use -api QuickTimeUTC on your exiftool command and -utc on matchDateTime.

Cheers
Hayo

P.S. I will also think about adding an option so matchDateTime will use the file modification time instead of CreateDate.

[Hayo Baan]

matchDateTime now explicitly checks (and updates) the language variants of the CreationDate tag as well. You can furthermore specify to use the file modification time instead of the CreateDate (use the -modtime option).

Note: matchDateTime will not update the XMP version of CreationDate (nor any other datetime tags) that you (accidentally) created. You will still need to use exiftool for that. I actually suggest you remove the XMP version altogether. Note that if you do that, you (magically) get the QuickTime version back. I'm not sure how this is possible (@Phil?), but it's what happens

[reggoboy]

Holy heck, I think we have success!!

First of all, you are correct, Hayo, that I mistakenly started with a clip that I had already "played with", so I wasn't starting with a clean slate. That was my oversight ;-)

So I went back to the original clip, and here's what I did to get it to work:

Daves-MBP:DV dkitabji$ exiftool -a -G -s small-MODIFIED.mov | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2016:12:18 12:16:19-05:00
[File]          FileInodeChangeDate             : 2016:12:18 12:16:13-05:00
[File]          FileModifyDate                  : 2006:11:30 13:42:00-05:00
[QuickTime]     CreateDate                      : 2016:12:11 22:49:02
[QuickTime]     CreationDate                    : 2016:12:11 17:37:56-05:00
[QuickTime]     CreationDate-und-US             : 2016:12:11 17:37:56-05:00
[QuickTime]     MediaCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     MediaModifyDate                 : 2016:12:11 22:49:08
[QuickTime]     ModifyDate                      : 2016:12:11 22:49:08
[QuickTime]     TrackCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     TrackModifyDate                 : 2016:12:11 22:49:08

Daves-MBP:DV dkitabji$ exiftool -P '-QuickTime:CreateDate<File:FileModifyDate' ./small-MODIFIED.mov
    1 image files updated

Daves-MBP:DV dkitabji$ exiftool -a -G -s small-MODIFIED.mov | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2016:12:18 12:17:15-05:00
[File]          FileInodeChangeDate             : 2016:12:18 12:17:11-05:00
[File]          FileModifyDate                  : 2006:11:30 13:42:00-05:00
[QuickTime]     CreateDate                      : 2006:11:30 13:42:00
[QuickTime]     CreationDate                    : 2016:12:11 17:37:56-05:00
[QuickTime]     CreationDate-und-US             : 2016:12:11 17:37:56-05:00
[QuickTime]     MediaCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     MediaModifyDate                 : 2016:12:11 22:49:08
[QuickTime]     ModifyDate                      : 2016:12:11 22:49:08
[QuickTime]     TrackCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     TrackModifyDate                 : 2016:12:11 22:49:08

Daves-MBP:DV dkitabji$ ~/matchDateTime small-MODIFIED.mov
Updated CreationDate 2016-12-11T17:37:56-0500 => 2006-11-30T13:42:00-0500 in small-MODIFIED.mov (1 times)
OK: Changed date 1 times in small-MODIFIED.mov

Daves-MBP:DV dkitabji$ exiftool -a -G -s small-MODIFIED.mov | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2016:12:18 12:18:43-05:00
[File]          FileInodeChangeDate             : 2016:12:18 12:18:43-05:00
[File]          FileModifyDate                  : 2006:11:30 13:42:00-05:00
[QuickTime]     CreateDate                      : 2006:11:30 13:42:00
[QuickTime]     CreationDate                    : 2006:11:30 13:42:00-05:00
[QuickTime]     CreationDate-und-US             : 2006:11:30 13:42:00-05:00
[QuickTime]     MediaCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     MediaModifyDate                 : 2016:12:11 22:49:08
[QuickTime]     ModifyDate                      : 2016:12:11 22:49:08
[QuickTime]     TrackCreateDate                 : 2016:12:11 22:49:02
[QuickTime]     TrackModifyDate                 : 2016:12:11 22:49:08


And sure enough, when I import this newly-modified clip into Apple Photos, Photos sees it as taken 2006:11:30 13:42:00.

So happy to have this working! And what a pain that all these software apps put us through all this hassle just to have things "work right"!

So Hayo, you nailed it and are truly the man. I wonder if Phil (the other man ;-) would consider building your product into ExifTool? Why not have all this functionality in one place?

Anyway, thanks so much to all of you. What an adventure! If other matters come up, you'll be the first to hear from me! And I hope this detailed thread benefits someone else :-)

[Hayo Baan]

Holy heck, I think we have success!!

First of all, you are correct, Hayo, that I mistakenly started with a clip that I had already "played with", so I wasn't starting with a clean slate. That was my oversight ;-)

Great! Good to hear.

And your "mistake" actually made the script better; the localised dates could be different for some people/circumstances, so explicitly checking them is better, plus it takes care of the situation where the original CreationDate got hidden by exiftool.

So happy to have this working! And what a pain that all these software apps put us through all this hassle just to have things "work right"!

So Hayo, you nailed it and are truly the man. I wonder if Phil (the other man ;-) would consider building your product into ExifTool? Why not have all this functionality in one place?

It would be nice, of course, if this could be integrated into exiftool but I think this might require quite some (re)work in exiftool given the way changes to the metadata are handled there. In a way my approach is quite simplistic: I just look for byte sequences/strings that match a certain pattern (a date in this case) and then overwrite it with what we want there. Phil basically rebuilds the whole file from scratch using the metadata and image data as building block. That process is sometimes (too) complex/error prone for some of the structures so he leaves them alone. In case of these QuickTime tags, however, I think it might be possible for Phil to allow writing them without having to rework his approach.

Anyway, thanks so much to all of you. What an adventure! If other matters come up, you'll be the first to hear from me! And I hope this detailed thread benefits someone else :-)

It was fun to work on the script again, plus these additions make it more useful to others than before too.

Cheers,
Hayo

P.S. I had thought about having matchDateTime also automatically update e.g. createDate if you used the file modification time but in the end I decided against that as then there are so many other date tags we might need to consider updating too (Media and Track dates for instance). What do you think?

[skippingrock]

*** Edit: never mind, I realized that I didn't need to use the matchDateTime tool because my file doesn't have a CreationDate-und-US attribute. I added the files to Photos and they show up with the corrected date.

Hello, I've been going crazy with this very problem.
Why don't video have a standard date creation system?

Anyhow, I followed the last posting by @reggoboy but when I get to the last step using the matchDateTime tool, I get and error that the expected MakerNotes cannot be found in the file. I see CreateDate, so not sure what's it saying.

What am I doing wrong?
-skipR

MacBook-Pro:~ skippingrock$ exiftool -a -G -s /Users/skippingrock/Movies/New\ Folder\ With\ Items/20160728_025628.mov  | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2017:01:08 19:15:56-08:00
[File]          FileInodeChangeDate             : 2017:01:08 19:13:49-08:00
[File]          FileModifyDate                  : 2016:07:27 19:57:10-07:00
[QuickTime]     CreateDate                      : 2016:07:27 19:57:10
[QuickTime]     MediaCreateDate                 : 2017:01:09 00:04:28
[QuickTime]     MediaModifyDate                 : 2017:01:09 00:04:45
[QuickTime]     ModifyDate                      : 2017:01:09 00:04:45
[QuickTime]     TrackCreateDate                 : 2017:01:09 00:04:28
[QuickTime]     TrackModifyDate                 : 2017:01:09 00:04:45
MacBook-Pro:~ skippingrock $ /Users/skippingrock/bin/matchDateTime /Users/skippingrock/Movies/New\ Folder\ With\ Items/20160728_025628.mov
Skipped: Unable to find MakerNotes DateTimeOriginal or Creation Date in /Users/skippingrock/Movies/New Folder With Items/20160728_025628.mov
MacBook-Pro:~ skippingrock $

P.S. thank-you for coming out with these tools!

[Hayo Baan]

Yep, in your case you don't need MatchDateTime as all the dates in your file are in fact editable by exiftool already.

[reggoboy]

Okay, so I have an update!

Recall from my last post that I had come up with a working workflow to get footage dates ripped out of the raw DV footage and all the way into Apple Photos:

MiniDV Cassette -> LifeFlix -> *.mov files -> ExifTool  ('-QuickTime:CreateDate<File:FileModifyDate') -> matchDateTime -> Apple Photos

So I finally just got back on track ripping tapes using this workflow. But I forgot the matchDateTime step, AND IT WORKED FINE ANYWAY! In Photos, the clips sorted properly by date.

So what the heck happened?

When I inspect the new clips freshly ripped via LifeFlix, I see this:

Daves-Mac-mini:Scenes dkitabji$ exiftool -a -G -s "16-AJ 6-7mos - 3_h264.mov" | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2002:05:03 00:34:00-04:00
[File]          FileInodeChangeDate             : 2017:02:01 22:05:24-05:00
[File]          FileModifyDate                  : 2002:05:03 00:34:00-04:00
[QuickTime]     ContentCreateDate               : 2017:02:01 22:03:04-05:00
[QuickTime]     CreateDate                      : 2017:02:02 03:05:10
[QuickTime]     MediaCreateDate                 : 2017:02:02 03:05:10
[QuickTime]     MediaModifyDate                 : 2017:02:02 03:05:24
[QuickTime]     ModifyDate                      : 2017:02:02 03:05:24
[QuickTime]     TrackCreateDate                 : 2017:02:02 03:05:10
[QuickTime]     TrackModifyDate                 : 2017:02:02 03:05:24

Things have changed!

1. File:FileAccessDate now matches File:FileModifyDate, both showing the original footage date.
2. There's a new QuickTime:ContentCreateDate
3. But probably most noteworthy, the troublesome Quicktime:CreationDate-und-US is GONE.

The CreationDate-und-US, recall, is what Photos was using to sort by and which exiftool could not fix (enter matchDateTime to save the day).

So it seems that now, with simply QuickTime:CreateDate fixed, Photos is working fine.

But when I inspect a file that was ripped back during my earlier posts, it still shows Quicktime:CreationDate-und-US:

Daves-Mac-mini:DV dkitabji$ exiftool -a -G -s "test-ORIGINAL.mov" | grep 'Date' | sort | uniq
[File]          FileAccessDate                  : 2016:12:15 20:43:21-05:00
[File]          FileInodeChangeDate             : 2016:12:14 22:08:21-05:00
[File]          FileModifyDate                  : 2006:10:21 20:16:00-04:00
[QuickTime]     CreateDate                      : 2016:12:11 22:37:57
[QuickTime]     CreationDate                    : 2016:12:11 17:37:56-05:00
[QuickTime]     CreationDate-und-US             : 2016:12:11 17:37:56-05:00
[QuickTime]     MediaCreateDate                 : 2016:12:11 22:37:57
[QuickTime]     MediaModifyDate                 : 2016:12:11 22:39:34
[QuickTime]     ModifyDate                      : 2016:12:11 22:39:34
[QuickTime]     TrackCreateDate                 : 2016:12:11 22:37:57
[QuickTime]     TrackModifyDate                 : 2016:12:11 22:39:34

SO WHAT CHANGED??

Well, LifeFlix has not been updated. I am, however, running macOS Sierra 10.12.2 now, not 10.12.1. That may have updated the Photos app or may have also updated the video encoding libraries in macOS. Maybe they saw my complaints and fixed it in the OS? Maybe the fault hasn't been with LifeFlix or exiftool for not handling this properly, but the OS itself?

And Hayo was kind enough to do those upgrades to matchDateTime, but they don't appear to be needed for my application at this time!

So my new, working workflow is:

MiniDV Cassette -> LifeFlix -> *.mov files -> ExifTool  ('-QuickTime:CreateDate<File:FileModifyDate') -> Apple Photos

using:

    exiftool -P '-QuickTime:CreateDate<File:FileModifyDate' <filename>

Curious what you think!

And I'm again hoping this is helpful for others out there!

DK

[Hayo Baan]

Hi DK,

Interesting! So your newer files no longer have a Quicktime:CreationDate or Quicktime:CreationDate-und-US?

I doubt very much that it's a change in Photos that caused this: as far as I know it doesn't alter the original files. Also, if your workflow is exactly as you described, there's nothing apart from lifeflix and exiftool touching the data before finding the CreationDate tags...

Hmm, I'm wondering if the data on the MiniDV Cassette perhaps contained the additional information (or at least caused it to be created)?
This should be easy to figure out: just re-rip an earlier tape.

(Note: instead of going through sort and uniq, why not simply ask exiftool for only the date tags e.g. like so: exiftool -a -G -s '-*Date' "16-AJ 6-7mos - 3_h264.mov")

[Phil Harvey]

(Note: instead of going through sort and uniq, why not simply ask exiftool for only the date tags e.g. like so: exiftool -a -G -s '-*Date' "16-AJ 6-7mos - 3_h264.mov")

or like this: exiftool -a -G -s -time:all "16-AJ 6-7mos - 3_h264.mov"

- Phil