How to assign dates to a set of existing dates

[ronanleboite]

Hello,

I'm using Exiftool in my photo and videos management workflow.
So far I'm very happy with this tool and thank you so much for building it, maintaining it along with a great community!

I'm currently trying to figure out the best way to assign proper metadata when I process videos (AVI,MOV, etc) to mp4
I'm using Handbrake to convert those videos, using a standard preset, which is more than enough for my use.

Although I want to preserve the correct timestamp data at least and the tagsfromfile is just not working always in that situation.

For example, if I have a file in.AVI with the following metadata:

Code Select Expand


ExifTool Version Number         : 12.00
File Name                       : in.AVI
Directory                       : /Users/ronanleboite/Desktop/Exceptions/Mauvaises Dates
File Size                       : 33 MB
File Modification Date/Time     : 2007:02:20 09:48:28-08:00
File Access Date/Time           : 2020:12:19 18:38:22-08:00
File Inode Change Date/Time     : 2020:12:19 15:57:42-08:00
File Permissions                : rwxr-xr-x
File Type                       : AVI
File Type Extension             : avi
MIME Type                       : video/x-msvideo
Frame Rate                      : 30
Max Data Rate                   : 2337 kB/s
Frame Count                     : 559
Stream Count                    : 2
Stream Type                     : Video
Video Codec                     : mjpg
Video Frame Rate                : 30
Video Frame Count               : 559
Quality                         : 10000
Sample Size                     : Variable
BMP Version                     : Windows V3
Image Width                     : 640
Image Height                    : 480
Planes                          : 1
Bit Depth                       : 24
Compression                     : MJPG
Image Length                    : 921600
Pixels Per Meter X              : 0
Pixels Per Meter Y              : 0
Num Colors                      : Use BitDepth
Num Important Colors            : All
Audio Codec                     :
Audio Sample Rate               : 44100
Audio Sample Count              : 821730
Encoding                        : Microsoft PCM
Num Channels                    : 1
Sample Rate                     : 44100
Avg Bytes Per Sec               : 88200
Bits Per Sample                 : 16
Total Frame Count               : 559
Date/Time Original              : 2007:02:20 13:48:29
Software                        : CanonMVI06
Image Size                      : 640x480
Megapixels                      : 0.307
Duration                        : 18.63 s


after conversion using handbrake it gives:

NB: my script does try to preserve metadata usng a command with tagsfromfile, which fails with the following:
"Warning: No writable tags set from in.AVI"

Code Select Expand


ExifTool Version Number         : 12.00
File Name                       : in.mp4
Directory                       : /Users/ronanleboite/Desktop/test convert videos
File Size                       : 3.0 MB
File Modification Date/Time     : 2007:02:20 09:48:28-08:00
File Access Date/Time           : 2020:12:19 18:18:58-08:00
File Inode Change Date/Time     : 2020:12:19 18:39:23-08:00
File Permissions                : rw-r--r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 v2 [ISO 14496-14]
Minor Version                   : 0.2.0
Compatible Brands               : [isom,iso2,avc1,mp41]
Media Data Size                 : 3077163
Media Data Offset               : 48
Movie Header Version            : 0
Create Date                     : 2020:12:20 02:39:19
Modify Date                     : 2020:12:20 02:39:19
Time Scale                      : 1000
Duration                        : 18.69 s
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 3
Track Header Version            : 0
Track Create Date               : 2020:12:20 02:39:19
Track Modify Date               : 2020:12:20 02:39:19
Track ID                        : 1
Track Duration                  : 18.63 s
Track Layer                     : 0
Track Volume                    : 0.00%
Image Width                     : 640
Image Height                    : 480
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 640
Source Image Height             : 480
X Resolution                    : 72
Y Resolution                    : 72
Bit Depth                       : 24
Pixel Aspect Ratio              : 1:1
Video Frame Rate                : 30
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2020:12:20 02:39:19
Media Modify Date               : 2020:12:20 02:39:19
Media Time Scale                : 44100
Media Duration                  : 18.69 s
Media Language Code             : und
Handler Description             : Mono
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 2
Audio Bits Per Sample           : 16
Audio Sample Rate               : 44100
Track 2 Name                    : Mono
Track 2 Title                   : Mono
Handler Type                    : Metadata
Handler Vendor ID               : Apple
Encoder                         : HandBrake 1.3.3 2020111500
Image Size                      : 640x480
Megapixels                      : 0.307
Avg Bitrate                     : 1.32 Mbps
Rotation                        : 0


My python scripts  read the input files and are able to figure the best date to apply to an output file, although I tried this command

Code Select Expand


exiftool "-time:all=2007:02:20 13:48:29" out.mp4


and this assigns dates to a whole lot of fields.
I was looking forward to only assign dates to the dates that needed to be updated, and not the whole shebang.

Here are my two questions:

- Is there a command to only update the existing date fields in the metadata?
- Should I even worry about that and proceed to use the above command?

Thank you again for this great utility!

[StarGeek]

You don't give your exact command, but for the most part, the tags in the AVI don't exist in the MP4.  You'll need to state what tags you want to copy and where you want to put them.

Since your example matches the DateTimeOriginal tag in the AVI file, I'd suggest trying this
exiftool -TagsFromFile in.avi "-CreateDate<DateTimeOriginal" "-ModifyDate<DateTimeOriginal" -api QuickTimeUTC out.mp4

This will copy the DateTimeOriginal timestamp in the AVI to the two main timestamps in the MP4.  Additionally, since timestamps in MP4 files are supposed to be set to UTC, the -api QuickTimeUTC option will adjust the time properly, assuming the video was shot in the same timezone as the computer.

There is a DateTimeOriginal tag available in MP4 files, but I'd advise against using it.  It isn't used by many programs, but the main one that does, the Apple Photo app, will spit out nonsense timestamps unless a timezone is also included, which would not be as easy to add.

[ronanleboite]

Thanks a lot for your answer!
This brings me to some more questions.

The command I expected to use that I did not mention was:


exiftool -tagsfromfile "in.AVI" -all:all -r -overwrite_original "out.mp4"


As you're saying the exif information in both files are very different, and I'm not surprised it would fail.

My concern is that the mp4 contains a lot of other dates like Track Create Date, Track Modify Date etc... which now are set to the conversion date. I wanted to assign them too.
I guess I'll have to add them too to the command and assign them explicitely?

If I understand what you say about DateTimeOriginal, I should not set the DateTimeOriginal to mp4 videos.

I'm currently trying to correctly sort a lot of pictures and it's quite uneasy to merge all those files that might have been taken or received from different devices/sources

I have a couple questions:

- Does that mean that I should consider CreateDate and ModifyDate as correct time reference for mp4 videos?
- is that the same for photos? ie, should I always make sure to have a CreateDate in my photos? Should I even drop the DateTimeOriginal?
- In my script, I generally consider DateTimeOriginal as a correct UTC representation of the time the picture/video was taken. Is there anything else I should be aware of?

[StarGeek]

exiftool -tagsfromfile "in.AVI" -all:all -r -overwrite_original "out.mp4"

As you're saying the exif information in both files are very different, and I'm not surprised it would fail.

Using -All:All means that information is copied to the exact same place in the target file.  As DateTimeOriginal in an AVI is part of the RIFF group (RIFF:DateTimeOriginal) and a MP4 cannot contain a RIFF tag, it wouldn't be copied.

Minor nitpick, EXIF is not an all purpose term.  All EXIF data is metadata, but not all metadata is EXIF data. If you add the -G (-groupNames) option to your command, you'll see the groups that the tags belong to. Your base command for viewing data in a file should be exiftool -G -a -s (see FAQ #3).

My concern is that the mp4 contains a lot of other dates like Track Create Date, Track Modify Date etc... which now are set to the conversion date. I wanted to assign them too.
I guess I'll have to add them too to the command and assign them explicitely?

Yes.  Or don't bother.  I don't believe there are that many programs that actually use those tags.  Maybe in some sort of high end setting where you would be mixing different video and audio tracks and would need to keep track of the correct source.

If I understand what you say about DateTimeOriginal, I should not set the DateTimeOriginal to mp4 videos.

If you do, you should set the timezone as well, especially if it will be viewed by Apple Photos at some point.  In this post for example, setting DateTimeOriginal to "1995:01:18 18:50:00" results in "November 24, 183057   7:15:10 PM".  So in this example, the correct way to set it would be -Quicktime:DateTimeOriginal="1995:01:18 18:50:00-08:00".  Or you could do it as a second command after properly setting the CreateDate
exiftool -api QuickTimeUTC "-Quicktime:DateTimeOriginal<CreateDate" /path/to/files

- Does that mean that I should consider CreateDate and ModifyDate as correct time reference for mp4 videos?

Yes.  There are other timestamp tags in a video file, but most programs will read the Quicktime CreateDate and ModifyDate.

- is that the same for photos? ie, should I always make sure to have a CreateDate in my photos? Should I even drop the DateTimeOriginal?

No.  Most image programs will read and give priority to the EXIF:DateTimeOriginal.

- In my script, I generally consider DateTimeOriginal as a correct UTC representation of the time the picture/video was taken. Is there anything else I should be aware of?

Are you in a timezone that is usually the same as UTC?  Or did you have your camera set to UTC rather than the local time?  Your example AVI with the timestamp of 2007:02:20 13:48:29 would only be UTC if you were in WET time zone and would have to be adjusted during Summer time.

Images are normally set to local time.  MP4/MOV videos are supposed to be set to UTC time, but if the camera is not time zone aware, it would probably be incorrectly set to local time.

Also, when it comes to videos, Adobe programs do not properly read video timestamps.  They will display the time as is, which can cause problems in showing timelines, as most other things, such as Windows, MacOS, and Google Photos, will properly adjust for the time zone difference.

[ronanleboite]

That's all great information!

Thank you for the commands remarks, this will help me a lot!

Are you in a timezone that is usually the same as UTC?  Or did you have your camera set to UTC rather than the local time?  Your example AVI with the timestamp of 2007:02:20 13:48:29 would only be UTC if you were in WET time zone and would have to be adjusted during Summer time.

Unfortunately, no. most of my media have been taken in a bunch of timezones.
I'm thinking that I could eventually use the photos data to extrapolate the timezones I could set to matching videos...
Actually, correct me if I'm wrong, but I was assuming that DateTimeOriginal was a local time and as a consequence unaware of TimeZones.

Dealing with timezone in my current workflow (for past medias) is just very hairy and I was looking forward to only consider localtime and even trim the timezone in cases where I can only rely on tags that contain TimeZone information.

So my understanding is that with my workflow I should do:
- Photos: use my best guess local time date and write it to EXIF:DateTimeOriginal. I'm also planning to override CreateDate and ModifyDate, just to be sure to include tags that might be used by more applications and services.
- Videos: use best guess local time date to write in CreateDate and ModifyDate, and drop Quicktime:DateTimeOriginal if it's ever set.

[StarGeek]

Actually, correct me if I'm wrong, but I was assuming that DateTimeOriginal was a local time and as a consequence unaware of TimeZones.

EXIF:DateTimeOriginal in an image file is assumed to be set to local time and does not have a time zone component built in.  The EXIF standard holds the time zone in a separate tag, OffsetTimeOriginal. Exiftool has a helpful shortcut, SubSecDateTimeOriginal, which will display or set the EXIF:DateTimeOriginal, EXIF:OffsetTimeOriginal, and EXIF:SubSecTimeOriginal all at once.

The IPTC Photo Metadata Standard uses XMP:DateCreated as the equivalent tag, but the EXIF standard for XMP uses XMP:DateTimeOriginal.  The latter isn't a well supported in most software, though.  Both of the XMP tags support the inclusion of a time zone as an option.

DateTimeOriginal in a video (Quicktime:DateTimeOriginal) can include a time zone and I assume it's meant to be in local time.  It would be very confusion if it was supposed to be UTC and include a time zone.

The tl;dr is:  All metadata is a hot mess and will drive you mad if you think about it too much.

Dealing with timezone in my current workflow (for past medias) is just very hairy and I was looking forward to only consider localtime and even trim the timezone in cases where I can only rely on tags that contain TimeZone information.

You can decide what standards you want to follow and which you want to edit for your own purpose.  If it's easier to just use local time in the videos, go for it.  Just understand you might get different results in some software.  As an example, I've heard that some photographers who travel a lot will set their cameras to UTC and leave it at that. 

- Photos: use my best guess local time date and write it to EXIF:DateTimeOriginal. I'm also planning to override CreateDate and ModifyDate, just to be sure to include tags that might be used by more applications and services.

Exiftool has the AllDates shortcut which will set all three of those tags at once.

[ronanleboite]

The tl;dr is:  All metadata is a hot mess and will drive you mad if you think about it too much.

That is exactly what I want to avoid haha!
It boils down to what level of madness I'm willing to endure in order to satisfy my organization expectations

Thank you for all the help, I have a good track now I can follow.
That will help me sort and fix better my media and leave just a handful of exceptions.

[StarGeek]

It boils down to what level of madness I'm willing to endure in order to satisfy my organization expectations

Exactly!