Greetings everyone,
Having dedicated an entire day to resolving a persistent issue, I find myself reaching out to this community in the hope that someone might offer a solution.
Here's the challenge at hand:
In my Mac Photos app, I've encountered a situation where videos from my iPhone are consuming a significant amount of space. To address this, I've been exporting the original, untouched files and converting them to h.265 mp4 using Shutter Encoder while preserving metadata. However, upon re-importing these files back into the Photos app, I've noticed a peculiar issue — no location data is displayed. It appears as though the Photos app is ignoring the GPS information from the converted files. This behavior persists even when inspecting the file in Quicktime Player and accessing the information panel, where no GPS data is shown.
Curiously, when I use Terminal with ExifTool to retrieve metadata, the location data is indeed present. Same when importing into Google Photos. Right after uploading I can see the correct location.
I hope someone can help me to figure out what exactly is the problem here.
Maybe try copying the GPS from the original using ExifTool before importing the video. Some software is very particular about the number of digits in the GPS coordinates, I have seen this before, and the way that ExifTool writes this may be more compatible with Apple Photos.
exiftool -tagsfromfile SOURCEFILE "-all:gps*" TARGETFILE
- Phil
Thank you Phil. I did as you said and ExifTool does it successful, but when I open the updated file in Quicktime Player or import it in Photos app, it won't read the GPS data.
Photos.app reads GPS location from 'Keys:GPSCoordinates' from .mp4, .m4v and .mov (or 'UserData:GPSCoordinates' from .mov). Keys overrides UserData if both exist. Photos ignores this UserData tag in .mp4 and .m4v.
- Matti
After converting the file I look up the location data with exiftool -a -G1 -s -n -ee -Location:all
This is what the tool says:
[Keys] GPSCoordinates : 52.5335 12.9967 46.347
[Composite] GPSAltitude : 46.347
[Composite] GPSAltitudeRef : 0
[Composite] GPSLatitude : 52.5335
[Composite] GPSLongitude : 12.9967
[Composite] GPSPosition : 52.5335 12.9967
But when I import it to Apple Photos, no GPS is shown.
Yes, I see the same thing:
Original where QuickTime Player and Photos read GPS:
exiftool -a -G1:07 -s -n -Location:All movie.mp4
[Keys:ID-location2eISO6709] GPSCoordinates : 41 -74
Re-encoded with Shutter Encoder to .h265 with "Preserve metadata" seems to have the same details:
exiftool -a -G1:07 -s -n -Location:All movie_H.265.mp4
[Keys:ID-location2eISO6709] GPSCoordinates : 41 -74
But macOS 14 QuickTime Player or Photos does not read GPS from the re-encoded movie because obviously some detail (QuickTime:HandlerType or some offset?) is not correct for those apps.
But this fixes it (I have to clear that "substandard" GPS before it works):
exiftool -overwrite_original -Keys:GPSCoordinates= movie_H.265.mp4
1 image files updated
exiftool -overwrite_original -TagsFromFile movie.mp4 -Keys:GPSCoordinates movie_H.265.mp4
This is why I always use exiftool to copy metadata to the re-encoded movies.
- Matti
Perfect! This is working :) Thank you!
Could you please help me finding a workflow for doing this to around 1000 video files? I have the original .mov files and the converted .mp4 files in one folder (but could easily put them in two seperate folders). .mov and .mp4 files have the same filename. Only the extension mov/mp4 is different.
PS: Would is also be possible to include all original metadata from the source file? I noticed that camera information is also not imported in the Photos app - only the original picture taken date seems to work. Is it maybe better to NOT preserve metadata from Shutter Encoder and copy the metadata via ExifTool?
To be honest, I think if someone could create a simple script for this (something like: select original folder, select target folder), it could be quite popular to Photos app users.
Try something like this (backup the originals before doing this, just in case):
Copy all from .mov to a same name .mp4 in the same folder and set file dates (all Track*Date, Media*Date, Keys might not be updated if they do not exist in the source):
exiftool -m -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -TagsFromFile '%-.0f.mov' -All:All '-FileCreateDate<QuickTime:CreateDate' '-FileModifyDate<QuickTime:CreateDate' *.mp4
If Shutter Encoder has written metadata to the movie, it seems you have to clear the Keys first because otherwise Apple's apps do not read them (be careful not to delete tags from the originals!):
exiftool -m -overwrite_original -api LargeFileSupport=1 -Keys:All= *.mp4
I usually check which tags the original has and selectively copy only the desired tags with something like:
Copy some 'Keys' from .mov to a same name .m4v in the same folder:
exiftool -m -P -overwrite_original -api LargeFileSupport=1 -TagsFromFile '%-.0f.mov' -Keys:GPSCoordinates -Keys:Make -Keys:Model -Keys:Software *.mp4
- Matti
Thank you so much. I'm not at home at the moment but will test it later. Just one question: how do I point ExifTool to the folder with the files instead of a single file?
Is it for example filepath/*.mp4
Update: I figured it out and it's working. I just had to understand some context related things that I have to put the filepath in front of '%-.0f.mov' and in front of *.mp4.
Thank you so much. I'm sure this will help some other people as well!
This seems to work (somebody please chime in if there is something to be corrected). source_folder with .mov and destination_folder with .mp4 on the Desktop, replace 'user' with your username or otherwise adjust the paths.
Copy all from .mov to same name .mp4 in a different folder and set file dates (all Track*Date, Media*Date, Keys might not be updated if they do not exist in the source):
exiftool -m -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -ext mp4 -TagsFromFile '/Users/user/Desktop/source_folder/%-.0f.mov' -All:All '-FileCreateDate<QuickTime:CreateDate' '-FileModifyDate<QuickTime:CreateDate' '/Users/user/Desktop/destination_folder'
- Matti
Thanks Matti for solving this, I wouldn't have been able to figure this out myself.
Would it be possible to attach the exiftool -v3 output for a file that Photos doesn't like, and the same one after fixing GPSCoordinates with ExifTool? I would like to see if I can figure out what the problem was.
- Phil
Quote from: Phil Harvey on November 22, 2023, 10:52:02 AMWould it be possible to attach the exiftool -v3 output for a file that Photos doesn't like, and the same one after fixing GPSCoordinates with ExifTool?
macOS 14 QuickTime Player displays all following tags, and Photos.app displays Date, Title, Description, Keywords, and GPS in the original 2001-0101-1200-00.mp4.
[Keys] Author : Author
[Keys] CreationDate : 2001:01:01 12:00:00+02:00
[Keys] Description : Description
[Keys] Keywords : Keywords 1,Keywords 2
[Keys] GPSCoordinates : -36.6101 -66.91515 119.9
[Keys] Title : TitleI re-encoded the original to H.265 with Shutter Encoder's "Preserve metadata" option. BTW it encodes with ffmpeg and seems to have exiftool built-in.
But in the re-encoded .mp4 none of those tags are displayed anymore (unfixed.txt).
I could fix it with:
exiftool -Keys:All= 2001-0101-1200-00_H.265.mp4
1 image files updated
exiftool -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -TagsFromFile 2001-0101-1200-00.mp4 -All:All 2001-0101-1200-00_H.265.mp4
1 image files updatedThe Keys had to be cleared first for the fix to succeed (fixed.txt).
The movies are so small that I attached them as well. I hope this helps.
unfixed.txt
fixed.txt
2001-0101-1200-00.mp4.zip
2001-0101-1200-00_H.265.mp4.zip
p.s. thanks for these 20 years with exiftool!
- Matti
Quote from: wywh on November 22, 2023, 09:49:50 AMThis seems to work (somebody please chime in if there is something to be corrected). source_folder with .mov and destination_folder with .mp4 on the Desktop, replace 'user' with your username or otherwise adjust the paths.
Copy all from .mov to same name .mp4 in a different folder and set file dates (all Track*Date, Media*Date, Keys might not be updated if they do not exist in the source):
exiftool -m -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -ext mp4 -TagsFromFile '/Users/user/Desktop/source_folder/%-.0f.mov' -All:All '-FileCreateDate<QuickTime:CreateDate' '-FileModifyDate<QuickTime:CreateDate' '/Users/user/Desktop/destination_folder'
- Matti
I'm sorry, I'm currently learning on the fly the commands for working with terminal and ExifTools without any pre-knowledge.
How do I have to use the string instead or in addition to the 3 strings from above that worked for adding GPS?
PS: Regarding my question: I tried it with the string and GPS wasn't included.
Quote from: gpsaddict on November 22, 2023, 11:57:11 AMI tried it with the string and GPS wasn't included
I am sorry but could you re-phrase the question and explain what you were trying to do? What "strings", what did not include GPS where?? ???
Of course, sorry. I lack coding experience or any background with the tool what adds some complexity for me.
Regarding my question: You provided a solution that worked for me, and subsequently, you shared the following code:
exiftool -m -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -ext mp4 -TagsFromFile '/Users/user/Desktop/source_folder/%-.0f.mov' -All:All '-FileCreateDate<QuickTime:CreateDate' '-FileModifyDate<QuickTime:CreateDate' '/Users/user/Desktop/destination_folder'
I thought it might be a more refined version of the previous solution. After testing it by creating two folders on my desktop, etc., I noticed that the resulting files did not contain GPS data. I don't understand if and how I can modify the command you wrote (is this the correct term? I've been using 'string,' akin to constructing a Google search with operators) to include the GPS data. I thought it's a "one-line solution". Sorry, it's challenging to describe without the knowledge of how to write commands in Terminal with ExifTool. I hope that makes sense.
Quote from: gpsaddict on November 22, 2023, 02:10:37 PMI thought it might be a more refined version of the previous solution
The last version was meant for a scenario where the original .mov files are in another folder and the re-encoded same-name .mp4 destination files in another folder. I thought you asked that, but now re-reading your old post, maybe that was not the case because you wrote:
> how do I point ExifTool to the folder with the files instead of a single file?
I usually cd to the folder (i.e. "working directory") where the files are and then issue the command. For example, to then check almost all location data from all files (the dot '.' at the end means "this folder"):
exiftool -a -G1 -s -api LargeFileSupport=1 -Location:All -n -ee .I already mentioned this version where the source .mov files and the same-name .mp4 files are in the same folder. So cd to that folder and then:
exiftool -m -overwrite_original -api QuickTimeUTC=1 -api LargeFileSupport=1 -TagsFromFile '%-.0f.mov' -All:All '-FileCreateDate<QuickTime:CreateDate' '-FileModifyDate<QuickTime:CreateDate' *.mp4I hope this helps,
- Matti
Matti, thanks for the samples. The problem file has the Keys tags embedded in a UserData atom for some reason. I tested this and the good news is that you can fix it and preserve all of the known Keys tags with a single command:
exiftool -keys:all= -tagsfromfile @ -keys:all bad.mp4
The unknown keys tags that would be lost with this command are: MajorBrand, MinorVersion, CompatibleBrands, CreationTime and Encoder. I will do some checking to see if I should add support for any of these.
- Phil
Quote from: Phil Harvey on November 22, 2023, 10:19:59 PMThe problem file has the Keys tags embedded in a UserData atom
Thanks for solving what causes this issue.
It seems to be related to ffmpeg because if I use the options that should preserve metadata (Keys in the input.mp4):
ffmpeg -i input.mp4 -c copy -movflags use_metadata_tags -map_metadata 0 output.mp4...I get the same kind of UserData as with Shutter Encoder's preserve metadata option (exiftool v3 output attached), and it can be fixed with that simple TagsFromFile command.
ffmpeg.txt
- Matti
I wonder why they are doing this.
BTW, for this specific issue, the exiftool -G5 output gives a clear indication of where this is stored:
[MOV-Movie-UserData-Meta-Keys] Major Brand : mp42
[MOV-Movie-UserData-Meta-Keys] Minor Version : 1
[MOV-Movie-UserData-Meta-Keys] Compatible Brands: isommp41mp42
[MOV-Movie-UserData-Meta-Keys] Creation Time : 2001-01-01T10:00:00.000000Z
[MOV-Movie-UserData-Meta-Keys] Author : Author
[MOV-Movie-UserData-Meta-Keys] Creation Date : 2001:01:01 12:00:00+02:00
[MOV-Movie-UserData-Meta-Keys] Description : Description
[MOV-Movie-UserData-Meta-Keys] Keywords : Keywords 1,Keywords 2
[MOV-Movie-UserData-Meta-Keys] GPS Coordinates : 36.610100 S, 66.915150 W, 119.9 m Above Sea Level
[MOV-Movie-UserData-Meta-Keys] User Rating : 3
[MOV-Movie-UserData-Meta-Keys] Title : Title
[MOV-Movie-UserData-Meta-Keys] Encoder : Lavf60.14.100
and after this command (ExifTool 12.70):
exiftool test.mp4 -keys:all= -tagsfromfile @ -keys:all
you get this:
[MOV-Movie-Meta-Keys] Author : Author
[MOV-Movie-Meta-Keys] Creation Date : 2001:01:01 12:00:00+02:00
[MOV-Movie-Meta-Keys] Description : Description
[MOV-Movie-Meta-Keys] Keywords : Keywords 1,Keywords 2
[MOV-Movie-Meta-Keys] GPS Coordinates : 36.610100 S, 66.915150 W, 119.9 m Above Sea Level
[MOV-Movie-Meta-Keys] User Rating : 3
[MOV-Movie-Meta-Keys] Title : Title
- Phil
P.S. I'll add support for writing "Keys:Encoder" so it will be preserved with this command for ExifTool 12.71 and later, but I'm not going to add the other unknown tags yet because they seem like more of a hack to me.
Quote from: Phil Harvey on November 23, 2023, 08:45:53 AMexiftool -G5 output gives a clear indication of where this is stored
Thanks for the hint about G5! So this seems to be the problem with ffmpeg (somebody please chime in if I am missing some option that might preserve metadata):
ffmpeg -i input.mov -c:v libx265 -tag:v hvc1 -c:a aac -movflags use_metadata_tags -map_metadata 0 ffmpeg_output.mp4
ffmpeg version 4.4.4 Copyright (c) 2000-2023 the FFmpeg developers
exiftool -a -G5 -s -n -api QuickTimeUTC=1 -Keys:All -QuickTime:CreateDate .
======== ./input.mov
[MOV-Movie-Meta-Keys] Author : Author
[MOV-Movie-Meta-Keys] Description : Description
[MOV-Movie-Meta-Keys] Keywords : Keyword 1,Keyword 2
[MOV-Movie-Meta-Keys] UserRating : 3
[MOV-Movie-Meta-Keys] Title : Title
[MOV-Movie-Meta-Keys] GPSCoordinates : -36.6101 -66.91515 119.9
[MOV-Movie-Meta-Keys] CreationDate : 2001:01:01 12:00:00+02:00
[MOV-Movie-MovieHeader] CreateDate : 2001:01:01 12:00:00+02:00
======== ./ffmpeg_output.mp4
[MOV-Movie-UserData-Meta-Keys] MajorBrand : qt
[MOV-Movie-UserData-Meta-Keys] MinorVersion : 0
[MOV-Movie-UserData-Meta-Keys] CompatibleBrands : qt
[MOV-Movie-UserData-Meta-Keys] CreationTime : 2001-01-01T10:00:00.000000Z
[MOV-Movie-UserData-Meta-Keys] Author : Author
[MOV-Movie-UserData-Meta-Keys] Description : Description
[MOV-Movie-UserData-Meta-Keys] Keywords : Keyword 1,Keyword 2
[MOV-Movie-UserData-Meta-Keys] UserRating : 3
[MOV-Movie-UserData-Meta-Keys] Title : Title
[MOV-Movie-UserData-Meta-Keys] GPSCoordinates : -36.6101 -66.91515 119.9
[MOV-Movie-UserData-Meta-Keys] CreationDate : 2001:01:01 12:00:00+02:00
[MOV-Movie-UserData-Meta-Keys] Encoder : Lavf58.76.100
[MOV-Movie-MovieHeader] CreateDate : 2001:01:01 12:00:00+02:00p.s. it seems ffmpeg '-movflags use_metadata_tags'
tries to preserve those Keys, and '-map_metadata 0' only preserves 'QuickTime:CreateDate'.
- Matti
ffmpeg 6.1.1 still copies Keys:GPSCoordinates embedded in UserData atom that Apple's apps do not read (the same applies to all Keys tags like CreationDate, Title, DisplayName, Author, Description, Keywords, Make, Model, Software etc):
ffmpeg6 -i input.mp4 -c copy -movflags use_metadata_tags -map_metadata 0 output.mp4
exiftool -a -G5 -s -n -Keys:GPSCoordinates .
======== ./input.mp4
[MOV-Movie-Meta-Keys] GPSCoordinates : -36.6101 -66.91515 119.9
======== ./output.mp4
[MOV-Movie-UserData-Meta-Keys] GPSCoordinates : -36.6101 -66.91515 119.9
-> Surprisingly subsequent exiftool updates also go to the same incorrect place in that movie:
exiftool -m -P -overwrite_original -Keys:GPSCoordinates='14.79354, 75.40448, 99.9' .
exiftool -a -G5 -s -n -Keys:GPSCoordinates .
======== ./input.mp4
[MOV-Movie-Meta-Keys] GPSCoordinates : 14.79354 75.40448 99.9
======== ./output.mp4
[MOV-Movie-UserData-Meta-Keys] GPSCoordinates : 14.79354 75.40448 99.9
...rebuilding metadata fixes it:
exiftool -m -P -overwrite_original -All= -tagsFromFile @ -All:All -Unsafe -ICC_Profile output.mp4
exiftool -a -G5 -s -n -Keys:GPSCoordinates .
======== ./input.mp4
[MOV-Movie-Meta-Keys] GPSCoordinates : 14.79354 75.40448 99.9
======== ./output.mp4
[MOV-Movie-Meta-Keys] GPSCoordinates : 14.79354 75.40448 99.9
- Matti
Hi Matti
Video metadata is such a hairball. ExifTool only creates a new Keys section if it didn't already exist, otherwise it writes to the existing one. My thought was that it would be better to be flexible like this rather than force it into a specific location, in keeping with the untamed nature of the video metadata landscape.
- Phil