Writing GPSPosition, then reading GPSPosition returns old GPSPosition

Started by tipa, August 19, 2023, 02:45:22 AM

Previous topic - Next topic

tipa

I am using ExifTool in one of my apps to read & edit location information / geotags of photos & videos. One of the app's users approached me and reported a case where this was not possible. I was able to reproduce the problem and it looks like a bug in the ExifTool to me.

Information to reproduce problem:
- System type & version: macOS 13.5.1
- ExifTool version: 12.65


Steps to reproduce:
exiftool -GPSPosition='11.1, 11.1' -v video.mov

Rewriting video.mov...
  FileType = MOV
  FileTypeExtension = MOV
  MIMEType = video/quicktime
  Editing tags in: Composite GPS MIE-GPS MOV Movie UserData XMP
  Rewriting Movie
  Rewriting MovieHeader
  Rewriting Track
  Rewriting TrackHeader
  Rewriting Media
  Rewriting MediaHeader
  Rewriting MediaInfo
  Rewriting DataInfo
  Rewriting DataRef
  Rewriting SampleTable
  Rewriting Track
  Rewriting TrackHeader
  Rewriting Media
  Rewriting MediaHeader
  Rewriting MediaInfo
  Rewriting DataInfo
  Rewriting DataRef
  Rewriting SampleTable
  Rewriting Meta
  Rewriting Keys
  Rewriting ItemList
  Rewriting UserData
  Rewriting XMP
    1 image files updated

exiftool -GPSLatitude -GPSLongitude -GPSPosition ./video.mov -n -v

  ExifToolVersion = 12.65
  FileName = video.mov
  Directory = .
  FileSize = 4273035
  FileModifyDate = 1692426934
  FileAccessDate = 1692426934
  FileInodeChangeDate = 1692426934
  FilePermissions = 33216
  FileType = MOV
  FileTypeExtension = MOV
  MIMEType = video/quicktime
  FileType (SubDirectory) -->
  + [BinaryData directory, 12 bytes]
  | MajorBrand = qt 
  | MinorVersion =
  | CompatibleBrands = qt 
  Movie (SubDirectory) -->
  + [Movie directory]
  | MovieHeader (SubDirectory) -->
  | + [BinaryData directory, 100 bytes]
  | | MovieHeaderVersion = 0
  | | CreateDate = 3773973166
  | | ModifyDate = 3773973166
  | | TimeScale = 600
  | | Duration = 1381
  | | PreferredRate = 65536
  | | PreferredVolume = 256
  | | MatrixStructure = 1 0 0 0 1 0 0 0 16384
  | | PreviewTime = 0
  | | PreviewDuration = 0
  | | PosterTime = 0
  | | SelectionTime = 0
  | | SelectionDuration = 0
  | | CurrentTime = 0
  | | NextTrackID = 3
  | Track (SubDirectory) -->
  | + [Track directory]
  | | TrackHeader (SubDirectory) -->
  | | + [BinaryData directory, 84 bytes]
  | | | TrackHeaderVersion = 0
  | | | TrackCreateDate = 3773973166
  | | | TrackModifyDate = 3773973166
  | | | TrackID = 1
  | | | TrackDuration = 1381
  | | | TrackLayer = 0
  | | | TrackVolume = 0
  | | | MatrixStructure = 1 0 0 0 1 0 0 0 16384
  | | | ImageWidth = 125829120
  | | | ImageHeight = 94371840
  | | TrackAperture (SubDirectory) -->
  | | + [TrackAperture directory]
  | | | CleanApertureDimensions = 0 1440 1080
  | | | ProductionApertureDimensions = 0 1440 1080
  | | | EncodedPixelsDimensions = 0 1440 1080
  | | Unknown_edts = .elst..e.
  | | Media (SubDirectory) -->
  | | + [Media directory]
  | | | MediaHeader (SubDirectory) -->
  | | | + [BinaryData directory, 24 bytes]
  | | | | MediaHeaderVersion = 0
  | | | | MediaCreateDate = 3773973166
  | | | | MediaModifyDate = 3773973166
  | | | | MediaTimeScale = 90000
  | | | | MediaDuration = 207207
  | | | | MediaLanguageCode = 21956
  | | | Handler (SubDirectory) -->
  | | | + [BinaryData directory, 41 bytes]
  | | | | HandlerClass = mhlr
  | | | | HandlerType = vide
  | | | | HandlerVendorID = appl
  | | | | HandlerDescription = .Core Media Video
  | | | MediaInfo (SubDirectory) -->
  | | | + [MediaInfo directory]
  | | | | VideoHeader (SubDirectory) -->
  | | | | + [BinaryData directory, 12 bytes]
  | | | | | GraphicsMode = 64
  | | | | | OpColor = 32768 32768 32768
  | | | | Handler (SubDirectory) -->
  | | | | + [BinaryData directory, 48 bytes]
  | | | | | HandlerClass = dhlr
  | | | | | HandlerType = alis
  | | | | | HandlerVendorID = appl
  | | | | | HandlerDescription = .Core Media Data Handler
  | | | | DataInfo (SubDirectory) -->
  | | | | + [DataInfo directory]
  | | | | | DataRef (SubDirectory) -->
  | | | | | + [DataRef directory]
  | | | | | | Unknown_alis = .
  | | | | SampleTable (SubDirectory) -->
  | | | | + [SampleTable directory]
  | | | | | VideoSampleDesc (SubDirectory) -->
  | | | | | + [BinaryData directory, 216 bytes]
  | | | | | | CompressorID = avc1
  | | | | | | VendorID =
  | | | | | | SourceImageWidth = 1440
  | | | | | | SourceImageHeight = 1080
  | | | | | | XResolution = 72
  | | | | | | YResolution = 72
  | | | | | | CompressorName = .H.264
  | | | | | | BitDepth = 24
  | | | | | VideoFrameRate = .E..
  | | | | | CompositionTimeToSample = C........E.......E.......E.......E.......E.......[snip]
  | | | | | CompositionToDecodeTimelineMapping = .....E...)g
  | | | | | SyncSampleTable = ...=
  | | | | | IdependentAndDisposableSamples =  ............................. ...........[snip]
  | | | | | SampleToChunk = ................
  | | | | | SampleSizes = E..;..7.!..F.....cp.8.g1...5g.@5...I..pHb..J...B..~M...O...qN[snip]
  | | | | | ChunkOffset = ........s)..9..
  | Track (SubDirectory) -->
  | + [Track directory]
  | | TrackHeader (SubDirectory) -->
  | | + [BinaryData directory, 84 bytes]
  | | | TrackHeaderVersion = 0
  | | | TrackCreateDate = 3773973166
  | | | TrackModifyDate = 3773973166
  | | | TrackID = 2
  | | | TrackDuration = 1381
  | | | TrackLayer = 0
  | | | TrackVolume = 0
  | | | MatrixStructure = 1 0 0 0 1 0 0 0 16384
  | | | ImageWidth = 0
  | | | ImageHeight = 0
  | | Unknown_edts = .elst..e.
  | | Media (SubDirectory) -->
  | | + [Media directory]
  | | | MediaHeader (SubDirectory) -->
  | | | + [BinaryData directory, 24 bytes]
  | | | | MediaHeaderVersion = 0
  | | | | MediaCreateDate = 3773973166
  | | | | MediaModifyDate = 3773973166
  | | | | MediaTimeScale = 90000
  | | | | MediaDuration = 207207
  | | | | MediaLanguageCode = 21956
  | | | Handler (SubDirectory) -->
  | | | + [BinaryData directory, 45 bytes]
  | | | | HandlerClass = mhlr
  | | | | HandlerType = tmcd
  | | | | HandlerVendorID = appl
  | | | | HandlerDescription = .Core Media Time Code
  | | | MediaInfo (SubDirectory) -->
  | | | + [MediaInfo directory]
  | | | | GenMediaHeader (SubDirectory) -->
  | | | | + [GenMediaHeader directory]
  | | | | | GenMediaInfo (SubDirectory) -->
  | | | | | + [BinaryData directory, 16 bytes]
  | | | | | | GenMediaVersion = 0
  | | | | | | GenFlags = 0 0 0
  | | | | | | GenGraphicsMode = 64
  | | | | | | GenOpColor = 32768 32768 32768
  | | | | | | GenBalance = 0
  | | | | | TimeCode (SubDirectory) -->
  | | | | | + [TimeCode directory]
  | | | | | | TCMediaInfo (SubDirectory) -->
  | | | | | | + [BinaryData directory, 32 bytes]
  | | | | | | | TextFont = 22
  | | | | | | | TextFace = 0
  | | | | | | | TextSize = 12
  | | | | | | | TextColor = 0 0 0
  | | | | | | | BackgroundColor = 65535 65535 65535
  | | | | | | | FontName = Courier
  | | | | Handler (SubDirectory) -->
  | | | | + [BinaryData directory, 48 bytes]
  | | | | | HandlerClass = dhlr
  | | | | | HandlerType = alis
  | | | | | HandlerVendorID = appl
  | | | | | HandlerDescription = .Core Media Data Handler
  | | | | DataInfo (SubDirectory) -->
  | | | | + [DataInfo directory]
  | | | | | DataRef (SubDirectory) -->
  | | | | | + [DataRef directory]
  | | | | | | Unknown_alis = .
  | | | | SampleTable (SubDirectory) -->
  | | | | + [SampleTable directory]
  | | | | | OtherSampleDesc (SubDirectory) -->
  | | | | | + [BinaryData directory, 34 bytes]
  | | | | | | OtherFormat = tmcd
  | | | | | | PlaybackFrameRate = 29.97002997
  | | | | | TimeToSampleTable = ...)g
  | | | | | SampleToChunk = ....
  | | | | | SampleSizes = ..
  | | | | | ChunkOffset = ...o
  | Meta (SubDirectory) -->
  | + [Meta directory]
  | | Handler (SubDirectory) -->
  | | + [BinaryData directory, 26 bytes]
  | | | HandlerClass =
  | | | HandlerType = mdta
  | | | HandlerVendorID =
  | | | HandlerDescription =
  | | Keys (SubDirectory) -->
  | | + [Keys directory]
  | | | Added ItemList Tag 1.1 = (mdta) location.ISO6709
  | | | Added ItemList Tag 1.2 = (mdta) creationdate
  | | ItemList (SubDirectory) -->
  | | + [ItemList directory]
  | | | GPSCoordinates = +29.0695-082.4271/
  | | | CreationDate = 2023-07-19T12:17:37-04:00
  | UserData (SubDirectory) -->
  | + [UserData directory]
  | | XMP (SubDirectory) -->
  | | + [XMP directory, 2850 bytes]
  | | | XMPToolkit = Image::ExifTool 12.65
  | | | GPSLatitude = 11,6.0N
  | | | GPSLongitude = 11,6.0E
  MediaDataSize = 4267447
  MediaDataOffset = 5588
  MediaData = .......0]!Kq.q,....)....%.@....5.....\|.N..)..!N*F.."..3..m.r...+/....Q..[snip]
GPS Latitude                    : 29.0695
GPS Longitude                  : -82.4271
GPS Position                    : 29.0695 -82.4271

As can be seen in the output, the new geotag seems to be written into the XMP subdirectory, but not into the "ItemList" subdirectory, where it is read from. Example file: https://1drv.ms/v/s!Ao11bhuirfKfzuhU-6wB2zV823rutw?e=EoS7mj

StarGeek

Can you share the file on some other site such as dropbox or google drive?  I tried Firefox, Chrome, and Edge to download it, but none of them will download the file when I click on the download button.

You might try reverting to 12.64
https://exiftool.org/ExifTool-12.64.dmg
as there there is a bug in 12.65 which affects the output.
"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

StarGeek

Was about to download a video using a download manager, but its definitely not the original file.  It's a re-encoded MP4 with no data, not a MOV file.

If you don't have access to other sites, try zipping the file up, compression set to 0/storage, before uploading it.
"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

tipa

Sure, here is the file shared via Google Drive: https://drive.google.com/file/d/1lAL1jHZAcpE-4l9q4ostrd5SN8qS29fF/view?usp=drive_link

I also tried v12.64 (and another, much older version), but they show the same behavior.
It's true that the video file is "weird", but it plays in the media player on both Windows and macOS.

StarGeek

That link says
"You need access
Request access, or switch to an account with access"

It's been a while since I shared a GDrive link, but it looks like the default setting for "Copy Link" is restricted.  In order to make a public link from the website, you have to Right Click->Share->Share and set "General Access" to "Anyone with the link".  I don't have GDrive installed on my computer, so I don't know the procedure with it.
"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

tipa

Sorry, haven't shared a file though Google Drive since a while and wasn't aware of that default. I adjusted the access setting for the file

StarGeek

Sorry, I should have paid more attention to your first post.  You're writing to GPSPosition, which is a Composite tag consisting of GPSLatitude and GPSLongitude.  In an image file, writing to this will write to the EXIF GPS tags and the corrisponding reference direction tags.  For most other files, it will write to the XMP:GPSLatitude/XMP:GPSLongitude tags.

The tag you want to write in a video is the GPSCoordinates tag.  In this file, it is currently in Keys:GPSCoordinates, but the GPSCoordinates tag can also appear in ItemList:GPSCoordinates and UserData:GPSCoordinates.

Always use the command in FAQ #3 so you can view all the data, including duplicates, and their locations.

Quote from: tipa on August 20, 2023, 10:17:05 AMSorry, haven't shared a file though Google Drive since a while and wasn't aware of that default.

Neither did I.

"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

tipa

Thanks for your reply.

Ideally, I would not have to check for the file type in my application in order to use different commands/composite tags for the writing operation. For the large majority of files (photo + video, incl. QuickTime), I can write to the GPSPosition composite tag and subsequently read it (returning the same values that I have written) - this is what a new user without deeper knowledge of the individual tags would be expecting to see and it is also suggested in FAQ #14, without any mention of the GPSCoordinate-limitation. The problem I am facing with this particular file is that the writing operation reports "success" (1 image files updated), but then reading the very same composite tag reports other values.

I think a better, more consistent behaviour would be if the "GPSPosition" composite tag would ignore the QuickTime:GPSCoordinates tag when reading - because it is currently not able to write it.

The best solution however would be if the GPSCoordinates tag would be written when writing to the GPSPosition tag. Then there wouldn't have to be changes in the reading logic and there would only be one GPS location present in the meta tags.

Is there a particular reason that this isn't possible?
Would there be a downside to changing my geotagging code to always write to both the GPSPosition & GPSCoordinates tags, no matter if I am editing a movie or picture file?

Phil Harvey

Quote from: tipa on August 21, 2023, 03:40:03 AMThe best solution however would be if the GPSCoordinates tag would be written when writing to the GPSPosition tag.

I agree that this would be best.  Unfortunately, looking at my samples, GPSCoordinates isn't stored in a consistent location.  About half of them are stored in the UserData, and the other half as Keys.  I don't see any clean way to deal with this.  :(

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

Quote from: Phil Harvey on August 21, 2023, 08:51:29 AMlooking at my samples, GPSCoordinates isn't stored in a consistent location.  About half of them are stored in the UserData, and the other half as Keys.

With, as I mentioned above, ItemList:GPSCoordinates also being a possibility, albeit an apparently rare location.  Welcome to the Multiverse of Maddness that is video metadata (unlike image metadata, which is simply a Universe of Maddness).

And as I always post, Obligatory xkcd
"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

tipa

Thanks for your comments!

  • It appears to be possible to write to "GPSCoordinates". So there must be some existing logic. Isn't it possible to call that logic when writing to "GPSPosition"?
  • Would it be worth considering to remove the inclusion of GPSCoordinates when reading GPSPosition? Then it would at least be a consistent behavior when writing + reading "GPSPosition"
  • Or would it be possible to prefer returning the XMP GPS tags over the QuickTime GPS tags (if both are present in the file) when reading the GPSPosition composite tag? That would also give a more consistent behavior when writing + reading to the GPSPosition composite tag.

wywh

I use this command when I want to se what locations the movie might have (here all locations are deliberately set different):

exiftool -a -G1 -s -api LargeFileSupport=1 -Location:All -n -ee 2006-0606-1206-06.mp4
[UserData]      GPSCoordinates                  : 36.6101 -66.91515 119.9
[ItemList]      GPSCoordinates                  : -36.6101 66.91515 119.9
[UserData]      LocationInformation             : 36.6101, 66.91515, 119.9 Role=shooting Lat=999.00000 Lon=999.00000 Alt=0.00 Body= Notes=
[Keys]          GPSCoordinates                  : -36.6101 -66.91515 119.9
[Keys]          LocationName                    : Keys:LocationName
[XMP-iptcCore]  Location                        : XMP-iptcCore:Location
[XMP-exif]      GPSAltitude                     : 119.9
[XMP-exif]      GPSAltitudeRef                  : 0
[XMP-exif]      GPSLatitude                     : 18.6101
[XMP-exif]      GPSLongitude                    : 33.91515
[XMP-exif]      GPSProcessingMethod             : MANUAL
[XMP-mediapro]  Location                        : XMP-mediapro:Location
[Composite]     GPSAltitude                     : 119.9
[Composite]     GPSAltitude                     : 119.9
[Composite]     GPSAltitude                     : 0
[Composite]     GPSAltitudeRef                  : 0
[Composite]     GPSAltitudeRef                  : 0
[Composite]     GPSLatitude                     : -36.6101
[Composite]     GPSLatitude                     : 999.00000
[Composite]     GPSLongitude                    : -66.91515
[Composite]     GPSLongitude                    : 999.00000
[Composite]     GPSLatitudeRef                  : N
[Composite]     GPSLongitudeRef                 : E
[Composite]     GPSPosition                     : 999.00000 999.00000

I usually leave just Keys:GPSCoordinates, and delete those UserData, ItemList, XMP location tags that my apps do not support.

When writing movie locations, I explicitly use G1 'Keys:GPSCoordinates' tag because by default plain G0 '-GPSCoordinates' goes to ItemList which none of my apps read:

exiftool -m -P -overwrite_original_in_place -Keys:GPSCoordinates='-36.6101, -66.91515, 119.9' movie.mp4
After cleaning off those other tags, and with images in the mix I sometimes take advantage of the Composite tags so the location is neatly displayed in the same manner for both:

exiftool -a -G1 -s -api LargeFileSupport=1 -GPSPosition -n -ee .
======== ./2006-0606-1206-06.mp4
[Composite]     GPSPosition                     : -36.6101 -66.91515
======== ./2006-0606-1206-06.jpg
[Composite]     GPSPosition                     : -36.6101 -66.91515

- Matti

tipa

Sorry to bother you again, just wanted to ask for some clarification if one of the suggestions I made above could possibly be implemented.

I still think it's very confusing behaviour when a composite tag returns different data after a successful write:
exiftool -GPSPosition='11.1, 11.1' video.mov ----> 1 image files updated
exiftool -GPSPosition ./video.mov -n ----> 29.0695 -82.4271

If there are no plans to fix this, do you see any potential problems if I adjusted my code to also always write to GPSCoordinates (no matter the file type)?

Thanks again, your comments are much appreciated.

StarGeek

Phil is away for a few weeks, so he won't be able to respond for a while.  But I get the impression that this is unlikely to change.
"It didn't work" isn't helpful. What was the exact command used and the output.
Read FAQ #3 and use that cmd
Please use the Code button for exiftool output

Please include your OS/Exiftool version/filetype

Phil Harvey

Yes, sorry that this is confusing, but as I said I don't see a clean way to fix this.  (Actually, it is worse than I mentioned.  GPSCoordinates may be written in 2 different ways as UserData, as well as existing in Keys, and for some reason Google Photos writes this in the ItemList tags. :( )

Quote from: tipa on August 24, 2023, 02:20:32 AMdo you see any potential problems if I adjusted my code to also always write to GPSCoordinates (no matter the file type)?

GPSCoordinates may be written only to QuickTime-based file formats.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).