ValueConv is not applied to digikam's XMP-video:GPSLatitude in an xmp sidecar.

[lsmith23]

Summary: In an xmp sidecar file with XMP-video:GPSLatitude tag before an XMP-exif:GPSLatitude tag, retrieving
"GPSLatitude#" does not convert to a signed decimal degrees number.  If XMP-exif:GPSLatitude is first,
then it does convert correctly.  Examples and details below.

This problem differs from FAQ #3 since exiftool is reading the GPSLatitude but is not allowing it to be formatted.

I think if exiftool uses XMP-video:GPSLatitude to return the "GPSLatitude" tag then it should allow printConv.


I've been using exiftool for many years but have recently encountered a problem after I started using digikam to create an xmp sidecar.
I'm using the latest exiftool version (12.67) and the latest digikam version (8.1.0).

Digikam created an xmp file containing a "video" section followed by an "exif" section.  Both sections contain GPS co-ordinates.

However, doing a GetValue('GPSLatitude', 'ValueConv') or running exiftool with "-n" does not convert the co-ordinates to a single numeric value (signed decimal degrees), and instead returns a string such as "62,36.15000000N".
Doing a coordFormat (FAQ #14) also does not work.

It seems that exiftool is returning XMP-video:GPSLatitude in preference to XMP-exif:GPSLatitude because the video group is the first one in the file, but then does not apply the conversion.
Using MWG does not make any difference.

Manually removing the "video" section causes the co-ordinates to be converted as expected.

Swapping the order of the groups in the file so that "video" is after "exif" also works.

I've shrunk down the xmp file to a minimum example below, but kept all the namespaces just for reference.
Note that I've put different values in GPSLatitude/GPSLongitude to determine which values are being used.

Is there any way to get the GPSLatitude converted to a numeric value on the current exiftool version?
I tried "-api IgnoreTags=XMP-video:GPSLatitude", but it seemed to cause exiftool to also ignore exif:GPSLatitude.

Thanks.


A minimum working example mwe.xmp:
(For some reason my browser is showing an unprintable character between the quote marks of the begin attribute, but there is not one there).

Code Select Expand

<?xpacket begin="�" id="W5M0MpCehiHzreSzNTczkc9d"?>
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 4.4.0-Exiv2">
 <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
  <rdf:Description rdf:about=""
    xmlns:video="http://www.video/"
    xmlns:xmpDM="http://ns.adobe.com/xmp/1.0/DynamicMedia/"
    xmlns:audio="http://www.audio/"
    xmlns:exif="http://ns.adobe.com/exif/1.0/"
    xmlns:tiff="http://ns.adobe.com/tiff/1.0/"
    xmlns:xmp="http://ns.adobe.com/xap/1.0/"
    xmlns:Iptc4xmpExt="http://iptc.org/std/Iptc4xmpExt/2008-02-29/"
    xmlns:LImage="http://ns.leiainc.com/photos/1.0/image/"
    xmlns:pdf="http://ns.adobe.com/pdf/1.3/"
    xmlns:digiKam="http://www.digikam.org/ns/1.0/"
    xmlns:MicrosoftPhoto="http://ns.microsoft.com/photo/1.0/"
    xmlns:lr="http://ns.adobe.com/lightroom/1.0/"
    xmlns:mediapro="http://ns.iview-multimedia.com/mediapro/1.0/"
    xmlns:acdsee="http://ns.acdsee.com/iptc/1.0/"
    xmlns:photoshop="h|tp://ns.adobe.com/photoshop/1.0/"
    xmlns:dc="http://purl.org/dc/elements/1.1/"
   video:GPSLatitude="62,36.15000000N"
   video:GPSLongitude="22,0.56400000W"
   video:GPSMapDatum="WGS-84"
   video:GPSVersionID="2.0.0.0"
   exif:GPSLatitude="61,36.1500000N"
   exif:GPSLongitude="21,0.5640000W"
   exif:GPSMapDatum="WGS-84"
   >
   </rdf:Description>
 </rdf:RDF>
</x:xmpmeta>
<?xpacket end="w"?>


Code Select Expand

$ exiftool -a -G1 -s  -n mwe.xmp
ExifTool]      ExifToolVersion                 : 12.67
[ExifTool]      Warning                         : [minor] Fixed incorrect URI for xmlns:MicrosoftPhoto
...
[XMP-x]         XMPToolkit                      : XMP Core 4.4.0-Exiv2
[XMP-video]     GPSLatitude                     : 62,36.15000000N
[XMP-video]     GPSLongitude                    : 22,0.56400000W
[XMP-video]     GPSMapDatum                     : WGS-84
[XMP-video]     GPSVersionID                    : 2.0.0.0
[XMP-exif]      GPSLatitude                     : 61.6025
[XMP-exif]      GPSLongitude                    : -21.0094
[XMP-exif]      GPSMapDatum                     : WGS-84
[Composite]     GPSLatitudeRef                  : N
[Composite]     GPSLongitudeRef                 : W
[Composite]     GPSPosition                     : 62,36.15000000N 22,0.56400000W
Note that XMP-video:GPSLatitude is a string, but XMP-exif:GPSLatitude is a number.

Code Select Expand

$ exiftool -G1 -s  -n -gpslatitude mwe.xmp 
[XMP-video]     GPSLatitude                     : 62,36.15000000N
Note that GPS Latitude contains comma and "N" since we've retrieved XMP-video and not applied the ValueConv.


Now, if we remove the video:GPSLatitude, etc., from the mwe and re-run:

Code Select Expand

exiftool -G0   -n mwe.xmp
[ExifTool]      ExifTool Version Number         : 12.67
[ExifTool]      Warning                         : [minor] Fixed incorrect URI for xmlns:MicrosoftPhoto
...
[XMP]           XMP Toolkit                     : XMP Core 4.4.0-Exiv2
[XMP]           GPS Latitude                    : 61.6025
[XMP]           GPS Longitude                   : -21.0094
[XMP]           GPS Map Datum                   : WGS-84
[Composite]     GPS Latitude Ref                : N
[Composite]     GPS Longitude Ref               : W
[Composite]     GPS Position                    : 61.6025 -21.0094
Note that GPS Latitude contains just a number (as expected).

[StarGeek]

Exiftool doesn't convert it because exiftool doesn't have a definition for the tag and it can't do a conversion on a tag it doesn't know about.  If you look at the Tag Groups table, group 1 (Specific Location), you will see that XMP-video isn't listed.  See also the XMP tags page.

Do you have a source for the XMP-video that lists all the tags available? A config file could be created to allow reading and writing of this tag group.

[StarGeek]

Did some searching for XMP-video and couldn't find any data at all.  I found a post of mine from a couple years ago, but that's it.  Any other result in my Google search is just separate "XMP" and "Video" on the page, nothing for "XMP-video".

[StarGeek]

(For some reason my browser is showing an unprintable character between the quote marks of the begin attribute, but there is not one there).

I've noticed this before, but never looked at it.  I was able to find this in the Adobe XMP specs

Here is an outline of an XMP Packet, showing the text of the header and trailer:

Code Select Expand

<?xpacket begin='■' id='W5M0MpCehiHzreSzNTczkc9d'?>
...Where '■' represents the Unicode "zero width non-breaking space character" (U+FEFF) used as a byte-order marker.

[StarGeek]

Here's a config file that will allow you to read/write the 4 XMP-video tags listed in your XMP.

Code Select Expand

# copy/pasted from XMP.pm
# conversions for GPS coordinates
%latConv = (
    ValueConv    => 'Image::ExifTool::GPS::ToDegrees($val, 1)',
    ValueConvInv => 'Image::ExifTool::GPS::ToDMS($self, $val, 2, "N")',
    PrintConv    => 'Image::ExifTool::GPS::ToDMS($self, $val, 1, "N")',
    PrintConvInv => 'Image::ExifTool::GPS::ToDegrees($val, 1, "lat")',
);
%longConv = (
    ValueConv    => 'Image::ExifTool::GPS::ToDegrees($val, 1)',
    ValueConvInv => 'Image::ExifTool::GPS::ToDMS($self, $val, 2, "E")',
    PrintConv    => 'Image::ExifTool::GPS::ToDMS($self, $val, 1, "E")',
    PrintConvInv => 'Image::ExifTool::GPS::ToDegrees($val, 1, "lon")',
);

# The %Image::ExifTool::UserDefined hash defines new tags to be added
# to existing tables.
%Image::ExifTool::UserDefined = (
# new XMP namespaces (eg. xxx) must be added to the Main XMP table:
'Image::ExifTool::XMP::Main' => {
# namespace definition for examples 8 to 11
video => { # <-- must be the same as the NAMESPACE prefix
SubDirectory => {
TagTable => 'Image::ExifTool::UserDefined::video',
# (see the definition of this table below)
},
},
},
);

%Image::ExifTool::UserDefined::video = (
    GROUPS => { 0 => 'XMP', 1 => 'XMP-video', 2 => 'Image' },
    NAMESPACE => { 'video' => 'http://www.video/' },
#PRIORITY => 0, # copy/pasted from XMP-exif, which says "not as reliable as actual EXIF tags"
    WRITABLE => 'string', # (default to string-type tags)
GPSVersionID    => { Groups => { 2 => 'Location' } },
    GPSLatitude     => { Groups => { 2 => 'Location' }, %latConv },
    GPSLongitude    => { Groups => { 2 => 'Location' }, %longConv },
GPSMapDatum     => { Groups => { 2 => 'Location' } },
);
#------------------------------------------------------------------------------
1;  #end

[lsmith23]

Thanks for the quick reply.

(I was typing this before I saw your config file.  It works perfectly, thanks!)


I looked yesterday about Xmp-video but couldn't find much information either.

As you say, Xmp-video isn't on the tag groups table.  I think that should mean exiftool will ignore the tag.
However, it seems exiftool ignores the namespaces in the xmp and just uses the unknown tag anyway.

In theory it could cause issues.  For example, I created a test namespace and a tag myns:GPSLatitude="Yes".
Then exiftool will return this tag when I request "GPSLatitude", even though it's not a latitude.

However, perhaps a lot of existing xmp files have bad/inconsistent namespaces and so it would cause issues if we explicitly parsed the namespaces and ignored unknown namespaces.

A compromise could be to assume the same tag names always means the same things (GPSLatitude is always a GPSLatitude).



Anyway, I took a loop in the digikam code.  It doesn't seem to define a schema.

I include the information in case it's of any use.

Code Select Expand

git clone https://invent.kde.org/graphics/digikam.git     
...
git log -p --reverse -i -SXmp.video.GPSLatitude

Commit: 7898c334def397183e41eea10d746a98b392458a
Author: Gilles Caulier <...@gmail.com>
Date:   Wed Feb 28 18:02:50 2018 +0100
    more ffmpeg tags backported to XMP

diff --git a/libs/dmetadata/dmetadata_ffmpeg.cpp b/libs/dmetadata/dmetadata_ffmpeg.cpp

That file is now in digikam/core/libs/metadataengine/dmetadata/dmetadata_video.cpp

There are a few links at the top of that page.  The second one (FFMpeg MP4 parser) has the latitude retrieval code.

It seems there are a large number of tags used.

git grep --fixed-string  --cached Xmp.video. | grep -v '=' | sed 's/.*\(Xmp.video.[^"]*\)".*/\1/'  | sort -u

Code Select Expand

Xmp.video.Album
Xmp.video.ArchivalLocation
Xmp.video.Artist
Xmp.video.AspectRatio
Xmp.video.BaseURL
Xmp.video.BitDepth
Xmp.video.Cinematographer
Xmp.video.Codec
Xmp.video.CodecDescription
Xmp.video.ColorMode
Xmp.video.ColorSpace
Xmp.video.Comment
Xmp.video.Commissioned
Xmp.video.CompatibleBrands
Xmp.video.Composer
Xmp.video.Copyright
Xmp.video.CostumeDesigner
Xmp.video.Country
Xmp.video.Cropped
Xmp.video.DateTimeDigitized
Xmp.video.DateTimeOriginal
Xmp.video.DateUTC
Xmp.video.DefaultStream
Xmp.video.Dimensions
Xmp.video.Director
Xmp.video.DistributedBy
Xmp.video.DotsPerInch
Xmp.video.Edit%1   <- number from 1 to 9
Xmp.video.EditedBy
Xmp.video.EncodedBy
Xmp.video.Encoder
Xmp.video.EndTimecode
Xmp.video.Engineer
Xmp.video.ExtendedContentDescription
Xmp.video.FileID
Xmp.video.FileName
Xmp.video.FileSize
Xmp.video.FileType
Xmp.video.FirmwareVersion
Xmp.video.Format
Xmp.video.FrameHeight
Xmp.video.FrameRate
Xmp.video.FrameSize
Xmp.video.FrameWidth
Xmp.video.GPSAltitude
Xmp.video.GPSCoordinates
Xmp.video.GPSLatitude
Xmp.video.GPSLongitude
Xmp.video.GPSMapDatum
Xmp.video.GPSVersionID
Xmp.video.Genre
Xmp.video.Grouping
Xmp.video.HandlerDescription
Xmp.video.Height
Xmp.video.ISRCCode
Xmp.video.InfoBannerImage
Xmp.video.InfoBannerURL
Xmp.video.InfoText
Xmp.video.InfoURL
Xmp.video.Information
Xmp.video.Language
Xmp.video.Length
Xmp.video.Lightness
Xmp.video.LocationInfo
Xmp.video.LogoIconURL
Xmp.video.LogoURL
Xmp.video.Lyrics
Xmp.video.MajorBrand
Xmp.video.Make
Xmp.video.MaxBitRate
Xmp.video.MediaCreateDate
Xmp.video.Medium
Xmp.video.MimeType
Xmp.video.MinorVersion
Xmp.video.Model
Xmp.video.ModificationDate
Xmp.video.MusicBy
Xmp.video.Name
Xmp.video.NumOfColors
Xmp.video.NumOfParts
Xmp.video.Organization
Xmp.video.Orientation
Xmp.video.Part
Xmp.video.PerformerKeywords
Xmp.video.Performers
Xmp.video.ProducedBy
Xmp.video.Producer
Xmp.video.Product
Xmp.video.ProductionDesigner
Xmp.video.ProductionStudio
Xmp.video.Rate
Xmp.video.Rated
Xmp.video.Rating
Xmp.video.Requirements
Xmp.video.RippedBy
Xmp.video.SchemeTitle
Xmp.video.SecondaryGenre
Xmp.video.Sharpness
Xmp.video.SoftwareVersion
Xmp.video.Source
Xmp.video.SourceForm
Xmp.video.SourceImageHeight
Xmp.video.SourceImageWidth
Xmp.video.Starring
Xmp.video.StartTimecode
Xmp.video.Statistics
Xmp.video.StreamCount
Xmp.video.SubTCodec
Xmp.video.SubTCodecInfo
Xmp.video.SubTLang
Xmp.video.Subject
Xmp.video.Subtitle
Xmp.video.TapeName
Xmp.video.Technician
Xmp.video.Title
Xmp.video.TrackCreateDate
Xmp.video.TrackNumber
Xmp.video.URL
Xmp.video.VegasVersionMajor
Xmp.video.VegasVersionMinor
Xmp.video.WatermarkURL
Xmp.video.Width
Xmp.video.WrittenBy
Xmp.video.Year
Xmp.video.duration
Xmp.video.videoFrameRate


[StarGeek]

As you say, Xmp-video isn't on the tag groups table.  I think that should mean exiftool will ignore the tag.
However, it seems exiftool ignores the namespaces in the xmp and just uses the unknown tag anyway.

This is complicated.

Exiftool will list unknown XMP tags because both the group and the name are included directly in the data. This is different than unknown EXIF and IPTC tags because neither of those groups include the name in the actual data.  This is unlikely to change because doing so is likely to break innumerable scripts that have been created over the 19 year since the exiftool has been created.

(side note, coming up on the 20th anniversary of exiftool, Nov 19th)

The PrintConv is done at the tag level.  Since there isn't a definition for the XMP-video tags, no PrintConv is done to the data.

Now, because there are so many locations where GPSLatitude/GPSLongitude can exist, the Composite:GPSPosition only looks for the tag by name, not the group it belongs to.  See its entry on the Composite tags page and compare it to the other Composite GPS tags.  Those will pull from named groups, but even then, it doesn't necessarily pull from an exact tag, as in the case of QuickTime:GPSCoordinates, which can appear in the Quicktime Keys, ItemList, and UserDate sub-groups.

However, perhaps a lot of existing xmp files have bad/inconsistent namespaces and so it would cause issues if we explicitly parsed the namespaces and ignored unknown namespaces.

These are actually very rare.  Most programs use the standard namespaces and the ones that don't *cough*ACDSee*cough* are common enough that they have either been added or a config file has been created.  This is only the second time this namespace has come up and we still don't know where it came from.  Your title suggests it's from Digikam, but searching their site doesn't reveal the name anywhere.  I'm guessing Digikam is only reporting the data, not creating it.

A compromise could be to assume the same tag names always means the same things (GPSLatitude is always a GPSLatitude).

As pointed out above, this is what is actually happening here.

There are a few links at the top of that page.  The second one (FFMpeg MP4 parser) has the latitude retrieval code.

Going off of that, I think the data is actually from exiv2.  In the exiv2 source, Line 115 in properties.cpp is

Code Select Expand

    {"http://www.video/", "video", xmpVideoInfo, N_("XMP Extended Video schema")},

The XMP-audio namespace which is in your example raw XMP is also listed there.

The full code is at line 3424.  Still no idea as to the source.

I have no idea how to search the revision history on github, but I can see that the namespace was added before March 2018, as there is a small patch to a tag in this group, but still no actual source.

[lsmith23]

Thanks for the info.

It seems to me that the namespace (and an audio namespace too) are exiv2-specific created for GSoC 2012 at:
https://dev.exiv2.org/issues/813

They probably should have used a schema namespace with a better url to make it clearer on the ownership (instead of www.video).


Anyway, I'll put the information I found here in case it's useful in the future.

So, following on from your finding the xmpVideoInfo, I searched the exiv code:

Code Select Expand

$ git clone https://github.com/Exiv2/exiv2.git
$ git log --reverse -SxmpVideoInfo -p
commit 9b131f3c612ece1b1bca79fd2045006b7be8ef8b
Author: Andreas Huggel <....@gmx.net>
Date:   Mon Aug 20 15:33:32 2012 +0000

    Merge branch 'gsoc2012'

The gsoc2012 branch does not appear to exist in the repository, maybe it was dropped when it was migrated to git.

But, searching for "digikam gsoc2012" leads to discussions:
   
https://community.kde.org/Digikam/GSoC2012
==>
https://bugs.kde.org/show_bug.cgi?id=164442
==>
https://invent.kde.org/graphics/digikam/-/commit/9c361bc4b3d4a625210c75aea67b66ab35ab7300
==>
https://invent.kde.org/graphics/digikam/-/blob/9c361bc4b3d4a625210c75aea67b66ab35ab7300/libs/dmetadata/dmetadata.cpp

So, digikam relied on exiv to implement the code.

So, searching for exiv gsoc2012 leads to: https://dev.exiv2.org/issues/813 (Video metadata support).

This linked to:

Code Select Expand

Revision 2755 (diff)
Added by Abhinav Badola over 9 years ago
[gsoc2012] #813: Created Custom Namespace Xmp.video and made all entries in riffvideo.cpp

And, alas, all those links are broken and not archived at archive.org.

So, back to exiv git to see what the person did.

Code Select Expand

git log --author Abhinav --reverse -p

commit a721d58cd23837dd6811eb3477a01b6c579a9d8f
Author: Abhinav Badola <...@gmail.com>
Date:   Sun Jul 8 06:41:58 2012 +0530

    Added support for Xmp.video and Xmp.audio namespace in metadata toolbar

...
diff --git a/libs/widgets/metadata/xmpwidget.cpp b/libs/widgets/metadata/xmpwidget.cpp
...
+    "video",           // Exiv2 Video Metadata Schema
+    "audio",           // Exiv2 Audio Metadata Schema

That file is now called: ./core/libs/widgets/metadata/exiv2/xmpwidget.cpp

It contains a list of their supported schemas and origins.  So, it seems there is also a "kipi" private xmp namespace too.

Code Select Expand

static const char* StandardXmpEntryList[] =
{

    "acdsee",          ///< Schema for ACDSee.
    "audio",           ///< Exiv2 Audio Metadata Schema
    "aux",             ///< Schema for Additional Exif Properties.
    "crs",             ///< Camera Raw schema.
    "dc",              ///< Dublin Core schema.
    "digiKam",         ///< Our Xmp schema used to store private information (see MetaEngine classes for details).
    "dwc",             ///< Qualified Dublin Core schema.
    "exif",            ///< Schema for Exif-specific Properties.
    "iptc",            ///< IPTC Core schema.
    "iptcExt",         ///< IPTC Extension schema.
    "kipi",            ///< Xmp schema used to store private information from tools.
    "lr",              ///< Adobe LightRoom schema.
    "MicrosoftPhoto",  ///< Microsoft schema.
    "MP",              ///< Microsoft Photo 1.2 schema.
    "mwg-rs",          ///< Metadata Working Group schema.
    "pdf",             ///< Adobe PDF schema.
    "photoshop",       ///< Adobe Photoshop schema.
    "plus",            ///< PLUS License Data Format Schema.
    "tiff",            ///< Schema for TIFF Properties
    "video",           ///< Exiv2 Video Metadata Schema
    "xmp",             ///< Basic schema.
    "xmpBJ",           ///< Basic Job Ticket schema.
    "xmpDM",           ///< Dynamic Media schema.
    "xmpMM",           ///< Media Management schema.
    "xmpRights",       ///< Rights Management schema.
    "xmpTPg",          ///< Paged-Text schema.

    "-1"
};