Add Hierarchical Tags to Images: Tag Structure Issue

[RossTP]

Hi all,

I hope I'm not repeating an old topic, but I figured my situation is quite unique, albeit a bit of an odd question, so here goes:

I'm using ExifTool to add custom XMP tags to thousands (>500k) of images. I've managed to figure out how to achieve this using a custom config file (attached). The config file seems to add the custom tags to each image (which is great!), but when I view these images in a tag viewing software (DigiKam, in my case), the tag structure is messed up. Instead of me trying to explain how they look, I figured it would be easier to show with an image (attached).

In the attached image, you'll notice that the custom tags have been updated with some hierarchical data (e.g., Hierarchical Subject : Site|Timbavati_2013, Species|QQ, Individual|NA, Number|1, Gender|Unknown, Position|Left). This was done from Terminal (Mac OSX El Capitan) using ExifTool (using a csv to update the modified metadata).

So my question is...is the tag structure issue a result of something I did in the config file, or is it an issue with DigiKam? Is there someway that I can change the config file so that DigiKam reads it correctly, and structures the tag info correctly?

Hope someone can assist?

Thanks in advance!

Ross

[StarGeek]

If you run this command
exiftool -Sep "##" -TagsList -LastKeywordXMP -HierarchicalSubject -CatalogSets -Subject File
You'll see that the keywords are not separated by the hash marks
Output:

Code Select Expand

Tags List                       : Site/Timbavati_2013, Species/QQ, Individual/NA, Number/1, Gender/Unknown, Position/Left
Last Keyword XMP                : Site/Timbavati_2013, Species/QQ, Individual/NA, Number/1, Gender/Unknown, Position/Left
Hierarchical Subject            : Site|Timbavati_2013, Species|QQ, Individual|NA, Number|1, Gender|Unknown, Position|Left
Catalog Sets                    : Site|Timbavati_2013, Species|QQ, Individual|NA, Number|1, Gender|Unknown, Position|Left
Subject                         : Timbavati_2013, QQ, NA, 1, Unknown, Left

This means that all these tags have only a single entry in them.  Hierarchical Subject, for example, has only one entry containing only
Site|Timbavati_2013, Species|QQ, Individual|NA, Number|1, Gender|Unknown, Position|Left
not individual entries of
Site|Timbavati_2013
Species|QQ
Individual|NA
Number|1
Gender|Unknown
Position|Left

It seems to me that the list type tags are being written incorrectly.  They could be fixed by copying the tags back into the file and using the -sep option.
exiftool -sep ", " -tagsfromfile @ -TagsList -LastKeywordXMP -HierarchicalSubject -CatalogSets -Subject FileOrDir

Are you writing these tags through the Panthera config somehow?  Or are they written separately?  Or is the Panthera config just for grouping the tags for reading? 

[RossTP]

StarGeek – that worked perfectly!! Thank you so much!!

In an attempt to make things more efficient, how would I apply the separator in my current situation? Given that the code used to apply the data to each hierarchical tag is this:

Code Select Expand

exiftool -config  /DirectoryToCustomTag/customTag.cfg  -csv=/DirectoryToCSV/CamCAT.timbavati.2017.updated.CamCAT.csv /DirectoryToImageFolder/Timbavati_2017 -m -r -overwrite_original

At the moment I have to run your line of code after the above code to fix the separator issue, but it would be great to combine both processes. If possible?

To answer your question (provided I understand you correctly) – the config file simply adds the XMP tags, then the above code adds the data. I do this all within R via the system() command.

Many thanks!

[StarGeek]

Since you're adding tags from the CSV file, just add the -sep ", " to your command.  Test it out first, of course, but I think that's all you'll need to do.

[RossTP]

Hi StarGeek,

It seems my earlier jubilation was somewhat premature. Your earlier code certainly got me closer to the final product, but for some reason some tags aren't being written correctly. I wondered if you wouldn't mind having a look at two images (one made via ExifTool, and the other made via DigiKam), in an attempt to identify something that the ExifTool image is missing? Currently, two tags that were written to the ExifTool image (Species/Human/Number/1 & Species/Human/Position/Left) aren't showing up when I view them in DigiKam. I cannot figure out what this is happen, especially since Species/Human/Gender/Female is working perfectly.

Any help would be really appreciated.

Thanks in advance!

Ross

[Hayo Baan]

I had a look at your files and while they both have different tags filled with what I think are your hierarchical lists. The problem is that not all of them are properly encoded as lists in the exiftool generated version. Instead of a / as separator some also use a |, but I guess that's OK.

Code Select Expand

exiftool -a -G0:1 -sep "##" -"*subject*" -CatalogSets -LastKeywordXMP -TagsList made_via_*
======== made_via_DigKam.jpg
[XMP:XMP-lr]    Hierarchical Subject            : Species|Human##Species|Human|Position|Left##Site|Timbavati##Individual|NA##Species|Human|Number|1##Species|Human|Gender|Female
[XMP:XMP-dc]    Subject                         : Human##Left##Timbavati##NA##1##Female
[XMP:XMP-mediapro] Catalog Sets                 : Species|Human##Species|Human|Position|Left##Site|Timbavati##Individual|NA##Species|Human|Number|1##Species|Human|Gender|Female
[XMP:XMP-microsoft] Last Keyword XMP            : Species/Human##Species/Human/Position/Left##Site/Timbavati##Individual/NA##Species/Human/Number/1##Species/Human/Gender/Female
[XMP:XMP-digiKam] Tags List                     : Species/Human##Species/Human/Position/Left##Site/Timbavati##Individual/NA##Species/Human/Number/1##Species/Human/Gender/Female
======== made_via_exiftool.JPG
[XMP:XMP-Panthera] Hierarchical Subject         : Site|Timbavati, Species|Human, Individual|NA, Species|Human|Gender|Female, Species|Human|Number|1, Species|Human|Position|Left
[XMP:XMP-Panthera] Subject                      : Timbavati, Human, NA, 1, Female, Left
[XMP:XMP-dc]    Subject                         : Timbavati##Human##NA##1##Female##Left
[XMP:XMP-lr]    Hierarchical Subject            : Site|Timbavati##Species|Human##Individual|NA##Species|Human|Gender|Female##Species|Human|Number|1##Species|Human|Position|Left
[IPTC]          Catalog Sets                    : Site|Timbavati##Species|Human##Individual|NA##Species|Human|Gender|Female##Species|Human|Number|1##Species|Human|Position|Left
[XMP:XMP-microsoft] Last Keyword XMP            : Site/Timbavati##Species/Human##Individual/NA##Species/Human/Gender/Female##Species/Human/Number/1##Species/Human/Position/Left
[XMP:XMP-Panthera] Last Keyword XMP             : Site/Timbavati, Species/Human, Individual/NA, Species/Human/Gender/Female, Species/Human/Number/1, Species/Human/Position/Left
[XMP:XMP-Panthera] Tags List                    : Site/Timbavati, Species/Human, Individual/NA, Species/Human/Gender/Female, Species/Human/Number/1, Species/Human/Position/Left
[XMP:XMP-digiKam] Tags List                     : Site/Timbavati##Species/Human##Individual/NA##Species/Human/Gender/Female##Species/Human/Number/1##Species/Human/Position/Left
    2 image files read


You should start by rewriting the tags that aren't lists (i.e. the tags without ## in the lists). However, as you can see the problem tags are all in your Panthera group. I had a look at the config file you provided, and it's not surprising they are not properly recognised as lists: they are not defined as lists... So you should fix that first. Actually, why are you creating a new group for these tags in the first place? There are already tags with the same name present (and those are already properly defined as lists too!).

[RossTP]

Hi Hayo,

Thanks very much for your response.

The reason why I'm using the "|" is because DigiKam uses this as a hierarchical delimiter (as far as I'm aware).

Now this just goes to show my naïvety, as I wasn't aware the those tags were already present on the images. I'm guessing these must be defaults? That config file took me ages (I mean weeks) to figure out 

I wouldn't know how to define the Panthera group tags as lists, but given that those tags already exist, I skipped that step and went ahead and tried to add the data using the csv. It looks like I was able to do that semi-successfully, but there's still something wrong because when I import those modified images into DigiKam it doesn't recognise the tags. I've attached a file as an example (note: when I run "exiftool -Sep "##" -TagsList -LastKeywordXMP -HierarchicalSubject -CatalogSets -Subject File" on this new image I still get one "Last Keyword XMP" without the ##, suggesting it still isn't being recognised as a list, right?

I'm at a loss here.

Perhaps I'm going about this the wrong way? All I'm trying to do is add hierarchical data (e.g., Species/Human/Number/1, Species|Human|Position|Left, etc.) using a csv to a few tags (namely the "Hierarchical Subject" tag) so that DigiKam will recognise the information. Is my approach efficient, or have I been on a wild goose chase?

Many thanks for any assistance!
Ross

[Hayo Baan]

You still have the Panthera tags in this file. Remove them with

Code Select Expand

exiftool -XMP-Panthera:all= FILEorDIR and I think you should be fine (the other tags are in proper list format).

[RossTP]

Thanks Hayo! I realised that I accidentally left in "-config" on one of my lines of code (that's why XMP-Panthera was still lurking). Deleted it now and everything is working.

Thanks again (and thanks to StarGeek also!)

Ross

[Hayo Baan]

You're welcome! Glad it's solved for you now.

By the way, making your custom tags of list type was actually quite straightforward. All you needed to do was replace the { } in their definition with { List => 'Bag' } (see the example tag NewXMPxxxTag3 (line 254) in the example.cfg file). But I think you'll agree it's better to use the standard tags to begin with. Exiftool will actually automatically create any known tag if it did not already exist. These tags will always be created at the preferred location/group so you normally never need to worry about this at all.