ExifTool Forum

I have some experience with ExifTool, but I cannot get this to work.
I start with a JPG file which has no XMP data yet.
I run the following commands in an .args file:

-XMP-iptcExt:ArtworkCreator=
-XMP-iptcExt:ArtworkSource=
-XMP-iptcExt:ArtworkTitle=

-XMP-iptcExt:ArtworkCreator=C1
-XMP-iptcExt:ArtworkCreator=C2
-XMP-iptcExt:ArtworkCreator=C1

-XMP-iptcExt:ArtworkSource=S1
-XMP-iptcExt:ArtworkSource=S2
-XMP-iptcExt:ArtworkSource=S1

-XMP-iptcExt:ArtworkTitle=T1
-XMP-iptcExt:ArtworkTitle=T2
-XMP-iptcExt:ArtworkTitle=T3
Fetching the result via

exiftool -xmp-iptcext:all image.jpg
yields

Artwork Creator                 : C1, C2, C1
Artwork Source                  : S1, S2, S1
Artwork Title                   : T1, T2, T3
which looks correct. But looking at the structured XMP output

exiftool -xmp-iptcext:all -struct -X image.jpg
I get

...
 <XMP-iptcExt:ArtworkOrObject>
  <rdf:Bag>
   <rdf:li rdf:parseType='Resource'>
    <XMP-iptcExt:AOCreator>
     <rdf:Bag>
      <rdf:li>C1</rdf:li>
      <rdf:li>C2</rdf:li>
      <rdf:li>C1</rdf:li>
     </rdf:Bag>
    </XMP-iptcExt:AOCreator>
    <XMP-iptcExt:AOSource>S1</XMP-iptcExt:AOSource>
    <XMP-iptcExt:AOTitle>T1</XMP-iptcExt:AOTitle>
   </rdf:li>
   <rdf:li rdf:parseType='Resource'>
    <XMP-iptcExt:AOSource>S2</XMP-iptcExt:AOSource>
    <XMP-iptcExt:AOTitle>T2</XMP-iptcExt:AOTitle>
   </rdf:li>
   <rdf:li rdf:parseType='Resource'>
    <XMP-iptcExt:AOSource>S1</XMP-iptcExt:AOSource>
    <XMP-iptcExt:AOTitle>T3</XMP-iptcExt:AOTitle>
   </rdf:li>
  </rdf:Bag>
 </XMP-iptcExt:ArtworkOrObject>
...

or, easier to read in JSON:

[{
  "SourceFile": "C:/data/Image Tests/Faces/B/A/Artwork.jpg",
  "ArtworkOrObject": [{
    "AOCreator": ["C1","C2","C1"],
    "AOSource": "S1",
    "AOTitle": "T1"
  },{
    "AOSource": "S2",
    "AOTitle": "T2"
  },{
    "AOSource": "S1",
    "AOTitle": "T3"
  }]
}]

which does not seem correct?

When I look in Adobe Photoshop at the metadata written by ExifTool, I get the expected (wrong) result:

1.jpg

How do I create 3 separate elements, each with its own Creator, Source and Title?
Where is my mistake?

I think this falls under the "Tricky" section (https://exiftool.org/struct.html#Writing) on the Structured Information page.  And I'll admit I'm struggling to figure this out correctly.

But this works correctly I believe
C:\Programs\My_Stuff>type temp.txt
-XMP-iptcExt:ArtworkOrObject={AOCreator=C1}
-XMP-iptcExt:ArtworkOrObject={AOCreator=c2}
-XMP-iptcExt:ArtworkOrObject={AOCreator=c3}

-XMP-iptcExt:ArtworkSource=S1
-XMP-iptcExt:ArtworkSource=S2
-XMP-iptcExt:ArtworkSource=S3

-XMP-iptcExt:ArtworkTitle=T1
-XMP-iptcExt:ArtworkTitle=T2
-XMP-iptcExt:ArtworkTitle=T3

C:\Programs\My_Stuff>exiftool -P -overwrite_original  -@ temp.txt  y:\!temp\Test4.jpg
    1 image files updated

C:\Programs\My_Stuff>exiftool -G1 -a -s -xmp-iptcext:all -j -struct y:\!temp\Test4.jpg 
[{
  "SourceFile": "y:/!temp/Test4.jpg",
  "XMP-iptcExt:ArtworkOrObject": [{
    "AOCreator": ["C1"],
    "AOSource": "S1",
    "AOTitle": "T1"
  },{
    "AOCreator": ["c2"],
    "AOSource": "S2",
    "AOTitle": "T2"
  },{
    "AOCreator": ["c3"],
    "AOSource": "S3",
    "AOTitle": "T3"
  }]
}]
But of course, that probably isn't the best option for working in IMatch, as that would require trying to figure the correlation between the structured tag name and the flattened tag name.  I'm not sure there's a way to do that.

Hopefully, Phil will have a way.

Nested lists are a problem for the flattened tags.  Items are added to the lowest-level list as you have seen.  To do this properly you must write as a structure, either all at once using list notation:

-artworkorobject=[{AOCreator=C1,AOSource=S1,AOTitle=T1},{AOCreator=C2,AOSource=S2,AOTitle=T2},{AOCreator=C3,AOSource=S3,AOTitle=T3}]
or as separate items:

-artworkorobject={AOCreator=C1,AOSource=S1,AOTitle=T1}
-artworkorobject={AOCreator=C2,AOSource=S2,AOTitle=T2}
-artworkorobject={AOCreator=C3,AOSource=S3,AOTitle=T3}
- Phil

Quote from: Phil Harvey on October 27, 2022, 07:50:28 AMTo do this properly you must write as a structure

Do any of the -list options give the names of structure components such as AOCreator, AOSource, or AOTitle?

I believe, and correct me if I'm wrong, Mac2 is extracting tag lists from exiftool for IMatch using -list*.  And my quick checks of the -list options doesn't seem to have structured tags details. So basically, is there a way to programmably extract a structure's details from exiftool or do the structures have to be researched and separately coded.

Quote from: StarGeek on October 27, 2022, 10:46:48 AMDo any of the -list options give the names of structure components such as AOCreator, AOSource, or AOTitle?

The -listx option gives you this information in the "tag id", preprended by the structure name.  For example, with -listx -f:

<tag id='ArtworkOrObject' name='ArtworkOrObject' type='struct' writable='true' flags='Bag,List'>
  <desc lang='en'>Artwork Or Object</desc>
 </tag>
 <tag id='ArtworkOrObjectAOCreator' name='ArtworkCreator' type='string' writable='true' flags='Flattened,List,Seq'>
  <desc lang='en'>Artwork Creator</desc>
  <desc lang='de'>Artwork Ersteller</desc>
  <desc lang='fr'>Créateur de l&#39;Illustration</desc>
 </tag>


QuoteI believe, and correct me if I'm wrong, Mac2 is extracting tag lists from exiftool for IMatch using -list*.  And my quick checks of the -list options doesn't seem to have structured tags details. So basically, is there a way to programmably extract a structure's details from exiftool or do the structures have to be researched and separately coded.

Complex XMP structures do make this difficult.

- Phil

Thanks to both of you for providing insight and help.
#StarGeek is correct, I import the flattened tags into the database. I use -listx to retrieve information about groups and tags and store it into the database. For example, I have:

ArtworkOrObject           ArtworkOrObject (struct)
ArtworkOrObjectAOCreator  ArtworkCreator (string)
ArtworkOrObjectAOSource      ArtworkSource (string)
ArtworkOrObjectAOTitle      ArtworkTitle (lang-alt)
The database knows that ArtworkOrObject is a structure, but it does currently not know that ArtworkOrObjectAOCreator is part of a structure, or which.

When I understand Phil correctly, I could determine if and of which structure a tag belongs to by

a) finding all structure tags in the database (this currently finds 243 tags)
b) finding all tags which have an id that starts with the id of a structure =>
ArtworkOrObjectAOCreator
ArtworkOrObjectAOSource
...
c) "link" these flattened tags to the original structure tag in the database somehow

This would allow my code to know that when the user makes a modification to ArtworkOrObjectAOCreator that the output has to be written using the structure syntax if the user has provided more than one value.
So if the user changed ArtworkOrObject to  A;B;C, it would be written as
-XMP-iptcExt:ArtworkOrObject={AOCreator=A}
-XMP-iptcExt:ArtworkOrObject={AOCreator=B}
-XMP-iptcExt:ArtworkOrObject={AOCreator=C}

Quote from: Mac2 on October 29, 2022, 04:29:24 AMThe database knows that ArtworkOrObject is a structure, but it does currently not know that ArtworkOrObjectAOCreator is part of a structure

If you use the -f option with -listx, the elements of a structure will have the "Flattened" tag set.  This allows you to identify which tags are part of a structure.

QuoteI could determine if and of which structure a tag belongs to by

a) finding all structure tags in the database (this currently finds 243 tags)
b) finding all tags which have an id that starts with the id of a structure =>
ArtworkOrObjectAOCreator
ArtworkOrObjectAOSource
...
c) "link" these flattened tags to the original structure tag in the database somehow

I understand that this is not ideal.  The "Flattened" flag should help a bit here.  We could maybe think about making this easier somehow.  Would it help to add a "struct" element that points up to the containing structure (ie. "flags='Flattened,List,Seq' struct='ArtworkOrObject'")?

- Phil

Quote from: Mac2 on October 29, 2022, 04:29:24 AMa) finding all structure tags in the database (this currently finds 243 tags)
b) finding all tags which have an id that starts with the id of a structure =>
ArtworkOrObjectAOCreator
ArtworkOrObjectAOSource
...
c) "link" these flattened tags to the original structure tag in the database somehow

Unfortunately, there are structures where this falls apart, usually due to shortened flattened name tags and structures within sturctures.  The most obvious being MWG:RegionInfo, where most of the tags don't contain the name of the parent structure, which doesn't contain the name of its parent structure.

Quote from: Phil Harvey on October 29, 2022, 09:55:16 AMWe could maybe think about making this easier somehow.  Would it help to add a "struct" element that points up to the containing structure (ie. "flags='Flattened,List,Seq' struct='ArtworkOrObject'")?

I like this idea, but then I'm not the one coding for it.  I once had the thought of making a simple autohotkey gui for inputting structures, but that would had to have been hard coded.

Quote from: StarGeek on October 29, 2022, 10:22:42 AMUnfortunately, there are structures where this falls apart, usually due to shortened flattened name tags and structures within sturctures.

True the tag names may be shortened, but not the ID's.

- Phil

Quote from: Phil Harvey on October 29, 2022, 02:59:38 PMTrue the tag names may be shortened, but not the ID's.

Ah, yes, I didn't take the time to look closer at the -listx output for these. My mistake.

Quote from: Phil Harvey on October 29, 2022, 09:55:16 AMI understand that this is not ideal.  The "Flattened" flag should help a bit here.  We could maybe think about making this easier somehow.  Would it help to add a "struct" element that points up to the containing structure (ie. "flags='Flattened,List,Seq' struct='ArtworkOrObject'")?

I know about the Flattened tag. My code parses it but so far did not have a use for it.

I like your idea of structure members having a 'pointer' to their containing parent element. This would work as well for nested structures, where code just could follow the parent chain to determine all containing structures until there is no parent anymore.

This would make it easy to 'collect' all values for a structured tag from its flattened tags in the database and then dynamically producing the optimal input for ExifTool during write-back.

OK.  Will do.  The -listx -f output will look like this for ExifTool 12.50:

<tag id='ArtworkOrObject' name='ArtworkOrObject' type='struct' writable='true' flags='Bag,List'>
  <desc lang='en'>Artwork Or Object</desc>
 </tag>
 <tag id='ArtworkOrObjectAOCreator' name='ArtworkCreator' type='string' writable='true' flags='Flattened,List,Seq' struct='ArtworkOrObject'>
  <desc lang='en'>Artwork Creator</desc>
  <desc lang='de'>Artwork Ersteller</desc>
  <desc lang='fr'>Créateur de l&#39;Illustration</desc>
 </tag>
Where the "struct" attribute is the ID of the parent structure tag.  For nested structures you will have to work your way up the hierarchy as you mentioned.  Cyclical references shouldn't be a problem since ExifTool already breaks those.

Adding this "struct" attribute makes the Flattened flag redundant, but I'll have to leave it in for backward compatibility.

- Phil

That is excellent! Thank you very much.
I will use this to enhance future versions of my software.

I've always had the pending to-do for a dynamic structured tag editor (mainly for XMP/IPTC), to allow for easy input of repeatable structures. This will make this easier to implement in a generic way.

One more question:
When I use -f -t -l -listx, I get results like this:

<tag id='ArtworkOrObjectAOCreator' name='ArtworkCreator' type='string' writable='true' flags='Flattened,List,Seq'>
but when writing the structure, the tag must be named AOCreator.

XMP-iptcExt:ArtworkOrObject={AOCreator=ABC}
Where does AOCreator come from?

Is is the right way to derive it from the tag id by removing the name of the containing struct:
"ArtworkOrObjectAOCreator" => "AOCreator" ?

Quote from: Mac2 on November 02, 2022, 09:54:35 AMWhere does AOCreator come from?

Is is the right way to derive it from the tag id by removing the name of the containing struct:
"ArtworkOrObjectAOCreator" => "AOCreator" ?

Yes, I believe that's what Phil was saying above when he corrected me above in this post (https://exiftool.org/forum/index.php?topic=14089.msg75919#msg75919).  The tag ID will be the Structure name+Structure element, so as you say, remove the structure name to get the element.

Quote from: StarGeek on November 02, 2022, 10:51:27 AMYes, I believe that's what Phil was saying above when he corrected me above in this post (https://exiftool.org/forum/index.php?topic=14089.msg75919#msg75919).  The tag ID will be the Structure name+Structure element, so as you say, remove the structure name to get the element.

Yes, that's simple enough, if it is consistent and reliable.

Looking at the XML output for a file, we get

<XMP-iptcExt:ArtworkCreator>
 <rdf:Description et:id='ArtworkOrObjectAOCreator' et:table='XMP::iptcExt'>
  <et:prt>
    <rdf:Bag>
      <rdf:li>C1</rdf:li>
    </rdf:Bag>
  </et:prt>
 </rdf:Description>
</XMP-iptcExt:ArtworkCreator>
...
but the structured tag output uses

<XMP-iptcExt:AOCreator>
  <rdf:Bag>
    <rdf:li>C1</rdf:li>
   </rdf:Bag>
 </XMP-iptcExt:AOCreator>

When we have the new struct attribute for tags in a future ExifTool version (I have already implemented this in my parser and the database), we can at least tell for each flattened tag how to produce the tag name required for writing the tag by removing the parent's id from the flattened tag's id.

Really looking forward to how you implement this in IMatch.

Quote from: StarGeek on November 02, 2022, 11:20:55 AMReally looking forward to how you implement this in IMatch.

This is not something that will affect many users (It came up for Artwork recently - one user).
But I prefer to make this work in general. The current method of just separating multiple elements via ; in the MD panel failed for this structure, which is why this came up.

If I make it work for multiple entries separated via ; this will be a win.
The plan is when the user modifies one or more elements of a structured tag to always write the full tag. With the new struct attribute the code can find all tags for the structure and produce the optimal arguments for the args file. I don't think partial updates will work, according to the info on https://exiftool.org/struct.html

The next step would be a generic multi-value nested structure editor, but these are a) quite a pain to implement and b) not really needed or useful for the majority of users. This would probably be a nice to have, 'rainy afternoons for two weeks' project.

Can this be a problem?

The combination of group id and tag id is not always unique.
There are 277 cases by my counts.

For example the group DNG::AdobeData contains 91 tags with the id 'MakN'. The tags have different 'tag' Attributes, though. Exif::Main contains 91 tags with the id 37500 but different values in 'tag'.
There are also some (rare) cases where group id, tag id and tag are identical and only the index is different.

Only the combination of (group id, tag id, tag name, index) is truly unique.

Looking up a struct members parent tag only by group id and tag id (from the new struct attribute) is thus not necessarily unambiguous. I don't know yet if this affects any structured tags. Seems more like maker notes and other obscure tags.

Is it save to assume that I can always lookup a parent tag using

group: same as tag
id: value delivered by ExifTool in the struct member
tag: same as id
index: 0

For example, for tag id 'ArtworkOrObjectAOTitle' with struct = 'ArtworkOrObject', I find the tag's struct parent with

group: XMP::iptcExt
id: ArtworkOrObject
tag: ArtworkOrObject
index: 0

The non-unique entries (tags with an "index") are ones where the actual tag is different depending on context.  This does not happen for any XMP tag, and structures are only implemented for XMP.

- Phil

Thanks, Phil!

I have planned for the "struct" change introduced in the 12.50 and simulated it using the ArtworkOrObject tag set.
Today I finally could make some time to update to ExifTool 12.51 and the new struct attribute works just great.
My database now lists 243 structs with 2,375 elements.

Thank you again!
Knowing the parent tag for structures makes things a lot easier!

Great!