Difference in XML output between 11.54 and 11.55

[Mac2]

How did you deal with the "-2", "-3", etc suffixes of 11.54?   (Note that these suffixes would change depending on the order the modules were loaded, which is the reason for the change.)

Rather simple. My databases stores the data under a tag "key" made up composed from the group name, the id and the tag. To make each data value uniquely identifiable.

When I import data for a file, each value delivered by ExifTool is checked.
If the tag key (group|id|tag) exists in the database, they data linked to the existing tag. Else a new tag record is created and the data is linked to this tag.

Since ExifTool now delivers data with the key

Composite|Nikon::LensID|LensID

instead of

Composite|LensID|LensID,

my database creates a new tag, without knowing that this is just a new name for an existing tag.
Data for new files will be associated with the new tag. Data for files processed previously will remain associated to the original tag. Although both are the same, just with different ids.

I need to 'fold' the old tag and new tag together in the databases. Basically linking all data associated with "Composite|City|City" to "Composite|MWG::City|City" and all will be well.

This can be done with a simple update statement for the database.
I only need to find a way to know how the the id has changed. Or I need to make a manual "old tag key => new tag key" bridging table in the code.

I also have to deal with user presets, templates, layouts etc. which use the old tag keys, e.g. in variables. A variable like {File.MD.Composite.City.City} will no longer work. Or at least not anymore for databases migrated to the new ExifTool version. I need to communicate this to my users so they know that they have to check.

Not many users use Composite tags directly, though.

Luckily I have a "short keys" mechanism in place already which allows users to write {File.MD.city} and IMatch internally maps this to the actual tag. So I can change this in a central place.
But this does not work everywhere and I'm still figuring out how many things in my software this tag id change breaks...

[Phil Harvey]

I see.  For your purposes then it sounds like you don't care, for example, which Composite LensID tag was generated.  A quick patch for you could be to set the id to the tag name for all Composite tags when parsing the -listx or -X output.  There should only be 2 places in your code which would need to be changed.

- Phil

[Mac2]

I did ponder this. It would be an easy solution.
But then the 'real' tag ids used by ExifTool and what my software uses would no longer match. And I try to keep this layer as close to ExifTool as possible. This is better, long-term.

I will create a manual old id => new id bridging table manually from listx outputs and then write some code which uses this to migrate existing databases on first open. From then on my software will only use the new tag id throughout.

Most of the work for this is done by running

exiftool -s1 -listx -composite:all

for both versions and a diff, I guess.

I will use the results of this also to write some documentation for my users. To help them figure out if they are affected by the name changes and how fix.

[Phil Harvey]

Sounds good.

Most of the work for this is done by running

exiftool -s1 -listx -composite:all

for both versions and a diff, I guess.

Be sure to add -use MWG to this command to load the optional MWG Composite tags.

- Phil

[Mac2]

Right. Thanks for reminding me about the -MWG. I use them in my code normally.

Just to be sure: The tag with the ids
id            name
Duration-2    Duration
Duration-3    Duration

would also become AIFF::Duration-2 and AIFF::Duration-3 and similar for other tags which a numbered that way?

[Phil Harvey]

Right. Thanks for reminding me about the -MWG. I use them in my code normally.

Just to be sure: The tag with the ids
id            name
Duration-2    Duration
Duration-3    Duration

would also become AIFF::Duration-2 and AIFF::Duration-3 and similar for other tags which a numbered that way?

No.  This is what happens:

Code Select Expand

> Image-ExifTool-11.54/exiftool -listx -composite:all -s1 | grep Duration
<tag id='Duration' name='Duration' type='?' writable='false'/>
<tag id='Duration-2' name='Duration' type='?' writable='false'/>
<tag id='Duration-3' name='Duration' type='?' writable='false' g2='Audio'/>
<tag id='Duration-4' name='Duration' type='?' writable='false' g2='Video'/>
<tag id='Duration-5' name='Duration' type='?' writable='false'/>
<tag id='Duration-6' name='Duration' type='?' writable='false'/>
<tag id='Duration-7' name='Duration' type='?' writable='false'/>

> Image-ExifTool-11.55/exiftool -listx -composite:all -s1 | grep Duration
<tag id='AIFF::Duration' name='Duration' type='?' writable='false'/>
<tag id='APE::Duration' name='Duration' type='?' writable='false' g2='Audio'/>
<tag id='FLAC::Duration' name='Duration' type='?' writable='false'/>
<tag id='MPEG::Duration' name='Duration' type='?' writable='false' g2='Video'/>
<tag id='RIFF::Duration' name='Duration' type='?' writable='false'/>
<tag id='RIFF::Duration2' name='Duration' type='?' writable='false'/>
<tag id='Vorbis::Duration' name='Duration' type='?' writable='false'/>

But your guess is as good as mine as to which Duration tag from the old version maps into which one from the new version because as I said the suffix numbers depend on the order in which the modules were loaded.

- Phil

[Mac2]

Ah, that's what these numbers mean...Mhm.

I use exiftool.exe for Windows.
Is there a determined sequence in which these 'modules' are loaded?
Or does ExifTool load them in the order in which it encounters files of a specific format?
Then it would be random and no way to tell which module has created Duration-2 or GPSAltitude-2  ...

[Phil Harvey]

Or does ExifTool load them in the order in which it encounters files of a specific format?
Then it would be random and no way to tell which module has created Duration-2 or GPSAltitude-2  ...

Exactly.  That's the reason that this was fixed.

- Phil

[Phil Harvey]

Oh.  I just realized that earlier you may have been trying to point out that there are two RIFF Duration tags (in version 11.55 and later).  One of them has an ID of RIFF::Duration2, but in this case the suffix is permanent, and not generated by the loader.

- Phil

[Mac2]

Such things just happen.

I'll bridge all existing Composite tag ids to the new format in my next release.
I'll leave the dynamic ids (with the -nn suffixes) alone. I doubt that any user has used such tags suffixes for anything.
The old data will be still accessible with the original tag id (with suffix) if really needed.

Then a FAT entry in the release notes and an explicit popup message in the software. That should do it.

[Phil Harvey]

Sorry about this, but ExifTool 12.03 has another change to the Composite tag ID's which may affect you.  I've changed the "::" to "-" in all of the Composite tag ID's to make them conform more easily to group names since they are now exposed via the new family 7 groups.

- Phil

[Mac2]

Oh, no! That's bad news indeed...

I'm still getting bug reports caused by the old name changes. So many users have used Composite tags in their layouts, staistics, apps, exports, metadata templates and whatnot.
And these all break when ExifTool changes tag names. And it sometimes takes weeks or months before a user recognizes that something no longer works, and then opens a bug report.

I have a set of standard tags which I maintain in a central configuration file. Basically these tags map short names like "title" to a corresponding ExifTool tag name. Mostly XMP, but also some Composite tags.
As long as users use these standard tags, a renamed Composite tag only requires me to update the configuration file and then ship an update. No problem.
But when they use composite tags directly, things will break. And that's just what users do. I did so too, in the past. Never anticipated tag names could change. My error.

I perfectly understand that you want to 'cleanup' ExifTool or make changes to incorporate new things.

My application just allows users to do amazing things with metadata delivered by ExifTool, much more than can be done in other applications. Unfortunately, that also means that the user base may be hit hit harder by breaking changes.

Thanks for the warning. When I have a free slot I will download the 12.03 and implement all required changes in my software.

[Phil Harvey]

The tag names are not changing.  It is the tag ID's.  The users should be interacting with the tag names.

But if I understand correctly your are using the tag ID's from the -listx output in some sort of database, and so I am giving you this heads up.

- Phil

Edit.  I see you explained all this above.