XMP becomes data duplicated when writing to DNG files produced by iPhone 14

[Mac2]

A user reported that writing metadata changed in my software to the DNG files created by his new iPhone causes the written values (XMP-dc:Title, XMP-dc:Description,...) to be duplicated.
I did run some tests and I can reproduce the problem. After writing back, the DNG file contains two XMP records with duplicate data.

The DNG contains a rudimentary XMP record with only 3 tags, standard EXIF and GPS data.
You can download the original image file from here: https://1drv.ms/u/s!Atck8IqFtmUwgaZPCNP_rjCg8DiK0w?e=RghLZQ

My software writes metadata using the arguments in the attached text file (it's long-ish so I've attached it).
After the write back, tags like title, description etc. exist twice in the file.

I found that when I delete the existing XMP data in the file before writing back using:

Code Select Expand

-overwrite_original_in_place
-xmp=
c:\temp\ATEST.DNG
the duplication does not longer occur and only one XMP record is created.
I guess something in the existing XMP record, or some other data, and the way my software writes metadata confuses ExifTool for this particular DNG file.

[Phil Harvey]

The non-standard XMP is the issue.  ExifTool will update this to avoid conflicting information while creating XMP in the correct location, resulting in duplicate tags.

Code Select Expand

> exiftool IMG_9138.DNG -validate -warning -a
Validate                        : 4 Warnings (all minor)
Warning                         : [minor] Non-standard SubIFD tag 0xcd2d ProfileGainTableMap
Warning                         : [minor] Non-standard XMP at TIFF-IFD0-SubIFD1-XMP
Warning                         : [minor] Non-standard SubIFD1 tag 0xcd2e SemanticName
Warning                         : [Minor] Not decoding some large array(s). Ignore minor errors to decode
- Phil

[Mac2]

OK.
When I understand you correctly is the only way to 'fix' this is to tell the user to first wipe existing XMP data from the file.
And that this is something Apple must fix in their firmware (like, writing a standard-compliant XMP record).

[Phil Harvey]

The alternative is to leave the XMP where it is, and add the ExifTool -wm wc option to avoid creating new XMP.

- Phil

[Mac2]

I first had to look this parameter up. Leaning something new every day...

My software deals with a wide range of metadata, written over the past 20 years, by dozens of applications.
I'm reluctant to change something regarding how my software utilizes ExifTool - because it works just fine as it is.

When I understand you correctly, not creating groups in this particular case would solve this problem.
But it would probably break XMP writing for other files with existing standard-compliant XMP data or no prior XMP data?

I guess I just tell the user with this problem to wipe the XMP data after importing the files. This is easy to do and nothing of value would be lost at that time.

[Phil Harvey]

Yes.  It would work for this particular case, but break just about everything else.

It is just as well to wipe the XMP from the SubIFD because I'm sure you would run into compatibility issues with other software if you left it there.

- Phil

[Mac2]

Thanks for your support, Phil.

I won't invest any more time in this.
The user knows that deleting the partial XMP record written by this phone solves the problem and it's now working for him.

Another typical example for metadata mess