"Non-standard header for APP1 XMP segment"; some tags can't be modified

Started by mjs9876543210, December 23, 2017, 01:50:06 AM

Previous topic - Next topic

mjs9876543210

Background: I am trying to clear all face information from files. I've mostly been using Picasa writing to XMP tags, but I've used differing tools over the years. I would like to start over from scratch (most likely using Lightroom).

Question 1:

A large fraction of the image files I'm processing result in the following:
    Warning: Non-standard header for APP1 XMP segment - IMG_9998.JPG
and extracting tag information from those files doesn't work (and I assume that updating tags would also fail).

Is there anything simple I can do to continue working with these files?


Question 2:

For files which don't have the problem above, I'm running into another issue:

Here's a representative example of using exiftool to identify region-related XMP tags, which is how I understand faces information is typically stored. (If I'm wrong about this I'd be happy to be educated.)


$ exiftool -s -a -G1 -XMP-mwg-rs:all *.JPG
======== IMG_0001.JPG
[XMP-mwg-rs]    RegionAppliedToDimensionsW      : 3264
[XMP-mwg-rs]    RegionAppliedToDimensionsH      : 2448
[XMP-mwg-rs]    RegionAppliedToDimensionsUnit   : pixel
[XMP-mwg-rs]    RegionName                      : Mike
[XMP-mwg-rs]    RegionType                      : Face
[XMP-mwg-rs]    RegionAreaX                     : 0.625
[XMP-mwg-rs]    RegionAreaY                     : 0.372549
[XMP-mwg-rs]    RegionAreaW                     : 0.175245
[XMP-mwg-rs]    RegionAreaH                     : 0.280229
[XMP-mwg-rs]    RegionAreaUnit                  : normalized
======== IMG_0013.JPG
[XMP-mwg-rs]    RegionType                      : Face
[XMP-mwg-rs]    RegionAreaY                     : 0.693500
[XMP-mwg-rs]    RegionAreaW                     : 0.095000
[XMP-mwg-rs]    RegionAreaX                     : 0.319500
[XMP-mwg-rs]    RegionAreaH                     : 0.127000
[XMP-mwg-rs]    RegionAreaUnit                  : normalized
[XMP-mwg-rs]    RegionExtensionsAngleInfoYaw    : 0
[XMP-mwg-rs]    RegionExtensionsAngleInfoRoll   : 270
[XMP-mwg-rs]    RegionExtensionsConfidenceLevel : 1000
[XMP-mwg-rs]    RegionExtensionsTimeStamp       : 1900772309528
[XMP-mwg-rs]    RegionExtensionsFaceID          : 12
[XMP-mwg-rs]    RegionAppliedToDimensionsH      : 2448
[XMP-mwg-rs]    RegionAppliedToDimensionsW      : 3264
[XMP-mwg-rs]    RegionAppliedToDimensionsUnit   : pixel


(Part of the reason I want to start over - note that in one case the RegionName is present and in the other RegionExtensionsFaceID is used - and the latter has no name.)

My specific question is why some tags are removable and others do not appear to be.

I can remove (for example) the RegionAppliedToDimensionsH without issue:
   
$ exiftool -s -a -G1 -XMP-mwg-rs:RegionAppliedToDimensionsH= *.JPG
    2 image files updated


However I cannot remove RegionExtensionsAngleInfoYaw:

   
$ exiftool -s -a -G1 -XMP-mwg-rs:RegionExtensionsAngleInfoYaw= *.JPG
Warning: Tag 'XMP-mwg-rs:RegionExtensionsAngleInfoYaw' is not defined
Nothing to do.


I'm concerned I won't be able to truly start from a clean slate if I can't remove all of these tags.

Thank you all for any help you can provide.

~Mike


Phil Harvey

Hi Mike,

Quote from: mjs9876543210 on December 23, 2017, 01:50:06 AM
    Warning: Non-standard header for APP1 XMP segment - IMG_9998.JPG
and extracting tag information from those files doesn't work

ExifTool should extract the XMP even with this warning.  Post a sample file and I'll take a look.

QuoteHowever I cannot remove RegionExtensionsAngleInfoYaw:

A tag must be defined to be able to remove it.  Unfortunately no specific region extensions are pre-defined.  So your option is to remove the entire structure like this: -regionextensions=  See this section of the MWG tags documentation for more information.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

mjs9876543210

Thank you for getting back to me.

I realized I should have included the command line too:
    exiftool -s -a -G1 -m -XMP-mwg-rs:all IMG_9998.JPG

The file is too large to attach - you can use this link (click on the "Download" button in the upper right): https://www.dropbox.com/s/ipm4tmyssfjy3n1/IMG_9998.JPG?dl=0

I will experiment with the -regionextensions= approach.

Thanks again,
Mike

Phil Harvey

Hi Mike,

I've taken a look at your file.  ExifTool warns about the bad XMP header (apparently written by Picasa), but does extract all of the available XMP.  Only the XMP:ModifyDate and XMP:Creator exist in this file.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rumbunny12

Sorry to tag on to an old thread but I was interested in that last response.  I am having the same issue with Picasa writing a bad XMP Header.  I can read the data but I would like to fix the problem, if possible.  This will avoid issues with some other software's that I am using with the photos. 

Any advice to a complete beginner on how to do determine the exact issue and then fix it?

Thanks!

Phil Harvey

I would need a sample to test this, but try writing any XMP to see if ExifTool automatically fixes this problem.  If it doesn't, then rewriting it all should do the trick:

exiftool -xmp:all= -tagsfromfile @ -xmp:all FILE

But this will have the effect of deleting any tags not pre-defined in ExifTool.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

PersistentPacifier

Hi, I'm trying to fix this issue by using that command but it doesn't seem to fix it, after validating the file it still outputs the same message, I'm attaching one file, please take a look.

booklet-inside.jpg

As a side question, what would be the difference between doing this instead:

exiftool -xmp:all= -tagsfromfile @ -all:all FILE
Thanks.

StarGeek

Phil is away at the moment, so he won't be able to look at the file until next week.

The file has ended up with two separate copies of the XMP data.  I've had this problem before with bad software.  The first command in FAQ #20 was the only way I found to fix it, but you do lose some of the Photoshop specific tags.  It's up to you whether they are worth keeping or not.

Example of the lost Photoshop tags.  The "ICC_Profile deleted" warning can be ignored since it is copied back into the file.
C:\>exiftool -G1 -a -s -photoshop:all Y:\!temp\dd\booklet-inside.jpg
[Photoshop]     IPTCDigest                      : 00000000000000000000000000000000
[Photoshop]     XResolution                     : 200
[Photoshop]     DisplayedUnitsX                 : inches
[Photoshop]     YResolution                     : 200
[Photoshop]     DisplayedUnitsY                 : inches
[Photoshop]     PrintStyle                      : Centered
[Photoshop]     PrintPosition                   : 0 0
[Photoshop]     PrintScale                      : 1
[Photoshop]     GlobalAngle                     : 120
[Photoshop]     GlobalAltitude                  : 30
[Photoshop]     CopyrightFlag                   : False
[Photoshop]     SlicesGroupName                 : Untitled-5
[Photoshop]     NumSlices                       : 1
[Photoshop]     PixelAspectRatio                : 1
[Photoshop]     PhotoshopThumbnail              : (Binary data 5867 bytes, use -b option to extract)
[Photoshop]     HasRealMergedData               : Yes
[Photoshop]     WriterName                      : Adobe Photoshop
[Photoshop]     ReaderName                      : Adobe Photoshop CS3
[Photoshop]     PhotoshopQuality                : 12
[Photoshop]     PhotoshopFormat                 : Standard

C:\>exiftool -all= -tagsfromfile @ -all:all -unsafe -icc_profile Y:\!temp\dd\booklet-inside.jpg
Warning: Non-standard header for APP1 XMP segment - Y:/!temp/dd/booklet-inside.jpg
Warning: ICC_Profile deleted. Image colors may be affected - Y:/!temp/dd/booklet-inside.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -photoshop:all Y:\!temp\dd\booklet-inside.jpg
[Photoshop]     CopyrightFlag                   : False
[Photoshop]     GlobalAngle                     : 120
[Photoshop]     GlobalAltitude                  : 30
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

PersistentPacifier

Thank you! I was trying to avoid losing any metadata but those PS tags don't look important.