Trying to get started with ExifTool

[emccainaz]

Hi Phil,

Just a little postscript in case you are interested. I posted the file on the Photo Mechanic forum and Kirk Baker there couldn't parse the file with Photo Mechanic, Photoshop CS5 or Apple Preview. Add to that Media Pro and Bridge CS4 that I tried. I'm will be needing to be able to edit the metadata in the files with (hopefully) Photo Mechanic and Media Pro, so it would be super helpful to figure out where this problem is coming from. I surely don't have the expertise to figure out something like this, so in case you might take pity on me <grin>.

Note: I tried to attach one of the files that isn't getting parsed in hopes that you might have time to get to the bottom of the case of the disappearing metadata but got a message from Simple Machines that the jpeg didn't pass the security check for uploading! Hmm, so the files must have some kind of problem, but I am not sure how to determine what it might be.

I remain sincerely grateful for all your help with my little project.

Edward

[Phil Harvey]

Hi Edward,

This type of problem is serious.  Please mail me the original and edited images and I will look into this right away.  (philharvey66 at gmail.com).

Thanks.

- Phil

[Phil Harvey]

Thanks for the samples.

There were some problems with the original versions of these files (2 of them had an invalid XMP header, and some others had corrupted maker notes), but other than this I can't figure out what Kirk could have been talking about.  I tried preview and Photoshop CS4, and they both didn't have a problem loading all of the images after I edited them with exiftool using the CSV file you posted.

However, the information from your CSV file wasn't displayed by CS4 for these images.  This is because the IPTC in these files is not properly synchronized with the XMP to begin with.  Here, the IPTC takes precedence over the XMP because the IPTCDigest doesn't agree with the IPTC content.  Since you are writing only XMP from your CSV file, your information is not displayed.  There are a number of ways you could fix this.  Which one to use depends on how important the IPTC information is to you:

1) Delete all IPTC and the IPTCDigest when writing your XMP:

    exiftool -iptc:all= -iptcdigest= -csv=CSVFILE DIR

2) Synchronize the IPTC to the XMP after writing your XMP:

    exiftool -@ xmp2iptc.args DIR

3) Synchronize the XMP to the IPTC before writing your XMP:

    exiftool -@ iptc2xmp.args DIR

The 2nd and 3rd options use .args files which are available in the full exiftool distribution.

- Phil

[emccainaz]

Thanks for getting back to me on this. The first option should be fine, as the more critical metadata is contained in the csv file. I'll need to investigate the IPTCDigest function to understand how metadata gets synchronized.

As for the second and third options, I'm not sure what you mean by the full exiftool distribution. Is that the Perl library version that requires the developer tools for OS X?

Thanks again,

Edward

[Phil Harvey]

Hi Edward,

Yes, I mean the full Perl distribution.  You only need the developer tools to do the standard installation for this version, but the installation doesn't install the .args files anyway.  You just extract them from the archive and use them as needed.

I thought of another option which may be preferable, rather than bluntly deleting all of the IPTC:

4) Update the IPTCDigest to reflect the current IPTC content when writing your XMP:

    exiftool -iptcdigest= '-iptcdigest<currentiptcdigest' -csv=CSVFILE DIR

(the iptcdigest= was necessary to delete the IPTCDigest in the case where IPTC does not exist)

This is somewhat simpler, and has the advantage that potentially useful IPTC information is not lost, but the disadvantage that the IPTC and XMP remain unsynchronized.

Read the MWG recommendations for details about the use of the IPTCDigest.

- Phil

[Phil Harvey]

After visiting the Photo Mechanic forum, it seems there may be a real problem with the XMP here.

But I didn't see this problem with the samples you sent after I edited them with exiftool using your CSV file.

I would like to understand exactly what happened because there is a chance that you have found a bug in ExifTool.

I have the original FrostedFlatwoodsSalamander_c_MichaelGraziano_1TP.jpeg, but can't re-create this problem XMP.  Can you tell me the exact steps you took to produce the file you posted in the Photo Mechanic forum (starting from the original you sent me)?  Thanks.

- Phil

[Phil Harvey]

This may be difficult to track down...

I have done a more detailed comparison of the image you sent me and the one you posted to the PhotoMechanic forum.  Between these two versions of the image, metadata was updated not only by ExifTool, but also by Photoshop CS4 and PhotoMechanic.

- Phil

[emccainaz]

First of all, I don't know the complete history of how these files have been treated because I have "inherited" them for a project with a local nonprofit that works to protect endangered species. The files come from a variety of sources and have been stored and accessed in Microsoft Sharepoint as a sort of makeshift DAM. I received the images from a download off the server where the files have resided for a few years. In order to work with them a bit easier, I have been using Photo Mechanic as a browser. The only metadata that I can remember may have been added would be to "tag" the images using the PM tag feature for making selections. Other than that, I have also looked at them in Bridge CS4 but not edited them. I have found a backup of the image that I posted on the Photo Mechanic forum that I have not handled other than to copy it as a backup in case of some sort of failure or loss. It looks like it was last opened in November of 2009.

Note: I tried attaching that image here in case it would help you figure out what may have happened. At least it might serve as a baseline for what the file looked like before I looked at it in PM or Bridge. But, like the file I tried sending before, I got this error message: "Your attachment has failed security checks and cannot be uploaded. Please consult the forum administrator." So, apparently the file had something wrong with it when I received it. Let me know if you want me to email that file to you.

Thanks,

Edward

[Phil Harvey]

Hi Edward,

I feared that this may be difficult to reproduce.  But all is not lost, I have a theory about what could have happened:  If PhotoMechanic wrote an empty ContactInfo structure (I have never seen anything write an empty structure, but it is not outside the realm of possibility), and you subsequently wrote to that structure with ExifTool, then we would get the problem we see here.  I will add a patch in ExifTool 8.70 that will prevent this from happening.

Re: attachments in this forum.  Many people seem to have problem with attachments, but apparently it isn't a systemic SMF problem because the PhotoMechanic forum works for you and they use SMF too.  I just checked my attachment security settings and everything is turned off but one option to recode suspicious images, which I just now disabled as well.  However, I prefer samples to be mailed to me instead of posted on the board anyway.  But I have the original sample you mentioned, so there is no need to mail it.

- Phil

[emccainaz]

But if the file has something wrong with it to begin with, it wouldn't be Photo Mechanic. I'm wondering if SharePoint might have done something to it, although I don't think SharePoint has the ability to work with embedded metadata.

Thanks for looking into this.

Edward

[Phil Harvey]

Hi Edward,

I don't know if an empty structure is technically wrong.  Photoshop will read the file with an empty structure.  The problem is that ExifTool didn't know what to do with it, so it just left alone when it wrote more information.  Now comes the problem because Photoshop will not read XMP if there is both an empty and non-empty structure of the same type in a file.  It doesn't bother ExifTool because ExifTool is very tolerant of XMP formatting problems when reading.

At any rate, I have just released ExifTool 8.70 which should avoid this type of problem.   If you use this version to re-write your XMP metadata for the problem files I think this particular issue will be solved.  There is of course the other very similar issue about Photoshop ignoring XMP if the IPTCDigest is incorrect, but I think you know how to solve this too.  Interesting that we had two very different problems with an identical symptom.  This lead to a bit of confusion with my posts in the PhotoMechanic forum.

- Phil

[emccainaz]

Hi Phil,

I just ran ExifTool v8.70 and indeed it did fix the problem I had been seeing. Wow - you were fast at figuring that out and fixing it. I'm really enjoying working with ExifTool and appreciate your concern and responsiveness.

Edward