Fun with MPO files and MPImage2 errors

[Indy]

Hi, Phil!

I've enjoyed exiftool for many years and have written quite a few handy scripts (which I really need to make publicly available, at some point) thanks to your work. For the past couple years, I've been fighting with an unusual issue with some MPO files. Tonight, I've decided to finally ask for assistance instead of pounded at my keys, give up, forget about the problem, and then repeat the cycle again months later.

I run a couple different scripts to quickly process the photos I take. Unfortunately, those scripts have been causing problems for eachother when it comes to MPO files. The "workaround" I settled on for the past year was to throw a "--ext mpo" to the first script and just forget about it. I think I'd like to attempt a real solution now though.

So, here are the simplified purposes (and related lines of code) from the two scripts dealing with mpo files:

SCRIPT 1:
Adjusts image timestamps, using (very basically) this -->

Code Select Expand

exiftool -AllDates+=1 $filename

SCRIPT 2:
Attempts to split an MPO file into to JPEGs like this -->

Code Select Expand

exiftool -all= --exif:all -o $right $mpo
exiftool -mpimage2 -b -w $left -if '\$mpimage2:mpimagetype==0x020002' $mpo
exiftool -mpimage2 -b -w $left -if '\$mpimage3:mpimagetype==0x020002' $mpo
(This was made possible thanks entirely to a thread in this forum a few years ago.)

Each script works great independently and I've enjoyed easily processing thousands of photos because of the effectiveness of exiftool. The errors occur when I try running a file through script 1 first and then script 2. I'll attach an MPO image which was taken today.

If, somehow, anyone is able to get this file to work through those two simplified scripts -- in order -- without errors, I will post more code from the scripts to see if it is the cause and not the above.

Thank you for looking this over.

[Phil Harvey]

Thanks for this report, but I can't reproduce this problem:

Code Select Expand

> exiftool -ver
9.23
> exiftool -AllDates+=1 20130315_154027.MPO
    1 image files updated
> exiftool -all= --exif:all -o 20130315_154027_right.jpg 20130315_154027.MPO
    1 image files created
> exiftool -mpimage2 -b -w _left.jpg -if '$mpimage2:mpimagetype==0x020002' 20130315_154027.MPO
    1 output files created

And the two output jpg images look fine.  The only change I had to make was to not escape the "$", but presumably this is because you're using a Bourne shell and I'm using a csh.

- Phil

[Indy]

That's great to hear that you were able to run those lines without errors, that tells me that there is light at the end of the tunnel.
Unfortunately, I'm still getting errors. I've tested it on multiple platforms over the years, but I forgot to mention that last night. After seeing your version number, I decided to check the version I have.

The aging server which has the scripts is running version 8.65. I'll see if I can upgrade it, but since that system can't support the next level of its OS it may be stuck at that version of exiftool too. Anyway, that would be an easy fix, so I'll keep at that. However, in the interest of testing, I decided to run the code on an available Windows system using version 9.23. Unfortunately, that system still encountered an error. I ran the same lines that you listed in your reply and received this result after the third command:

Code Select Expand

exiftool -mpimage2 -b -w _left.jpg -if '$mpimage2:mpimagetype==0x020002' 20130315_154027.MPO
    1 files failed condition
    0 output files created

Also, last night after writing my first post, I tried running the "left" command without the if to see the specific error:

Code Select Expand

Warning: Error reading MPImage2 from file - 20130315_154027.MPO

So, it seems that regardless of the version, the file is still being corrupted after the date correction. I'm attaching the MPO file (and it's _original, for good measure) that I have after running the AllDates command.

As an aside, I've tried this splitting process using other commands and utilities which rip into the MPO files binary and pull out the right and left images from the file using the binary "markers". If I remember correctly, those sometimes failed too, leading me to believe that the binary markers are corrupted when the AllDates command is run. That seemed strange to me though and I'm convinced that this should be safely possible using only exiftool. I hope your success is proof of that possibility.

Thanks again!

[Phil Harvey]

You should be able to run the new version of ExifTool on any platform.  I have never dropped support for old platforms.

Your command that failed the condition:  What shell are you using?  Do you need to escape the "$" symbol like you did before?

The error reading MPImage2 from the file you posted:  I can extract this image without errors with ExifTool 9.23

- Phil

[Indy]

I finished upgrading exiftool to 9.23 on the aging server. The scripts now work on that system. It's strange that the 9.23 version on Windows gave me an issue, but -- since I only ran it there while doing the upgrade process on the real system -- I'm not concerned about that error. Maybe I copied over a previously corrupted version of the file or something.

In any case, version 9.23 is running the commands successfully and in order. It doesn't surprise me that it ended up being something so simple, but it's also slightly embarrassing to document that I didn't try upgrading to the current version at first.

Thank you for taking the time to look into this. At the least, your testing was able to confirm that the code should work as expected and that it was something I was doing incorrectly on my system.

[Phil Harvey]

It's strange that the 9.23 version on Windows gave me an issue,

The quoting is different in the Windows cmd shell.  You need to use double quotes instead of single.

- Phil

[Indy]

Ah. Good point.
I rushed through testing it on that platform, mainly because I was just doing that while I upgraded the other system to 9.23.
I'm glad you were able to verify it so quickly on your system and I'm also glad that I was able to upgrade to the latest version.

I can confirm that the MPImage2 error does not occur on any system (that I currently have) using exiftool 9.23.

[Indy]

Here's something odd....
I have three MPO files, taken during a narrow span of time, which aren't able to get through those scripts without error. I'm attaching one of the MPO files to this reply.

To be clear, the issue is non-existent with any recent MPOs, so I still consider it resolved. At this point, I'm simply curious what could be causing these specific files to fail. They seem to be normal MPO files, yet exiftool is unable to see MPImage2 when attempted to pull the left image.

I think the files have already had their dates modified, perhaps more as well, but I can't remember what I may have attempted after taken the photos last summer.

[Phil Harvey]

This MPO image has been edited by some software that has removed the MPO image trailer.

- Phil

[Indy]

Aha! Thank you. That would explain why it only happened to those three images. I must have used some utility or an incorrect command while diagnosing my MPO scripts during that time.

What did you use to diagnose the missing image trailer? Is there a command I can run through exiftool to attempt a repair?

(Again, this is mostly an academic discussion for me as these images are non-critical and, better still, my camera creates a "backup" single-image JPEG for every MPO.)

[Phil Harvey]

I used the -htmlDump to see what was in the file.  You'll notice there is no data at all after the JPEG EOI in this file, so the image can not be recovered.  It is just gone.  You could possibly do this by accident in ExifTool if you forgot the -o option in your "exiftool -all=" command, but in this case you would have an "_original" backup file to save your ass.

- Phil

[Indy]

OK. Good to know the usefulness of -htmlDump for future diagnosis.

To summarize future readers who will likely find this thread, as I myself have benefited from many other threads:
- Make sure you're running the best/recent version of exiftool
- If using another utility at any point, make sure to keep a backup copy of your files until everything looks good. (If using exiftool for all lines, avoid using -overwrite_original until you're equally confident with the code.)
- Test your code one line at a time and in a different order (if possible) to find the specific command causing the error.

Phil, thank you so much for taking the time to look this over and creating such a great tool for the world to use.
Have a good evening!