Fixing Corrupted Sony ARW Files

[milliman]

I have several images where I attempted to write a keyword to my Sony ARW files that are now corrupted. I'd like to use exiftool to fix them if possible. The message is

[minor] Bad format (27957) for SR2Private entry 0

from the PrintMatchingImage IFD. I know I won't be able to recover the data in that block, but is there a way I can remove the block or at least fix it so I can use the images? I've tried the traditional methods of rewriting the EXIF data but the process encounters an error on my test data image.

[StarGeek]

I have several images where I attempted to write a keyword to my Sony ARW files that are now corrupted.

Was this using exiftool? Or some other program?  If exiftool, I'm pretty sure Phil would want to know more details

I've tried the traditional methods of rewriting the EXIF data but the process encounters an error on my test data image.

It's usually not a good idea to rewrite the EXIF block on a RAW file.  See the Note: under FAQ #20.

Not much help I can provide here, but if you can make a corrupt sample available, maybe someone can take a look.

[Phil Harvey]

Yes.  What starGeek said:

1. Exactly how were the files corrupted?

2. Can you provide a sample?

- Phil

[milliman]

I used exiv2 instead of exiftool to write keywords to the IPTC block. The command used was
exiv2 -M"add Iptc.Application2.Keywords String Photos" -k -q filename.ARW

Now the ARW files are corrupted even though Preview can open the file with a green tint and the exiftool executes with it stating

Code Select Expand

[minor] Bad format (27957) for SR2Private entry 0

I'd like to recover as much of the data as possible. I could not attach a sample image due to the size but you can download it here.

Thanks guys for your assistance. I admit that I am still a of a noob concerning the data structures of the EXIF data.

[Phil Harvey]

The corruption of the maker notes is a bit surprising.  Some of the encrypted records are completely scrambled while others seem OK.  I would expect it to be all or nothing.  In theory this file should be salvageable, but the original WB information is probably lost.  The best you could do may be to copy the WB information from a good ARW file shot in similar conditions.  Unfortunately Exiftool can't be used to do this because it won't write a file with errors like this due to the possibility of further corruption.

Is this your Exiv2 issue?:  (But the title is completely unrelated to ARW corruption)

  https://github.com/Exiv2/exiv2/issues/1419

If so, I'll try to follow that issue and help if I can.

- Phil

[milliman]

Phil:

Thanks for the response on Christmas. Merry Christmas!

You found the exact issue I encountered. I should post a more detailed question against that bug on GitHub. The developer said that he does not have time to work on supporting writing to AWR files but maybe he'll have a suggestion on how to fix someone who did write to one. I originally posted a comment there but it wasn't that detailed.

I have additional examples of the WB data that can be used. Can Exiftool fix the errors or remove a corrupted IFD so I can rebuild the file? it seems smart enough to find the corruption and work around it to display as much non-corrupted data as possible. I'd be content with some data loss as long as I could recover the photo.

I should have stuck with exiftool to add the data. All I wanted to do was add a keyword after a file was detected in a directory and added to Photos. I've learned a lesson that I should always have the original RAW files around as backups.

-Mark

[Phil Harvey]

Can Exiftool fix the errors or remove a corrupted IFD so I can rebuild the file?

No.  Unfortunately, as I said, ExifTool can not be used to edit a file that is corrupted like this.

I may be able to patch ExifTool for this specific case.  What system are you running?

- Phil

[milliman]

I know that you stated that fact in your reply Phil, and I obviously have experienced it myself when I tried everything I could think of to remove the corruption in the file. I was being that kid that asks the question many times hoping that the answer would be different. 

That would be amazing if you'd be able to add that capability in a patch to exiftool. I'm on macOS 11.2 with Bash version 5.1.4. I use exiftool on Ubuntu too but very rarely. I do 99.9% of my photo processing on my Mac.

I just have to commend you on your dedication to exiftool. It has been a savior and powerful utility for many years. I should have never strayed and I don't even know why I did it other than I think I was struggling with it to add a keyword that macOS Photos would recognize at the time.

Thanks,
Mark

[Phil Harvey]

Hi Mark,

Well, bad news.  I hacked ExifTool to push past this first error and ran into other problems in the file that foiled my efforts.  The files may still be repairable but it wouldn't be an easy fix.  My time has been very limited recently so I'm not able to go any further with this right now.  Definitely push Exiv2 to at least disable ARW writing until they fix this issue -- it shouldn't have gone on so long with this serious problem.

- Phil

[milliman]

Phil:

I sincerely appreciate your efforts especially on Christmas. I didn't expect you to hack at it today! Thank you so much. I will add to my comment on GitHub that he stop treating these files like a TIFF and prevent writing to them.

Thanks so much for your efforts,
Mark