Help with corrupt D800 NEF files (How to repair file header)

Started by macadphotos, August 25, 2015, 04:06:34 AM

Previous topic - Next topic

macadphotos

Hi,
I have some D800 NEF files that I copied onto an external hard disk and when I tried to restore them after a HDD failure, the files could not be opened. When I examined the files with a hex editor, I found that the header of the corrupt files were mostly zeros. Interestingly, the rest of the file are exactly the same. (See below for the header differences) I happened to have some original files intact so I wanted to see if I could restore the file by copying a generic header. It worked if I copied the header of A.NEF to CorruptedA.NEF, but not if I copied from B.NEF to CorruptedA.nef. When using the repair tool, I got this error:

Warning: [minor] Entries in SubIFD were out of sequence. Fixed. - /Volumes/Photos/2015/2015_07_03 NY Independence/Macad_150705_DSC3566aaa.NEF
Error: Bad format (7881) for SubIFD entry 0 - /Volumes/Photos/2015/2015_07_03 NY Independence/Macad_150705_DSC3566aaa.NEF

Is there a way to repair the header using exiftools?

Thanks!

____________________________________


Phil Harvey

No, sorry.  ExifTool can not be used to repair hard errors like this in TIFF-format files (which includes most RAW formats).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

macadphotos

Hmm, it is a pity since most of the files are still intact. Thanks for the reply!

macadphotos

I figured it out. The header part of a NEF can be reused between different files as long as the subIFD tag (ID:0x14A) values are changed to reflect the location of the subIFD in each individual file.

So generate the 3 values for subIFD tag, here are my steps:

1) Copy the header from a healthy NEF file (  from MM... to ...<?xpacket begin )
2) Search for the location of the first 000800FE occurrence, which marks the start of the subIFD. Write down the location.
3) Add 120 to generate the second value, and another 228 to generate the third value.
4) Convert the 3 values to hex and replace the header with these new values instead.

The file can now be opened!

Caveats:
1) Strip offsets (ID:0x111), GPSinfo (ID:8255) and date are supposed to be unique to each individual file. While leaving them as is doesn't affect the function of the file, it should eventually be changed.
2) This only works if the header (Position 0-539) of the file is corrupted. Doesn't work for more extensive corruption.
3) I have only tried this for a D800 NEF. Will probably be different for other file types.

To be done: Need to automate the process and use exiftools to edit the fields.

Thanks for the website http://www.exiftool.org/TagNames/EXIF.html. That really helped a lot in figuring out what is what.

Phil Harvey

...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

millipede

Hello,

sorry for bumping this old thread but it really was key to fixing my problem and it got me onto the right track after a lot of research with no success.

I have a WD drive which had some Nikon D7000 and D60 (not a D90 but still a good camera!) raw NEF and JPG files on it. No backup of those files (yes, I know!) and disk started dying. I managed to get the files off the disk, but appr. 40% of the files were corrupted. Upon closer inspection (hex editor) it seems that each file's first block = first 512 bytes got wiped. For the JPG files this was easy to fix, I just copied the first 6 bytes back in from a healthy Nikon JPG and accepted that most EXIF data got wiped (after all, there were corresponding NEF files for each of these).

What was much trickier was fixing the NEF files. After looking at some healthy D7000 & D60 NEF files it seems the first 512 bytes are pretty static and some kind of TIFF header. However just copying the header from a healthy file didn't work.

After reading this thread I learned how to locate the start of the subIFD. For both, the D7000 and the D60, you need to provide the start and end offsets (120 bytes), similar to the way outlined here. Just the location within the header where to add the offsets differs on the D60 files. This fixed the files so they can be opened again.

On a roll, I wanted to fix the preview as well. You have to locate the end of the preview "FF D9" and for the D60 add two bytes count, for the D7000 18/19 byte count (not sure on what that depends, the incorrect byte count causes the preview colours to be off) and then provide that offset in the header (same location in header for both models).

As a last fix for the pedantic I copied the date and time from the EXIF data to the TIFF header.

Result: Fixed NEF including preview. I have attached screenshots showing corrupted files and offset locations in healthy file for the D7000 & the D60.

Now for scripting it to fix the other 3000 files :-)


Thanks again @Phil for an amazing tool and @macadphotos for the solution path!

Christian

Phil Harvey

Very interesting.  I hadn't seen this specific type of corruption before.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

ClayGop

Quote from: Phil Harvey on May 09, 2017, 07:14:39 AM
Very interesting.  I hadn't seen this specific type of corruption before.

- Phil

Me neither Phil, what does it mean?

MrStupendo

Same Issue here. 2023. Can someone of you buffs help me. Photographer - no experience using a HEX-editor...  :-\

Phil Harvey

1. How did your files get corrupted?

2. What output do you get if you run ExifTool on one of them?

3. If you zip one of the files, how does the size change?

4. Can you upload a sample so we could take a look

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).