fix_corrupted_nef.pl : support for DX Images May Be Added if there is a demand

[PascalPradier]

hello,
I call Pascal and I lost over 3500 images with the TransfertNX software. The image format is lossless compressed RAW with a Nikon D4 12bits.
I got about 2900 pictures in FX format with your fix_corrupted_nef.pl process and I thank you infinitely. I read that "support for DX Images May Be Added if there is a demand." I made ​​you officialy request to retrieve the 600 remaining.

Thank you for your help and your work.

[Phil Harvey]

Sure.  I can add support for the DX format if you send me a set of 12 test images.  One each corrupted and uncorrupted at each of the compression settings (uncompressed, lossless compressed and compressed) for 12 and 14 bit.  My email is philharvey66 at gmail.com .  I'm away this week, so the turn around at my end won't be quite as quick as usual.

- Phil

[PascalPradier]

Thank Phil

I send you today a set of 12 wrong pictures and another one with goods pictures

[Phil Harvey]

Great.  I'll post back here after I've had a chance to look at them (next week sometime).

- Phil

[Phil Harvey]

I just took a look for them in my mail, but didn't see them.  I don't think they made it through.  Did you mail them directly?  If so, maybe try uploading them and just sending me a link.

- Phil

[PascalPradier]

Hey Phil,

My D4 is now being repaired at Nikon.
I can not you create the 12-bit DX format without transfer error.
I should wait for my D4 is back. I'll see if I can speed up things at Nikon.

I can just send you the pictures in DX format error

[Phil Harvey]

Hi Pascal,

I may be able to figure things out with just the corrupted images, but I can't find them in my mail.  (philharvey66 at gmail.com)  Perhaps try sending them a different way.

- Phil

[Phil Harvey]

I have updated fix_corrupted_nef to (hopefully) fix D4 DX images.

- Phil

[PascalPradier]

Thank you Phil, I try now and I will send you my answer about your évolution

[PascalPradier]

I have try but without success. I send you by email my corrupt pictures!

[PascalPradier]

I have try only on my Windows computer this morning from my job. I will try tonight with my Mac from my home.
I have send you a set of 12 pictures at your email adress

[PascalPradier]

error Message is:

Error : This crop size is not yet supported for the D4 - <file adress>

[Phil Harvey]

Hi Pascal,

I got the images this time, thanks.  It looks like my guess was wrong.  I'll come out with a new version of the utility as soon as I can.

- Phil

[Phil Harvey]

Hi Pascal,

I've taken a look at the images.  I still need some uncorrupted D4 DX samples to be able to implement this.  Four images should do it:  two uncompressed (one 12 bit and one 14 bit), and two compressed (one lossless 12 bit, and one lossy 14 bit).

Thanks.

- Phil

[PascalPradier]

I'll try to send one of each pictures uncompress for 12 and 14 bits formats. But for the compress format you will have wait because my camera is in the nikon shop for doing  repaired.

[PascalPradier]

Phil,
you find in your mail box 2 files Nikon D4 14bit Lossless Compressed RAW (DX &FX)

[Phil Harvey]

Thanks.  Now I only need 3 more uncorrupted samples to add support for D4 DX images:

1) 12 bit lossy compressed
2) 12 bit uncompressed
3) 14 bit compressed

- Phil

[Hayo Baan]

Hi Phil,

I just sent you 12 and 14 bit DX sample files taken with my D4. Hope this helps you solve Pascal's (and other's) corrupted files.

Cheers,
Hayo

[Phil Harvey]

Thanks Hayo!

Well, darn.  I did guess correctly the first time.

The NEF images Hayo sent match my guess at what the DX parameters should be.  The most telling one is:

Crop Hi Speed: On (4992x3292 cropped to 3264x2140 at pixel 840,576)

However, the files that Pascal is having troubles with are not this size.  They are:

Crop Hi Speed: On (4992x3292 cropped to 4160x2732 at pixel 392,280)

I don't have D4 manual handy, so I can't tell what mode this is (apparently not DX format), but I need the samples at this crop size to be able to add support for fixing Pascal's images.

- Phil

[Hayo Baan]

Hi Phil, ah, those were shot at the 1.2x crop setting. I just sent you those too. For completeness I have sent you 5:4 images as well.
(hope you have a big inbox )

Cheers,
Hayo

[Phil Harvey]

Hi Hayo,

Excellent, thanks!  I'll take a look at these as soon as I can, and add support for all of these crop sizes.

My inbox is huge, thanks.   But your zipped images are well compressed, so their size is no problem at all.

- Phil

[Phil Harvey]

Thanks to Hayo, I have the samples I needed to add support for D4 1.2x and 5:4 cropped images.  I've just released fix_corrupted_nef version 2.3, which should now support these crop sizes.

Note that some of the test files Pascal sent were transferred with a newer version of Nikon Transfer, and didn't seem corrupted.  Unfortunately, the fix_corrupted_nef utility incorrectly identifies these as corrupted NEF images (since the IFD structure has been re-arranged so that it no longer looks like an out-of-camera NEF).  So it will try to fix them, actually corrupting them in the process.  However, you always need to check your "fixed" images anyway, to be sure you selected the correct bit depth.  So if you find that the "fixed" images are corrupted for both bit depths, then the original may be one of these non-corrupted-but-NikonTransfer-modified images.

- Phil

Edit:  Just FYI:  I need the uncorrupted samples to get the uncompressed image lengths in bytes, and the StripByteCounts and RowsPerStrip for each of the different compression types.  This information is lost by the Nikon software corruption, and is needed to rebuild the raw image IFD (SubIFD1).  My earlier guess for these parameters for D4 DX images was that they were the same as for the Df (which turned out to be correct).  Since these two models are so similar, I have used Hayo's D4 1.2x and 5:4 samples to also add support for these Df crop sizes. (Before this update, the Df supported DX crop, but not 1.2x or 5:4.)  I use the corrupted samples to verify that the fix is working properly.

[PascalPradier]

Thankyou Phil for your job, I tried your new version where you have fixed the bug for 12 bits compress with DX format and it's the very best success. 
I send you my contribution by paypal. Thank you very much for all