Hi all,
I use a netbook to store images when I am on trips. Unfortunately I did not update the freaking Nikon Transfer from version 1.5.3 to the new one on the netbook and ended up with a batch of corrupted files. Thanks to the nice little utility from Phil I could recuperated the majority of them. One thing is frustrating me, however. There are a great number of macros I shot one day, and they file sizes are 52+/-Mb (usually my file sizes are around 74+/- MB ... I shoot RAW, uncompressed). When I tried to convert/repair them, a notice came up stating "Error: This size of D800 cropped image is not yet supported"; I do not remember cropping any images while I was shooting them, so this really baffles me. Weird thing is: I can view the images in ViewNX2 from Nikon, it tells me some details of them, but I cannot export/convert/manipulate them. I am somewhat at a loss. Being not very good at editing EXIF files I was wondering if there was a quick fix for that, if any one else had encountered this?
A very frustrated Matt
Hi Matt,
Odd. What is the value of the CropHiSpeed tag for these images?
And if you want to send me a sample, I'll take a look. My email is philharvey66 at gmail.com
- Phil
Hi Phil,
using your exiftool, what I can see here is the following:
Crop Hi Speed: On (7424x4924 cropped to 6784x3788 at pixel 296,568. I have meanwhile downloaded PhotoME as well, and I am comparing the EXIF on a healthy (fixed) file with this sick one ... and under their heading "additional image data (2) is where the differences come in. The healthy file will show width in px and height in px, whereas the sick file does not have those tags and goes directly to "compression scheme", which is jpeg -old style ...
hope this helps.
I will attach a file to an email, but it is rather large ... we'll see if it goes through ...
Thanks
Would not attach to outlook .... too big, sent you a link to a file I uploaded to dropbox. Hope this works for you.
Hi Matt,
The damage to these files is in SubIFD1. Use this command to compare the files:
exiftool -subifd1:all -a FILE
I don't know why this image was cropped, but we should be able to recover the raw image. If you can reproduce the conditions under which the cropped image was generated, and send me a full set of samples with the same CropHiSpeed, then I will add support for this to the fix_corrupted_nef utility. I will need original and corrupted samples at each of the 3 compression settings and each of the 2 bit depths (for a total of 12 samples).
- Phil
Hi Phil,
all of these images I have were shot the same way, with a tele extender mounted to the macro lens, on tripod, using live view to manually focus. I wonder if Nikon automatically crops images when you magnify live view to more than 100% in order to focus. That could be an issue, but all the documentation form Nikon does not mention it. I will look through the files to see if the issue is always the same. I might not have samples of all three corrupted compression settings, will have a look. May take me a bit.
Matt
Hi Phil,
I am having trouble using the command line in the cmd window. tells me "exiftool is not a recognized as an internal or external command, operable program or batch file. I have followed your instructions to place the file on the desktop. tried it in its original format, renamed it by deleting the (-k), even tried to rename it exiftool.exe ....to no avail.
Looked through those corrupt files. They all seem to have the same thing showing: Crop Hi Speed: On (7424x4924 cropped to 6784x3788 at pixel 296,568. I don't even know how the tag HiCropSpeed is generated and what exactly it would refer to. Sorry, I am of little help and many questions ....
Regards,
Matt
Hi Matt,
Another way to see the SubIFD1 information I mentioned is to change the name of "exiftool(-k).exe" to "exiftool(-subifd1:all -a -k).exe" and drop the file on this app.
- Phil
Edit: Darn. This won't work because ":" is not allowed in Windows file names. :( So maybe better to read this post (https://exiftool.org/forum/index.php/topic,4888.0.html) for some help.
Phil,
as far as crop modes are concerned, I had a look at my camera setting. You can shoot in either FX mode (36x24), or 1.2 x (30x20), DX (24x16) of 5:4 (30x24) neither of these setting work our mathematically to the crop factor reducing the image from 7424x4924 to 6784x3788. And the file size is a given, so it will not change. What I found weird is that the image resolution height and width direction is 0 or 1095139329.5dpi ... very strange.
I'll read the link --- oops, have already done that .... if anything fails, read the instructions ... has not helped me. I used to be quite good at DOS (in the 80ies), but my mind is not that young anymore.
Hi again, Phil,
got the exiftool working ... needed to put it into the windows directory (duhh). checked three files, they are all the same, except the "other image start" shows different values. All other values are the same.
Right. The differences will be between corrupted and uncorrupted files. You can look at the tags in SubIFD1 to see if the file has been corrupted. An original NEF should have the raw data in SubIFD1, but a corrupted NEF will say that a JPEG is stored here. It is all of the missing/incorrect tags in this IFD that the fix_corrupted_nef utility repairs. The fact that there is an OtherImageStart tag in this IFD is a symptom of a corrupted NEF. And you are right, the X/YResolution tags are also messed up.
- Phil
Hi Phil,
i sure wished there was a way of copy/pasting a screenshot in here so i'll just attach it. Since this is pretty much all i can supply you (I don't think I can reproduce corrupted files simply since I got rid of the purpetrator .....) the last file on the printed screen was a healthy one. Any ideas?
Well, having one corrupted image is better than none.
If you send me a set of 6 uncorrupted images for this crop setting (12 and 14-bit depth at each of the 3 compression settings), then I will see what I can do. I won't be able to test it for the other types, but I should be able to add support for them. I actually use the corrupted OtherImageLength to identify the 12 and 14-bit uncompressed images, but I should be able to figure out what this would be for 12-bit uncompressed type.
- Phil
Phil, your efforts are highly appreciated. I can surely shoot a picture in six different settings, that should not be a problem. The trick is to figure out how this crop setting actually manifested itself. Since the size of it does not mathematically correspond with any of the crop settings the camera is capable off. So lemme go and shoot .... anything in particular? :)
Phil,
sent 8 files your way on dropbox. Link should show up in your email. there are 4 -14bit images, and 4 - 12 bit images
Matt
Hi Matt,
Great. I'll take a look at those as soon as I get a chance. I can't do it from home because I don't have a high-speed connection here, so it may have to wait until Monday unless I drop in to work on the weekend.
- Phil
no worries, i have waited long enough to start with, your help is greatly appreciated.
Hi Phil,
I see little chance for recouping the files. When I looked at the EXIF of those I sent you, they do not talk about "cropped" at all, they refer to every size as being full size. So something must have happened in the camera to crop the images at the given point. Usually, the D800 will only switch to crop mode if you enable DX format automatic cropping. Since I do not use DX lenses on this camera, it is a real mystery to me why the camera would decide to crop an image out of the blue. Nikon is no help, typically. All they did was to advise me to uninstall the outdated software, and I should not have that problem anymore - duh. Other than that they ignored me or made no effort to help at all. Makes one wonder if one is using the right equipment.
Cheers,
Matt
Hi Matt,
If those files are important to you, then I am pretty sure that we can get them fixed. It may take some trial and error though, and probably won't result in a production version of fix_corrupted_nef, but I won't give up on you.
Yes, Nikon isn't much help. As per my comment on the fix_corrupted_nef web page (https://exiftool.org/fix_corrupted_nef.html):
It is very sad that this Nikon software bug penalizes only loyal Nikon owners (since it affects only people who had an older copy of the Nikon software from a previous camera), yet Nikon customer support provides no help other than suggesting that you contact your memory card manufacturer to see if the original images can be restored.
- Phil
Unless there is a relatively quick fix, Phil, don't bother. These are macros of orchids and a night heron. In all there are 36 files ... I will have a look if there is anything else that can be established from the EXIF on those files. Seems to me, since they were relatively random in between the good files, that there may be a clue in the way I took them. I suspect it could have to do with manual focus, for one reason or another. I did use manual focus using the view screen on the back of the camera rather than the viewfinder, since one can zoom in on the focal point and then really get it much sharper than the autofocus would allow. Lemme see if adjacent healthy files were autofocus or not.
Matt
Just checked through several of the files ... there is a mixture of auto-focus/manual focus images adjacent to the corrupted files .. so no help from that point of view ... However, I found some files that had the tag HiSpeedCrop in them where the high speed crop was "Off" and the size showed 7424x4924 cropped to 7424x4924 @ pixel0,0). Some files have this tag in them, others do not ... anyway ....
Matt
OK. Well this is interesting anyway. I haven't before encountered this Nikon crop mode.
- Phil
Hi Phil,
After writing a nasty letter to Nikon they have advised me to try the new software .... Capture NX-D, which I downloaded. This software actually will let me save the images in TIFF format or JPEG. Which is one giant step forward. However, the actual size is limited to 1920x1080 pixels, a far cry from what it should be. But at least I can get at them. They are also mentioning another software from Nikon, View NX-i which I have not tried yet. I will play around with those and see what the EXIF will have to say. Will let you know.
Hi Matt,
Quote from: Matt Kubitza on April 20, 2015, 06:14:55 PM
Capture NX-D [...] size is limited to 1920x1080 pixels,
1920x1080. That's ridiculous. You would think that Nikon owners would be very upset that they bought such an expensive 2 megapixel camera.
- Phil
it is that, indeed ... when I can see that the cropped image should be at least 6784 x 3788 as per EXIF .... and with a file having a size of 53.149kb one would think one would be able to get a larger image. Btw, the metadata for this particular file says
File info 1
File size 51.9Mb
image size 1920x1080
which does not show up when I put it onto your exiftool ... to boot it says under
File Info 2
Image quality RAW (12bit) ... and i know for a fact I shot with 14 bit ... and in exiftool it says NEF bit depth :n/a (JPEG) .... sigh so three ....
Matt
Yes. In this characteristic type of corrupted image the bit depth and raw image size information is lost from SubIFD1, and the raw data is hidden as if it were an embedded JPEG.
- Phil
Quote from: Matt Kubitza on April 20, 2015, 06:14:55 PM
Hi Phil,
After writing a nasty letter to Nikon they have advised me to try the new software .... Capture NX-D, which I downloaded. This software actually will let me save the images in TIFF format or JPEG. Which is one giant step forward. However, the actual size is limited to 1920x1080 pixels, a far cry from what it should be. But at least I can get at them. They are also mentioning another software from Nikon, View NX-i which I have not tried yet. I will play around with those and see what the EXIF will have to say. Will let you know.
I'm sure the size limit is a user error as this is the first time I heard from it...
Quote from: Hayo Baan on April 21, 2015, 02:30:39 AM
I'm sure the size limit is a user error as this is the first time I heard from it...
It is likely due to the corruption then. It may be just taking one of the embedded JPEG images:
> exiftool corrupted.nef -b -otherimage | exiftool - -imagesize
Image Size : 1920x1080But in this case, I don't know why it doesn't take the larger one:
> exiftool corrupted.nef -b -jpgfromraw | exiftool - -imagesize
Image Size : 6720x3776- Phil
Hi Phil/ Hayo
Nikon has come back with this:
"Hi Matt,
Unfortunately, you may need to follow steps to recover the corrupted files. Nikon does not offer file recovery services, you can check with your memory card company for further advice. The cause of the corruption is unknown.
We apologize,"
No kidding: .... may need to follow steps to recover the corrupted files. Ha! I have told them WHY the files seem to be corrupted, more than once. The fact that I unfortunately used the wrong NIKON Transfer software (being unaware that the old one would corrupt the files) is to blame for the corrupted files, imho.
I guess Phil has a good point, why does the new Capture NX-D not use the larger file? Anyway, only 36 files or so .. so no biggie, I can live with the smaller images :)
Matt
Matt,
At least Nikon customer service is consistent (see my blue comment above). :(
But you don't have to live with the small image. ExifTool will extract the larger image:
exiftool -jpgfromraw -b -w _jfr.jpg DIR
This is the next best thing to having the raw data.
- Phil
Whooohooo, thanks, Phil, this is great. Yes, much better, indeed .. did not pull the RGB info with it, but that does not matter since I can assign this in photoshop, anyway, but a much larger image, indeed ... thanks a million, you're the best!
Matt