Nikon now using little endian for maker note

[camerabits]

Phil,

It appears that Nikon now uses little endian for formatting the maker note (and Exif wrapper).  We've been hardcoding big endian for some things like the FileInfo structure and ISO expansion (as it appears you also do).  But it looks like the D7100 is Intel (also I think the D5200 as well).  So you will probably want to change your code to follow the endian logic of the maker note.

Regards,

--dennis

[camerabits]

Phil,

Hmmm - it looks like Nikon Capture NX 2 still writes some of these fields (e.g. FileInfo and ISO expansion) using a big endian format even though CNX2 rewrites saved NEF files as little endian.  So this gets messy.  I guess one could look at the "software" tag and if it starts with "Capture NX 2" then stick with big endian regardless of the fact the NEF wrapper says little endian.  Or one could put some logic in to swap bytes until a reasonable value pops out (not always possible for everything).

Thoughts?

--dennis

[Phil Harvey]

Hi Dennis,

Thanks.  You're right, this affects the D7100 and D5200.  Luckily it only affects a few tags in the FileInfo and ISOInfo structures (please let me know if you find any others).

I think the only thing to do for now is to hard-code these structures to little-endian for these two models (based on model name).  If would be great if there was a more general way to do this that would be forward compatible with future versions, but I don't see it right now.

The byte order of these structures is hard-coded specifically because of the liberties that the Nikon software takes with the maker notes, as you have discovered

- Phil

[camerabits]

I think the only thing to do for now is to hard-code these structures to little-endian for these two models (based on model name).  If would be great if there was a more general way to do this that would be forward compatible with future versions, but I don't see it right now.

Phil,

Yeah Nikon owns the format so they own their maker note!

Well we don't update Photo Mechanic as often as you update ExifTool so I'm a little hesitant to start white-listing models.  Also, I haven't tried loading a D7100 NEF into CNX2 to see if it indeed turns this data back into big endian when it saves a NEF out.  If it does then you wouldn't be able to hard-code based on the model alone.

I think for these two structures I am going to write some auto-sense code.  For the FileInfo, the directory should be a number starting at 100 and (I hope) only one byte.  So based on the directory number (which PM doesn't use) I could figure out how to byte-swap the file number (because otherwise it isn't obvious other than it should be less than 10000, right?).

For the ISO expansion, the base ISO multiplier is just a byte so that isn't an issue.  But if you want the multiplier or "expansion string" (e.g. Hi 1.3) then you need the right byte order for the 16-bit ISOExpansion.  Fortunately there are a limited set of values so I will just swap until I get a value like 0x10m or 0x20n.  Not pretty but it should work.

The other option as I mentioned before is to look at the software tag and if Capture NX 2 then assume big endian.  But that may be wrong as well if for some reason CNX2 does NOT convert a D7100 NEF to big endian.

So for these two fields I think the safest route is to auto-sense.  However, you have a whole bunch of fields you track where this may not be possible.  Fun fun...

Regards,

--dennis

[Phil Harvey]

Hi Dennis,

Well we don't update Photo Mechanic as often as you update ExifTool so I'm a little hesitant to start white-listing models.

It only affects a small number of not-very-significant tags, so getting these wrong isn't a really big deal.

Also, I haven't tried loading a D7100 NEF into CNX2 to see if it indeed turns this data back into big endian when it saves a NEF out.

I tried this with ViewNX, and it changed the byte order of the maker notes, but NOT these structures.  It does change the byte order of some structures though, and it is simply trial and error then hard-coding to determine which ones are changed and which aren't.  (It would make sense to me that only records stored as multi-byte integers would be swapped, but sadly it isn't this simple.)

I think for these two structures I am going to write some auto-sense code.  For the FileInfo, the directory should be a number starting at 100 and (I hope) only one byte.  So based on the directory number (which PM doesn't use) I could figure out how to byte-swap the file number (because otherwise it isn't obvious other than it should be less than 10000, right?).

Right.  I thought about this.  Your DirectoryNumber heuristic will work for directories numbered 100-25599, which is certainly reasonable (particularly since I'm guessing this tag has a max value of 999).

For the ISO expansion, the base ISO multiplier is just a byte so that isn't an issue.  But if you want the multiplier or "expansion string" (e.g. Hi 1.3) then you need the right byte order for the 16-bit ISOExpansion.  Fortunately there are a limited set of values so I will just swap until I get a value like 0x10m or 0x20n.  Not pretty but it should work.

This won't work.  Unfortunately numbers like 0x0102 and 0x0201 are both valid for ISOExpansion.  If this tag came after the FileInfo then you could just use the same byte order here, but unfortunately the ISOInfo comes first (at least if you parse the maker notes IFD in order, as ExifTool does).  Also, I looked at the rest of the structure hoping to find another telltale multi-byte value, but no luck.  So I have no good ideas for this one.

- Phil

[camerabits]

Also, I haven't tried loading a D7100 NEF into CNX2 to see if it indeed turns this data back into big endian when it saves a NEF out.

I tried this with ViewNX, and it changed the byte order of the maker notes, but NOT these structures.  It does change the byte order of some structures though, and it is simply trial and error then hard-coding to determine which ones are changed and which aren't.  (It would make sense to me that only records stored as multi-byte integers would be swapped, but sadly it isn't this simple.)

I'll try with CNX2 and see what it does.  So you are saying ViewNX left these as little endian for the D7100?  If I load a big endian NEF file (e.g. D800) into CNX2 it converts everything to little endian but leaves these structures as big endian.  But does this mean CNX2 always has these structures as big endian or does this mean it doesn't change the byte order for these structures?  If the latter then a D7100 should still have these structures in little endian.  Very untrustworthy.

For the ISO expansion, the base ISO multiplier is just a byte so that isn't an issue.  But if you want the multiplier or "expansion string" (e.g. Hi 1.3) then you need the right byte order for the 16-bit ISOExpansion.  Fortunately there are a limited set of values so I will just swap until I get a value like 0x10m or 0x20n.  Not pretty but it should work.

This won't work.  Unfortunately numbers like 0x0102 and 0x0201 are both valid for ISOExpansion.  If this tag came after the FileInfo then you could just use the same byte order here, but unfortunately the ISOInfo comes first (at least if you parse the maker notes IFD in order, as ExifTool does).  Also, I looked at the rest of the structure hoping to find another telltale multi-byte value, but no luck.  So I have no good ideas for this one.

Yeah I ran into this when trying to code.  This is the only ambiguous case.  Fortunately we know what the base ISO is (as it is calculated by a single byte) and so one could say that if the ISOExpansion is 0x102 (Hi 0.5) then the base ISO should be greater than something like 400; conversely, if the ISOExpansion is 0x201 (Lo 0.3) then the base ISO should be less than something like 400.  So if these tests fail then we need to swap bytes.  All the other cases we need to have MSB be 1 or 2 and if this isn't true then we need to swap (after checking).

Messy but it should work...

--dennis

[Phil Harvey]

Hi Dennis,

So you are saying ViewNX left these as little endian for the D7100?

Yes.  I am using ViewNX on OS X, and it changed the little-endian D7100 maker notes to big endian.  (Weird, I know.)   But these structures were not changed.  I did a detailed comparison of what changed, and aside from all of the multi-byte formats, there were a few undefined-format structures that were modified: ColorBalance, AFInfo2, PictureControl and UnknownInfo.  Other undefined structures were either not changed, or the change just swapped zeroed bytes so I didn't notice (like ISOExpansion for instance, which I didn't notice because ISO expansion was not used in the sample I tested).  Also, UnknownInfo2 had one byte changed from 1 to 0, and ExposureTuning had an extra null byte added.  And one more change: all of the rational values throughout were reduced to the lowest common denominator.

If I load a big endian NEF file (e.g. D800) into CNX2 it converts everything to little endian but leaves these structures as big endian.  But does this mean CNX2 always has these structures as big endian or does this mean it doesn't change the byte order for these structures?

I think that it doesn't change the byte order of these.

If the latter then a D7100 should still have these structures in little endian.  Very untrustworthy.

Yes, this is what happens.  And I agree.

Fortunately we know what the base ISO is (as it is calculated by a single byte) and so one could say that if the ISOExpansion is 0x102 (Hi 0.5) then the base ISO should be greater than something like 400; conversely, if the ISOExpansion is 0x201 (Lo 0.3) then the base ISO should be less than something like 400.  So if these tests fail then we need to swap bytes.  All the other cases we need to have MSB be 1 or 2 and if this isn't true then we need to swap (after checking).

Messy but it should work...

I thought of this too, but didn't like the idea so I didn't mention it.

- Phil

[camerabits]

Phil,

It looks like the ISOExpansion is always big endian.  So perhaps it is better to update ISOExpansion as two bytes rather than a 16 bit value:

High or Low: 1 byte (1 = Hi, 2 = Lo)
Variant: 1 byte (adjusts multiplier to baseISO and also variant reported e.g. 1 = 0.3, 2 = 0.5, 3=0.7, 4 = 1.0 etc)

So then we don't have to worry about endian for this record.

It looks like the FileInfo does indeed follow the endian of the maker note / raw wrapper.

(BTW - I put in code to handle auto-sense these fields just in case)

Regards,

--dennis

[Phil Harvey]

Hi Dennis,

OK, now I'm confused.  The ISOInfo was already hard-coded to big-endian.  So you're saying this wasn't a problem after all?

I located some samples with ISO expansion enabled so I could try this myself, but now for some reason I can't get ViewNX to change the byte order of this file... I'll work on this some more and post back here.

- Phil

Edit:  Ah.  I figured it out.  ViewNX changes the byte order of the maker notes to be the same as the EXIF.  In the D7100 sample I tested earlier, Windows Photo Viewer had changed the EXIF to big-endian.  I can confirm that ViewNX doesn't change the byte ordering of the ISOInfo record.  Also, I can confirm that ISOExpansion is big-endian for the D5200.  So the only problem is with the FileInfo, and you have a good solution for that.

P.S. Just FYI:  I have always tried to discourage image editors from changing the byte order.  There is no way to do this without jeopardizing the data integrity.  It was my prompting that resulted in this addition to the MWG recommendations:

Exif metadata is formatted as a TIFF stream, even in JPEG files. TIFF streams have an explicit indication of being big endian or little endian. A Changer SHOULD preserve the existing byte-order.

[Phil Harvey]

Oops.  I was just thinking some more about DirectoryNumber.  Unfortunately your idea doesn't work.  Again, there are problems for numbers like 0x0102 and 0x0201, for which both 258 and 513 are valid directory numbers.

- Phil

[camerabits]

OK, now I'm confused.  The ISOInfo was already hard-coded to big-endian.  So you're saying this wasn't a problem after all?

Phil,

What I am saying is that the ISOInfo structure is probably better described as an array of bytes rather than containing 16-bit (always) big-endian values for ISOExpansion.  The first byte of ISOExpansion can be described as Hi or Lo (1 or 2) and the second byte can be described as the variant or amount of expansion.  Since the other maker note fields like FileInfo follow the endian of the Exif wrapper but ISOExpansion doesn't indicates to me that it isn't meant to be viewed as 16-bit values.  Obviously you can hard-code these as always big-endian (i.e. leave the code alone) and you will be okay.  But it is an exception and therefore better described as individual bytes don't you think?  It made my code a lot cleaner looking at it this way.

Oops.  I was just thinking some more about DirectoryNumber.  Unfortunately your idea doesn't work.  Again, there are problems for numbers like 0x0102 and 0x0201, for which both 258 and 513 are valid directory numbers.

I was thinking that the folder would likely never go above 255 (I mean who would have 156 folders from 100-255 on their flash card?) and then I could just verify that the MSB is zero.  But since the camera (e.g. D800) does allow someone to manually select a folder up to 999 then I guess we will just have to assume that the FileInfo structure follows the endian of the wrapper, and this appears to be the case (other than Microsoft's software doing something idiotic), right?

--dennis

[Phil Harvey]

But it is an exception and therefore better described as individual bytes don't you think?  It made my code a lot cleaner looking at it this way.

The proof is in the pudding.  If it makes your code cleaner, then this is the way to do it.  But I don't think this is an exception... structures like the ShotInfo and ColorBalance, are always big-endian.

I guess we will just have to assume that the FileInfo structure follows the endian of the wrapper, and this appears to be the case (other than Microsoft's software doing something idiotic), right?

I only needed to invoke Microsoft's stupidness to coax ViewNX to change the D7100 maker note byte order.  But I suspect there are other ways you can get Nikon software to change the maker note byte order, and I think you'll wind up with the same problem then.

- Phil