Warning: Unknown format (32) for IFDO tag 0x10f

[pixelperfect]

Hi Phil,

I have a problem where the embedded thumbnails from my camera's JPGs will not show up in XnView.  However, other software seems to have no trouble displaying the Exif thumbnails.  So, I used ExifTool to try to extract a thumb, but I am getting the message - Warning: Unknown format (32) for IFDO tag 0x10f.  

1. What does this error mean and why would my camera produce files with such an error?  
2. How might I go to correct the problem, if possible?  
3. Finally, why can most software read the thumbnail, but XnView can't?

Xnview and ExifTool seem to identify the thumbnail's properties, but, XnView still doesn't totally recognize it, apparently.  JPEGsnoop doesn't show a thumbnail section in its analysis, like other pictures do, either. The jhead program also came up with the same error as ExifTool - Nonfatal Error : '2008.11.06 (006).jpg' Illegal number format 32 for tag 010f.  Are some viewer programs just more forgiving of this type of problem?

Here is a sample file (I only renamed it) from the camera:

http://public.bay.livefilestore.com/y1p6FoVgxBnBbguYeJFcXZ9OMTHFXUoC-2uJQ5mCCJHnJOyNVttPVa1XrAoONbx2k-cvrI1Ole5W1P1fqpY4Rsc_w/2008.11.06%20(006).jpg?psid=1

Thanks for your help.

[Phil Harvey]

What make and model of camera is this?

Some software will barf and die (ie. stop processing the metadata) on this type of format error, which could easily cause the thumbnail information to be ignored.  More robust readers may be able to read past this type of problem to extract the thumbnail.

The problem is caused by a bug in your camera firmware. You might be able to fix this if there is a firmware update available for your camera.  Otherwise, you can repair the image using exiftool -- see FAQ number 20 for details.

- Phil

[pixelperfect]

Hello,

The camera is an Aiptek Pocket DV3100+ which I don't really use much anymore.  However, I have taken thousands of images with it already, so I need to be able to read the thumbs in the problem files fast, especially over USB 1.x.  Unfortunately, I don't want to make a second set of all these images, since maintaining and keeping backups of all those would be quite unwiedly.

I see that the thumbnail extracts if I use the -m option, but convincing the developer of XnView to make his parser more robust to deal with such rarities may be futile.  I liked your idea of rewriting the JPGs, and it definitely works, but I noticed that it leads to some loss of the original Exif data, which is not something I like.  So, is this format error very simple?  Can I just edit out a byte in a hex viewer to isolate the problem, or does the error compound itself later on making it too complicated?  I really would just like to fix the files themselves, even if I had to initially make two copies.  Otherwise, I think it is just easier using another viewer that can overlook this deficiency.  If it is a nonfatal error though, why should some software die anyway?

You haven't provided much of a technical description of what the bug actually is?  Does it look like an oversight, a nonconformity, or plain incompetence?  If you can describe it better, I could try reporting it to Aiptek.  I don't think there is a firmware update for the camera (unless it is kept private and must be requested).  The issue may never have arisen to necessitate one either, or they've declined.  May also be that the camera doesn't use firmware, I suppose, and updates wouldn't be possible.  It doesn't look promising though, since only one camera seems to have a patch on their site.

[Phil Harvey]

I liked your idea of rewriting the JPGs, and it definitely works, but I noticed that it leads to some loss of the original Exif data, which is not something I like.

In your case the only lost data is the erroneous IFD entry (the Make tag -- see below) that is causing the problem.

So, is this format error very simple?  Can I just edit out a byte in a hex viewer to isolate the problem, or does the error compound itself later on making it too complicated?

The exact problem is that the format code is 32 for IFD0 entry with tag ID 0x10f, which is the Make tag.  If the byte for the format is changed from 32 to 2 (the format code for ASCII), then the problem is fixed and the Make reads as "Camera".

If it is a nonfatal error though, why should some software die anyway?

It is just how the author of the software decided to handle the error condition.  At some point when you read enough invalid data all software should declare the data as corrupted and stop trying to read it (either that or spew endless warnings, which isn't very useful).

- Phil

[pixelperfect]

Thanks for the quick response.  I really appreciate you helping me out with this.

In your case the only lost data is the erroneous IFD entry (the Make tag -- see below) that is causing the problem.

Not from what I've found.  Several entries are either missing or altered.  Here is the Exif output of the two versions after applying the generic command:

exiftool -all= -tagsfromfile @ -all:all -unsafe bad.jpg

To mention a few, notice that YCbCrPositioning has changed from Co-sited to Centered, while Thumbnail YCbCrPositioning and CompressedBitsPerPixel: 2 are missing altogether.

======== 2008.11.06 (006).jpg
---- EXIF ----
ImageDescription                : Camera
Model                           : Camera
Orientation                     : Horizontal (normal)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ModifyDate                      : 2001:01:01 00:00:00
YCbCrPositioning                : Co-sited
ExposureTime                    : 1/18
FNumber                         : 2.0
ExposureProgram                 : Program AE
ISO                             : 100
ExifVersion                     : 0210
DateTimeOriginal                : 2001:01:01 00:00:00
CreateDate                      : 2001:01:01 00:00:00
ComponentsConfiguration         : YCbCr
CompressedBitsPerPixel          : 2
ExposureCompensation            : +0.0909
MaxApertureValue                : 1.7
MeteringMode                    : Center-weighted average
LightSource                     : Unknown
Flash                           : No Flash
FocalLength                     : 7.5 mm
FlashpixVersion                 : 0100
ColorSpace                      : sRGB
ExifImageWidth                  : 2048
ExifImageHeight                 : 1536
InteropIndex                    : R98 - DCF basic file (sRGB)
InteropVersion                  : 0100
FileSource                      : Digital Camera
SceneType                       : Directly photographed
Compression                     : JPEG (old-style)
Orientation                     : Horizontal (normal)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ThumbnailOffset                 : 715
ThumbnailLength                 : 12618
YCbCrPositioning                : Co-sited
======== Rebuild of 2008.11.06 (006).jpg
---- EXIF ----
ImageDescription                : Camera
Model                           : Camera
Orientation                     : Horizontal (normal)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ModifyDate                      : 2001:01:01 00:00:00
YCbCrPositioning                : Centered
ExposureTime                    : 1/18
FNumber                         : 2.0
ExposureProgram                 : Program AE
ISO                             : 100
ExifVersion                     : 0210
DateTimeOriginal                : 2001:01:01 00:00:00
CreateDate                      : 2001:01:01 00:00:00
ComponentsConfiguration         : YCbCr
ExposureCompensation            : +0.0909
MaxApertureValue                : 1.7
MeteringMode                    : Center-weighted average
LightSource                     : Unknown
Flash                           : No Flash
FocalLength                     : 7.5 mm
FlashpixVersion                 : 0100
ColorSpace                      : sRGB
ExifImageWidth                  : 2048
ExifImageHeight                 : 1536
FileSource                      : Digital Camera
SceneType                       : Directly photographed
Compression                     : JPEG (old-style)
Orientation                     : Horizontal (normal)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ThumbnailOffset                 : 618
ThumbnailLength                 : 12618

The exact problem is that the format code is 32 for IFD0 entry with tag ID 0x10f, which is the Make tag.  If the byte for the format is changed from 32 to 2 (the format code for ASCII), then the problem is fixed and the Make reads as "Camera".

OK, I fixed it.  I must confess I don't have any background knowledge pertaining to how to read Exif hex data, so I had to guess.  Changing the first 0x20 value I found to 0x02 seemed to work.  Can you recommend though a good place to learn how to systematically read the structure of the hex data?  I found a few sources of info, but it would be nice to see something with real examples that is more understandable than just the Exif or TIFF spec.

What is your choice for a command-line hex editor?  So far, I've found Hexalter, which seems to work.

Thanks again, Phil.

[Phil Harvey]

Several entries are either missing or altered.

You're right that the the unsafe IFD1 tags are not copied with this command (IFD1:YCbCrPositioning in this case).  I will think about adding them to the Unsafe shortcut.

But the IFD1:YCbCrPositioning is the only tag that isn't copied when I do my tests.  What version of exiftool are you using?  It sounds like you are using a version older than 7.70 if the other Unsafe tags aren't being copied.

And it is expected that the ThumbnailOffset will change.  See FAQ number 13 for details.

Can you recommend though a good place to learn how to systematically read the structure of the hex data?

The exiftool -htmldump option gives you a unique way to view the TIFF structure that I find very useful in understanding what is going on.  But I don't have any good suggestions for documentation other than the EXIF and TIFF standards.

What is your choice for a command-line hex editor?

I use a utility I wrote myself.

- Phil

[pixelperfect]

But the IFD1:YCbCrPositioning is the only tag that isn't copied when I do my tests.  What version of exiftool are you using?  It sounds like you are using a version older than 7.70 if the other Unsafe tags aren't being copied.

You are right!  I'm totally sorry.  I should have at least tried the latest release out, but I didn't think you'd still have those kinds of problems after so many versions, and only recently fix them between last year and now.  You spotted it; I have version 7.69.  I'm not a compulsive updater, I know, but now I have 8.29.

The exiftool -htmldump option gives you a unique way to view the TIFF structure that I find very useful in understanding what is going on.  But I don't have any good suggestions for documentation other than the EXIF and TIFF standards.

I'm so glad I asked.  The -htmldump feature is simply fantastic!  It really is the tool I needed to visualize what is going on.  It is unbelievable how much easier it is to work with the hex data now.  Thanks Phil for such a great tool. 

However, now that I see the htmldump, there are indeed some altered values in the rebuilt file.  The rational format values are not completely matching the originals.  It appears that they have rounding errors most likely due to reversing the calculations of the approximated display values.  For instance, take a look at MaxApertureValue, which is 3/2 or 1.5.  The displayed value would be 1.7 (rounded from 1.682....) after internal calculation.  But, the new file writes to MaxApertureValue: 8747/5713 or 1.531....  So, now it will read differently in programs that use more decimal places.  I can't examine all the possiblities, but several entries do this.  Less important (but still a difference, nonetheless) are the equivalent fractions that get written as irreducible ones (e.g. 2/36 becomes 1/18).  The values probably should be maintained accurately when copying.

I use a utility I wrote myself.

I don't suppose you will ever be offering it by itself or incorporating it into Exiftool?

[Phil Harvey]

However, now that I see the htmldump, there are indeed some altered values in the rebuilt file.  The rational format values are not completely matching the originals. 

By default, the print-converted values are copied.  These values have been rounded off for display.  You can maintain more precision by specifically copying the unconverted values (ie. -maxaperturevalue#), but the fractions will always be reduced.  For instance, cameras often write FNumber with 10 as a denominator, so F2.0 would be 20/10.  ExifTool will always write this as 2/1.  I understand your desire to keep the original rational values, but the current behaviour allows for much more flexibility since it automatically converts values to different formats when transferring between different types of meta information.

I use a utility I wrote myself.

I don't suppose you will ever be offering it by itself or incorporating it into Exiftool?

No.  It is a set of bare-bones command-line utilities I wrote in C and I don't want to spend the time supporting them on various platforms or adding new features.

- Phil