News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Bad xxxxxSubdir offset for tag 0x002b error

Started by rondomatika, September 04, 2013, 04:34:05 AM

Previous topic - Next topic

rondomatika

Can someone please explain this error to me
I get it when I modify dates...
thank you

Phil Harvey

This indicates that the metadata in your image is corrupted.  Without more details I can't be more specific.

If you send me the image (mail to philharvey66 at gmail.com), I will take a look.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

thank you...
any information that i can give to make it more specific?
I would like to know, as I get the warning in thousands of files, although it seems like the operation is successful:
exiftool "-filemodifydate<datetimeoriginal" *\*.*

Phil Harvey

Aside from sending me a sample, knowing the exact warning message would be useful.  Is it a warning or an error?  Is it "[minor]" or not?

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

i understand..the DNG file is rather large, but I am sending you a link for the file (via email)
many thanks for your time (and also for developing such an amazing tool: i modified 32,000 files in few minutes!

Phil Harvey

Sorry to say, but the Adobe DNG converter has really messed up your original Ricoh GR Digital 3 DNG images.  Aside from corrupting the maker note offsets which is the reason for the error reported by ExifTool, it has also changed the byte ordering of the TIFF structure, which is a REAL NO-NO!  (Microsoft does this too.  You think they would both learn, because both are members of the Metadata Working Group which recommends "A Changer SHOULD preserve the existing byte-order".)

Anyway, read FAQ number 15 for more information about the maker notes corruption.  In your case, adding -m to the command will allow the file to be written (at the possible risk of losing some maker note information).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

Wow...i don't know what to say...
i had to delete the original DNGs as i was running out of room...(the DNG files are anywhere 30-60% smaller now and they looked OK to me when I compared them with the original DNGs.)

i wonder if this would cause more problems down the line that i am not aware of.
I tried both the -m and -F and i still get the very same error.
The files open fine in my editing software.
Am I facing a big problem now? There is no way to decompress these DNGs to their original, or?
(I realize this is another issue)

Phil Harvey

This probably isn't an issue if you only use Adobe products to develop these modified DNG's.  But it is possible that you could have problems if you use other software, particularly Ricoh software.  There is no easy way to repair the damage done by the Adobe DNG converter.

Note that this really isn't a fault of the Adobe DNG coverter, except that it probably should have warned you that some maker note information could be lost, because the MakerNoteSafety tag is explicitly set to "Unsafe" for the GR DIGITAL 3 DNG images, which indicates that it is unsafe to rewrite the maker notes.

However, ExifTool, of course, rewrites the original maker notes just fine.  But then, ExifTool is smarter than Adobe software.

Adobe has never shown much concern for loss of proprietary metadata.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

I appreciate your time and help.
Lesson learned.
but a bit too late.
I will move on.
As I said above, I tried both -m and -F and I still get the very same error, although the date change seems to work fine.
I did two operations:
-filecreatedate<datetimeoriginal
-filemodifydate<datetimeoriginal
(maybe there was a way to do both at the same time, doesn't matter as your software is amazingly fast)
anyhow, i get no errors for "filecreatedate", but only for modify. I must also add: not in all DNGs. Only the ones from last year gave this error. I believe it has to do with a firmware update of the camera.
Thanks again

Phil Harvey

Without -m, you should get a minor error and the file will not be written:

> exiftool a.dng -artist=me
Error: [minor] Bad MakerNotes offset for Ricoh_Subdir_0x002b - a.dng
    0 image files updated
    1 files weren't updated due to errors


With -m, minor errors are downgraded to warnings, allowing the file to be written:

> exiftool a.dng -artist=me -m
Warning: Bad MakerNotes offset for Ricoh_Subdir_0x002b - a.dng
    1 image files updated


- Phil

Edit:  Oh.  I just noticed you aren't writing the file anyway, since you are only modifying the filesystem information, not tags in the file itself.  So you are right, the -m isn't necessary.  I don't understand why the difference between FileModifyDate and FileCreateDate as you mentioned.
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

sorry for continuing here, but i am a bit confused:
without m, i was still able to get the date update on the converted files.
it gave me a warning but still completed the task.
does that make sense?

rondomatika


rondomatika

and you are correct:
creation date update gave me the same error....

Phil Harvey

...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

#14
because i happen to have straight out of camera jpegs of the corrupt DNGs, i tried the following:

exiftool -tagsfromfile 1.jpg -makernotes 1.dng

the old offset error is gone now, but DNG gives me a different kind of error:

Warning: Bad MakerNotes offset for FirmwareVersion - 1.dng

is this fixible? in genral, is using the jpeg tags to fix the adobe converted dng's a good idea, or am i potentially causing more damage?
thanks

edit:
i used -a parameter to list all warnings...this is not the only warning there are four others which are not displayed after regular use.
so i guess the JPEG tags are not good to fix maker notes.


Phil Harvey

Interesting idea, and it comes close to working, but ExifTool doesn't fix up the offsets properly when you do this.  I'm not sure why... give me a bit of time to look into this.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

phil, sure please take your time...
i am very proud to hear that you thought this was an interesting idea.
after all i lack the most basic computer skills, and my experience with command line programs started yesterday morning with exiftool :)
anyhow, going back to your initial analysis of the DNG file that i sent you: you wrote the following:
Aside from corrupting the maker note offsets which is the reason for the error reported by ExifTool, it has also changed the byte ordering of the TIFF structure, which is a REAL NO-NO! 
do i understand this right? even if it would be possible to fix the maker notes, the "TIFF structure" will still remain damaged, or?
thanks again

Phil Harvey

Quote from: rondomatika on September 05, 2013, 07:39:46 AM
do i understand this right? even if it would be possible to fix the maker notes, the "TIFF structure" will still remain damaged, or?

Changing the byte ordering doesn't necessarily damage the file, but the problem is that you must be very, very careful when doing this, and it is very easy to break things, especially when it comes to proprietary metadata.  So even if we fix the maker notes, we could easily still have other problems.  For example, Ricoh software may expect the maker notes to be in the same byte order as the rest of the file.  But it is impossible to change the byte order of the maker notes without a complete knowledge of the data structures (which probably only Ricoh has), so it may be that even if you manage to "fix"  the maker notes, they could be unreadable by Ricoh software anyway.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Phil Harvey

#18
I looked into your idea of copying the maker notes.  Actually, the problem was that Adobe has also moved the maker notes from IFD0/ExifIFD/MakerNotes to the proprietary IFD0/DNGAdobeData, so you wind up with duplicate maker notes, including the old Adobe version with all of its problems.  And unfortunately ExifTool doesn't currently have the ability to add or delete the DNGAdobeData.

Just for fun, I will add the ability to do this in ExifTool 9.36, and with this version you will be able to use this command to try to recover the maker notes from a JPEG:

exiftool -tagsfromfile test.jpg -makernotes -dngadobedata= test.dng

However, there are other changes that the Adobe DNG converter made to the structure of this image, so it still will be different from the original Ricoh DNG.  So I don't see any real benefit to doing this to all of your images, and I don't think I would recommend it since it may now lead to and Adobe incompatibility.

(btw, this is the reason that I use the PEF image format instead of DNG for my Pentax cameras -- the Adobe utilities modify the DNG without warning, but they don't touch the PEF.)

- Phil

...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

rondomatika

Phil, thank you, very interesting.
I may also try this "just for fun", with the new version. But I got your message: sounds like a bad idea to use it for global "fix" of files. I have been experimenting with a lot of image editors since you made me aware of the potential problems, and so far it seems fine.
Still very troubled by all these...
So after all, it looks like we really need a truly open source alternative to DNG. Or, a third party DNG converter that is less invasive?