ExifTool misses an embedded preview in DNG files if they are missing extension

[shouburu]

We are leveraging exiftool to find embedded previews in raw files and extract them to be used as previews. We are using "Debian GNU/Linux 9 (stretch)" images in a with exiftool version 10.40 but I've replicated the issue with the latest exiftool on a Mac.  I've attached verbose logs for the two outputs.

exiftool -j -s -n -preview:all -orientation -filetype -imageheight -imagewidth <imageName>
{
  "SourceFile": "<imageName>.dng",
  "ThumbnailTIFF": "(Binary data 131544 bytes, use -b option to extract)",
  "Orientation": 1,
  "FileType": "DNG",
  "ImageHeight": 2954,
  "ImageWidth": 4476
}
exiftool -j -s -n -preview:all -orientation -filetype -imageheight -imagewidth <imageName>.dng
{
  "SourceFile": "<imageName>.dng",
  "PreviewImage": "(Binary data 43608 bytes, use -b option to extract)",
  "ThumbnailTIFF": "(Binary data 131544 bytes, use -b option to extract)",
  "Orientation": 1,
  "FileType": "DNG",
  "ImageHeight": 2954,
  "ImageWidth": 4476
}



It seems that without the DNG extension that exiftool was interpreting the image as a TIFF and going that route.  I was attempting to pass the "dng" extension with -ext but then exiftools expects the file to have the extension.

I was thinking of using exiftools to extract the "DNG" file type back into itself so it would treat the extensionless file as a DNG and not a TIFF so I get the line '"PreviewImage": "(Binary data 43608 bytes, use -b option to extract)"' while retaining the extensionless name.  Sorry if this is a repeat question, I searched for about an hour through the forum and 3 on documentation to look for the answer.

[Phil Harvey]

Thanks for this report.

The problem is that without an extension ExifTool doesn't know this is a DNG file until after it extracts DNGVersion, which comes after the preview image in the file.

But I have patched ExifTool to extract this preview from TIFF images as well, and the new version (11.56, just released) should do this for you.

- Phil

[shouburu]

That's awesome, thanks so much Phil! 

In the short term we are going to do some renaming of the file with this command I figured out yesterday, "exiftool '-filename<$filename.$filetypeextension' -ext . <imageNameNoExtension>" as part of the pipe.  Once your changes are in the next production release we'll take out our renaming.

Exiftool is such an awesome tool, appreciate all of your hard work!

[yetithefoot]

Thanks for the explanation.

I just dived out from a long debugging the code that operates extensionless files via exiftool. It rewrites tags in extensionless files (which are actually Sony ARW) by running:

exiftool -Compression="Uncompressed" -overwrite_original -fixBase -ignoreMinorErrors ./extensionless_file

This leads to incomplete metadata set (in compare to original file) in target file because it recognizes extensionless files as TIFFs.

Rewriting extensionless gives:

Code Select Expand


Warning: Can't convert IFD0:Compression (not in PrintConv)
======== ./extensionless
Rewriting ./extensionless...
  Editing tags in: IFD0 TIFF XMP
  FileType = TIFF
  FileTypeExtension = TIF
  MIMEType = image/tiff
  Rewriting IFD0
  Rewriting SubIFD
  Rewriting ExifIFD
  Rewriting MakerNoteSony5
  Rewriting ShotInfo
  Rewriting Tag9400c
  Rewriting Tag9401
  Rewriting Tag9402
  Rewriting Tag9403
  Rewriting Tag9404b
  Rewriting Tag9405b
  Rewriting Tag9406
  Rewriting Tag940c
  Rewriting Tag940e
  Rewriting Tag2010g
  Rewriting Tag9050a
  Rewriting InteropIFD
  Rewriting IFD1
  Copying 1 image data blocks
    1 image files updated


after adding extension

Code Select Expand


Warning: Can't convert IFD0:Compression (not in PrintConv)
======== ./extensionless.arw
Rewriting ./extensionless.arw...
  Editing tags in: IFD0 TIFF XMP
  FileType = ARW
  FileTypeExtension = ARW
  MIMEType = image/x-sony-arw
  Rewriting IFD0
  Rewriting SubIFD
  Rewriting ExifIFD
  Rewriting MakerNoteSony5
  Rewriting ShotInfo
  Rewriting Tag9400c
  Rewriting Tag9401
  Rewriting Tag9402
  Rewriting Tag9403
  Rewriting Tag9404b
  Rewriting Tag9405b
  Rewriting Tag9406
  Rewriting Tag940c
  Rewriting Tag940e
  Rewriting Tag2010g
  Rewriting Tag9050a
  Rewriting InteropIFD
  Rewriting SR2Private
  Rewriting IDC_IFD
  Deleting IDC_IFD
  Rewriting MRWInfo
  Rewriting SR2SubIFD
  Rewriting SR2DataIFD
  Rewriting MakerNotes1
  Rewriting MakerNotes2
  Rewriting MakerNotes3
  Rewriting MakerNotes4
  Rewriting MakerNotes5
  Rewriting MakerNotes6
  Rewriting MakerNotes7
  Rewriting MakerNotes8
  Rewriting MakerNotes9
  Rewriting MakerNotes10
  Rewriting MakerNotes11
  Rewriting MakerNotes12
  Rewriting IFD1
  Copying 1 image data blocks
    1 image files updated


Would you advice how to understand which tags from ARW will be wiped out in such case? Only EXIF or IPTC and XMP as well?

In my specific case adding the argument to force pass the file type (instead of TIFF) would be helpful.

Anyway, thanks. Exiftool is amazing and the only tool.

[Phil Harvey]

The problem is not writing an extensionless file.  The problem is that you are changing the Compression tag, which breaks the ARW file-type recognition:

Code Select Expand

> cp tmp/sony_a7_iii_01.arw t0
> exiftool t0 -filetype
File Type                       : ARW
> exiftool t0 -artist=me
Warning: [minor] Oversized SubIFD StripByteCounts (48674304 bytes, but expected 42590016) - t0
Warning: [minor] Entries in SubIFD were out of sequence. Fixed. - t0
    1 image files updated
> exiftool t0 -filetype
File Type                       : ARW
> exiftool t0 -compression=uncompressed
Warning: [minor] Oversized SubIFD StripByteCounts (48674304 bytes, but expected 42590016) - t0
    1 image files updated
> exiftool t0 -filetype
File Type                       : TIFF


- Phil

[yetithefoot]

Thanks Phil,

What would be your advice to inform the exiftool about correct type while writing such tags?
Would the adding extension to files solve that or it's still possible that exiftool in some cases will not recognize type even with extension?

[Phil Harvey]

The ARW file type is a bit of an anomaly because it is more difficult than other TIFF-based raw formats to distinguish from a TIFF file.

In this case, the best thing to do is make sure the file has an .arw extension.  Either that, or don't mess with the Compression tag.  ExifTool also uses the Compression tag to recognize other RAW formats, but for many of these the value is proprietary and unmistakable (see the values > 32k here).

- Phil