Using exiftool to list [& later rename] photos with DNG Lossy vs DNG NonLossy?

[clem]

As I'm cleaning up my photo folders, I have realized that I have been inconsistent in my file creation routines. So the result is that I converted camera raw files from their native format to DNG and sometimes to DNG Lossy. How can I go about generating a recursive list of files that are created with DNG Lossy IFF there is a corresponding DNG file (with the same exposure data (f-stop/shutter-speed/ISO/date/lens) in the same directory? (I'm comfortable with using 'terminal' in OSX)?

Thanks for any pointers

[Phil Harvey]

You could try something like this:

exiftool -p "$directory/$filename" -if "$ATAG =~ /ASTRING/" -ext EXT -srcfile %d%f.DNG -r DIR

where ATAG is the name of a tag that allows you to recognize the lossy DNG file when it contains ASTRING, and EXT is the extension of your camera raw files.  Here I assume that the raw and DNG files are in the same directory and have the same name, but you can change the -srcfile argument accordingly if that isn't the case.

- Phil

Edit: Oh, darn.  After re-reading your post I see you want to recognize the corresponding DNG by exposure data.  Really?  You don't have a better handle than that?  What about DateTimeOriginal for example?  Even so, the process becomes much more difficult if the file names aren't consistent between the raw and DNG.  In this case, maybe just create a spreadsheet containing the relevant information using the -csv option, the sort it by the exposure settings (?!) to make the matching files easier to find.

[clem]

Thank you for your help... I'm still working on how to do this in a practical way that doesn't require a huge amount of manual intervention with the # of files involved.


I'm not sure if what I'm trying to say is clear...but I'll try anyhow...

A follow-up question: as exiftool recognizes (as per http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) DNGs as being stored as lossy vs lossless...

Code Select Expand

DNG Version 1.4.0.0 adds support for the following compression codes:
• Value = 8: Deflate (ZIP)
• Value = 34892: Lossy JPEG
Deflate (8) compression is allowed for floating point image data, 32-bit integer image data and transparency mask data.
Lossy JPEG (34892) is allowed for IFDs that use PhotometricInterpretation = 34892 (LinearRaw) and 8-bit integer data. This new compression code is required to let the DNG reader know to use a lossy JPEG decoder rather than a lossless JPEG decoder for this combination of PhotometricInterpretation and BitsPerSample.

(page 19)



so according to my reading, there would be 4 values available for Compression tag: 1 (Uncompressed), 7, JPEG compressed, 8, Deflate, and 34892, Lossy JPEG and it's indicated in exiftool's output:

Code Select Expand

Compression: JPEG.

Is there a way to get exiftool to express its DNG compression output as 'Lossy' or 'Lossless' JPEG for the main image as opposed to the JPEG-embedded previews in some DNG? If so, can I amend the advised command above to (instead of doing an exact filter for exposure etc) simply append 'DNG-Lossy' to the file name before its DNG extension? Then I could manually filter through folders for files of similar base file name (since they are named by YYYYMMDD-HHMMSS convention) and remove the duplicates that way after further inspection.

thanks as always for your help!

[Hayo Baan]

Yes, it is possible to determine lossy vs non lossy and I have done so myself. In fact, I amended the original tag output in my exiftool.config so it now shows me if it was lossy or lossless jpg compression. When I have access to my system again, I will post a version here. You'll need a fairly recent exiftool as Phil has made some enhancements to exiftool to better support this

[Hayo Baan]

OK, here's the code I use in my .ExifTool_config. It adds lossless/lossy to the existing compression tag. If this isn't what you want, you should be able to create a custom tag from the code below too, I think (Phil actually developed quite a nifty custom tag for this, but I can't find his code any longer ).

Code Select Expand

# Return the value of the tag in the given group (family 0:1 format)
sub GetValueInGroup($$$;$) {
    my ($et, $group, $tag, $type) = @_;
    my $value = $$et{VALUE};
    for (my $i=0; ; ++$i) {
        my $key = $tag . ($i ? " ($i)" : '');
        last unless defined $$value{$key};
        if ($et->GetGroup($key, '0:1') eq $group) {
            return $et->GetValue($key, $type || 'ValueConv');
        }
    }
    return undef;
}

# Add indication of JPEG compression type (lossless vs lossy)
require Image::ExifTool::Exif;
$Image::ExifTool::Exif::Main{0x103}->{PrintConv} = q{
    my $compression = $Image::ExifTool::Exif::compression{$val};
    if ($val == 7) {
        my $grp = $self->GetGroup($tag, '0:1');
        my $pi  = GetValueInGroup($self, $grp, 'PhotometricInterpretation');
        my $bps = GetValueInGroup($self, $grp, 'BitsPerSample');
        if ($pi && $bps) {
            if (($pi == 6 && $bps eq '8 8 8') || ($pi == 1 && $bps eq '8')) {
                $compression .= ' (Lossy';
            } else {
                $compression .= ' (Lossless';
            }
            $compression .= '?' if $$self{TIFF_TYPE} ne 'DNG';
            $compression .= ')'
        }
    }
    return $compression;
};


P.S. the above code makes use of quite some internal exiftool information so might change with future versions of exiftool.