I was really surprised (nay, shocked: I don't want my raw data to be lossy) by the fact that the raw data inside a DNG was reported as JPEG compressed. Since I changed my workflow to DNG, it was very important to me to be sure I wasn't losing any raw data information. So I started to read up on the DNG standard a bit more and found something really interesting: JPEG can also be lossless!
In a DNG, depending on the Photometric Interpretation and Bits Per Sample, the specification says the JPEG compression must either be "baseline DCT JPEG" (lossy) or "lossless Huffman JPEG".
It would be nice if exiftool told us this without having to derive this ourselves. This would require an enhancement to the extraction of the Compression tag(s), also incorporating the values of the other tags in the final result.
I can, of course, also create a custom (composite) tag to do this, but the problem is that there are multiple SubIFDs that all can have a different Compression/PhotometricInterpretation/BitsPerSample, so this doesn't work well.
Hi Hayo,
Quote from: Hayo Baan on January 09, 2017, 03:31:05 AM
It would be nice if exiftool told us this without having to derive this ourselves. This would require an enhancement to the extraction of the Compression tag(s), also incorporating the values of the other tags in the final result.
What other tags? This does sound like a job for a Composite tag, but it may require a bit of advanced wizardry.
- Phil
Quote from: Phil Harvey on January 09, 2017, 07:24:45 AM
What other tags? This does sound like a job for a Composite tag, but it may require a bit of advanced wizardry.
I meant Photometric Interpretation and Bits Per Sample: If PhotometricInterpretation = 6 (YCbCr) and BitsPerSample = 8/8/8, or if PhotometricInterpretation = 1 (BlackIsZero) and BitsPerSample = 8, then it's lossy JPEG. In all other cases is's lossless JPG. It would be nice if this could be noted alongside the Compression tag. (See page 19 of the DNG spec (http://wwwimages.adobe.com/content/dam/Adobe/en/products/photoshop/pdfs/dng_spec_1.4.0.0.pdf) on Compression)
The problem with a composite tag is that you can only define one, but as I said there are multiple SubIFDs that have the Compression tag so this doesn't really work :(
Note: I tried to create my own $Image::ExifTool::Exif::Main{0x103}->{PrintConv} but as soon as I call GetInfo there to attempt to get e.g. the matching PhotometricInterpretation ExifTool no longer shows tag values...
Trial code in
.ExifTool_config:
require Image::ExifTool::Exif;
$Image::ExifTool::Exif::Main{0x103}->{PrintConv} = q{
my $grp = $self->GetGroup($tag, '0:1');
my $pi = $self->GetInfo("$grp:PhotometricInterpretation");
return $Image::ExifTool::Exif::compression{$val};
};(incomplete, of course, but it already bombs out when you enable the GetInfo line).
I need to use GetInfo, not GetValue (which I think works at this spot) as I need to be able to direct exiftool to the proper tag (i.e., matching group), or do you know another way that enables me to determine the value of an exactly specified tag?
OK, I played a bit more and found a way to get what I want:
require Image::ExifTool::Exif;
$Image::ExifTool::Exif::Main{0x103}->{PrintConv} = q{
my $compression = $Image::ExifTool::Exif::compression{$val};
if ($val == 7) {
my $grp = $self->GetGroup($tag, '0:1');
my ($pi_tag) = grep { /PhotometricInterpretation( \(.*)?/ && $self->GetGroup($_, '0:1') eq $grp } keys($$self{VALUE});
my ($bps_tag) = grep { /BitsPerSample( \(.*)?/ && $self->GetGroup($_, '0:1') eq $grp } keys($$self{VALUE});
my $pi = $$self{VALUE}{$pi_tag} // "";
my $bps = $$self{VALUE}{$bps_tag} // "";
if ($pi && $bps) {
if (($pi == 6 && $bps eq "8 8 8") || ($pi == 1 && $bps eq "8")) {
$compression .= " (Lossy)";
} else {
$compression .= " (Lossless)";
}
}
}
return $compression;
};
This will add (Lossy) or (Lossless) after the JPEG compression type indicator to give more detail on the type of the compression.
The code seems to work well on the files I tested with. I'm not completely certain it is 100% correct in all cases though. For instance an image I scanned with Apples Image Capture as tiff is reported as JPEG compressed too and using the above code as guideline it is reported now as lossless compressed. Could this indeed be true? (I hope so, otherwise there's no benefit in scanning tiff versus jpg).
But this leads me to the question: how can you determine the compression type of a JPG (other than how I do it now)?
Hi Hayo,
I think you probably want to look at only the compression of the main (full-resolution) image. How about this config file to define a Composite MainImageCompression tag?
- Phil
Quote from: Phil Harvey on January 09, 2017, 10:01:52 AM
I think you probably want to look at only the compression of the main (full-resolution) image. How about this config file to define a Composite MainImageCompression tag?
Neat! Contains a couple of handy features I might want to use at a later stage 8)
But I actually want to know it for
all embedded images. Your config file, for instance, does not report the type of JPG compression for the TIFF scan I mentioned, and I'd like to know that too :)
(also: any chance the JPEG compression type can be determined otherwise, I'd like to confirm the JPEG compression in the TIFF is indeed Lossless compressed)
Hi Hayo,
OK, here you go. In this version I generate MyCompression tags for all embedded images and set the family 1 group of the composite tag according to the original IFD.
Quote from: Hayo Baan on January 09, 2017, 11:37:20 AM
(also: any chance the JPEG compression type can be determined otherwise, I'd like to confirm the JPEG compression in the TIFF is indeed Lossless compressed)
I haven't had the time to research this, so I can't answer this question right now.
- Phil
Edit: Fixed to always return MyCompression even if other tags not defined.Edit2: Removed requirement for SubFileType
Nice, more tricks I can use in future coding!
I needed to remove the requirement for SubFileType to get the config file to report the MyCompression tag for the scanned TIFF (the Desired tags aren't used so they can go too).
The group they are reported in now is e.g. ExifTool:IFD0, not EXIF:IFD0. How do I fix that?
Update: I noticed that with the Compression tag in SubIFD, MyCompression reports it as coming from IFD0...
PS. I guess you prefer this custom tag approach instead of my hack as this way is more future proof?
Hi Hayo,
Quote from: Hayo Baan on January 09, 2017, 12:51:47 PM
I needed to remove the requirement for SubFileType to get the config file to report the MyCompression tag for the scanned TIFF (the Desired tags aren't used so they can go too).
The group they are reported in now is e.g. ExifTool:IFD0, not EXIF:IFD0. How do I fix that?
I've updated the attached config file in my last post accordingly.
QuoteUpdate: I noticed that with the Compression tag in SubIFD, MyCompression reports it as coming from IFD0...
Can you send me the sample (philharvey66 at gmail.com)?
QuotePS. I guess you prefer this custom tag approach instead of my hack as this way is more future proof?
Yes. But not completely since we are still accessing some ExifTool internals.
- Phil
Hi Phil,
Keeping you quite busy, sorry :)
Quote from: Phil Harvey on January 09, 2017, 01:33:37 PM
Quote from: Hayo Baan on January 09, 2017, 12:51:47 PM
I needed to remove the requirement for SubFileType to get the config file to report the MyCompression tag for the scanned TIFF (the Desired tags aren't used so they can go too).
The group they are reported in now is e.g. ExifTool:IFD0, not EXIF:IFD0. How do I fix that?
I've updated the attached config file in my last post accordingly.
This works! (but what if the Compression tag is not in EXIF? Isn't it safer to use GetGroup($key, 0) and use that as the SetGroup? I attached my change.
Quote from: Phil Harvey on January 09, 2017, 01:33:37 PM
QuoteUpdate: I noticed that with the Compression tag in SubIFD, MyCompression reports it as coming from IFD0...
Can you send me the sample (philharvey66 at gmail.com)?
Take for instance the DNG from the DNG preview discussion (https://exiftool.org/forum/index.php/topic,7928.msg40660.html#msg40660), but any file that has the compression tag in a SubIFD shows the same problem (e.g. NEFs).
The problem seems to be that (without showing duplicates), a different composite tag (the last instead of the first?) is used instead of the one selected for the non-composite... I'm not sure how easy this is to fix.
Quote from: Phil Harvey on January 09, 2017, 01:33:37 PM
QuotePS. I guess you prefer this custom tag approach instead of my hack as this way is more future proof?
Yes. But not completely since we are still accessing some ExifTool internals.
Hmm, given all the extra (though for me VERY worthwhile) work we now have gone through, my original quick hack still seems easier though (and almost as maintainable). ;D
Besides, it doesn't introduce a new tag but integrates seamlessly :)
P.S. I tried to understand the TIFF way of handling the JPEG compression by reading the TIFF specs, but I haven't found a definite answer there (probably meaning that the tags alone don't contain the answer). Perhaps the answer lies in the JPEG standard?
Quote from: Hayo Baan on January 09, 2017, 02:47:29 PM
This works! (but what if the Compression tag is not in EXIF? Isn't it safer to use GetGroup($key, 0) and use that as the SetGroup? I attached my change.
Good.
QuoteThe problem seems to be that (without showing duplicates), a different composite tag (the last instead of the first?) is used instead of the one selected for the non-composite... I'm not sure how easy this is to fix.
Right. Changing the order to match those of the Compression tags is actually fairly straightforward, but setting the group of the final tag is a bit of a problem (that is until ExifTool 10.39 is released).
QuoteHmm, given all the extra (though for me VERY worthwhile) work we now have gone through, my original quick hack still seems easier though (and almost as maintainable). ;D
Besides, it doesn't introduce a new tag but integrates seamlessly :)
You might be right. Your way is somewhat simpler.
- Phil
Quote from: Phil Harvey on January 09, 2017, 03:16:26 PM
QuoteThe problem seems to be that (without showing duplicates), a different composite tag (the last instead of the first?) is used instead of the one selected for the non-composite... I'm not sure how easy this is to fix.
Right. Changing the order to match those of the Compression tags is actually fairly straightforward, but setting the group of the final tag is a bit of a problem (that is until ExifTool 10.39 is released).
Interesting, will be looking forward to it :)
Quote from: Phil Harvey on January 09, 2017, 03:16:26 PM
QuoteHmm, given all the extra (though for me VERY worthwhile) work we now have gone through, my original quick hack still seems easier though (and almost as maintainable). ;D
Besides, it doesn't introduce a new tag but integrates seamlessly :)
You might be right. Your way is somewhat simpler.
I guess so (though your code taught me more!). Anyway, as long as I'm not certain that the trick is also applicable to non dngs we can't really use either (for non DNG)
Here is a version of the config file that generates the MyCompression tags in the same order as the Compression tags. The groups won't be set properly for one of the tags if using a version of ExifTool earlier than 10.39.
- Phil
Oh, wow! It works (apart from, as you mentioned, the chosen one showing as "Composite" now – I'm on 10.38 still, of course ;)).
Boy, has this solution grown to be extremely complex! Very helpful for future coding though as it shows a lot of tricks to get things show up where (and how) you want them! 8)
Just to let you know: with 10.39 this works as advertised :)
Thanks for the confirmation. :)