Inability of exiftool to parse Auto Focus info from Canon EOS-R DNG

Started by ralfbergs, October 24, 2020, 10:23:18 AM

Previous topic - Next topic

ralfbergs

Hi there.

I use the old Lightroom 6.14, and the Show Focus Point plugin that under the hood uses exiftool to extract the focus point info.

My new camera, a Canon EOS-R, creates CR3 files, that LR can't natively read. So I converted them to DNG, using Adobe's DNG Converter 12.4.0.555, running under Windows.

However, exiftool 12.07 is either unable to parse the focus points info from the DNG, or this info is not even included in the DNG.

C:\Users\rabe\Pictures\2020\202010\New folder (2)>"C:\Program Files (x86)\Tools\exiftool.exe" -*Focus* -*AF* 0N8A0074.dng
Approximate Focus Distance      : 0.58


Here's the info in the CR3 (massively trimmed...):

C:\Users\rabe\Pictures\2020\202010\New folder (2)>"C:\Program Files (x86)\Tools\exiftool.exe" -*Focus* -*AF* 0N8A0074.cr3
[...]
AF Area Widths                  : 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362 362
AF Area Heights                 : 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334 334
AF Area X Positions             : -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736 -2736 -2280 -1824 -1368 -912 -456 0 456 912 1368 1824 2280 2736
AF Area Y Positions             : 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 2006 1605 1605 1605 1605 1605 1605 1605 1605 1605 1605 1605 1605 1605 1204 1204 1204 1204 1204 1204 1204 1204 1204 1204 1204 1204 1204 803 803 803 803 803 803 803 803 803 803 803 803 803 402 402 402 402 402 402 402 402 402 402 402 402 402 0 0 0 0 0 0 0 0 0 0 0 0 0 -401 -401 -401 -401 -401 -401 -401 -401 -401 -401 -401 -401 -401 -802 -802 -802 -802 -802 -802 -802 -802 -802 -802 -802 -802 -802 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1203 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -1604 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005 -2005
AF Points In Focus              : 55,68
AF Points Selected              : 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,74,75,76,77,78,79,80,81,82,83,84,85,86,87,88,89,90,91,92,93,94,95,96,97,98,99,100,101,102,103,104,105,106,107,108,109,110,111,112,113,114,115,116,117,118,119,120,121,122,123,124,125,126,127,128,129,130,131,132,133,134,135,136,137,138,139,140,141,142
[...]


(The JPG version ooc seems to contain the very same info... Yes, I know that the Show Focus Point plugin can read the points from the JPG, but I only show a single photo in Lightroom for both the DNG and JPG versions, and that's always the DNG...)

I know from my own research, that at least for some Canon cameras (5D Mark 2), and with some DNG Converter versions, the focus point info IS included.

How can I (we?) found out whether the info is at all included in the DNG?

If it's not included, then there's nothing we can do (apart from getting in touch with Adobe, with I would then try...). But if it's included, but exiftool fails to properly parse it, then there's obviously hope...

So how can we further investigate it? Let me know what I could do to help...

I also noticed the following weird things:

For the DNG file, the following info is shown, but for the CR3 and JPG files the info is not shown:
Approximate Focus Distance : 0.58
I'm wondering where this info even comes from, because the EOS-R's manual mentions that the focus distance would be displayed only for RF lenses, and I use EF lenses, so my guess was that these lenses do not even deliver the focus distance (or why would the cam not display it, if the lense delivers it?). And why is it only shown for the DNG, and not for the other flavors?

Many thanks in advance.

Kind regards,

Ralf

Phil Harvey

Hi Ralf,

I didn't read your whole post, but it is well known that the DNG converter doesn't preserve the Canon makernote information when converting from CR3.  I have it on my list of things to do to add an ExifTool feature to allow you to copy maker notes from CR3 to DNG.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

ralfbergs

Hi Phil.

Quote from: Phil Harvey on October 25, 2020, 05:59:08 PM
it is well known that the DNG converter doesn't preserve the Canon makernote information when converting from CR3.  I have it on my list of things to do to add an ExifTool feature to allow you to copy maker notes from CR3 to DNG.

Sorry, I'm completely new to this, and I was using the DNG Converter for the first time, so to me it was not well-known. ;-)

It would be a great feature to be able to copy the missing info with exiftool (so many thanks in advance for your efforts on this!), and it would be definitely better than having nothing at all, but I feel the "best" solution would be if the Adobe DNG Converter was copying that data right away. Do we know why Adobe chose not to implement it? Would it make sense to get in touch with them and ask to add the missing info?

Kind regards,

Ralf

greybeard

Its not just Canon and its not just the DNG Raw Converter.

The most frustrating thing is that the maker notes aren't passed across during an image export from Lightroom.

People post images on forums thinking they are showing the "Straight out of Camera" image because they haven't explicitly made a change but Adobe will have stripped out the most useful metadata.

ralfbergs

Quote from: greybeard on October 26, 2020, 03:47:47 AM
Its not just Canon and its not just the DNG Raw Converter.
That doesn't make it any better... ;-)

greybeard


Phil Harvey

Hi Ralf,

Quote from: ralfbergs on October 26, 2020, 03:13:09 AM
Do we know why Adobe chose not to implement it? Would it make sense to get in touch with them and ask to add the missing info?

Yes.

- Phil

Edit: I forgot to answer the first question:  Lazy, incompetent, and/or underpaid programmers would be my guess.
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Phil Harvey

Quote from: greybeard on October 26, 2020, 03:47:47 AM
Its not just Canon and its not just the DNG Raw Converter.

The most frustrating thing is that the maker notes aren't passed across during an image export from Lightroom.

I am currently using the DNG converter for my D500 NEF files and it does preserve the maker notes (although it changes them to a different Adobe-specific format).  And when I checked this years ago, it also preserved the maker notes for every other type of raw file I tested (including CR2).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Phil Harvey

OK.  I've taken a closer look at the CR3 maker notes.  The maker notes are split into 4 sections in a CR3.  The leading static section has most of the relevant information.  The other 3 parts are in the time metadata, and the only records here that aren't in the static section are:

ColorData9
Flavor
CanonCameraInfoUnknown
VignettingCorrUnknown1
Canon_0x4033
Canon_0x402e

The ColorData9 is fairly significant, but not a deal breaker if it is missing.  So it seems that just extracting the static section may be sufficient, and this would save me a lot of work.  This section does contain the auto focus info you want.  I'll look into adding the feature to copy this in ExifTool 12.09.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

greybeard

Quote from: Phil Harvey on October 26, 2020, 06:52:09 AM
Quote from: greybeard on October 26, 2020, 03:47:47 AM
Its not just Canon and its not just the DNG Raw Converter.

The most frustrating thing is that the maker notes aren't passed across during an image export from Lightroom.

I am currently using the DNG converter for my D500 NEF files and it does preserve the maker notes (although it changes them to a different Adobe-specific format).  And when I checked this years ago, it also preserved the maker notes for every other type of raw file I tested (including CR2).

- Phil

You're right - I got carried away - its only the Lightroom export that removes the maker notes (for FujiFilm anyway) - not the DNG Converter

ralfbergs

Quote from: Phil Harvey on October 26, 2020, 09:42:47 AM
I'll look into adding the feature to copy this in ExifTool 12.09
That would be really great, thank you so much in advance for your efforts!

And if you ever come close to Düsseldorf, I'll buy you a beer.😁

PS: I've posted into Adobe's support forum and raised a feature request.

Phil Harvey

ExifTool 12.09 (just released) has the ability to copy the static makernotes from a CR3 file to other file types (including DNG, JPEG and TIFF).

Note that I am writing them using the standard Canon makernote convention, and not trying to implement Adobe's butchery of the makernote structure in DNG's.  (Their engineers are idiots for doing this in the first place.)

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

ralfbergs

Quote from: Phil Harvey on October 29, 2020, 10:10:01 AM
ExifTool 12.09 (just released) has the ability to copy the static makernotes from a CR3 file to other file types (including DNG, JPEG and TIFF).

Amazing service... Thank you so much...

I will try it out tomorrow... I suppose if it works well (i. e. the Lightroom plugin will be able to extract the focus point info from the DNG) I will have to write myself a wrapper that runs over all DNG files to copy the data from the corresponding CR3 files, but that will not be a big problem...

Quote from: Phil Harvey on October 29, 2020, 10:10:01 AM
Note that I am writing them using the standard Canon makernote convention, and not trying to implement Adobe's butchery of the makernote structure in DNG's.  (Their engineers are idiots for doing this in the first place.)
Cannot really comment on this, as I'm clueless about EXIF, but I strongly believe in standards, and if Adobe really violate them they cannot be blamed enough for doing so...

Many thanks again. I'll let you know how it goes...

ralfbergs

Quote from: ralfbergs on October 31, 2020, 03:33:55 PM
I will try it out tomorrow...

I'm sorry that it took much longer than intended, but I've finally found the time to try it out.

And it seems to work!!!

I checked only a single file, by using the following invocation:

"\Program Files (x86)\Tools\exiftool(-k).exe" -tagsFromFile 0N8A0994.cr3 0N8A0994.dng


Then I checked in Lightroom, and presto -- the plugin showed the focus point. For plausibility, I checked some other DNGs, and they were lacking the focus point info. So the new functionality you kindly added seems to work well.

Could you please confirm that this is the right option? I have absolutely no clue about the different kinds of meta information, so I just picked the first option that would likely do something similar to what I want, and it worked... :-)

I also noticed that exiftool seems to be able to work on whole directories, as least for select options. So maybe instead of writing a "wrapper" for it, it can also do something like

"\Program Files (x86)\Tools\exiftool(-k).exe" -tagsFromFile *.cr3 *.dng


(That's for sure the wrong way to do it, but you get the idea... I would like to copy the tags from all CR3 files to the DNG file with the same base name...)

Many thanks again.

And remember: If you ever come to Germany and will stay close to Düsseldorf, you'll get the beer I promised... :-)

Phil Harvey

That looks good.  It will probably copy more than you want though.  To copy just the focus points, you could do -tagsfromfile %d%f.cr3 -makernotes ...

To apply to a whole directory:

exiftool -tagsfromfile %d%f.cr3 -makernotes -ext dng DIR

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).