Basics - extracting sub-sub-folder maker notes?

Started by Archive, May 12, 2010, 08:54:39 AM

Previous topic - Next topic

Archive

[Originally posted by billp on 2009-08-07 19:13:50-07]

Hi,

If I do a -v3 -s on a Canon image, I can see the tree well enough:

Code:
 MakerNoteCanon (SubDirectory) -->
  | |     - Tag 0x927c (2408 bytes, undef[2408])
  | | + [MakerNotes directory with 28 entries]
  | | | 0)  CanonCameraSettings
  | | | 1)  CanonFocalLength (SubDirectory) -->
  | | | 3)  CanonShotInfo (SubDirectory) -->
  | | | | ExposureTime = 426

etc.

There's no less than 4 tags with the name ExposureTime, two of them are tag 6 under CanonShotInfo and some other subdirectory, and another is tag 22 under CanonCameraInfoPowerShot subdirectory...

I can only reference the last one by -canon:exposuretime or -makernotes:exposuretime... this is probably by design. But my question is - how do I reference a specific one?  In the example above, how do I pull that value?

Thanks for such a useful tool Phil, and thanks for reading everyone.

-bp

Archive

[Originally posted by exiftool on 2009-08-11 11:18:51-07]

With exiftool 7.83 or later you can extract specific instances
of a tag by specifying a family 4 group name:

Code:
> exiftool ../pics/CanonEOS500D.jpg -exposuretime -G1:4
[ExifIFD]       Exposure Time                   : 1/320
[Canon:Copy1]   Exposure Time                   : 1/304
[Canon:Copy2]   Exposure Time                   : 1/332

> exiftool ../pics/CanonEOS500D.jpg -copy1:exposuretime
Exposure Time                   : 1/304

- Phil

Archive

[Originally posted by exiftool on 2009-08-11 11:20:12-07]

Oops.  The first command is missing a -a

Archive

[Originally posted by billp on 2009-08-11 15:24:13-07]



Hi Phil, welcome back. Hope you enjoyed your vacation.

&gt&gt With exiftool 7.83 or later you can extract specific instances of a tag by specifying a family 4 group name:

Yea, I figured part of it out while you were gone, I have been reading the archives back over a year. Educational! But no light-bulb moment for my current problem. I can't figure it out and am well aware it's probably my own idiocy. So every MakerNotes "subdirectory" is considered a duplicate block, which can be referenced by instance:tagname only? Obviously with MakerNotes, where it's hard-coded deciphering routines per-camera, they always will be in the same place for the same camera. I just get twitchy around assumptions and coding practices.

Code:
$ exiftool -a -e -s -G:4:3:2:1:0 -ExposureTime testimage.jpg

[Main:Image:ExifIFD:EXIF] ExposureTime          : 1/10000
[Copy1:Main:Image:Canon:MakerNotes] ExposureTime: 1/10173
[Copy2:Main:Image:Canon:MakerNotes] ExposureTime: 1/8422

And while I can retrieve a duplicate tag using the instance reference, I cannot push it:

Code:
$ exiftool -Copy2:Main:Image:Canon:MakerNotes:ExposureTime testimage.jpg
Exposure Time                   : 1/8422

$ exiftool -Copy2:Main:Image:Canon:MakerNotes:ExposureTime=1/5000 testimage.jpg
Tag 'Copy2:Main:Image:Canon:MakerNotes:ExposureTime' does not exist
Nothing to do.

$ exiftool -a -copy2:exposuretime=1/5000 testimage.jpg
Tag 'copy2:ExposureTime' does not exist
Nothing to do.

This part really confuses me. Since I cannot reference them uniquely, if I set it via -MakerNotes:ExposureTime=<value&gt it sets both? I guess because of the duplicating names? But not just that, it appears it's programatically derived like Composite Tags are, because this is not the value I passed, even with -e. I understand one is probably APEX of 1/25000th below, but what's that other one?

Code:
$ exiftool -makernotes:exposuretime=1/25000 testimage.jpg
    1 image files updated
$ exiftool -a -e -s -G:4:3:2:1:0 -ExposureTime testimage.jpg
[Main:Image:ExifIFD:EXIF] ExposureTime          : 1/10000
[Copy1:Main:Image:Canon:MakerNotes] ExposureTime: 1/26008
[Copy2:Main:Image:Canon:MakerNotes] ExposureTime: 1/25006

Looking into Canon.pm, I find these lines related for ShotInfo:22 and CameraInfoPowerShot:6, respectively:

Code:
   22 => [
            Name => 'ExposureTime',
            # approximate big translation table by simple calculation - PH
            ValueConv => 'exp(-Image::ExifTool::Canon::CanonEv($val)*log(2))',
 
    0x06 => {
        Name => 'ExposureTime',
        ValueConv => 'exp(-($val+24)/100*log(2))',

I'm sure there's a reason, but I don't understand the math additions to tag 6. So in this case, 22 is probably the (most) correct one?

OK realworld, let's say I want ShotInfo:22, but I'm no Perl guru nor shell master, so here's my entirely inefficient solution for fetching one and overwriting the relevant EXIF tags with it.

Code:
$ MYSPEED=`exiftool -a -H -exposuretime testimage.jpg | \
grep 0x0016 | cut -d ":" -f2 | tr -d " "`; \
exiftool -EXIF:ShutterSpeedValue="${MYSPEED}" -EXIF:ExposureTime="${MYSPEED}" testimage.jpg

While this works, it's messy, I'd really like to be able to reference each individually, as theres 3 of everything shot related; ISO, FNumber, etc. I also cannot find a way to reference a tag by it's tag ID (via the standalone). I guess that doesn't matter without being able to traverse the MakerNotes subdirectories, as Tag ID 6 under one subdirectory will be ExposureComp and ExposureTime in another, and it will be per-camera anyway, so I guess I have to decide which one is correct and reference by instance:tagname?

I think I'm more confused than when I started.

Thanks again!

Archive

[Originally posted by exiftool on 2009-08-11 17:15:38-07]

I can see why you get twitchy.  The instance number may well change
when new (duplicate) information is discovered.  And yes, the instance
number can not be used when writing.  The idea is that you write
all instances of a tag with a given name -- this ensures that things
are consistent (well, as consistent as possible) between the different
instances.  The only way around this is to override tag values using
user-defined tags.  This will also give you a work-around to allow
you to extract a specific tag by a unique name.

About the difference in decoding of 22 vs. 0x06:
I decoded 22 very carefully so I'm pretty sure this is correct.  The reason
for the CanonEv routine is to handle the rounding off of the 1/3 EV values.
It may well be that the 0x06 value (for PowerShot
models) could be improved by applying a similar rounding routine, but I
don't have the ability to test this in enough detail to determine whether
or not this is the case.

Archive

[Originally posted by billp on 2009-08-12 17:41:53-07]

Hi again,

I'm sorry, I don't understand. (This is a normal condition.) If I stick in a userdef tag:

Code:
'Image::ExifTool::Composite' => {
    MyExposureTime => {
      Require => 'Canon:ExposureTime (1)',
      PrintConv => 'Image::ExifTool::Exif::PrintExposureTime($val)',
    },
 },

... it works, but that still relies on instancing and is no better than the command line method. I tried many times to re-create the Image::ExifTool::Canon::ShotInfo tag 22 as an override, stealing from your example in the thread write unwritable DNG-Tags, but cannot get it to play along. I thought Image::ExifTool::MakerNotes::Main would work, but not for me.

 
Thanks!

Archive

[Originally posted by exiftool on 2009-08-13 10:57:11-07]

Sorry, this is a little confusing.  Especially since ShotInfo tag 0x22 is an
array, and array-type user-defined tags aren't yet supported.  So
taking the common case (most EOS models), this is what I meant:
(copying the tag definition from Canon.pm)

Code:
%Image::ExifTool::UserDefined = (
    'Image::ExifTool::Canon::ShotInfo' => {
        22 => {
            Name => 'MyExposureTime',
            Priority => 0,
            # apparently a value of 0 is valid in a CRW image (=1s, D60 sample)
            RawConv => '($val or $$self{FILE_TYPE} eq "CRW") ? $val : undef',
            # approximate big translation table by simple calculation - PH
            ValueConv => 'exp(-Image::ExifTool::Canon::CanonEv($val)*log(2))',
            ValueConvInv => 'Image::ExifTool::Canon::CanonEvInv(-log($val)/log(2))',
            PrintConv => 'Image::ExifTool::Exif::PrintExposureTime($val)',
            PrintConvInv => 'eval $val',
        },
    },
);
1;

- Phil

Archive

[Originally posted by billp on 2009-08-15 22:53:00-07]



Hi again Phil,

Thanks for the example, that works great.

Interesting! Regarding the comment & RawConv line in that block:

Code:
# apparently a value of 0 is valid in a CRW image (=1s, D60 sample)
RawConv => '($val or $$self{FILE_TYPE} eq "CRW") ? $val : undef',

Of the PowerShot cameras I have here; SX100, SD1000, SD1100, and A720 - *all* of them use 0 for one second exposures, and with that line present will end up being undefined and destroys querying off that tag, user-defined name or not.

When I comment it out, it'll display for shots with one second shutter speeds, but then it messes up your other calculations off of Ev. See below:

Scratch that. cpanforum destorys my formatting even with <code> tags, so if you wouldn't mind, here's a hosted link:

exiftool table of values.txt

Thanks!

PS: regarding "I don't have the ability to test this in enough detail to determine whether or not this is the case." -- If you would like this kind of data, or MIEs, or stripped jpegs replaced with 8x8 images like in the sample image archives you make available, I can amass and format said data from 50+ unique PowerShot cameras, and various firmware revisions of each model. I can automate the picture taking process using CHDK scripting, and post requests for assistance on the CHDK wiki and forum. Many would help by submitting the results of their model(s) when the end result will be more accurate data. Further discussion should probably be off-list but the offer stands: If you'll correct it, I'll collect it.

Archive

[Originally posted by exiftool on 2009-08-16 00:17:19-07]

I'll have to go back and re-think the 0 value problem.  I must have thought
that some cameras wrote 0 for an invalid value or I wouldn't have done this.
What do you mean that it messes up my other calculations off of Ev when
0 is allowed?

Your table of values is great!  From this, I can improve the ExposureTime
value (and the FNumber value too) in the PowerShotInfo:

Code:
   0x05 => {
        Name => 'FNumber',
        Groups => { 2 => 'Image' },
        Priority => 0,
        ValueConv => 'exp($val/192*log(2))',
        ValueConvInv => 'log($val)*192/log(2)',
        PrintConv => 'sprintf("%.2g",$val)',
        PrintConvInv => '$val',
    },
    0x06 => {
        Name => 'ExposureTime',
        Groups => { 2 => 'Image' },
        Priority => 0,
        ValueConv => 'exp(-$val/96*log(2))',
        ValueConvInv => '-log($val)*96/log(2)',
        PrintConv => 'Image::ExifTool::Exif::PrintExposureTime($val)',
        PrintConvInv => 'eval $val',
    },

Thanks!

Also, it looks like the Powershots don't need the special 1/3-stop logic,
which makes sense because 96 is evenly divisible by 3 (wheras the EOS
models use a divisor of  32, which isn't).

- Phil

Archive

[Originally posted by billp on 2009-08-16 03:12:41-07]

Hi,

>> What do you mean that it messes up my other calculations off of Ev when 0 is allowed?

 
After a bit of research, I know that according to APEX system rules, Tv=log2(1/shutterspeed), so that ExposureTime=0 for one second exposures is valid and will have to be accommodated... but I don't know why, when I remark out that RawConv line in ShotInfo:22, that the FNumber values of both ShotInfo:21 and InfoPowerShot:5 change after the 1s point. Please refer to that table above.

And if it breaks that, what else does it break? etc.

Thanks again. If there's any other data you need... :^)

Archive

[Originally posted by exiftool on 2009-08-16 10:17:16-07]

There is no way that removing this value should affect
the FNumber.  There must be something else happening.
Could you send me the EXIF from the sample images?

Code:
exiftool -w exif/%f.exif -exif -b FILES

Then send me the contents of the "exif" directory.
My mail is philharvey66 at gmail.com.

Thanks.

- Phil

Archive

[Originally posted by billp on 2009-08-16 19:00:41-07]

Sent.

Yes, it struck me as odd, even with the modified routines for 21 & 22 you posted above, it does still does it. ??

Thanks!