Strange bug with Perl processing and LensModel

[philW]

I'm experiencing a really strange bug while writing to the LensModel tag.

Basically, the first modified LensModel value gets written to *every* file processed thereafter, instead of only to the corresponding file!

I'm stumped because all 'my'ed' variables get reset for every new record, so no value should be able to creep over to the next processed record. 

Example of data being written to image with exif_lens_tagger - LensModel is correct:
Minolta AF DT 18-70mm F3.5-5.6 (D) ## 40 ## DT 18-70mm F3.5-5.6 ## 18-70mm f/3.5-5.6 ## 18.0 mm ## 27 mm ## 10.0 ## APS-CSizeCapture: On, UHBS ## 2005:10:09 13:59:15 ## DYNAX 5D ## /Comm/PICT0033.tif

Example of data being read afterwards from image with exif_tag_parser - LensModel is now wrong:
Minolta AF DT 18-70mm F3.5-5.6 (D) ## 40 ## Loxia 25mm F2.4 ## 18-70mm f/3.5-5.6 ## 18.0 mm ## 27 mm ## 10.0 ## APS-CSizeCapture: On, UHBS ## 2005:10:09 13:59:15 ## DYNAX 5D ## /Comm/PICT0033.tif


My Perl script exif_tag_parser reads the EXIF data from all supplied images (thousands of Minolta & Sony TIF files saved via Photoshop), processes the data, fixes or modifies some, and finally writes one record for each image to a text database file.   

Each image record consists of these tab delimited fields:
Lens, XMP-aux:LensID, LensModel, LensInfo, FocalLength, FocalLengthIn35mmFormat, Aperture, Keywords, DateTimeOriginal, Model, File path

Example:
Minolta AF DT 18-70mm F3.5-5.6 (D) ## 40 ## DT 18-70mm F3.5-5.6 ## 18-70mm f/3.5-5.6 ## 18.0 mm ## 27 mm ## 10.0 ## APS-CSizeCapture: On, UHBS ## 2005:10:09 13:59:15 ## DYNAX 5D ## /Comm/PICT0033.tif

All records from the text database file are copied to the __DATA__ section of the other Perl script exif_lens_tagger and are processed as follows.

Example of records:
__DATA__
Zeiss Loxia 25mm F2.4   49236   Loxia 25mm F2.4   loxia 25mm f/2.4   25.0 mm   25 mm   10.0   agriculture   2023:05:04 12:33:28   ILCE-7RM4   /Volumes/Data/Photos/PW Photos/Travel/_DSC6664.tif
Minolta AF 28-85mm F3.5-4.5   25521   28-85mm F3.5-4.5   28-85mm f/3.5-4.5   35.0 mm   35 mm   4.0   Australia   1990:10:09 01:13:49   DYNAX 7000i   /Volumes/Data/Photos/PW Photos/Travel/Australia 1990/010 1a-1.tif
Sony DT 18-70mm F3.5-5.6 (SAL1870)   40   DT 18-70mm F3.5-5.6   18-70mm f/3.5-5.6   18.0 mm   27 mm   16.0   APS-CSizeCapture: On, work   2006:02:19 15:34:46   DYNAX 5D   /Volumes/Data/Photos/PW Photos/Business/PICT0074.tif
Sigma AF 12-24mm F4.5-5.6 EX DG   40   12-24mm F4.5-5.6 EX DG   12-24mm f/4.5-5.6   12.0 mm   18 mm   4.5   APS-CSizeCapture: On, work   2006:08:14 16:35:01   DYNAX 5D   /Volumes/Data/Photos/PW Photos/Business/room_flash.tif
 

Example script exif_lens_tagger:

Code Select Expand

#! /usr/bin/perl
use strict;
use Image::ExifTool;
my $exifTool = new Image::ExifTool;
$exifTool->Options( IgnoreMinorErrors => 1, overwrite_original => 1 );
while ( <DATA> ) {
...
my @fields = split /\t/;
my $file = pop @fields;
my %data;
$data{'Lens'} = shift @fields;
$data{'LensID'} = shift @fields;
$data{'LensModel'} = shift @fields;
$data{'LensInfo'} = shift @fields;
...
my $info = $exifTool->ImageInfo( $file );
for ( keys %data ) {
next unless (exists $data{$_} && $data{$_}) or $_ eq 'Keywords';
# only overwrite if tag exists and only if different
next if $_ eq 'LensID' and not $data{'LensID'};
next if $_ eq 'LensModel' and (not exists $$info{'LensModel'} or not $data{'LensModel'});
next if $_ eq 'LensInfo' and (not exists $$info{'LensInfo'}  or not $data{'LensInfo'});
...
if ( $_ eq 'Keywords' ) {
if ( $data{'Keywords'} =~ /[,;]/ ) {
my @keywords_orig = $exifTool->GetValue('Keywords', 'ValueConv');
my $keywords_orig = join ', ', @keywords_orig;
next if $keywords_orig eq $data{'Keywords'};
$exifTool->SetNewValue( 'Keywords' ); # reset - delete information for specified tag
my ($success, $err_msg) = $exifTool->SetNewValue( 'Keywords' => $data{'Keywords'}, 'AddValue' => 0, 'Group' => 'IPTC' );
$err_msg and print STDERR "## exiftool Keywords error: $err_msg ### $file\n";
} elsif ( $data{'Keywords'} ne $$info{'Keywords'} ) {
...
}
} elsif ( $_ eq 'LensID' ) {
my $LensID_orig = $exifTool->GetValue('XMP-aux:LensID');
next if $LensID_orig eq $data{$_};
my ($success, $err_msg) = $exifTool->SetNewValue('XMP-aux:LensID');
($success, $err_msg) = $exifTool->SetNewValue('XMP-aux:LensID' => $data{$_}, AddValue => 0);
$err_msg and print STDERR "## exiftool LensID 2 error: $err_msg ### $file\n";
} else {
my $val_orig = $exifTool->GetValue( $_ );
next if $val_orig eq $data{$_};
my ($success, $err_msg) = $exifTool->SetNewValue( $_ => $data{$_}, 'AddValue' => 0 );
$err_msg and print STDERR "## exiftool tag $_ error: $err_msg ### $file\n";
}
}
$exifTool->WriteInfo( $file );
$errorMessage = $exifTool->GetValue('Error');
$warningMessage = $exifTool->GetValue('Warning');
...
}
So, my question is how can the LensModel value of a previous record survive and be written to all files following that Zeiss Loxia 25mm F2.4 record? Technically, this should be impossible due to burner variables. Is Exiftools doing some woodoo thing behind the curtain ?

And another question: do I understand it correctly that the ExifTool binary is executed (launched) for every new image instead of being kept open until the script finishes? That would be indeed unfortunate for performance.

[Phil Harvey]

(TLDR)

Once you call SetNewValue(TAG, VALUE), the value remains queued until you reset values from previous calls by calling SetNewValue() with no arguments.

- Phil

[philW]

Thanks, that's a rather unexpected behavior.

And strange that the problem only happens with the LensModel tag.

But what does this mean in a while/foreach loop, where  multiple tags of a file are processed each with a SetNewValue, but the actual WriteInfo to file happens first after the loop? I would have to wait with resetting the current record's tag/value with $exifTool->SetNewValue() until after the WriteInfo, I assume?

And that would require that I'd have to duplicate the loop processing just to reset the tag values again.

[Phil Harvey]

I guess I don't understand.  LensModel is not special.  Use SetNewValue() with no arguments to clear all previous new values that have been set.  If you want to write the same values to another file, then don't do this, but I was keying on your statement that LensModel was written to all files.  If you want to clear just this value, use SetNewValue(LensModel => undef, Replace => 2).

And to answer this:

And another question: do I understand it correctly that the ExifTool binary is executed (launched) for every new image instead of being kept open until the script finishes?

No.  You aren't running the exiftool application, and your script is only launched once, so all ExifTool modules that you use are compiled only once.

- Phil

[philW]

I guess I don't understand.  LensModel is not special.  Use SetNewValue() with no arguments to clear all previous new values that have been set.  If you want to write the same values to another file, then don't do this, but I was keying on your statement that LensModel was written to all files.  If you want to clear just this value, use SetNewValue(LensModel => undef, Replace => 2).

I'm batch-processing many files and each file has it's own unique EXIF tag values. Per file I process its EXIF tags in a foreach tag loop, and write the tags to the file after the loop. So I take it then that I can clear all tag values of the file after the final WriteInfo with just one SetNewValue(), which would be great.

And to answer this:

And another question: do I understand it correctly that the ExifTool binary is executed (launched) for every new image instead of being kept open until the script finishes?

No.  You aren't running the exiftool application, and your script is only launched once, so all ExifTool modules that you use are compiled only once.

- Phil

Good to know, thanks Phil!

[philW]

Yep, problem solved - thanks!