News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Reported number of tags changed different since 11.40

Started by Hayo Baan, May 21, 2019, 05:39:59 AM

Previous topic - Next topic

Hayo Baan

Hi Phil,

I noticed the reported tag count of the changed tags when writing the file updates using the API has changed. Where I previously got only 5 tags reported (I update 4 tags & delete 1 tag so this is the expected number), I now get 7 reported. Any idea what's going on and what caused the change in the number?

Note: looking at the output of exiftool run on the before and after files, I notice a couple more tag changes, but these are all internal (e.g. offsets and xmp toolkit) and imo shouldn't be counted (and some of these definitely aren't or otherwise the count would have been 9).

Here's the diff of the (sorted) output of exiftool -a -G0:1 when run on the before and after files (I removed the reported changes in the Composite and File:System tags from the output since these are inevitable and therefore uninformative):
26c26
< [EXIF:ExifIFD]  Create Date                     : 2015:01:02 17:32:59
---
> [EXIF:ExifIFD]  Create Date                     : 2015:01:02 16:32:59
28c28
< [EXIF:ExifIFD]  Date/Time Original              : 2015:01:02 17:32:59
---
> [EXIF:ExifIFD]  Date/Time Original              : 2015:01:02 16:32:59
58d57
< [EXIF:ExifIFD]  User Comment                    :
63c62
< [EXIF:IFD0]     Copyright                       : Copyright Hayo Baan - Photography
---
> [EXIF:IFD0]     Copyright                       : © Hayo Baan – Photography. All Rights Reserved.
76c75
< [EXIF:IFD1]     Thumbnail Offset                : 14766
---
> [EXIF:IFD1]     Thumbnail Offset                : 14728
114c113
< [MPF:MPImage2]  MP Image Start                  : 93092
---
> [MPF:MPImage2]  MP Image Start                  : 93058
320c319
< [MakerNotes:PreviewIFD] Preview Image Start     : 15736
---
> [MakerNotes:PreviewIFD] Preview Image Start     : 15698
325,326c324,325
< [XMP:XMP-x]     XMP Toolkit                     : Image::ExifTool 9.77
< [XMP:XMP-xmp]   Create Date                     : 2015:01:02 16:32:59+02:00
---
> [XMP:XMP-x]     XMP Toolkit                     : Image::ExifTool 11.43
> [XMP:XMP-xmp]   Create Date                     : 2015:01:02 16:32:59.30+01:00


Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

Are you talking about the count returned by SetNewValue()?  This is the number of new tag values set and queued to be written, but it has nothing to do with the actual number finally written to the file.  The final number depends on specifics of the file, most importantly the file type.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Quote from: Phil Harvey on May 21, 2019, 08:24:23 AM
Are you talking about the count returned by SetNewValue()?
Yes, I mean the return value of SetNewValue().

Quote from: Phil Harvey on May 21, 2019, 08:24:23 AM
This is the number of new tag values set and queued to be written, but it has nothing to do with the actual number finally written to the file.  The final number depends on specifics of the file, most importantly the file type.
Understood, but since I set just 5 tags, shouldn't that then have been the reported number? It was the number of reported tags before 11.40, so somewhere in 11.40 you changed the way the number is determined.
Not a big problem (I just needed to replace 5 with 7 in the expected test outcome), but I'm just curious as to how I should then interpret the number, because now it has nothing to do with
a) the number of tags (5 not 7) I specified to have changed (note that I specify the tag including the location, so there are no duplicates that can get set as part of the update), nor
b) the number of tags actually changed (9 in this case, not 7)
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

I likely just added some new writable tags.  You can use the application -v2 option to see what tags are being queued.  For example:

> exiftool a.jpg -artist=me -v2
Writing PNG:Artist
Writing ItemList:Artist
Writing Keys:Artist if tag exists
Writing UserData:Artist if tag exists
Writing Pentax:Artist if tag exists
Writing XMP-tiff:Artist if tag exists
Writing XMP-xmpDM:Artist if tag exists
Writing IFD0:Artist
======== a.jpg
Rewriting a.jpg...


In this output, ItemList, Keys and UserData tags would have been added to the list in recent versions of ExifTool.

Good question about how to interpret the number.  Certainly if it is zero then it isn't doing what you want.  If it is many more than you expect, then you may want to narrow things down by specifying a group name for the tag.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Quote from: Phil Harvey on May 21, 2019, 08:56:07 AM
I likely just added some new writable tags.  You can use the application -v2 option to see what tags are being queued.  For example:

[snip]

In this output, ItemList, Keys and UserData tags would have been added to the list in recent versions of ExifTool.

Good question about how to interpret the number.  Certainly if it is zero then it isn't doing what you want.  If it is many more than you expect, then you may want to narrow things down by specifying a group name for the tag.

But I am specifying the exact group! So there really can (should) be only one location the data is written to. Also when you look at what was actually written by exiftool it really was just those 5 tags (plus the internal tags I mentioned but those shouldn't be counted).
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

There are sometimes multiple tags with the same name in the same group.  MakerNotes are the worst offender here, but the new QuickTime tags are pretty bad too.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Quote from: Phil Harvey on May 21, 2019, 09:27:31 AM
There are sometimes multiple tags with the same name in the same group.  MakerNotes are the worst offender here, but the new QuickTime tags are pretty bad too.

That could explain things, but not in this case, I think. Only EXIF:ExifIFD:DateTimeOriginal, EXIF:ExifIFD:CreateDate, EXIF:IFD0:Copyright, XMP:XMP-xmp:CreateDate, and EXIF:ExifIFD:UserComment were touched. I don't think these exist multiple times (also, they only got reported once in the exiftool -a -G0:1 diff output I included, so that seems to confirm this).  I looked at the changes in 11.40, but I could not see a logical change that could have caused the increase from 5 (correct count) to 7 (unexplainable count), but it must be there as 11.39 still reported 5.
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

Try setting Verbose to 2 to see what it shows for both versions.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Hi Phil,

I can't see any repeat tags here either, just a reorder of the tags since one got deleted. Below is the full output of the diff of the output of exiftool -a -G0:1 -v2 on both files (the .tmp is the changed one). Strange...


2c2
<   FileName = 20150102_16325930_D4.jpg
---
>   FileName = 20150102_16325930_D4.jpg.tmp
4,5c4,5
<   FileSize = 193648
<   FileModifyDate = 1420471005
---
>   FileSize = 193614
>   FileModifyDate = 1558447172
7c7
<   FileInodeChangeDate = 1423596927
---
>   FileInodeChangeDate = 1558447173
12c12
< JPEG APP1 (20555 bytes):
---
> JPEG APP1 (20517 bytes):
35,36c35,36
<   | 10) Copyright = Copyright Hayo Baan - Photography
<   |     - Tag 0x8298 (34 bytes, string[34] read as undef[34])
---
>   | 10) Copyright = .. Hayo Baan ... Photography. All Rights Reserved.
>   |     - Tag 0x8298 (51 bytes, string[51] read as undef[51])
39c39
<   | + [ExifIFD directory with 41 entries]
---
>   | + [ExifIFD directory with 40 entries]
52c52
<   | | 6)  DateTimeOriginal = 2015:01:02 17:32:59
---
>   | | 6)  DateTimeOriginal = 2015:01:02 16:32:59
54c54
<   | | 7)  CreateDate = 2015:01:02 17:32:59
---
>   | | 7)  CreateDate = 2015:01:02 16:32:59
108c108
<   | | | | 4)  PreviewImageStart = 14812
---
>   | | | | 4)  PreviewImageStart = 14768
572,574c572
<   | | 17) UserComment = ASCII                                   
<   | |     - Tag 0x9286 (44 bytes, undef[44])
<   | | 18) SubSecTime = 30
---
>   | | 17) SubSecTime = 30
576c574
<   | | 19) SubSecTimeOriginal = 30
---
>   | | 18) SubSecTimeOriginal = 30
578c576
<   | | 20) SubSecTimeDigitized = 30
---
>   | | 19) SubSecTimeDigitized = 30
580c578
<   | | 21) FlashpixVersion = 0100
---
>   | | 20) FlashpixVersion = 0100
582c580
<   | | 22) ColorSpace = 1
---
>   | | 21) ColorSpace = 1
584c582
<   | | 23) ExifImageWidth = 1600
---
>   | | 22) ExifImageWidth = 1600
586c584
<   | | 24) ExifImageHeight = 1064
---
>   | | 23) ExifImageHeight = 1064
588c586
<   | | 25) InteropOffset (SubDirectory) -->
---
>   | | 24) InteropOffset (SubDirectory) -->
595c593
<   | | 26) SensingMethod = 2
---
>   | | 25) SensingMethod = 2
597c595
<   | | 27) FileSource = 3
---
>   | | 26) FileSource = 3
599c597
<   | | 28) SceneType = 1
---
>   | | 27) SceneType = 1
601c599
<   | | 29) CFAPattern = .....
---
>   | | 28) CFAPattern = .....
603c601
<   | | 30) CustomRendered = 0
---
>   | | 29) CustomRendered = 0
605c603
<   | | 31) ExposureMode = 1
---
>   | | 30) ExposureMode = 1
607c605
<   | | 32) WhiteBalance = 0
---
>   | | 31) WhiteBalance = 0
609c607
<   | | 33) DigitalZoomRatio = 1.53 (153/100)
---
>   | | 32) DigitalZoomRatio = 1.53 (153/100)
611c609
<   | | 34) FocalLengthIn35mmFormat = 72
---
>   | | 33) FocalLengthIn35mmFormat = 72
613c611
<   | | 35) SceneCaptureType = 0
---
>   | | 34) SceneCaptureType = 0
615c613
<   | | 36) GainControl = 0
---
>   | | 35) GainControl = 0
617c615
<   | | 37) Contrast = 0
---
>   | | 36) Contrast = 0
619c617
<   | | 38) Saturation = 0
---
>   | | 37) Saturation = 0
621c619
<   | | 39) Sharpness = 2
---
>   | | 38) Sharpness = 2
623c621
<   | | 40) SubjectDistanceRange = 0
---
>   | | 39) SubjectDistanceRange = 0
639c637
<   | 4)  ThumbnailOffset = 14754
---
>   | 4)  ThumbnailOffset = 14716
645,648c643,646
< JPEG APP1 (2897 bytes):
<   + [XMP directory, 2868 bytes]
<   | XMPToolkit = Image::ExifTool 9.77
<   | CreateDate = 2015-01-02T16:32:59+02:00
---
> JPEG APP1 (2901 bytes):
>   + [XMP directory, 2872 bytes]
>   | XMPToolkit = Image::ExifTool 11.43
>   | CreateDate = 2015-01-02T16:32:59.30+01:00
699c697
<   PreviewImage = SCALAR(0x7f905409ef40)
---
>   PreviewImage = SCALAR(0x7fafcca12340)
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

I meant to look at the verbose output from the SetNewValue() call, not WriteInfo().

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Ah, right, sorry ::)

Anyway, that doesn't help to explain either. Still only the tags that I explicitly wrote, 5 instead of 7.

JPEG DQT (130 bytes):
JPEG SOF0 (15 bytes):
  ImageWidth = 1600
  ImageHeight = 1064
  EncodingProcess = 0
  BitsPerSample = 8
  ColorComponents = 3
JPEG DHT (416 bytes):
JPEG SOS
  PreviewImage = SCALAR(0x7fcddb3fc0a8)
Writing ExifIFD:DateTimeOriginal if tag exists
Writing QuickTime:DateTimeOriginal if tag exists
Writing ExifIFD:CreateDate if tag exists
Writing QuickTime:CreateDate if tag exists
Writing IFD0:Copyright if tag exists
Writing QuickTime:CreateDate if tag exists
Writing XMP-xmp:CreateDate if tag exists
Deleting ExifIFD:UserComment
Writing IFD0:Copyright if tag exists
Rewriting corpus/20150102_16325930_D4.jpg.tmp...
  Editing tags in: APP0 APP1 ExifIFD IFD0 JFIF QuickTime XMP
  Creating tags in:
JPEG APP1 (20555 bytes):
  Rewriting IFD0
    - IFD0:Copyright = 'Copyright Hayo Baan - Photography'
    + IFD0:Copyright = '.. Hayo Baan ... Photography. All Rights Reserved.'
  Rewriting ExifIFD
    - ExifIFD:DateTimeOriginal = '2015:01:02 17:32:59'
    + ExifIFD:DateTimeOriginal = '2015:01:02 16:32:59'
    - ExifIFD:CreateDate = '2015:01:02 17:32:59'
    + ExifIFD:CreateDate = '2015:01:02 16:32:59'
  Rewriting MakerNoteNikon
  Rewriting PreviewIFD
  Rewriting VRInfo
  Rewriting PictureControlData
  Rewriting WorldTime
  Rewriting ISOInfo
  Rewriting DistortInfo
  Rewriting UnknownInfo
  Rewriting UnknownInfo2
  Rewriting HDRInfo
  Rewriting ShotInfoD4
  Rewriting ColorBalance0215
  Rewriting LensData0204
  Rewriting FlashInfo0103
  Rewriting MultiExposure
  Rewriting AFInfo2
  Rewriting FileInfo
  Rewriting AFTune
  Rewriting RetouchInfo
    - ExifIFD:UserComment = 'ASCII                                    '
  Rewriting InteropIFD
  Rewriting GPS
  Rewriting IFD1
JPEG APP1 (2897 bytes):
  Rewriting XMP
    - XMP-xmp:CreateDate = '2015-01-02T16:32:59+02:00'
    + XMP-xmp:CreateDate = '2015-01-02T16:32:59.30+01:00'
JPEG APP2 (4092 bytes):
JPEG DQT (130 bytes):
JPEG SOF0:
JPEG DHT (416 bytes):
JPEG SOS
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

I count 9:

Writing ExifIFD:DateTimeOriginal if tag exists
Writing QuickTime:DateTimeOriginal if tag exists
Writing ExifIFD:CreateDate if tag exists
Writing QuickTime:CreateDate if tag exists
Writing IFD0:Copyright if tag exists
Writing QuickTime:CreateDate if tag exists
Writing XMP-xmp:CreateDate if tag exists
Deleting ExifIFD:UserComment
Writing IFD0:Copyright if tag exists


At this point I'm lost because I don't know how you are setting these tags.  But I don't think we're going anywhere since the count of the number of new values being set isn't very significant anyway.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Hi Phil,

Indeed 9, that's what I counted too, these writes are conditionally though (if tag exists). Further on you can see actually only 5 are really set.

BUT looking again at the list, I think this actually does uncover a bug: As you notice there are 3 mentions of a conditional write to a QuickTime tag. However these should NOT have been chosen for a (conditional) overwrite at all since I specify the full tag name, including the group. E.g. I call SetNewValue("EXIF:ExifIFD:DateTimeOriginal", "2015:01:02 16:32:59+01:00",  Replace => 1, EditOnly => 1) in my code so the related QuickTime tag shouldn't have been a candidate!

Now I can also see why the count is 7: of the 9 conditional writes, there are 7 unique ones. IFD0:Copyright and QuickTime:CreateDate are "set" twice (the copyright because in my code I indeed have called SetNewValue for that tag twice, and the date because you matched it for the ExifIFD and XMP-xmp groups).
Hayo Baan – Photography
Web: www.hayobaan.nl

Phil Harvey

Hi Hayo,

Quote from: Hayo Baan on May 22, 2019, 02:15:09 AM
BUT looking again at the list, I think this actually does uncover a bug:

Nice catch.  Thanks!  I'll fix this in the next release.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Hayo Baan

Quote from: Phil Harvey on May 22, 2019, 07:31:44 AM
Nice catch.  Thanks!  I'll fix this in the next release.

Excellent! Glad that my persistence into getting to the bottom of something seemingly small actually led to some good in the end!
Hayo Baan – Photography
Web: www.hayobaan.nl