Hi Phil,
I noticed the reported tag count of the changed tags when writing the file updates using the API has changed. Where I previously got only 5 tags reported (I update 4 tags & delete 1 tag so this is the expected number), I now get 7 reported. Any idea what's going on and what caused the change in the number?
Note: looking at the output of exiftool run on the before and after files, I notice a couple more tag changes, but these are all internal (e.g. offsets and xmp toolkit) and imo shouldn't be counted (and some of these definitely aren't or otherwise the count would have been 9).
Here's the diff of the (sorted) output of exiftool -a -G0:1 when run on the before and after files (I removed the reported changes in the Composite and File:System tags from the output since these are inevitable and therefore uninformative):
26c26
< [EXIF:ExifIFD] Create Date : 2015:01:02 17:32:59
---
> [EXIF:ExifIFD] Create Date : 2015:01:02 16:32:59
28c28
< [EXIF:ExifIFD] Date/Time Original : 2015:01:02 17:32:59
---
> [EXIF:ExifIFD] Date/Time Original : 2015:01:02 16:32:59
58d57
< [EXIF:ExifIFD] User Comment :
63c62
< [EXIF:IFD0] Copyright : Copyright Hayo Baan - Photography
---
> [EXIF:IFD0] Copyright : © Hayo Baan – Photography. All Rights Reserved.
76c75
< [EXIF:IFD1] Thumbnail Offset : 14766
---
> [EXIF:IFD1] Thumbnail Offset : 14728
114c113
< [MPF:MPImage2] MP Image Start : 93092
---
> [MPF:MPImage2] MP Image Start : 93058
320c319
< [MakerNotes:PreviewIFD] Preview Image Start : 15736
---
> [MakerNotes:PreviewIFD] Preview Image Start : 15698
325,326c324,325
< [XMP:XMP-x] XMP Toolkit : Image::ExifTool 9.77
< [XMP:XMP-xmp] Create Date : 2015:01:02 16:32:59+02:00
---
> [XMP:XMP-x] XMP Toolkit : Image::ExifTool 11.43
> [XMP:XMP-xmp] Create Date : 2015:01:02 16:32:59.30+01:00
Are you talking about the count returned by SetNewValue()? This is the number of new tag values set and queued to be written, but it has nothing to do with the actual number finally written to the file. The final number depends on specifics of the file, most importantly the file type.
- Phil
Quote from: Phil Harvey on May 21, 2019, 08:24:23 AM
Are you talking about the count returned by SetNewValue()?
Yes, I mean the return value of
SetNewValue().
Quote from: Phil Harvey on May 21, 2019, 08:24:23 AM
This is the number of new tag values set and queued to be written, but it has nothing to do with the actual number finally written to the file. The final number depends on specifics of the file, most importantly the file type.
Understood, but since I set just 5 tags, shouldn't that then have been the reported number? It was the number of reported tags before 11.40, so somewhere in 11.40 you changed the way the number is determined.
Not a big problem (I just needed to replace 5 with 7 in the expected test outcome), but I'm just curious as to how I should then interpret the number, because now it has nothing to do with
a) the number of tags (5 not 7) I specified to have changed (note that I specify the tag including the location, so there are no duplicates that can get set as part of the update), nor
b) the number of tags actually changed (9 in this case, not 7)
I likely just added some new writable tags. You can use the application -v2 option to see what tags are being queued. For example:
> exiftool a.jpg -artist=me -v2
Writing PNG:Artist
Writing ItemList:Artist
Writing Keys:Artist if tag exists
Writing UserData:Artist if tag exists
Writing Pentax:Artist if tag exists
Writing XMP-tiff:Artist if tag exists
Writing XMP-xmpDM:Artist if tag exists
Writing IFD0:Artist
======== a.jpg
Rewriting a.jpg...
In this output, ItemList, Keys and UserData tags would have been added to the list in recent versions of ExifTool.
Good question about how to interpret the number. Certainly if it is zero then it isn't doing what you want. If it is many more than you expect, then you may want to narrow things down by specifying a group name for the tag.
- Phil
Quote from: Phil Harvey on May 21, 2019, 08:56:07 AM
I likely just added some new writable tags. You can use the application -v2 option to see what tags are being queued. For example:
[snip]
In this output, ItemList, Keys and UserData tags would have been added to the list in recent versions of ExifTool.
Good question about how to interpret the number. Certainly if it is zero then it isn't doing what you want. If it is many more than you expect, then you may want to narrow things down by specifying a group name for the tag.
But I am specifying the exact group! So there really can (should) be only one location the data is written to. Also when you look at what was actually written by exiftool it really was just those 5 tags (plus the internal tags I mentioned but those shouldn't be counted).
There are sometimes multiple tags with the same name in the same group. MakerNotes are the worst offender here, but the new QuickTime tags are pretty bad too.
- Phil
Quote from: Phil Harvey on May 21, 2019, 09:27:31 AM
There are sometimes multiple tags with the same name in the same group. MakerNotes are the worst offender here, but the new QuickTime tags are pretty bad too.
That could explain things, but not in this case, I think. Only
EXIF:ExifIFD:DateTimeOriginal,
EXIF:ExifIFD:CreateDate,
EXIF:IFD0:Copyright,
XMP:XMP-xmp:CreateDate, and
EXIF:ExifIFD:UserComment were touched. I don't think these exist multiple times (also, they only got reported once in the
exiftool -a -G0:1 diff output I included, so that seems to confirm this). I looked at the changes in 11.40, but I could not see a logical change that could have caused the increase from 5 (correct count) to 7 (unexplainable count), but it must be there as 11.39 still reported 5.
Try setting Verbose to 2 to see what it shows for both versions.
- Phil
Hi Phil,
I can't see any repeat tags here either, just a reorder of the tags since one got deleted. Below is the full output of the diff of the output of exiftool -a -G0:1 -v2 on both files (the .tmp is the changed one). Strange...
2c2
< FileName = 20150102_16325930_D4.jpg
---
> FileName = 20150102_16325930_D4.jpg.tmp
4,5c4,5
< FileSize = 193648
< FileModifyDate = 1420471005
---
> FileSize = 193614
> FileModifyDate = 1558447172
7c7
< FileInodeChangeDate = 1423596927
---
> FileInodeChangeDate = 1558447173
12c12
< JPEG APP1 (20555 bytes):
---
> JPEG APP1 (20517 bytes):
35,36c35,36
< | 10) Copyright = Copyright Hayo Baan - Photography
< | - Tag 0x8298 (34 bytes, string[34] read as undef[34])
---
> | 10) Copyright = .. Hayo Baan ... Photography. All Rights Reserved.
> | - Tag 0x8298 (51 bytes, string[51] read as undef[51])
39c39
< | + [ExifIFD directory with 41 entries]
---
> | + [ExifIFD directory with 40 entries]
52c52
< | | 6) DateTimeOriginal = 2015:01:02 17:32:59
---
> | | 6) DateTimeOriginal = 2015:01:02 16:32:59
54c54
< | | 7) CreateDate = 2015:01:02 17:32:59
---
> | | 7) CreateDate = 2015:01:02 16:32:59
108c108
< | | | | 4) PreviewImageStart = 14812
---
> | | | | 4) PreviewImageStart = 14768
572,574c572
< | | 17) UserComment = ASCII
< | | - Tag 0x9286 (44 bytes, undef[44])
< | | 18) SubSecTime = 30
---
> | | 17) SubSecTime = 30
576c574
< | | 19) SubSecTimeOriginal = 30
---
> | | 18) SubSecTimeOriginal = 30
578c576
< | | 20) SubSecTimeDigitized = 30
---
> | | 19) SubSecTimeDigitized = 30
580c578
< | | 21) FlashpixVersion = 0100
---
> | | 20) FlashpixVersion = 0100
582c580
< | | 22) ColorSpace = 1
---
> | | 21) ColorSpace = 1
584c582
< | | 23) ExifImageWidth = 1600
---
> | | 22) ExifImageWidth = 1600
586c584
< | | 24) ExifImageHeight = 1064
---
> | | 23) ExifImageHeight = 1064
588c586
< | | 25) InteropOffset (SubDirectory) -->
---
> | | 24) InteropOffset (SubDirectory) -->
595c593
< | | 26) SensingMethod = 2
---
> | | 25) SensingMethod = 2
597c595
< | | 27) FileSource = 3
---
> | | 26) FileSource = 3
599c597
< | | 28) SceneType = 1
---
> | | 27) SceneType = 1
601c599
< | | 29) CFAPattern = .....
---
> | | 28) CFAPattern = .....
603c601
< | | 30) CustomRendered = 0
---
> | | 29) CustomRendered = 0
605c603
< | | 31) ExposureMode = 1
---
> | | 30) ExposureMode = 1
607c605
< | | 32) WhiteBalance = 0
---
> | | 31) WhiteBalance = 0
609c607
< | | 33) DigitalZoomRatio = 1.53 (153/100)
---
> | | 32) DigitalZoomRatio = 1.53 (153/100)
611c609
< | | 34) FocalLengthIn35mmFormat = 72
---
> | | 33) FocalLengthIn35mmFormat = 72
613c611
< | | 35) SceneCaptureType = 0
---
> | | 34) SceneCaptureType = 0
615c613
< | | 36) GainControl = 0
---
> | | 35) GainControl = 0
617c615
< | | 37) Contrast = 0
---
> | | 36) Contrast = 0
619c617
< | | 38) Saturation = 0
---
> | | 37) Saturation = 0
621c619
< | | 39) Sharpness = 2
---
> | | 38) Sharpness = 2
623c621
< | | 40) SubjectDistanceRange = 0
---
> | | 39) SubjectDistanceRange = 0
639c637
< | 4) ThumbnailOffset = 14754
---
> | 4) ThumbnailOffset = 14716
645,648c643,646
< JPEG APP1 (2897 bytes):
< + [XMP directory, 2868 bytes]
< | XMPToolkit = Image::ExifTool 9.77
< | CreateDate = 2015-01-02T16:32:59+02:00
---
> JPEG APP1 (2901 bytes):
> + [XMP directory, 2872 bytes]
> | XMPToolkit = Image::ExifTool 11.43
> | CreateDate = 2015-01-02T16:32:59.30+01:00
699c697
< PreviewImage = SCALAR(0x7f905409ef40)
---
> PreviewImage = SCALAR(0x7fafcca12340)
I meant to look at the verbose output from the SetNewValue() call, not WriteInfo().
- Phil
Ah, right, sorry ::)
Anyway, that doesn't help to explain either. Still only the tags that I explicitly wrote, 5 instead of 7.
JPEG DQT (130 bytes):
JPEG SOF0 (15 bytes):
ImageWidth = 1600
ImageHeight = 1064
EncodingProcess = 0
BitsPerSample = 8
ColorComponents = 3
JPEG DHT (416 bytes):
JPEG SOS
PreviewImage = SCALAR(0x7fcddb3fc0a8)
Writing ExifIFD:DateTimeOriginal if tag exists
Writing QuickTime:DateTimeOriginal if tag exists
Writing ExifIFD:CreateDate if tag exists
Writing QuickTime:CreateDate if tag exists
Writing IFD0:Copyright if tag exists
Writing QuickTime:CreateDate if tag exists
Writing XMP-xmp:CreateDate if tag exists
Deleting ExifIFD:UserComment
Writing IFD0:Copyright if tag exists
Rewriting corpus/20150102_16325930_D4.jpg.tmp...
Editing tags in: APP0 APP1 ExifIFD IFD0 JFIF QuickTime XMP
Creating tags in:
JPEG APP1 (20555 bytes):
Rewriting IFD0
- IFD0:Copyright = 'Copyright Hayo Baan - Photography'
+ IFD0:Copyright = '.. Hayo Baan ... Photography. All Rights Reserved.'
Rewriting ExifIFD
- ExifIFD:DateTimeOriginal = '2015:01:02 17:32:59'
+ ExifIFD:DateTimeOriginal = '2015:01:02 16:32:59'
- ExifIFD:CreateDate = '2015:01:02 17:32:59'
+ ExifIFD:CreateDate = '2015:01:02 16:32:59'
Rewriting MakerNoteNikon
Rewriting PreviewIFD
Rewriting VRInfo
Rewriting PictureControlData
Rewriting WorldTime
Rewriting ISOInfo
Rewriting DistortInfo
Rewriting UnknownInfo
Rewriting UnknownInfo2
Rewriting HDRInfo
Rewriting ShotInfoD4
Rewriting ColorBalance0215
Rewriting LensData0204
Rewriting FlashInfo0103
Rewriting MultiExposure
Rewriting AFInfo2
Rewriting FileInfo
Rewriting AFTune
Rewriting RetouchInfo
- ExifIFD:UserComment = 'ASCII '
Rewriting InteropIFD
Rewriting GPS
Rewriting IFD1
JPEG APP1 (2897 bytes):
Rewriting XMP
- XMP-xmp:CreateDate = '2015-01-02T16:32:59+02:00'
+ XMP-xmp:CreateDate = '2015-01-02T16:32:59.30+01:00'
JPEG APP2 (4092 bytes):
JPEG DQT (130 bytes):
JPEG SOF0:
JPEG DHT (416 bytes):
JPEG SOS
I count 9:
Writing ExifIFD:DateTimeOriginal if tag exists
Writing QuickTime:DateTimeOriginal if tag exists
Writing ExifIFD:CreateDate if tag exists
Writing QuickTime:CreateDate if tag exists
Writing IFD0:Copyright if tag exists
Writing QuickTime:CreateDate if tag exists
Writing XMP-xmp:CreateDate if tag exists
Deleting ExifIFD:UserComment
Writing IFD0:Copyright if tag exists
At this point I'm lost because I don't know how you are setting these tags. But I don't think we're going anywhere since the count of the number of new values being set isn't very significant anyway.
- Phil
Hi Phil,
Indeed 9, that's what I counted too, these writes are conditionally though (if tag exists). Further on you can see actually only 5 are really set.
BUT looking again at the list, I think this actually does uncover a bug: As you notice there are 3 mentions of a conditional write to a QuickTime tag. However these should NOT have been chosen for a (conditional) overwrite at all since I specify the full tag name, including the group. E.g. I call SetNewValue("EXIF:ExifIFD:DateTimeOriginal", "2015:01:02 16:32:59+01:00", Replace => 1, EditOnly => 1) in my code so the related QuickTime tag shouldn't have been a candidate!
Now I can also see why the count is 7: of the 9 conditional writes, there are 7 unique ones. IFD0:Copyright and QuickTime:CreateDate are "set" twice (the copyright because in my code I indeed have called SetNewValue for that tag twice, and the date because you matched it for the ExifIFD and XMP-xmp groups).
Hi Hayo,
Quote from: Hayo Baan on May 22, 2019, 02:15:09 AM
BUT looking again at the list, I think this actually does uncover a bug:
Nice catch. Thanks! I'll fix this in the next release.
- Phil
Quote from: Phil Harvey on May 22, 2019, 07:31:44 AM
Nice catch. Thanks! I'll fix this in the next release.
Excellent! Glad that my persistence into getting to the bottom of something seemingly small actually led to some good in the end!