Is there a simple way to find out how many bytes my metadata occupies?
( I am trying to understand und resolve the
Error: Can't write multi-segment EXIF with external pointers
Is there a fixed maximum size (a segment) allocated for metadata?
)
I seem to recall having looked (many many years ago) at the beginning of a jpg file in a hex editor to find
the start of the image data. Anything before was metadata I think. Is this still doable?
For a jpeg, the largest a segment can be is 64k. With some metadata, XMP I think, exiftool can span multiple segments with the data.
You can see the size like this (see Extra tags (https://exiftool.org/TagNames/Extra.html) for all options)
C:\>exiftool -G1 -a -s -xmp -iptc -exif Y:\Pictures\Camera_Pictures\!Starting_Over\Hollywood_Show\2013-01\2013-01-12_10.59.19-2.Jpg
[XMP] XMP : (Binary data 4177 bytes, use -b option to extract)
[IPTC] IPTC : (Binary data 76 bytes, use -b option to extract)
[EXIF] EXIF : (Binary data 44973 bytes, use -b option to extract)
When I look at my exif block I see it is (for the file attached) 65k bytes.
Of that the thumbnail is 2351 bytes. When I list the exif data I can not see how this can use the remaining 63 k bytes.
There must be some hidden data in the EXIF block.
Somehow I need to be able to reduce the size of the EXIF block in the cases where it exeeds the 64k block boundary.
The attached file was generated from a Nikon NEF file using Adobe camera raw. I then had to rewrite it using irfanview in order
to be able to remove private information. This reduced the exif block from 79k to 65k - still unexplainably big.
PS: the Author/copyright info I removed using the Imatch image management program. Imatch can not handle the multi-segment exif in this case - this forced me to
first reduce the block by using an irfanview re-write.
Attached the jpg file in zipped form.
C:\>exiftool -G1 -a -s -MakerNotes "Y:\!temp\aa\2021-08-13 158 irfan.jpg"
[ExifIFD] MakerNoteNikon : (Binary data 61996 bytes, use -b option to extract)
MakerNotes are embedded in the EXIF block. So all that Nikon stuff is in that block. It doesn't seem that there is enough data there either (run exiftool -MakerNotes:all file.jpg to see the data).
Ah, found it. Adding in the -U (unknown) option (https://exiftool.org/exiftool_pod.html#u--unknown) and an addition 53,000 unknown tags show up. All starting with "Nikon_ShotInfoZ7"
Unfortunately, MakerNotes tags cannot be individually deleted, so I don't know if there's a way to deal with this without just deleting the MakerNotes (-MakerNotes=).
Phil would have to comment on any other possible options.
StarGeek you are a STAR!
The -U option revealed the problem.
I will see how I can remove the unwanted NIKON Makernotes.
Thanks
May I post a wish here to Phil:
Add the possibility to delete the Unknown Makernotes produced by a Nikon Z6 (presumably also by similar models)
This metadata tends to extend the EXIF block beyond the 64 k block size which causes downstream problems.
(Sample file in post above)
Individual MakerNotes cannot be deleted. Unfortunately, the only option would be to delete the MakerNotes in its entirety.
exiftool -MakerNotes= file.jpg
But you could always copy the MakerNotes back into the file after the other programs are done with it
exiftool -TagsFromFile original.jpg -MakerNotes new.jpg
I tried copying the MakeNotes back - but unfortunately this also copies the UNKNOWN fields.
I also tried :all, :Nikon and :VRMode - neither works.
Seems I/we do need Phil's help - or do with Makernotes altogether.
Quote from: ubacher on August 22, 2021, 02:48:55 AM
I tried copying the MakeNotes back - but unfortunately this also copies the UNKNOWN fields.
Yes, this can't be avoided. What I meant was to copy the MakerNotes back over after using the program that has problems with the file. That way the program that has a problem never sees the data.
Quote from: ubacher on August 20, 2021, 10:27:24 AM
PS: the Author/copyright info I removed using the Imatch image management program. Imatch can not handle the multi-segment exif in this case - this forced me to first reduce the block by using an irfanview re-write.
I finally noticed this. Unless you're using a very old version of Imatch, it uses exiftool to write the metadata and shouldn't have a problem with large EXIF data. Imatch3 was the last to write its own data, I think.
Imatch gives a warning that a mult-segment exif block is required (passed on from exiftool) - it requires a reload of the file to make the warning go away.
Actually just a nuisance - but - as I tried removing Author/Copyright from a file it does not work. I wanted to clear the situation
first at the exiftool forum before logging a bug report in Imatch.
Still - having a 60k bytes of non-visible/unused info in each jpg file seems/is a waste.
Is this an original JPG, or has the metadata been copied from a NEF file? In the past I have had to block some makernote data from being copied from NEF to JPG due to size restrictions. Also, I can add the ability to delete specific makernote data if it is necessary. I will look into this in more detail when I get a chance (probably in a couple of weeks).
- Phil
The makernotes was copied from the NEF file. (Using Imatch which uses exiftool)
QuoteIn the past I have had to block some makernote data from being copied from NEF to JPG due to size restrictions.
Looks like this is another of these cases. I suppose these UNKNOWN makernotes which Nikon Z6 adds specify settings in the camera -
to allow the looks in the camera to be duplicated when processing the raw file.
(Nikon Lens Data, File Info, Shot Info, Color Balance, Flash info.)