Questions about -overwrite_original -tagsfromfile usage (IFD0, SubIFD,...)

[Guillermo Luijk]

Hi, first of all thanks Phil for such a nice piece of software. Heard of it many years ago but (unfortunately) didn't decide to start using it until now.

I have edited a previous batch script by Anton Wolf: dng_stacker, adapting it from stacking several DNG RAW files to my personal needs, which are simply to be able to build an output DNG file based on a Bayer TIFF file, replicating the relevant image metadata found on an input DNG file. You can see some examples of this application here (Google Translation available) (HDR DNG, low ISO/ND filter simulation trough stacking and linear decomposition of light in a scene).



The point is that the script works perfectly fine so I'm not going to ask anything about debugging or issues. I just want to fully understand the syntax and exiftool commands.

To this piece of original code:

exiftool -n^
-IFD0:SubfileType#=0^
-overwrite_original -tagsfromfile !firstFile!.dng^
"-all:all>all:all"^
-DNGVersion^
-DNGBackwardVersion^
-ColorMatrix1 -ColorMatrix2^
"-IFD0:BlackLevelRepeatDim<SubIFD:BlackLevelRepeatDim"^
"-IFD0:PhotometricInterpretation<SubIFD:PhotometricInterpretation"^
"-IFD0:CalibrationIlluminant1<SubIFD:CalibrationIlluminant1"^
"-IFD0:CalibrationIlluminant2<SubIFD:CalibrationIlluminant2"^
-SamplesPerPixel^
"-IFD0:CFARepeatPatternDim<SubIFD:CFARepeatPatternDim"^
"-IFD0:CFAPattern2<SubIFD:CFAPattern2"^
-AsShotNeutral^
"-IFD0:ActiveArea<SubIFD:ActiveArea"^
"-IFD0:DefaultScale<SubIFD:DefaultScale"^
"-IFD0:DefaultCropOrigin<SubIFD:DefaultCropOrigin"^
"-IFD0:DefaultCropSize<SubIFD:DefaultCropSize"^
"-IFD0:OpcodeList1<SubIFD:OpcodeList1"^
"-IFD0:OpcodeList2<SubIFD:OpcodeList2"^
"-IFD0:OpcodeList3<SubIFD:OpcodeList3"^
!exposureTag!^
temp.dng >> dngmaker.log 2>>&1



I added (I needed the ForwardMatrices to be copied to properly render colours for the DNG files in ACR. Other metadata are already in the previous command but I wanted to summarize in a single command all camera-related metadata):

exiftool -overwrite_original -tagsfromfile !firstFile!.dng^
"-IFD0:AnalogBalance"^
"-IFD0:ColorMatrix1" "-IFD0:ColorMatrix2"^
"-IFD0:CameraCalibration1" "-IFD0:CameraCalibration2"^
"-IFD0:AsShotNeutral" "-IFD0:BaselineExposure"^
"-IFD0:CalibrationIlluminant1" "-IFD0:CalibrationIlluminant2"^
"-IFD0:ForwardMatrix1" "-IFD0:ForwardMatrix2"^
temp.dng >> dngmaker.log 2>>&1


Questions:

• If the -all:all>all:all command is present, why are any explicit references to some of the metadata needed? doesn't 'all' copy 'all'?
• What's the difference between -ColorMatrix1 and "-IFD0:ColorMatrix1"? do they do the same? what is -IFD0:SubfileType#=0?
• Why do some commands in Wolf's script appear with such a <SubIFD reference like in "-IFD0:CFARepeatPatternDim<SubIFD:CFARepeatPatternDim"? does it mean any difference to simply doing "-IFD0:CFARepeatPatternDim" or just  "-CFARepeatPatternDim"?

I have to say I tried to find out in the documentation but got a bit lost. Exiftool looks like a powerful tool and hence not so straightforward at a first glance.

Best regards

[Phil Harvey]

Hi Guillermo,

• If the -all:all>all:all command is present, why are any explicit references to some of the metadata needed? doesn't 'all' copy 'all'?

Because it copies to the same family 1 groups.  The explicit references are copying to another group.

• What's the difference between -ColorMatrix1 and "-IFD0:ColorMatrix1"? do they do the same? what is -IFD0:SubfileType#=0?

They are the same since EXIF is the preferred metadata type, and IFD0 is the default family 1 group (ie. location) in EXIF.  See the EXIF tags documentation for a list of default groups for all EXIF tags.

• Why do some commands in Wolf's script appear with such a <SubIFD reference like in "-IFD0:CFARepeatPatternDim<SubIFD:CFARepeatPatternDim"? does it mean any difference to simply doing "-IFD0:CFARepeatPatternDim" or just  "-CFARepeatPatternDim"?

This argument copies CFARepeatPatternDim from the SubIFD to IFD0.  See the paragraph starting with "A powerful redirection feature" in the the -tagsfromfile section of the exiftool application documentation.

- Phil

[StarGeek]

To give an example of -All vs -All:All, you can create a file with the XMP:Headline tag set

Code Select Expand

C:\>exiftool -P -overwrite_original -xmp:headline=Headline y:\!temp\Test4.jpg
    1 image files updated

C:\Programs\My_Stuff>exiftool -G1 -a -s -xmp:all y:\!temp\Test4.jpg
[XMP-x]         XMPToolkit                      : Image::ExifTool 12.25
[XMP-photoshop] Headline                        : Headline

If you copy data with just -All, then tags will be copied to tags with the same name, but in the preferred group

Code Select Expand

C:\>exiftool -P -overwrite_original -TagsFromFile y:\!temp\Test4.jpg -all y:\!temp\Test3.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -e y:\!temp\Test4.jpg  y:\!temp\Test3.jpg
======== y:/!temp/Test4.jpg
[ExifTool]      ExifToolVersion                 : 12.25
[System]        FileName                        : Test4.jpg
[System]        Directory                       : y:/!temp
[System]        FileSize                        : 432 KiB
[System]        FileModifyDate                  : 2021:04:17 21:41:57-07:00
[System]        FileAccessDate                  : 2021:04:28 18:33:03-07:00
[System]        FileCreateDate                  : 1899:12:31 23:59:59-08:00
[System]        FilePermissions                 : -rw-rw-rw-
[File]          FileType                        : JPEG
[File]          FileTypeExtension               : jpg
[File]          MIMEType                        : image/jpeg
[File]          ImageWidth                      : 1205
[File]          ImageHeight                     : 1744
[File]          EncodingProcess                 : Baseline DCT, Huffman coding
[File]          BitsPerSample                   : 8
[File]          ColorComponents                 : 3
[File]          YCbCrSubSampling                : YCbCr4:2:0 (2 2)
[XMP-x]         XMPToolkit                      : Image::ExifTool 12.25
[XMP-photoshop] Headline                        : Headline
======== y:/!temp/Test3.jpg
[ExifTool]      ExifToolVersion                 : 12.25
[System]        FileName                        : Test3.jpg
[System]        Directory                       : y:/!temp
[System]        FileSize                        : 648 KiB
[System]        FileModifyDate                  : 2020:07:07 08:35:22-07:00
[System]        FileAccessDate                  : 2021:04:28 18:33:03-07:00
[System]        FileCreateDate                  : 2019:10:11 15:30:27-07:00
[System]        FilePermissions                 : -rw-rw-rw-
[File]          FileType                        : JPEG
[File]          FileTypeExtension               : jpg
[File]          MIMEType                        : image/jpeg
[File]          CurrentIPTCDigest               : 3ca265e887dc741ac7f9a2d82c796727
[File]          ImageWidth                      : 1920
[File]          ImageHeight                     : 1080
[File]          EncodingProcess                 : Baseline DCT, Huffman coding
[File]          BitsPerSample                   : 8
[File]          ColorComponents                 : 3
[File]          YCbCrSubSampling                : YCbCr4:2:0 (2 2)
[IPTC]          Headline                        : Headline
[IPTC]          ApplicationRecordVersion        : 4
[XMP-x]         XMPToolkit                      : Image::ExifTool 12.25
[XMP-tiff]      BitsPerSample                   : 8
[XMP-tiff]      ImageHeight                     : 1744
[XMP-tiff]      ImageWidth                      : 1205
[XMP-tiff]      YCbCrSubSampling                : YCbCr4:2:0 (2 2)

As you can see, XMP-photoshop:Headline was copied to the IPTC:Headline tag, not the XMP-photoshop:Headline tag.  That is because IPTC is preferred over XMP by exiftool.  Additionally, you can see that several tags that were in the File group were copied to the XMP-tiff group, even though tags of these names exist in the EXIF group.  The reason they weren't copied to the EXIF tags is because those EXIF tags are marked as Unsafe to copy.  Unsafe tags are marked with an exclamation point in the EXIF tag page linked above.  So they end up in the XMP group instead.

With -All:All, the tags will be copied to the exact same group as they originally started in.  IPTC:Headline and the XMP-tiff tags will not be created.

[Guillermo Luijk]

Thanks a lot Phil and StarGeek, I think I'm already on the track to get a better understanding of exiftool.

Best regards