Help optimizing exiftool script

[vicmarto]

Hello!

Before run the ExifTool script I wrote to all my files, I would like to show it to all. Do you see any problem? Would be possible to optimise it a bit for speed? (I feel some executes can be merged, but my knowledge is so limited!)

The intended behaviour is:
  1) JPG: Rewrite all tags and delete "IPTC, XMP, Photoshop and PhotoMechanic" + Write a good FileModifyDate using some tags
  2) RAW: Delete "IPTC, XMP, Photoshop and PhotoMechanic" + Write a good FileModifyDate using some tags
  3) RAW + JPG: Write FileName using the FileModifyDate (that was updated before)
  4) REST (png, avi, mov, mp4, m4v) : Write a good FileModifyDate using some tags + Write FileName using the FileModifyDate (that was updated before)

Code Select Expand

exiftool \
    -ext 'jpg' -ext 'jpeg' -all= -tagsFromFile @ -all:all --IPTC:all --XMP:all --Photoshop:all --PhotoMechanic:all -unsafe \
    '-FileModifyDate<${MetadataDate;$_ = undef if /^0000/}' '-FileModifyDate<${ModifyDate;$_ = undef if /^0000/}' '-FileModifyDate<${CreateDate;$_ = undef if /^0000/}' '-FileModifyDate<${DateTimeOriginal;$_ = undef if /^0000/}' \
    -execute \
    -ext 'arw' -ext 'dng' -ext 'nef' -ext 'tif' -ext 'tiff' -IPTC:all= -XMP:all= -Photoshop:all= -PhotoMechanic:all= \
    '-FileModifyDate<${MetadataDate;$_ = undef if /^0000/}' '-FileModifyDate<${ModifyDate;$_ = undef if /^0000/}' '-FileModifyDate<${CreateDate;$_ = undef if /^0000/}' '-FileModifyDate<${DateTimeOriginal;$_ = undef if /^0000/}' \
    -execute \
    -ext 'arw' -ext 'dng' -ext 'jpg' -ext 'jpeg' -ext 'nef' -ext 'tif' -ext 'tiff' \
    -d "output_dir/%Y/%m/%Y-%m-%d %H.%M.%S" \
    '-FileName<$FileModifyDate %f.$FileTypeExtension' \
    '-FileName<$FileModifyDate ${SubSecTime;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<$FileModifyDate ${SubSecTimeDigitized;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<$FileModifyDate ${SubSecTimeOriginal;$_=substr($_.'0',0,2)}.$FileTypeExtension' \
    -execute \
    -ext 'png' -IPTC:all= -XMP:all= -Photoshop:all= \
    '-FileModifyDate<${MetadataDate;$_ = undef if /^0000/}' '-FileModifyDate<${DateCreated;$_ = undef if /^0000/}' \
    -execute \
    -ext 'png' \
    -d "output_dir/PNG/%Y-%m-%d %H.%M.%S" \
    '-FileName<$FileModifyDate %f.$FileTypeExtension' \
    -execute \
    -ext 'avi' \
    '-FileModifyDate<${DateTimeOriginal;$_ = undef if /^0000/}' \
    -execute \
    -ext 'avi' \
    -d "output_dir/%Y/%m/%Y-%m-%d %H.%M.%S" \
    '-FileName<$FileModifyDate %f.$FileTypeExtension' \
    -execute \
    -ext 'mov' -ext 'mp4' -ext 'm4v' \
    -api largefilesupport=1 '-FileModifyDate<${CreateDate;$_ = undef if /^0000/}' '-FileModifyDate<${CreationDate;$_ = undef if /^0000/}' \
    -execute \
    -ext 'mov' -ext 'mp4' -ext 'm4v' \
    -d "output_dir/%Y/%m/%Y-%m-%d %H.%M.%S" \
    -api largefilesupport=1 '-FileName<$FileModifyDate %f.$FileTypeExtension' \
    -common_args '-FilePermissions=rw-r--r--' -if '$FileName !~ /^\./' -overwrite_original -preserve -progress -recurse "input_dir"



Any idea? THANKS!!


--
This is under FreeBSD 11.2-RELEASE-p9 and exiftool 11.01

[Phil Harvey]

Your command looks pretty good (although I would probably use an -@ argfile for this instead of such a long command line).

The only optimization I can see is to combine setting the FileModifyDate and FileName in the same command for each type of file.  Unfortunately this would mean duplicating the FileName argument for each tag you are using to set FileModifyDate, so the command would get even more complicated.  This may gain you a factor of 2 in speed, but add complexity.  If you weren't using FileTypeExtension when setting FileName, you could speed these commands by adding -fast4, and get almost the same speed benefit as combining the commands.

- Phil

[vicmarto]

Thanks Phil for your suggestions.

Finally I have ditched the "IPTC, XMP, Photoshop and PhotoMechanic" elimination, and have combined the FileModifyDate and FileName settings in a (beautiful) long monster command:

Code Select Expand

exiftool \
        -ext 'arw' -ext 'dng' -ext 'jpg' -ext 'jpeg' -ext 'nef' -ext 'tif' -ext 'tiff' \
        -d "output_dir/3 FileName/%Y/%m/%Y-%m-%d %H.%M.%S" \
        '-FileName<$FileModifyDate %f.$FileTypeExtension' \
        '-FileModifyDate<${MetadataDate;$_ = undef if /^0000/}' '-FileName<${MetadataDate;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileName<${MetadataDate;$_ = undef if /^0000/} ${SubSecTime;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${MetadataDate;$_ = undef if /^0000/} ${SubSecTimeDigitized;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${MetadataDate;$_ = undef if /^0000/} ${SubSecTimeOriginal;$_=substr($_.'0',0,2)}.$FileTypeExtension' \
        '-FileModifyDate<${ModifyDate;$_ = undef if /^0000/}' '-FileName<${ModifyDate;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileName<${ModifyDate;$_ = undef if /^0000/} ${SubSecTime;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${ModifyDate;$_ = undef if /^0000/} ${SubSecTimeDigitized;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${ModifyDate;$_ = undef if /^0000/} ${SubSecTimeOriginal;$_=substr($_.'0',0,2)}.$FileTypeExtension' \
        '-FileModifyDate<${CreateDate;$_ = undef if /^0000/}' '-FileName<${CreateDate;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileName<${CreateDate;$_ = undef if /^0000/} ${SubSecTime;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${CreateDate;$_ = undef if /^0000/} ${SubSecTimeDigitized;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${CreateDate;$_ = undef if /^0000/} ${SubSecTimeOriginal;$_=substr($_.'0',0,2)}.$FileTypeExtension' \
        '-FileModifyDate<${DateTimeOriginal;$_ = undef if /^0000/}' '-FileName<${DateTimeOriginal;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileName<${DateTimeOriginal;$_ = undef if /^0000/} ${SubSecTime;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${DateTimeOriginal;$_ = undef if /^0000/} ${SubSecTimeDigitized;$_=substr($_.'0',0,2)}.$FileTypeExtension' '-FileName<${DateTimeOriginal;$_ = undef if /^0000/} ${SubSecTimeOriginal;$_=substr($_.'0',0,2)}.$FileTypeExtension' \
        -execute \
        -ext 'png' \
        -d "output_dir/3 FileName/PNG/%Y-%m-%d %H.%M.%S" \
        '-FileName<$FileModifyDate %f.$FileTypeExtension' \
        '-FileModifyDate<${MetadataDate;$_ = undef if /^0000/}' '-FileName<${MetadataDate;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileModifyDate<${DateCreated;$_ = undef if /^0000/}' '-FileName<${DateCreated;$_ = undef if /^0000/} %f.$FileTypeExtension' \
        -execute \
        -ext 'avi' \
        -d "output_dir/3 FileName/%Y/%m/%Y-%m-%d %H.%M.%S" \
        '-FileName<$FileModifyDate %f.$FileTypeExtension' \
        '-FileModifyDate<${DateTimeOriginal;$_ = undef if /^0000/}' '-FileName<${DateTimeOriginal;$_ = undef if /^0000/} %f.$FileTypeExtension' \
        -execute \
        -ext 'mov' -ext 'mp4' -ext 'm4v' \
        -d "output_dir/3 FileName/%Y/%m/%Y-%m-%d %H.%M.%S" \
        -api largefilesupport=1 \
        '-FileName<$FileModifyDate %f.$FileTypeExtension' \
        '-FileModifyDate<${CreateDate;$_ = undef if /^0000/}' '-FileName<${CreateDate;$_ = undef if /^0000/} %f.$FileTypeExtension' '-FileModifyDate<${CreationDate;$_ = undef if /^0000/}' '-FileName<${CreationDate;$_ = undef if /^0000/} %f.$FileTypeExtension' \
        -common_args '-FilePermissions=rw-r--r--' -if '$FileName !~ /^\./' -overwrite_original -preserve -progress -recurse "input_dir"

THANKS!

[vicmarto]

If you weren't using FileTypeExtension when setting FileName, you could speed these commands by adding -fast4, and get almost the same speed benefit as combining the commands.

About the -fast option, which is the fastest level (from -fast to -fast4) that can be used in these cases:

    1) exiftool FILE -all= -out - | md5 Here I'm trying to pipe to md5 to get something like a "fingerprint" of each picture

    2) exiftool -printFormat '$FileTypeExtension' FILE Here I'm trying to just get the -FileTypeExtension, as part of a more big command

Thanks!

[Phil Harvey]

-fast doesn't affect command 1.

Command 2 should work with -fast3 for most file types, but some file types (specifically some TIFF-based RAW formats) require more parsing to distinguish them from TIFF files.

- Phil

[vicmarto]

Command 2 should work with -fast3 for most file types, but some file types (specifically some TIFF-based RAW formats) require more parsing to distinguish them from TIFF files.

Thanks Phil. Yes, I plan to use this command with either tiff and raw files. Then, would be -fast2 enough just in case?

[Phil Harvey]

Sorry for the delay in responding.  Yes, -fast2 should do it.

- Phil