Efficiently correct time offset in huge directory tree

[feklee]

I forgot to adjust my Panasonic DMC-ZX1 from daylight savings time on December 31. Now I want to offset all affected files by one hour. The challenge is that they are spread over a big directory structure. As I learned in another thread, I can use exiftool to find files in a certain date range. To speed things up, I am thinking about combining find and exiftool where find looks at the last modification date. It's a bit convoluted, and I could split that up into several steps (Bash):

Code Select Expand


D=2021-10-31
M=DMC-ZX1
find . -iname "*.jpg" -newermt $D -exec exiftool -q -s3 -filename -if "\$model eq 'DMC-ZX1' and \$datetimeoriginal gt '${D//-/:}'" {} \; | while read a; do
    exiftool "-AllDates-=1" "$a"
    exiftool -"FileModifyDate<DateTimeOriginal" "$a"
done


What do you think, can that be simplified?

It's likely an issue that I'll face again.

[Phil Harvey]

You are suffering from common mistake #3.

Try this:

exiftool -if '$datetimeoriginal gt "2020:12:31"' -globaltimeshift -1 '-filemodifydate<datetimeoriginal' -alldates -r DIR

(quoting is for Mac/Linux.  Substitute the date you want in the -if expression.)

This command is very similar to the example in the -if section of the application documentation

                # add one hour to all images created on or after Apr. 2, 2006
                exiftool -alldates+=1 -if '$CreateDate ge "2006:04:02"' dir

The difference is that I am using the -globalTimeShift option to allow the times to be shifted before writing FileModifyDate.

- Phil

[feklee]

You are suffering from common mistake #3.

That was my suspicion, thank you!

Still I may combine your suggestion with find, because the directory tree is deep and I don't want ExifTool to waste time by parsing files that don't match based on their modification date and file name.

I did benchmarks in a sub directory:

Code Select Expand


$ time find . -iname "*.jpg" -newermt 2021-10-31
[...]

real    0m3.515s
user    0m0.000s
sys     0m0.244s
$ time exiftool -if '$filemodifydate gt "2021:10:31"' -s3 -filename -q -r .
[...]

real    0m31.241s
user    0m14.148s
sys     0m5.800s


I also tried telling ExifTool to only look for JPEGs, but there were no matches:

Code Select Expand


$ time exiftool -if '$filename ~= /jpg$/i and $filemodifydate gt "2021:10:31"' -s3 -filename -q -r .


What I did in the end:

Code Select Expand


$ D=2021-10-31
$ M=DMC-ZX1
$ find . -iname "*.jpg" -newermt $D | while read a; do
    DTO=${D//-/:}
    exiftool -if '$model eq "'$M'" and $datetimeoriginal gt "'$DTO'"' -globaltimeshift -1 '-filemodifydate<datetimeoriginal' -alldates -r "$a"
done


[Phil Harvey]

I don't want ExifTool to waste time by parsing files that don't match based on their modification date and file name.

Then use -if5 instead of -if

I also tried telling ExifTool to only look for JPEGs, but there were no matches:

Code Select Expand


$ time exiftool -if '$filename ~= /jpg$/i and $filemodifydate gt "2021:10:31"' -s3 -filename -q -r .


It is much more efficient to use -ext jpg than to do this in a -if argument.

What I did in the end:

I still think my way will be faster:

exiftool -ext jpg -if5 '$filemodifydate gt "2021:10:31"' -s3 -filename -q -r DIR

- Phil

[feklee]

exiftool -ext jpg -if5 '$filemodifydate gt "2021:10:31"' -s3 -filename -q -r DIR

Neat! find is still somewhat faster, but I understand that to the above command I can also add the editing steps, and then Exiftool only needs to start once.

Already I have the files fixed, but almost certainly I'll forget to adjust the time zone of my camera again. 

[Phil Harvey]

Yes.  My way executes ExifTool only once, which will result in a large time savings if there are a number of files to process.  But it will be slower than using "find" if there are very few files to edit.

A couple of quick tests here give a factor of 16x speed increase with -if5, plus an additional 2x with -ext jpg.

- Phil

[StarGeek]

While I always encourage the using native exiftool options, if find is easier for you to use, then what you should do is have find save the file list to a text file and then have exiftool use that list with the -@ (Argfile) option.  Something like
find . -iname "*.jpg" -newermt 2021-10-31 >Filelist.txt
exiftool -globaltimeshift -1 '-filemodifydate<datetimeoriginal' -alldates -@ Filelist.txt

[feklee]

-@ (Argfile) option

Thanks, this is certainly a nice alternative! That two step process feels safer.

[Phil Harvey]

@StarGeek: Good idea.  You could do it without a temporary file, like this:

find . -iname "*.jpg" -newermt 2021-10-31 | exiftool -globaltimeshift -1 '-filemodifydate<datetimeoriginal' -alldates -@ -

- Phil