Increase performance

[romsVLM]

Hello, I created a PHP script that uses exiftool. However this script is much too slow.

I have a folder of images (about 10000 images).

I have an Excel file of about 3000 lines containing for each line:
- a title
- longitude
- latitude
- keywords
- a list of paths to my photos folder

My PHP script goes through the Excel file line by line, and for each line, it goes through the list of paths to the photos.

For each path, I execute an exiftool command which will overload the photo by adding the exif data of my Excel line on which I am (title, longitude, latitude, keywords).

Given that I have 10000 photos, I therefore have 10000 executions of my exiftool command...

What can I do in my case to reduce the loading time of my script?

Here is my command executed for each photo:

Code Select Expand

eval(`exiftool -php -overwrite_original -stay_open -e -n -fast2 -XPKeywords="$keywords" -Keywords="$keywords" -XPAuthor="$title" -XPTitle="$title" -XPSubject="$tagline" -GPSLatitude=$lat -GPSLongitude=$long $path`);
Thank you !

[StarGeek]

Given that I have 10000 photos, I therefore have 10000 executions of my exiftool command...

What can I do in my case to reduce the loading time of my script?

This is Common Mistake #3.  Because the startup time is exiftool's biggest performance hit, you never want to loop exiftool and run it once for each file.  What you want to do is run exiftool on all the files at once.

Since you have the data in an Excel file, the easiest thing to do would be to export the data to a CSV file and then use the -csv option to import the changes.

This might require some temporary restructuring of the Excel file, so you probably want to make a copy of it.

The first column must have a header of "SourceFile" and contain either the full path to the files or the relative path from the directory you run exiftool in.  The header of the other files must be the exact names of the tags you want to write. Also, you need to make sure that when writing the GPS coordinates, you must also write the corresponding reference direction tags, GPSLatitudeRef/GPSLongitudeRef.  Otherwise, any coordinates that are supposed to be in the Western or Southern hemisphere will not be correct.  The easiest thing to do here is to copy/paste the GPSLatitudeRef/GPSLongitudeRef and then add Ref to the headers.  While the Ref tags are supposed to be N/S/E/W, exiftool is smart enough to accept the coordinates for the Ref tags and figure out the proper value.

Another problem will be your keywords.  You're writing to both XPKeywords and Keywords.  These tags are structured differently.  The first is a semicolon separated string and the second is a list type tag, where each entry is totally separate from the others.  The best thing to do here would be to make sure the keywords are separated by semicolons, no extra spaces, and then you would use the -sep option, e.g. -Sep ;

You should end up with a CSV file that looks like this

Code Select Expand

SourceFile,XPKeywords,Keywords,XPAuthor,XPSubject,GPSLatitude,GPSLatitudeRef,GPSLongitude,GPSLongitudeRef
y:/!temp/Test3.jpg,Keyword 1;Keyword 2;Keyword 3,Keyword 1;Keyword 2;Keyword 3,My Title,My tagline,40.6892,40.6892,-74.0445,-74.0445
y:/!temp/Test4.jpg,Keyword 4;Keyword 5;Keyword 6,Keyword 4;Keyword 5;Keyword 6,My other Title,My other tagline,40.748817,40.748817,-73.985428,-73.985428
At that point, your command would be
exiftool -csv=/path/to/file.csv /path/to/files/

[StarGeek]

Creating a separate post to avoid a bigger wall of text.

If saving to CSV isn't an option, then what you would want to do is write the commands to a temporary text file and use the -@ (Argfile) option and -execute option.

Your arg file would end up like this

Code Select Expand

y:/!temp/Test3.jpg
-XPKeywords=Keyword 1;Keyword 2;Keyword 3
-Keywords=Keyword 1;Keyword 2;Keyword 3
-XPAuthor=My Title
-XPSubject=My tagline
-GPSLatitude=40.6892
-GPSLatitudeRef=40.6892
-GPSLongitude=-74.0445
-GPSLongitudeRef=-74.0445
-execute
y:/!temp/Test4.jpg
-XPKeywords=Keyword 4;Keyword 5;Keyword 6
-Keywords=Keyword 4;Keyword 5;Keyword 6
-XPAuthor=My other Title
-XPSubject=My other tagline
-GPSLatitude=40.748817
-GPSLatitudeRef=40.748817
-GPSLongitude=-73.985428
-GPSLongitudeRef=-73.985428
And you would then run the command
exiftool -@ /path/to/temp.txt -Common_args -sep ;

The -Common_Args option is required to make sure -sep ; is applied to each operation.  You do not need an -execute at the end and doing so will be the same as running exiftool by itself, usually listing all the data for all the files in the directory.

You need to make sure there are not trailing spaces.

Doing this will avoid the startup time, as exiftool will be running constantly across all files in the text file.  The filename must be included and each write operation separated by the -execute option.