maximising performance

[quitetall]

Hi

I'm scraping EXIF data from my entire library, which is about 40,000 files. It takes about an hour and a half. I'm wondering if this is due to my script:

exiftool -if '$model eq "NIKON D300"' -T -r -directory -filename -createdate -FileModifyDate -ISO -shutterspeed -fnumber -focallength -exposureprogram -model /Photos/ > exif.D300.tab

and if I can speed it up? Does running a script like this call exiftool every time or does it stay running?

TIA!

[Phil Harvey]

That's 0.135 seconds per file.  Not bad really.  Your command loads exiftool only once, so the loading overhead is insignificant.  But since you aren't extracting any maker notes information you could add -fast2 to your command to speed things up.  It would be interesting to know how much faster it runs for you with this option.

- Phil

Edit: I tried this with a bunch of sample Nikon images and I got about a 50% speed improvement with -fast2.  If you get the same, your total time would drop to about an hour -- still a long time.  But on my system (a new iMac) the processing is about 10x faster (0.014 sec/file) to begin with.

[quitetall]

using the -fast2 switch seems to reduce the overall time by about a quarter.

I have a few more tests and queries so will have more data.

[quitetall]

better estimates:

I have 58,173 files, which are on a NAS which I mount by AFP on 802.11n.

Without the -fast2 switch it takes about 2 hours to run, with the switch about 1.5 hours. Thats with the '-if' switch as well.

As it happens I can also query locally via firewire drive, so I'll test that.

[quitetall]

Over Firewire the query takes 50 minutes to run