Please help! Need to Optimize Command

Started by blue-j, June 09, 2022, 12:25:13 AM

Previous topic - Next topic

blue-j

I've been asked to grab width, height and resolution from 28,000 files, across 30 folders.  I'm executing this and it's taking forever:

exiftool -csv -filename -imagewidth -imageheight -xresolution -r /path/to/Folder > /Desktop/all.csv

Is there a way to make this faster? -stay_open didn't seem right, but I've never used it.

- J

greybeard

It shouldn't take forever unless something is going wrong. How long have you let it run?

Are there a lot of other files on that drive? You could try restricting to just the file types you need (-ext jpg or whatever)

Is this a particularly slow drive? Have you looked at a monitor to make sure its continuing to actually read the drive?

Have you tried running against a subset to get an idea of how long it's taking? 

I tried running this command against a relatively slow external spinning disk with a random selection of files and restricted it to 20K FujiFilm raw files and it took just 17 minutes.

Phil Harvey

You may run into memory slow-downs using -csv with so many files.  Try this instead:

exiftool -p "$filename,$imagewidth,$imageheight,$xresolution" -fast2 -r /path/to/Folder > /Desktop/all.csv

(but note that you will have problems if any file names contain a comma.)

I have also added -fast2 to help speed things up a bit.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

Quote from: Phil Harvey on June 09, 2022, 08:04:20 AM
(but note that you will have problems if any file names contain a comma.)

See FAQ #12, part that starts "But note that any values containing commas"

;)
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Phil Harvey

Quote from: StarGeek on June 09, 2022, 10:02:26 AM
Quote from: Phil Harvey on June 09, 2022, 08:04:20 AM
(but note that you will have problems if any file names contain a comma.)

See FAQ #12, part that starts "But note that any values containing commas"

;)

Very good!

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

blue-j

Thank you so much!  I really appreciate it.  It did help.  I neglected to mention a major confounding variable - these files were on a cloud storage platform mirrored locally, so each one had to be downloaded before ET could read it!

-J

blue-j

Quote from: Phil Harvey on June 09, 2022, 08:04:20 AM
...Try this instead:

exiftool -p "$filename,$imagewidth,$imageheight,$xresolution" -fast2 -r /path/to/Folder > /Desktop/all.csv


When I use this command, if any of the images are missing any of the metadata, it returns all as blank.  Is there a way to mitigate this outcome?

- J

Phil Harvey

I think the -p option documentation explains this well:

            If a specified tag does not exist, a minor warning is issued and
            the line with the missing tag is not printed.  However, the -f
            option may be used to set the value of missing tags to '-' (but
            this may be configured via the API MissingTagValue option), or the
            -m option may be used to ignore minor warnings and leave the
            missing values empty.  Alternatively, -q -q may be used to simply
            suppress the warning messages.


- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

blue-j

Thanks so much!  I had just found the -f solution but the -m one is better still!  I didn't think to check the -p documentation; I hadn't realized it was the primary switch.

- J