Scanning thousands of files using goExifTool

Started by anton, July 01, 2020, 06:17:42 PM

Previous topic - Next topic

anton

hi,

i have an issue to scan ~50k images. For this purpose i'm using https://github.com/barasher/go-exiftool/.
It does not matter how the script looks like. It works for the small amount of files. When it reaches 7721 items, then it hangs with no response.

Actually everything what goExiftool does is calling exif command in go in one session. It means it does not close the command for every file.
I checked the code of this tool and see no problems there. So, the reasonable question is - can exiftool scan thousands of files in one session?


PS:
sometimes exiftool cannot detect headers, that's ok. But it's just for info.

PPS:
i tried to recreated command session for each 1000 items. and with no luck.

best regards,
Anton

Phil Harvey

Hi Anton,

Quote from: anton on July 01, 2020, 06:17:42 PM
So, the reasonable question is - can exiftool scan thousands of files in one session?

Yes.  ExifTool can scan an unlimited number of files in a single command.  Unless you use the -csv option, in which case the number of files is limited by the available memory.

There is also a command-line length limitation on most systems, so if you are specifying the filenames individually you should use the -@ option instead of trying to put them all on the command line.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).