Add Paginator

[laenglish]

We have a directory with 10k files in. Running EXIFTool on this directory takes a long time (understandable).

Would it be possible to add a paginator so we can pass in a sort, limit and offset so that it only runs on a selection of the files

For example

Code Select Expand

exiftool -sort name -limit 1000 -offset 500 /my/big/directory
Would output data for 1000 files starting a file 500 when ordered by name

[Phil Harvey]

The cost of adding ExifTool options is that the (already lengthy) documentation gets bloated... eventually it will get so long that nobody ever reads it. [We may have already passed that point.]

I think adding 2 options for this likely-very-seldom-used (perhaps even only-used-by-you) feature is too high of a cost.  Instead, doing something like this works in Mac/Linux:

ls /my/big/directory/* | sed -n '500,1500p;1501q' | exiftool -@ -

But there may be easier ways to do this.

- Phil

[StarGeek]

The ability to sort can be found in the -FileOrder option.

[laenglish]

The cost of adding ExifTool options is that the (already lengthy) documentation gets bloated... eventually it will get so long that nobody ever reads it. [We may have already passed that point.]

I think adding 2 options for this likely-very-seldom-used (perhaps even only-used-by-you) feature is too high of a cost.  Instead, doing something like this works in Mac/Linux:

ls /my/big/directory/* | sed -n '500,1500p;1501q' | exiftool -@ -

But there may be easier ways to do this.

- Phil

This is actually what are currently doing - but the issue we have is that ExifTool still runs its command on the 10k file, and we then just filter the output. So its extremely wasteful.

Could one param do it all?

-limit 1,500 (records 1 to 500)
-limit 500,500 (records 500 to 1000)

Similar to MYSQL LIMIT https://www.guru99.com/limit.html

[laenglish]

The ability to sort can be found in the -FileOrder option.

Thanks!

[laenglish]

Apologies Phil, we arent doign what you proposed actually.

We are runing exiftool on a directory and then piping that through jq which will indeed cause it to run on all the files.

exiftool my/directory/* | jq []

I will try your approach and report back, that looks like it would work just fine (providing I can sort the files before run sed on them)

[Phil Harvey]

(providing I can sort the files before run sed on them)

ls /my/big/directory/* | sort | sed -n '500,1500p;1501q' | exiftool -@ -

- Phil