-execute and 2 different format files and 2 output files ?

[Iwonder]

@StarGeek
Hello
coming back with the results !
Although I didn't manage this running all this in only 3 ligns with my environment(but it's not important), the The whole process only lasted... 15 minutes instead of 3.5 hours !!!!!
This is amazing !!!
Thank you so much !
One more question about this command line : Could you explain this -@ {} ?
On the help file I only can see that -@ is used for introducing an argument file, but there is no ARG file here...

[StarGeek]

Hmmm... looking at it now, I think that I made a mistake in adding the -@. For some reason, I was thinking that find would be piping the data. The -@ option can read data from a pipe, redirection, or STDIN, but that would have to be -@ - and I think find is directly providing the file list, which is inserted by find at the { }.

As a result, I think your processed list is one file short, as the -@ option would try to read the first file as an ARGS file.  Try running it again and dropping the -@.

Never mind, see below

[Iwonder]

in fact I tried omitting -@ {}
but it doesn't work, giving me an help page for ExifTool instead
but I don't know how to comment this

[StarGeek]

Yeah, it's needed.

Find is looking for the 1,000 line file list text files and exiftool is using -@ to read each of those for the list of files to process. I was taking a nap and remembering this woke me up to come make this post

[Iwonder]

thank you !
Hope you went back to your bed, with a rest mind ^^

[Phil Harvey]

Yes, -@ shouldn't be there.

Also, your "find" is looking for .txt files, but your exiftool command processes only .jpg files.

And I needed quotes around the semicolon because "find" needs it to terminate the arguments, and without quotes it was eaten by the shell.

Other than that, the command worked for me.

- Phil

[StarGeek]

No, -@ does need to be there. Find isn't feeding file names directly to exiftool. It's giving exiftool text files with 1,000 filepaths per text file.

Back in this post, I used find to gather all the filenames to be processed. It takes exiftool several minutes to generate a list for 28,000+ files, even with -fast5, while find was able to generate the list in a couple seconds.

Then split is used to split the results of that find into separate files with 1,000 filepaths per file named "OutputList##.txt"

find is used again to gather the names of each of these "OutputList##.txt" files and that is what is passed to exiftool with the -@, running exiftool once per "OutputList##.txt" file.

I don't know where the slow down is, maybe memory management like you said, but my 28K test directory was running for over 6 hours before I shut it down, while running it in 1,000 file batches only took 42 minutes.  And @Iwonder says splitting like this takes only 15 minutes instead of 3.5 hours.

[Phil Harvey]

No, -@ does need to be there. Find isn't feeding file names directly to exiftool. It's giving exiftool text files with 1,000 filepaths per text file.

Ah, sorry.  I missed that.  I didn't read the whole thread.

@Iwonder says splitting like this takes only 15 minutes instead of 3.5 hours.

When I get a chance I should look into this.

- Phil

[StarGeek]

When I get a chance I should look into this.

My first thought is that it's a problem with the Windows version, but I just realized I'm not sure what OS @Iwonder is using. For some reason I was thinking Windows, but Linux commands are listed.

[Iwonder]

@all

Yes I'm using Windows 10, with some .exe coming from UnixUtils for Windows

[StarGeek]

Regarding UnixUtils, you might want to take a look at MSYS2. MSYS2 versions are more up to date than UnixUtils.

[Iwonder]

@StarGeek
thank you for this suggestion.
if I knew this at the begining of this project, I surely would have used it.
have a nice day !