ExifTool Forum

ExifTool => Developers => Topic started by: jishi92 on November 09, 2022, 03:42:51 AM

Title: About exiftool stay_open using multi-process
Post by: jishi92 on November 09, 2022, 03:42:51 AM
Hi guys:
I use exiftool command line in Go.
I have used `stay_open` feature of exiftool to get a speed up compared to invoking one exiftool instance for each image.
However, the processing queue will accumulate if the speed of requests is greater than the speed of processing.Then the response-time will increase a lot.
I have too many images need to process.
Does the `stay_open` pattern support concurrent multi-process processing?
If yes, can you provide a demo code.
I would appreciate some guidance on where to look.
Title: Re: About exiftool stay_open using multi-process
Post by: Phil Harvey on November 09, 2022, 07:40:03 AM
All you need to do is create multiple instances of the exiftool application and send the commands to them in parallel.  You already have the code to do this if you have created one.

- Phil
Title: Re: About exiftool stay_open using multi-process
Post by: jishi92 on November 09, 2022, 10:59:41 PM
Thanks Phil, I already  have  the code and test it done. It's successful.
Title: Re: About exiftool stay_open using multi-process
Post by: jishi92 on November 10, 2022, 08:06:18 AM
Hey Phil:

I have another question. Does the `stay_open` pattern support input []byte data?
If I have []byte data from internet and now I have to save it a file and "exiftool -stay_open True -@ - filename -execute ".
I try write the file to ramdisk but it's slower than write it to disk immediately.
I don't know why that is.
Did I understand something wrong?
Title: Re: About exiftool stay_open using multi-process
Post by: Phil Harvey on November 10, 2022, 08:15:47 AM
Are you asking if you can send the image file directly to ExifTool rather than read it off a disk?  I don't know of a way to do this when using -stay_open.

- Phil
Title: Re: About exiftool stay_open using multi-process
Post by: jishi92 on November 15, 2022, 07:56:50 AM
Thank you Phil, I have process my images using multi-process processing in my code.
But I don't know is it concurrent safe?
I don't find any error in my concurrent test.
I want to confirm my result.
Title: Re: About exiftool stay_open using multi-process
Post by: Phil Harvey on November 15, 2022, 02:19:51 PM
Each thread must be running its own copy of the exiftool application.  If you do this, then the only possible conflict is if you try to process the same files simultaneously.

- Phil
Title: Re: About exiftool stay_open using multi-process
Post by: jishi92 on November 16, 2022, 07:33:25 AM
A file will be process in a thread and a thread will process many files.
So it's safe.
Thanks very much.