ExifTool Forum

ExifTool => The "exiftool" Application => Topic started by: grubin698 on March 13, 2025, 10:02:34 AM

Title: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 10:02:34 AM
I have a FLIR Radiometric JPG that contains embedded raw data. I want to get the exif data into a json file and the embedded raw data into an image file.  I am currently doing this successfully with two different CLI calls:
exiftool.exe -json "sourcefile.jpg" > "destinationFile.json"exiftool.exe -b -RawThermalImage "sourcefile.jpg" > "destinationFile.tiff"This works, but I feel like it would be more efficient to be able to do this in a single call.

I've tried several variations of the following, including sprinkling around some "-execute"s between the two different parts
exiftool.exe -json "sourcefile.jpg" > "destinationFile.json" -b -RawThermalImage "sourcefile.jpg" > "destinationFile.tiff"This doesn't get me the two different output files I am looking for.  Best I can do is one file that seems to contain the same binary data twice.   

Any advice?
Title: Re: Exporting multiple fields to different files
Post by: Phil Harvey on March 13, 2025, 10:26:05 AM
The only way you can do this is to use the -execute option to effectively run 2 commands together:

exiftool.exe -json -w "%ddestinationFile.json" -execute -b -RawThermalImage -w "%ddestinationFile.tiff" -common_args "sourcefile.jpg"
- Phil
Title: Re: Exporting multiple fields to different files
Post by: StarGeek on March 13, 2025, 11:15:57 AM
But also note that using the -execute option (https://exiftool.org/exiftool_pod.html#execute-NUM) isn't any different from running two separate commands except that you save the startup time, which would be negligible in this case.

I tend to recommend against using -execute in situations like this unless you are very knowledgeable about using exiftool. Of course, it's a purely selfish recommendation as it makes the command(s) more difficult to trouble shoot if there's a problem, and it would be more work for us to fix it for you.
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 11:25:52 AM
Thanks, Phil!
I've figured out how to write to a subfolder of the folder containing the source file - i.e. if the source jpg is located in "C:\Data" (Windows machine), then I can save to "C:\Data\Out" if I make my destination file arguments "%d\out\outfile.json" and "%d\out\outfile.tiff".

I haven't been able to figure out how to specify a destination folder that isn't a subfolder of the source folder.  I've been reading about the %d and %f arguments for -w, but so far my attempts have led to errors when it tries to append my attempts at a destination absolute path to the source file's location, i.e.:
exiftool.exe -json -w "%d\out\outfile.json" -execute -b -RawThermalImage -w "c:\temp\outfile.TIFF
" -common_args "H:\DATA\test\IR_0970.jpg"
yields
Error creating H:/DATA/test/IR_0970c:/temp/outfile.TIFF
Title: Re: Exporting multiple fields to different files
Post by: StarGeek on March 13, 2025, 11:30:15 AM
Your example shows you are using
-w "c:\temp\outfile.TIFF
but the error indicates you are using
-w "%dc:\temp\outfile.TIFF
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 11:34:28 AM
QuoteBut also note that using the -execute option isn't any different from running two separate commands except that you save the startup time, which would be negligible in this case.

I am running this from a Python script and I've benchmarked the execution time at about 1/3 of a second per call to exiftool. Not sure how much of that time is associated with exiftool vs. the Python subprocessing.call function. My small test dataset has 600 images so cutting the number of calls to exiftool in half saves me about 3 minutes of waiting. 

Future datasets are expected to be considably larger so I'm looking at anything that can shave significant time off the process. During my current data extraction loop, my CPU is sitting at about 10% and my SSD drive usage is just tickling 1-2%, so I don't believe I'm I/O limited at all.  Next step will be to try parallelizing the Python calls to Exiftool.
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 11:38:36 AM
Quote from: StarGeek on March 13, 2025, 11:30:15 AMYour example shows you are using
-w "c:\temp\outfile.TIFF
but the error indicates you are using
-w "%dc:\temp\outfile.TIFF


Here is what I'm seeing

EDIT: Sorry, I thought that screenshot would be inline (SG edit: it is now)
Screenshot 2025-03-13 113637.png
Title: Re: Exporting multiple fields to different files
Post by: StarGeek on March 13, 2025, 11:41:05 AM
Quote from: grubin698 on March 13, 2025, 11:34:28 AMI am running this from a Python script

I would suggest looking into PyExiftool (https://github.com/sylikc/pyexiftool) which is a wrapper around exiftool that keeps exiftool running in the background using the -stay_open option (https://exiftool.org/exiftool_pod.html#stay_open-FLAG).

Though I don't know if running it in parallel is possible using PyExiftool. But you would still want to use the -stay_option option.

As an example of running exiftool in parallel, on Windows, PhoTool's IMatch program uses the -stay_option and runs four instances of exiftool in the background,
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 11:44:00 AM
Quote from: StarGeek on March 13, 2025, 11:41:05 AMI would suggest looking into PyExiftool (https://github.com/sylikc/pyexiftool) which is a wrapper around exiftool

Thanks! I'll look into that
Title: Re: Exporting multiple fields to different files
Post by: StarGeek on March 13, 2025, 12:00:10 PM
Ah, got it. See Note #2 under the -w (-TextOut) option (https://exiftool.org/exiftool_pod.html#w-EXT-or-FMT--textOut)
Quote2) If the argument for -w does not contain a valid format code (eg. %f), then it is interpreted as a file extension,

Add %0f to the front of the path
-w! %0fc:\out\outfile.json
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 13, 2025, 12:05:49 PM
Quote from: StarGeek on March 13, 2025, 12:00:10 PMAdd %0f to the front of the path
-w! %0fc:\out\outfile.json

Thanks,  This is challenging coming from Python. Why would I represent my path string as a floating point number?  ;D
Title: Re: Exporting multiple fields to different files
Post by: grubin698 on March 18, 2025, 10:26:47 AM
StarGeek, I'd like to thank you again for pointing me toward PyExifTool.
My original processing loop using the command line calls was taking about 10 minutes.
PyExifTool not only removed the ~1/3 second overhead of each command-line call, but it also removed the need to use file I/O to pass the raw binary data between ExifTool and my Python code.

Just replacing my command line loop and subsequent .dat file read with calls to PyExifTool reduced my runtime from 10 minutes down to 30 seconds. Parallelization via Python's starmap got runtime down to under 10 seconds. :)
 
Title: Re: Exporting multiple fields to different files
Post by: StarGeek on March 18, 2025, 11:08:42 AM
I just love to hear that. I might just link to this post next time I have to explain Common Mistake #3, "Over-scripting" (https://exiftool.org/mistakes.html#M3).