Crash when Batch Processing many files

[MartinF]

Hi Phil,

I am using a powershell script to import images (e.g. a complete memory card folder structure) into a temporary Folder. Then, the script lets exiftool run over the folder to rename the image files by date and move them into another folder, according to their filetype (jpg, raw or video - exiftool is called seperately for each filetype); the original folder structure is discarded in the process. Since filenames are unique at that point, this should not cause any trouble.
This works perfectly on a moderate number of files (I have not yet seen any problems with less than ~400 files).

With more files, Occasionally, the Exiftool Process finishes its work (I can see in file manager, that all files are correctly renamed and moved) but then it does not close, i.e. the process stays alive as can be seen in the windows task manager.

Since my powershell script is waiting for the process to terminate, I then have to manually terminate the Process Structure in Windows Task Manager before my script will continue.

Notably, -IF- the problem occurs, it will occur on all 3 consequently called instances of ExifTool despite the fact, that the source folder is almost empty by the time the last instance is called, as jpgs and raws have already been moved out of the folder and only a few video files are remaining at that point. So it might not have anything todo with the number of files at all?

I must note that I am not a programmer, I am new to Powershell and although I have done a lot with exiftool, I am still a beginner here, too.
Anyway, As exiftool completes its task as desired in any case, I would implicate, that my Code is correct.

When trying to reproduce the error, I called ExifTool with the equivalent parameter from cmd.exe on a folder structure containing ~1400 files and it terminated correctly. I called it from powershell manually and it also terminated correctly. Here is what I called from cmd.exe

Code Select Expand

R:\>exiftool "r:/source/" "-directory=r:/target/" -ext jpg -ext jpeg "-filename<FileModifyDate" "-filename<datetimeoriginal" -d %Y-%m-%d_%H%M%S_%%f.%%e  -r
    5 directories scanned
1271 image files updated

And from Powershell (Code explained below):

Code Select Expand

32# & ExifT `"$dirNewTemp`" `"-directory=$dirNewJpg`" -ext jpg -ext jpeg --% "-filename<FileModifyDate" "-filename<datetimeoriginal" -d %Y-%m-%d_%H%M%S_%%f.%%e  -r
    5 directories scanned
1271 image files updated


However, under some circumstances which I am not yet able to reproduce, ExifTools Summary (5 directories scanned, 1271 image files updated) is not displayed in the command window and the Exiftool Process stays alive with 0% CPU usage and a memory usage of ~32-34MB, which is also stationary, so I assume that exiftool has indeed crashed and is not just still working.

So if the error is really not due to my coding, I would assume, there must be a bug after all files are processed and before the summary is displayed.


If it helps, Here are the relevant lines of my code. Exiftool is called 3 times by:

Code Select Expand


& ExifT `"$dirNewTemp`" `"-directory=$dirNewJpg`" -ext jpg -ext jpeg --% "-filename<FileModifyDate" "-filename<datetimeoriginal" -d %Y-%m-%d_%H%M%S_%%f.%%e  -r
& ExifT `"$dirNewTemp`" `"-directory=$dirNewRaw`" -ext arw -ext rw2 --% "-filename<FileModifyDate" "-filename<datetimeoriginal" -d %Y-%m-%d_%H%M%S_%%f.%%e  -r
& ExifT `"$dirNewTemp`" `"-directory=$dirNewMov`" -ext mts -ext mp4 --% "-filename<FileModifyDate" "-filename<datetimeoriginal" -d %Y-%m-%d_%H%M%S_%%f.%%e  -r


- The notation: & ExifT means for Powershell: "run external application ExifT"
- Thereby, "ExifT" is just an alias, pointing to the exiftool executable.
- the grave-accent(`) is Powershell's escape character.
- --% means that all further statements will be passed on without processing by powershell, i.e. escaping of characters is not necessary beyond this.
- $dirNewTemp , $dirNewJpg and so on are variables for folder names in the format: c:/directory/


Windows 7 x64, Powershell 4.0
Exiftool Version: 9.73
Exiftool freezes with a stationary memory-usage of around 32-34MB (0% CPU usage) after all processing is completed successfully and before displaying the result summary and terminating

Best regards from Berlin,
Martin

[Phil Harvey]

Hi Martin,

Are all of the symptoms consistent with exiftool stalling when it is trying to output the summary message?  I have no idea how a stdout write could stall like that, but I would blame the system, not exiftool.  Try adding the -q option to avoid the summary message to see if this helps.

- Phil

[MartinF]

Hi Phil,

I would say all symptoms are consistent, yes.
Thanks for that advice, I will try with -q.
Today I processed a large memory card without error.  If the problem occurs again, I will find a way to reproduce it and report back.

best regards,
Martin