News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

ExifTool crashes Win10 with large data sets

Started by georgejones314, February 22, 2020, 07:47:05 PM

Previous topic - Next topic

georgejones314

Phil good to see you're still here after so many years. I haven't used your marvelous tool for over two years.
My son bought me a new laptop (Dell Inspiron 3595) for Xmas and I've been slowly migrating my XP stuff over to Win10. It's pretty much a clean system (I wiped the hard drive and installed my own Win10 system).
My issue is the nice batch I used to check the track field of my podcasts on my XP no longer works.
I've reported all my 'attempts' at fixing it here:
https://stackoverflow.com/questions/60352323/memory-issues-with-dos-batch-command-line-use-of-exiftool
The gist is ExifTool is not releasing computer resources after execution (either in DOS or Batch).
Not knowing what it meant, I also tried using the -stay_open option without success.

Any thoughts?
Again, thanks for all your hard work on this great tool.

@echo off
chcp 1254>nul
setlocal disabledelayedexpansion
for /d %%f in (H:\Podcasts\*) do (
set buffer="%%f"
echo %%f
call:loop1
)
pause
EXIT /b

:loop1
c:\mPodcasts\software\exiftool\exiftool.exe -if "$track" -L -q -q -s3 -p $track~${directory;tr#/#\\#}\$filename %buffer%>>"c:\mPodcasts\exiftooltest.txt"

EXIT /b


StarGeek

You say in the comments there
" The -r option only gets as far as my /d before crashing"

I don't understand this.  The command I gave there was to be run by itself, not in a looping batch, so I don't see how that applies.

I've run exiftool on 10s of thousands of files in a single run and the memory use is pretty minimal.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

georgejones314

Sorry for the confusing wording. The Exiftool -r option only reads as many folders-files as does my batch file with the loop (and not using the -r option). That is to say, they both crash at about the same volume of reads. This is about 12,000 files spread across 150 folders.

Thx.

Phil Harvey

When any application terminates, it is up to the system to reclaim all used resources (ie. memory and file descriptors).  It isn't up to the application to do this.

If you execute exiftool many times in a batch file loop, then after each execution the resources should be reclaimed.  If not, it is a Windows bug.

However, running through all files with a single execution may result in more memory usage because then we would rely on the Perl memory garbage collection to reclaim memory as necessary while executing.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

georgejones314

Thanks for the prompt reply. It may be something in the way my Win10 came configured out of the box (off the stick really). I was hoping you or one of your fans may have come across this situation before. This batch did work on my meager Win/XP NC410c with 1gb memory and a 2ghz processor. The mp3's are all on an external HD ... the same one I used for the NC410c.
Actually there are fewer mp3's now on this hard drive, yet Exiftool is crashing the laptop at a mere 12K files.

BTW I tried this with the same version of Exiftool as on my XP (two years old) and the latest version.

obetz

for lack of a better idea you can try my Strawberry Perl based ExifTool: https://oliverbetz.de/pages/Artikel/ExifTool-for-Windows

But I'm afraid it will not solve your problem.

Did you watch in the task manager, exactly which process "chews up system resources"?

Oliver

georgejones314

Thanks obetz. In researching a response to your post I found the answer.

And the answer is: Acronis True Image 2020 Active Protection (specifically anti_ransomware_service.exe).
When this service kicked in it quickly drove CPU utilization to 100% and crashed the computer.
It's not even a part of the base True Image functionality, just an add-on.

FYI: the fact that commit and physical memory hit 100% is irrelevant. That's what they're supposed to do ... in case the previous processes using the memory want to use it again (so it doesn't have to be reloaded).

I stopped this service and all works as advertised.

Thanks all for your responses.