Getting "Out of Memory!" error with largefilesupport=1 set

Started by dvela, July 28, 2021, 03:33:59 PM

Previous topic - Next topic

dvela

This is my command line
exiftool -ee -api largefilesupport=1 -p json.fmt C:\Temp\RT622_20210702140911.mp4 C:\Temp\points.json

The mp4 file is a 5.4GB in size. The machine has 25 GB available in ram, 45GB available on the drive and 8 CPU barely running at 5%.

What do you think the issue is?

Phil Harvey

I think the issue is that you're running on Windows. :P

Does it run out of memory without the -ee option?

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

You might also try the alternative exiftool build.

Another option would be to install Perl and use the Perl version of exiftool.  It usually doesn't have the memory problems that can sometimes happen with the Windows executable.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

obetz

dvela, when you try my alternate build, please report whether it solved your issue and whether you used the 32 Bit or 64 Bit version reps. whether the 32 bit and 64 bit version behaved different.

dvela

Quote from: Phil Harvey on July 28, 2021, 07:11:38 PM
I think the issue is that you're running on Windows. :P

Does it run out of memory without the -ee option?

- Phil

When -ee is removed it gives this warning.
Warning: [minor] The ExtractEmbedded option may find more tags in the media data

If I suppress that warning. Then it outputs an empty file and doesn't process it.

dvela

Quote from: StarGeek on July 28, 2021, 10:25:52 PM
You might also try the alternative exiftool build.

Another option would be to install Perl and use the Perl version of exiftool.  It usually doesn't have the memory problems that can sometimes happen with the Windows executable.

This worked!! I used the Windows 64bit zip version. Thanks so much!

obetz

Quote from: dvela on July 29, 2021, 09:21:17 AM
This worked!! I used the Windows 64bit zip version. Thanks so much!

If it is not too much work - may I ask you to check whether my 32 bit version has the problem?


dvela

Quote from: obetz on July 29, 2021, 04:21:41 PM
Quote from: dvela on July 29, 2021, 09:21:17 AM
This worked!! I used the Windows 64bit zip version. Thanks so much!

If it is not too much work - may I ask you to check whether my 32 bit version has the problem?

The 32 bit did not work. Out of Memory error again. I used version 12.29 for 64 and 32 bit.

There is no way around this if you are loading the entire file into memory. The only way I can see a 32 bit working is if you chunk the file in and then dump the chunks after you process it. To keep it under the 32 bit limit of 2GB. Honestly I don't think its worth it. Microsoft has stopped supporting 32 bit versions of Windows 10.

obetz

Quote from: dvela on August 02, 2021, 09:36:50 AM
The 32 bit did not work. Out of Memory error again. I used version 12.29 for 64 and 32 bit.

thanks for your test. I was not sure whether 64 bit Perl still has limitations.