ExifTool Forum

ExifTool => Bug Reports / Feature Requests => Topic started by: Oleander on April 27, 2011, 04:01:09 AM

Title: Memory leak
Post by: Oleander on April 27, 2011, 04:01:09 AM
When starting ExifTool with the arguments '-stay_open 1 -@ -' and subsequently writing (XMP) tags to images memory is leaked.
The amount of memory leaked seems to correlate to the number of tags written.

OS: Win XP SP3
ExifTool version: Windows executable 8.50/8.56
Title: Re: Memory leak
Post by: Phil Harvey on April 27, 2011, 08:04:11 PM
I can pretty well guarantee that there is no memory leak, but it is normal for memory to grow to a point.  The memory handling is all done automatically by Perl.  At some point all unused memory is recovered by the regular garbage collection.  Most Perl implementations are very good, but the Active Perl (used by the Windows executable) seems to be more of a memory hog.

- Phil
Title: Re: Memory leak
Post by: Oleander on May 02, 2011, 06:54:23 AM
The memory usage of ExifTool grows until there is no memory left (2GB RAM + 2GB swapfile in this case) and then terminates.

Attached a crude sample batch file. Adjust paths as needed. Use a smaller test image / add more tags to speed up memory usage.
Title: Re: Memory leak
Post by: Phil Harvey on May 02, 2011, 07:42:55 PM
I can't test this until next week when I will have access once again to a Windows machine.

- Phil
Title: Re: Memory leak
Post by: Oleander on June 01, 2011, 03:26:59 AM
Any progress?
Title: Re: Memory leak
Post by: Phil Harvey on June 01, 2011, 07:55:55 AM
Thanks for reminding me about this.  I just ran a test extracting information from an image (832 KB jpeg image) repeatedly using the -stay_open option.  The system was Windows XP SP3, with the Windows executable version of ExifTool 8.57.  After each -execute, I checked the memory usage.  Here are the results (in KB):

16452
16432
16436
16440
16384
16396
16392
16372
16388
16372
16388
16388
16372

So it looks to me as if the ActivePerl memory reclamation is working properly, at least for this test.

I'll try your .bat file.

- Phil
Title: Re: Memory leak
Post by: Phil Harvey on June 01, 2011, 08:48:34 AM
OK, I can reproduce the leak using your .bat file.  For now, I suggest restarting exiftool every few hundred files or so, but I will continue to look into this to see if I can find the source of the problem.

- Phil
Title: Re: Memory leak
Post by: Phil Harvey on June 01, 2011, 10:32:48 AM
FOUND IT!!

It turns out that the Perl garbage collection algorithm has a known bug: It can't handle circular references when deallocating objects.

I was using a circular reference when storing information for tags to be written.  I now remove these references before the object goes out of scope and the problem is SOLVED!  (This also explains why I didn't see the problem when just reading information.)

This fix will appear in version 8.58 when it is released.

Thanks for pointing out this problem.  I learned something.

- Phil
Title: Re: Memory leak
Post by: Oleander on June 01, 2011, 11:16:08 AM
I'm glad I could help.