bug: hang or fail on large quantity multiple change in win xp

[pb]

I'll wait for your comment on "beta"... thanks.

Bogdan

Ok, I ran it with the beta.  Same exact hang, at the same exact spot.

[pb]

Here is the output from exiftool when I run it from the command line (actually from a windows cmd shell inside emacs).  (I obfuscated my name, to keep it off the internet, but still same number of chars.)  Note that exiftoolgui fails at file IMG_01771.jpg, i.e. IMG_01771.jpg is the last file where it modifies exif.

I've also attached a directory listing (emacs dired) of everything in the directory that I am running on.

Note that I have windows explorer set to use alphabetic (i.e. "stupid") (default in win2k and earlier) sort order, not "smart" (default in win xp) sort order where it sorts numbers by value rather than alphabetically.  Emacs always uses "stupid" sort order, I believe.

[pb]

Another curious thing I should mention, though probably not related to the problem, is that exiftool generates an error for the file "Picasa.ini" but not for the file ".picasa.ini".  (These are droppings from Picasa, one from an older version, one from a newer version.)  I guess exiftool does not attempt to process files that start with ".".

Also, incidentally, all the makernotes errors that exiftool discovers are generated by files that were written by some older version of Picasa, but files written by the most recent version do not generate the errors (i.e., exiftool "warning"s).

[pb]

One other piece of information.  If I kill the exiftool processes that were spawned by exiftoolgui (or more exactly, the exiftool process that gui spawned, and the exiftool process that exiftool spawned), then exiftoolgui unblocks and shows its message box with a (partial) list of exiftool warnings.  Clicking "ok" then restores a fully functioning exiftoolgui.  (But of course without modifying all the files that were requested.)

[BogdanH]

I can replicate the issue, so finding the cause should be easier now...

Added (about 12h later):
I'm out of ideas right now... for some reason, at some point, I simply can't catch error messages anymore, if too many files have errors -and increasing the buffer doesn't change a thing.
From my perspective, it looks like after writing "some" amount of error messages, ExifTool would suddenly change the way of writting them, and GUI can't cope with that. Yeah, I know it sounds stupid...
Reality is, in console, everything seems normal... so, I think, I need a break.

Bogdan

[pb]

This is actually good news, since I'm sure you will now be able to find the problem.  I predict it is due to some counterintuitive behavior of Windows when dealing with pipes.

I can replicate the issue, so finding the cause should be easier now...

Added (about 12h later):
I'm out of ideas right now... for some reason, at some point, I simply can't catch error messages anymore, if too many files have errors -and increasing the buffer doesn't change a thing.
From my perspective, it looks like after writing "some" amount of error messages, ExifTool would suddenly change the way of writting them, and GUI can't cope with that. Yeah, I know it sounds stupid...
Reality is, in console, everything seems normal... so, I think, I need a break.

Bogdan

[BogdanH]

I've found the cause of "hanging" and fixed it
It wasn't a bug actually.. if at all, I would blame MS API documentation. Anyway, because of it's nature, it's not possible to prevent "hangs" 100% -that is, I hope nobody keeps 10000 files (with corrupted metadata) in single folder 

Phil, thank you for providing these thousands of jpg examples (contaning "minor" errors) -without having them, I wouldn't be able to find the solution.

Bogdan

PS: Fixed GUI (v4.18) will be available for download within next 24h hours.

[pb]

Great!  Also, I'm glad I correctly predicted it would be some MS weirdness.

I've found the cause of "hanging" and fixed it
It wasn't a bug actually.. if at all, I would blame MS API documentation. Anyway, because of it's nature, it's not possible to prevent "hangs" 100% -that is, I hope nobody keeps 10000 files (with corrupted metadata) in single folder 

Phil, thank you for providing these thousands of jpg examples (contaning "minor" errors) -without having them, I wouldn't be able to find the solution.

Bogdan

PS: Fixed GUI (v4.18) will be available for download within next 24h hours.

[BogdanH]

To be honest, it's not that MS stuff wouldn't work as documented. It's just, (in this case) the thing I needed to know, isn't described precise enough.
Thanks again for all the info you gave me.

Bogdan
PS: new GUI v4.18 is ready for download.

[Phil Harvey]

Bogdan: I'm impressed you were able to fix this.  Bugs like this can be very difficult to track down.

- Phil

[BogdanH]

Thank you for compliment, Phil.
To tell the truth, I was very, very close to give up on this (+advice like "keep modest amount of files in single directory" ).

Bogdan

[MOL]

So what was the problem, Bogdan? Can you elaborate or provide a link that describes it in more detail? Remember that there are fellow programmers out there who also use EXIFTOOL in their applications...

Thanks!

[coz]

Bodgan,

Thanks for your persistance & patience.   Great fix.

Chris

[BogdanH]

@ coz:
Thank you.
@MOL:
There might be more ways/techniques to "talk" with ExifTool, but one can't avoid using at least two API functions: CreateProcess and CreatePipe.
In this case, after checking almost everything else involved in this (my own procedures/variables in first place), I came to my CreatePipe usage. I've re-read some tutorials on the web, plus MS quide:
http://msdn.microsoft.com/en-us/library/aa365152%28v=vs.85%29.aspx
-here, nSize parameter is described as follows:

The size of the buffer for the pipe, in bytes. The size is only a suggestion; the system uses the value to calculate an appropriate buffering mechanism. If this parameter is zero, the system uses the default buffer size.

The answer that I'm still looking for, is:
-how much buffer exactly do I get, if value is zero?
-how much buffer exactly do I get from buffering "mechanism", if value is non-zero?

That is, I don't like writting code by "try/error" until it works in particular case only. Of course, to be "sure" one could choose buffer size of 1GB, but... does "program being memory hog" sounds familiar? 

Bogdan

[MOL]

Thanks a lot, much appreciated!