Substitute exiftool.exe w/ perl version on Windows?

[Marsu42]

I am on windows and have multiple applications and plugins that provide their own copy of exiftool.exe. This is somewhat unsatisfactory since a) I have dozens of exiftool.exe to update and b) I have exiftool installed under ActivePerl (x64) and the local exiftool.pl works just fine.

Question: Is there any way to substitute all exiftool.exe with a transparent replacement that instead of being unpacked/cached/run in %temp% just calls the local perl version?

Admittedly the update problem could be solved with hard links to one exe, but anyway the perl version has the added gift to run as native x64 instead of x86.

[Phil Harvey]

Difficult question, but I have a sneaky idea that might work:

What happens if you replace exiftool.exe with a program that executes your version of perl?  This could be done with a simple .bat file like this

perl c:\windows\exiftool.pl %*

except that I suspect that you may need something with a .exe extension.  Building a .exe seems fairly simple, so I gave it a try.  In the attachment I have included the source code and executable.  Here are the steps to get this working:

1) Make sure that "perl" is somewhere in your PATH (hopefully the PATH environment is passed to exiftool.exe by your applications.  If not, then you may have to define a specific path for PERLNAME in exiftool.c and recompile it.)

2) Put "exiftool.pl" and the "lib" directory from the exiftool distribution into your "c:\windows" directory.  (If you want this located somewhere else, you will have to re-compile exiftool.c with another EXIFTOOL setting.)

3) Put "exiftool.exe" from the attachment in a place where your application will find it instead of the normal exiftool Windows executable.

4) Cross your fingers and pull the trigger.

- Phil

Edit: Changed exiftool path in .bat file to match the one I used in the C version.

[Marsu42]

4) Cross your fingers and pull the trigger.

I changed the script path & compiled it with mingw-w64 - and it works fine, however just as a stand-alone application and not as a drop-in replacement. But it's a step forward anyhow, thanks so far!

(Edit: my guess is:) The problem is that your original distributed exiftool.exe is blocking until the work is done, but the stub returns right after calling perl. With many applications calling exiftool in windows, this generates an error, they expect to have the output ready (i.e. as an xml file) when exiftool.exe quits. I'm not much of a programer, do you see a way to halt return of exiftool.exe until the perl process is done?

As for usability: Obviously, better than re-compiling the exe due to the changed path would be if it would use an env path or guess where the exiftool script is by reading the registry and looking relative to perl (my perl is in c:/perl, thus the script is in c:/perl/site/bin/exiftool (without .pl ending))

[Phil Harvey]

(Edit: my guess is:) The problem is that your original distributed exiftool.exe is blocking until the work is done, but the stub returns right after calling perl. With many applications calling exiftool in windows, this generates an error, they expect to have the output ready (i.e. as an xml file) when exiftool.exe quits. I'm not much of a programer, do you see a way to halt return of exiftool.exe until the perl process is done?

I noticed this too, but didn't understand why this was happening.  The Mac version didn't seem to behave this way.  I'll see if I can figure this out tonight when I have access to the PC again.  Maybe a solution will be to fork then exec, then join again before exitting.

As for usability: Obviously, better than re-compiling the exe due to the changed path would be if it would use an env path or guess where the exiftool script is by reading the registry and looking relative to perl (my perl is in c:/perl, thus the script is in c:/perl/site/bin/exiftool (without .pl ending))

I agree, but I don't think an environment variable will do because your applications may well set their own environment when running exiftool.exe.  The registry technique is probably the way to go, but I lack the programming skills/desire to start tinkering with the Windows registry.

- Phil

[Phil Harvey]

Further on this (quote from here):

On unix, execv() will replace the current process image with another.
But on window-mingw, it will create another process and terminate current
one

They also mention a work-around using spawnv.  I'll try this.

- Phil

[Marsu42]

I agree, but I don't think an environment variable will do because your applications may well set their own environment when running exiftool.exe. 

I don't see how this can happen - if e.g. a user or system env variable EXIFTOOLPERL=c:\perl\site\bin\exiftool is set (in the shell or via Control Panel -> System -> Advanced System Settings -> Environment Variables), nothing should change that by accident but everything can access it. This would enable your average Joe-Sixpack user without access to or knowledge of a compiler to run the replacement Win32Perl + exiftool.exe, too. Of course, I'm also fine with recompiling.

I'm looking forward to see if you figure out a way to solve the fork incompatibility!

[Phil Harvey]

I don't see how this can happen - if e.g. a user or system env variable EXIFTOOLPERL=c:\perl\site\bin\exiftool is set (in the shell or via Control Panel -> System -> Advanced System Settings -> Environment Variables), nothing should change that by accident but everything can access it.

For example, the execve() function sets the environment when it executes another application.  If your applications use this function to launch exiftool.exe then they can define any environment they want.  This doesn't change your system environment, just the environment in which the exec'd application runs.

- Phil

Edit: But this may be unlikely.  Perhaps I'll just try using environment variables to start, then we can see how well this works.  I hope to have a new version of the program for you to test by tonight.

[Marsu42]

On unix, execv() will replace the current process image with another.
But on window-mingw, it will create another process and terminate current
one[

Indeed it's a mingw issue: I compiled it with the Cygwin POSIX emulation layer and it works. Of course, the Cygwin dll has to be distributed along with the exe, and it cannot be x64 code (thus loading the Win32 emulation layer on a 64-bit OS which is regrettable when using 64bit Lightroom with 64bit Perl). However, the usual Cygwin performance hit shouldn't count here.

Concerning the environment issue: As I said, I have forgotten just about everything about C, but even if the app sets its own environment, there should be a way to still access the global system environment? Edit: I just read that you'll try this approach, I'll try it tomorrow.

[Phil Harvey]

Indeed it's a mingw issue: I compiled it with the Cygwin POSIX emulation layer and it works.

Very cool.  Well done.  This proves the concept.

This gives me more confidence that we can get the mingw version working now.

- Phil

[Phil Harvey]

Here you go. Attached is a new version of this program.  I'm quite interested to find out if this works.

It now uses 2 environment variables: PERLNAME and EXIFTOOL, plus I added a -h option which gives a brief description and shows you the current state of these environment variables.

Let me know how it goes in your various applications.

- Phil

Edit: very minor change to allow -h as first option in a normal exiftool command.  If -h is the only option it still shows the help for the environment variables.

[Marsu42]

Let me know how it goes in your various applications.

* Everyting works fine now - which is to be expected since the stub now is really transparent to the calling application.

* A quick benchmark (1000x reading tags from a file >nul) shows a ~15% speed gain when using the full x64 path in comparison with your original distribution auto-unpacked in %temp%. This is significant when using exiftool on a whole Lightroom catalog...

* One shouldn't tempt people to copy things in their windows directory nowadays, you might want to set the default path to something sane like #define EXIFTOOL "c:\\perl\\site\\bin\\exiftool" which is most likely location

* There already is a __MINGW32__ preprocessor symbol, no need to define your own MINGW

* For full drop-in compatibility, you might want to parse and add params given like exiftool(-add -something -here).exe

[Phil Harvey]

* A quick benchmark (1000x reading tags from a file >nul) shows a ~15% speed gain when using the full x64 path in comparison with your original distribution auto-unpacked in %temp%. This is significant when using exiftool on a whole Lightroom catalog...

Cool.

* One shouldn't tempt people to copy things in their windows directory nowadays, you might want to set the default path to something sane like #define EXIFTOOL "c:\\perl\\site\\bin\\exiftool" which is most likely location

I thought of doing exactly this, but in my install instructions I recommend c:\windows too.  I should probably change this as well.

* There already is a __MINGW32__ preprocessor symbol, no need to define your own MINGW

Thanks.  I quickly looked for this but didn't find it.

* For full drop-in compatibility, you might want to parse and add params given like exiftool(-add -something -here).exe

This is a feature designed to make drag-n-drop as simple as possible for people who would have trouble adding arguments to a Windows shortcut (which is the better way to do it).  But anyone running this version is likely to have this skill, so I don't feel guilty if I decline this suggestion.

- Phil

[Marsu42]

This is a feature designed to make drag-n-drop as simple as possible for people who would have trouble adding arguments to a Windows shortcut (which is the better way to do it).  But anyone running this version is likely to have this skill, so I don't feel guilty if I decline this suggestion.

No problem for me, I don't use it anyway - but you should mention this missing feature if you are going to distribute this stub to a wider public. Let me know if you need anything else tested concerning this stub or x64, I'll have a look here from time to time.

[Phil Harvey]

OK, here is the final version.  I had some spare time so I even added the parsing of arguments embedded in the application name.

DOWNLOAD final version of exiftool.exe stub
  |
  V

2012-09-12 - fixed problem with arguments containing spaces

[Marsu42]

* One more thing that might come in handy when trying to find the various versions of Exiftool floating around the system:

There is a trick on Windows to intercept a specified exe name like "exiftool.exe" and execute a substitute like "exiftool_stub.exe" instead. Both names have to be different or it loops & crashes with "The data area passed to a system call is too small.". I added a virtual option "-skip2args" which is not passed on to make this work, see the small addition in the source code. Now only a single registry key has to be added, see the .reg file and modified README.txt.

The whole thing works by mis-using the debugger entry and is often used to replace the default notepad.exe without touching the protected system areas (one wouldn't mess with one's Windows directory, right? :-)), see e.g. http://untidy.net/blog/2009/11/03/replacing-notepad-with-pn-via-image-file-execution-options/

The good part: No need to replace any actual files and the process survives auto-updates of each exiftool.exe by the various applications. The one downside: Since this mechanism can also be used for various other purposes like showing a message "Get a life!" when calling Solitaire sol.exe, the reg entry might alert some anti-malware apps.

Anyway, the new option is a very small addition, doesn't get in the way if ignored but might be of use to some. If you don't include it, no problem, it worksforme(tm) and that was to main reason to start this thread :-)

* I also compiled the stub as a pure x64 console app with x86_64-w64-mingw32, see inside .zip file.