Help with speeding up ExifTool

[kemsky]

yep, unicode is really annoying problem on windows.

I'm going to use ExifTool to extract info from ebooks, but without unicode it is very limited solution.
i could rename file to avoid unicode input problem, but extracted info (stdout) is ansi anyway...

i really wish that some day there will be dll-wrapper (like imagemagick has) supporting unicode arguments.

as a workaround exiftool could output info to file in unicode format (or even pipes/named pipes see CreateNamedPipe)

[cuagn]

I'm not really sure I want to reply ... but I still have a question using stay_open

I've try the proposed test of BogdanH

1/ Exiftool version 9.20 installed on Windows7 Pro (French)

2/ Create a file MyExifToolArgs.txt EMPTY

3/ Start a command line windows and key in

Code Select Expand

Microsoft Windows [version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Tous droits réservés.

C:\Program Files (x86)\Exiftool>exiftool -stay_open true -@ D:\MyExifToolArgs.txt


4/ No answer. That's normal. I Chech existing processes

Code Select Expand

exiftool.exe*32 .....   860K   Read and Write meta information
exiftool.exe*32 ..... 13900K   exiftool.exe

Seems OK ...

5/ Open MyExifToolArgs.txt (using Nootepad++) and add 3 lines (to have 2 EOL)

Code Select Expand

-ver
-execute123


6/ In the command line windows I've now

Code Select Expand

Microsoft Windows [version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Tous droits réservés.

C:\Program Files (x86)\Exiftool>exiftool -stay_open true -@ D:\MyExifToolArgs.txt
9.20
{ready123}

which are the expected answers for the version information and the ready info.

However, I've a question :

How can I receive these answers in a given file or appended to the input (MyExifToolArgs.txt) file?

I'm stuck... 

Thank you for your help.

Marc

[cuagn]

Made another test with strange results...

1/ Start exiftool with outfile defined (MyExifToolResult.txt)

Code Select Expand

Microsoft Windows [version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Tous droits réservés.

C:\Program Files (x86)\Exiftool>exiftool -stay_open true -@ D:\MyExifToolArgs.txt > D:\MyExifToolResult.txt


2/ Then, add 3 lines in the MyExifToolArgs.txt EMPTY file

Code Select Expand

-ver
-execute001


3/ Open MyExifToolResult.txt

Code Select Expand

{ready001}
No version indicated ...   

4/ What about echo ? Appended in args file

Code Select Expand

-ver
-execute001
-echo HELLO
-execute002

5/ Still no result in output except the ready message

Code Select Expand

{ready001}
{ready002}

6/ However Windows command line has an output

Code Select Expand

Microsoft Windows [version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. Tous droits réservés.

C:\Program Files (x86)\Exiftool>exiftool -stay_open true -@ D:\MyExifToolArgs.txt > D:\MyExifToolResult.txt
File not found: o HELLO


I know I make something wrong, but what ?

Regards

Marc

[Phil Harvey]

Hi Marc,

I just tried this under OS X, and it works fine.  Your echo command won't work as you wrote it though because each argument needs to be on a separate line, but with this fixed it works for me.  I don't understand why the -ver output didn't go to the redirected file.  This doesn't make sense to me.

- Phil

[cuagn]

Thank's for your answer.

May I make a specific test in order to give you some valuable information/feed back?

I'll try other tests this afternoon.

Thank you.

Marc

PS: in fact I want to invoke exiftool from a VB.NET application, but no being familiar with "pipe" under W7, I (try to) use a file based relation with exiftool.
A pipe guideline under .net will not solve the problem I've identified (at least in my environment), but will give me the required "knoledge" (by example).
Will go thru other topics...

[Phil Harvey]

I had a PC handy so I tried this myself.

Man.  Windows is weird.  Somehow some of the output is getting lost in the Windows pipe.

I believe that a simple redirection in Windows (ie. "> out.txt") will redirect both stdout and stderr.  Also, I think that Windows supports separate redirection of these as with a Unix Bourne shell (ie. "1> out.txt 2> err.txt"), so I tried this to redirect just stdout:

exiftool -stay_open true -@ a.args 1> out.txt

and it seems to work.  This indicates to me that there may be some sort of a bug in the Windows command shell where there is an interference that occurs when redirecting two streams to the same file.  But this is just a guess.

- Phil

[cuagn]

7 years ago I made an Home Automation based on Debian , with pipes (sorry queues...) and never had other difficulties than reading the documentation....

Unfortunately, my "family office" is under Windows and so are also the 14500 photos I would like to tag efficiently, thanks to exiftool.

I'll redo some tests and hope to find a bypass.

Thank you

Marc

[ryerman]

I won't pretend to know the intricacies of Windows re-direction, but there is good information here.

In summary:
1> redirects stdout    (> is merely an allowable short-form)
2> redirects stderr

To redirect to separate files use:
-exiftool some command line 1> out.txt 2> err.txt

To redirect to the same file use:
-exiftool some command line > both.txt 2>&1
or
-exiftool some command line 1> both.txt 2>&1
or
-exiftool some command line 2> both.txt 1>&2

Jim

[Phil Harvey]

Hi Jim,

Thanks for your input.  But now I'm even more confused because "1>" worked for me, while ">" didn't.  But I only ran a few tests, so maybe it is just inconsistent, however I hope this isn't the case.

- Phil

[ryerman]

....I know I make something wrong, but what ?...

Hi Marc.

After several hours of brute force trial & error , I can suggest this:
Every time you write to the arg file (even if it's empty), write one blank line first, before all the options.  This worked for me and I have no idea why.

Besides finally learning how -stay_open works, I also discovered this:

exiftool -stay_open true -@ Args.txt > Result.txt will append the information to the text file.
Without -stay_open, > will over-write and >> is used to append.  Both will create the file if necessary.

Why the differing actions concerning appending/over-writing?  Once again, I haven't a clue.

Jim

[Phil Harvey]

Hi Jim,

Yes, I noticed a similar effect...  Adding an extra option before -ver caused it to work.  The blank line is a good trick though.

But I'm sure you are fooling yourself with the append mode.  ">" will always overwrite, but if you don't clear your "Args.txt" before you re-run exiftool, then it will overwrite "Result.txt" with the same thing that it contained before, making it look like you are appending to the existing file when you aren't.  I tried this and verified that the output file was cleared when I used ">" with -stay_open (in Windows XP).

- Phil

[ryerman]

Hi Phil

Yes, I noticed a similar effect...  Adding an extra option before -ver caused it to work.  The blank line is a good trick though.

But I'm sure you are fooling yourself with the append mode.....

It wouldn't be the first time.

For reference: exiftool -stay_open true -@ "Args.txt" > "Result.txt"

I can't reproduce overwrite mode. When I delete the contents of the "Args.txt" and write another argument, ExifTool seems to freeze without accepting any new arguments.  I escape by killing the exiftool process.

How did you clear "Args.txt"?

However, if I write something in "Result.txt" beforehand, I can see that it is overwritten the first time I execute from "Args.txt".
But I want to try and make that happen after writing a 2nd or 3rd argument, so I do this:

1.  Execute some arguments from "Args.txt" and observe the output in "Result.txt".
2.  Change the arg file to "Newargs.txt" and delete the original "Args.txt".  Now, I assume Args.txt and all of its arguments are no longer involved.
3.  Execute an argument from "Newargs.txt".

The outcome is that "Result.txt" looks like it still contains the output produced when using "Args.txt", with the output produced with "Newargs.txt" appended.

Have I fooled myself again?

And btw, the necessity for a blank line or extra option mysteriously disappeared.

Jim

[Tarn]

exiftool -stay_open true -@ Args.txt > Result.txt will append the information to the text file.

Jim,
I'm a little confused. In this example you use only one greater than character ">". As of yet I don't understand -stay_open, or how to use -@, and arg files; but your example with the single ">" should overwrite Result.txt.

Without -stay_open, > will over-write and >> is used to append.  Both will create the file if necessary.

This makes sense.

In regards to the greater than character as a redirect, in Windows...
dir > Dir.txt
  1. Will create a file called "Dir.txt" and place in it the output that would have gone to the screen.
  2. Will overwrite a file called "Dir.txt" and place in it the output that would have gone to the screen.
dir >> Dir.txt
  1. Will add the output that would have gone to the screen to the end of the file "Dir.txt" (with no leading line).
  2. Will create a file called "Dir.txt" and place in it the output that would have gone to the screen; should the file not exist.

Maybe think of it this way, a single greater than sign will create a new file, fresh output, every time. Doubble greather than signs will add to the file. Unless it does not exist; then it will create it.

I'm not entirely sure that this was your question. But after reading this thread there does seem to be some confusion about how Windows redirects out put. So I thought I'd attempt to give a very simple clarification to you, or to anyone who might need, or want one.

For those of you who already knew and understood this, you can say to yourself something like: "I knew that", or "Tell me something I don't know", or even "Right, I thought that's how it worked."

Hope this helps someone.

P.S. At the risk of bursting people's bubbles here, I don't know everything about ExifTool. And, there is a possibility that I may have, completely, misunderstood the issue at hand. If so, feel free to enlighten me.

[Phil Harvey]

Tarn is correct.

There were 2 ways you could fool yourself.  The one I mentioned, or thinking that each -execute should clear the output file (which won't happen).

At the time you run exiftool, redirecting the output with ">" will start with a fresh output file.  The output accumulates in that file until exiftool terminates.

Similarly, the input will accumulate in args.txt.  You should not clear args.txt while ExifTool is reading from it.  If you do, then ExifTool won't read any more until the length of args.txt reaches the same size as before your cleared it.  But it doesn't make sense to do this.  If you want to clear args.txt you must first switch argument files (with -stay_open true -@ newfile.args [each argument on a separate line, of course]), or terminate the exiftool process (with -stay_open false).

- Phil

[Tarn]

So -stay_open keeps a file open. In other words to issue -stay_open on a file, my.args would prevent me from being able to delete that file with Del my.args. It simply opens the named file and keeps it that way until -stay_open is set to false, or until that process is terminated. Right?