Integrate ExifTool in stay_open mode with bash shell scripts

[dirkhillbrecht]

Hi all,

I use ExifTool from a variety of shell scripts. Always searching for speeding up my code, I discovered the stay_open "poor man's" server mode. I looked for examples how to use this within a plain shell script and found - nothing. So, I started programming on my own, and now I'd like to show this code here: First to make it public so that other people can build upon it, second to ask some questions.

Here we go:

The code

Put the following at the beginning of the script or in a seperate script file which is sourced in by the actual script at the beginning:

Code Select Expand

EXIN="/tmp/pipe.$$.EXIN"
EXOUT="/tmp/pipe.$$.EXOUT"
EXLAUNCHED=0

launchexiftool() {
  [ "$EXLAUNCHED" = "1" ] && return
  mkfifo $EXIN
  mkfifo $EXOUT
  ( exiftool -stay_open 1 -@ - <$EXIN >$EXOUT )&
  EXLAUNCHED=1
  trap "{ haltexiftool; exit; }" EXIT
}

haltexiftool() {
  [ "$EXLAUNCHED" = "0" ] && return
  echo "-stay_open" >$EXIN
  echo "false" >$EXIN
  sleep 1
  rm -f $EXIN $EXOUT
  EXLAUNCHED=0
}

callexiftool() {
  for p ; do echo "$p" >$EXIN ; done ; echo "-execute" >$EXIN
  while [ true ] ; do
    read line <$EXOUT
    case "$line" in
      {ready}) break;;
      *{ready}) echo "$line" | cut -c1-$[ ${#line} - 7 ] ; break;;
      *) echo $line ;;
    esac
  done
}

launchexiftool

Usage

Use this by calling the "callexiftool" function as a drop-in repladement for calling exiftool directly. So, if you wrote

exiftool -title -b picture.jpg

so far, you can now just write

callexiftool -title -b picture.jpg

instead. That works also with more complex calls like

thetitle=`callexiftool -title -b picture.jpg`

Implementation

The shell interface uses two named pipes for interacting with exiftool: One for the input, one for the output. exiftool itself is started during initialisation into a seperate process. It is stopped by passing it the "-stay_open false" option. The result is received by scanning the according amed pipe until "{ready}" is printed out by exiftool. In case of binary result requests ("-b" option to exiftool), the "{ready}" may not be on its own line. For this case, a special line evaluation is put into the "callexiftool" function.

The asynchronous exiftool process is killed by an "exit trap" which is installed during the exiftool initialisation. If the script ends by any means, it will shutdown the exiftool.

Questions

- Is there another way to interface shell and exiftool? Is that other way better?

- If the script is interrupted with Ctrl-C, exiftool might hang. That seems to happen if a new command is just emit and the exit trap comes in between and passes it "-stay_open" in the wrong moment. Then, the whole script hangs and can only be revived by some sequence of Ctrl-C, Ctrl-D and kill commands. Can this be done in a better way?

- Is my basic concept of passing the commands ("for p"-loop in callexiftool) flexible enough for all use cases? I am using exiftool right now mainly for (manual) geotagging and title addition, so: Should the parameter transmission be programmed differently to be more powerful?

ExifTool is a real cool tool! Many thanks for this! Hopefully, this posting also helps someone or the other to realize their ideas around exiftool more easily.

Best regards,
Dirk

[Phil Harvey]

Hi Dirk,

This looks great!  Thanks for sharing your code.  The mkfifo trick is new to me... very interesting.

- Is there another way to interface shell and exiftool? Is that other way better?

Instead of a shell script, a Perl script would be fine for most people because the Image::ExifTool Perl libraries do all of the heavy lifting.  The only case where a shell script is preferable is when you want to take advantage of some added functionality in the exiftool application.

- If the script is interrupted with Ctrl-C, exiftool might hang. That seems to happen if a new command is just emit and the exit trap comes in between and passes it "-stay_open" in the wrong moment. Then, the whole script hangs and can only be revived by some sequence of Ctrl-C, Ctrl-D and kill commands. Can this be done in a better way?

I haven't done signal handling in scripts myself, but I imagine that it is possible.  The problem could be solved if you catch the CTRL-C (SIGINT) in your script and use it to send a SIGINT to exiftool then abort the script.

- Is my basic concept of passing the commands ("for p"-loop in callexiftool) flexible enough for all use cases? I am using exiftool right now mainly for (manual) geotagging and title addition, so: Should the parameter transmission be programmed differently to be more powerful?

I am a  bit surprised that this seems to work with -n as the first argument.  I expected echo to eat it, but it doesn't.

However, there is a problem with arguments that contain spaces, ie:

exiftool -comment="a comment" a.jpg

or,

exiftool -p 'This is the file name: $filename' a.jpg

But this could just as well be a problem with my implementation and the way I'm handling the arguments in my script:

Code Select Expand

#!/bin/sh
source dirks_code
callexiftool $*
haltexiftool

But there is another more serious problem:  The script worked fine the first few times I ran it, but after that it would hang every time when trying to read from $EXOUT when the output is long.  I think there could be something funny happening with the fifo on my system (OS X 10.8.2).  It will read the first 50 or so lines of metadata (about 1 kB) before hanging.  I initially guessed that the problem was related to buffering, but adding the exiftool -v0 option didn't fix it, so that wasn't the problem.  This was using "t/images/ExifTool.jpg" from the ExifTool distribution as the test file, with no other arguments.

- Phil

[Phil Harvey]

I did some experimenting, and the following solves the hang problem.  The only catch is that you must use /bin/bash (/bin/sh doesn't understand this syntax):

Code Select Expand

EXIN="/tmp/pipe.$$.EXIN"
EXLAUNCHED=0

launchexiftool() {
  [ "$EXLAUNCHED" = "1" ] && return
  mkfifo $EXIN
  exec 3< <( exiftool -stay_open 1 -@ - <$EXIN )
  EXLAUNCHED=1
  trap "{ haltexiftool; exit; }" EXIT
}

haltexiftool() {
  [ "$EXLAUNCHED" = "0" ] && return
  echo "-stay_open" >$EXIN
  echo "false" >$EXIN
  sleep 1
  rm -f $EXIN
  EXLAUNCHED=0
}

callexiftool() {
  for p ; do echo "$p" >$EXIN ; done ; echo "-execute" >$EXIN
  while [ true ] ; do
    read line <&3
    case "$line" in
      {ready}) break;;
      *{ready}) echo "$line" | cut -c1-$[ ${#line} - 7 ] ; break;;
      *) echo $line ;;
    esac
  done
}

launchexiftool

- Phil

Edit: And yes, the quoting problem was in my script.  If I call your "callexiftool" function with properly quoted arguments then everything works fine.

Also, I should have given a link to the example where I found this technique.  I don't know enough about bash scripting to have figured this out by myself.