Piping vs Shelling, basics ways of calling Exiftool from code

[Skippy]

  It seems to me that exiftool provides 3 options for processing a folder full of photos:
1. Stay_open mode and Piping, which is where Exiftool is used to generated the binary data for a jpeg and the calling program manages saving that data to a file (& interprocess communications are via StdIn);
2. Stay_open mode, where exiftool stays open in the background and activates whenever the argument file it is watching is overwritten; and
3. Processing a list of photos in one invocation where the list of photos and the tags to be processed is specified in a txt, json or xml file.

I have gone down the third route as I find this one the easiest but I would like to know more about the pro's and con's of each options.  I have not been able to find a basic overview of the alternatives.

Perhaps the biggest concern I have at present is how to know when exiftool has finished a process as I would like to call exiftool again within the same procedure to do another process. 

Exiftool does have a rather long learning curve and I find that I have to basically browse everything and then assemble bits and pieces to get where I need to go or at least understand the answers that other of the forum provide.  I am an intermediate programmer so need a bit more detail that most.

[Phil Harvey]

I'm not sure exactly what you mean by -stay_open with piping, but if you are piping the input file then I don't think this will work for multiple files.

The way I would recommend is to use -stay_open and piping the command-line arguments (ie. -@ -).

You know when exiftool is done processing because it returns "{ready}" as per the -stay_open documention.

- Phil

[Skippy]

I am pretty sure that StdIn and StdOut not available within the ms-access environment so to sum up, piping does not seem to be an option in this environment if multiple files need to be processed.  Various forums suggest that StdOut can be captured in a file then consumed, however exiftool exports json files and this seems to be a more robust way of capturing information.

[Phil Harvey]

If piping is a problem, then option 3 avoid the problem and your biggest concern seems to be how to determine when the exiftool process has finished.  Can't you just check to see if the process is still running?  When exiftool is done the process should terminate.

- Phil

Edit:  Ah.  It looks like you have already found a solution to this.

[Skippy]

I managed to get piping to work from the ms-access environment.  My starting point was the VB6 code written by Michael Wandel that is provided in a link on the exiftool home page.  https://exiftool.org/modExiftool_101.zip.  Unfortunately, there are a couple of procedures missing from the module and they had to be rewritten or references to these procedures removed.  I will try and post a fully self contained ms-access exiftool tool kit when I get everything fully worked out. 

Piping just redirects the output from exiftool from the command window directly into ms-access where I can store the exiftool output to a string variable.  As exiftool output is mostly a list of tag : value pairs, I can easily write some code to parse the string and pull out the information I am looking for.  The alternative to piping is to get exiftool to output to a text file, such as a json file and to then read the json file.  There does not seem to be a lot of difference between the two approaches from a programming perspective.  The code for piping is advanced and hard to understand and the code for shelling to exiftool and then importing a json file is much easier to understand.  Shelling is asynchronous unless you use the shellwait version of shelling.  Asynchronous means that ms-access starts exiftool, then proceeds without waiting for exiftool to do its work.  That is a problem when you need to get some information from exiftool to use in your procedure.  I am new to piping but it does not seem to be asynchronous and this is an advantage for piping. 

With Michael's code, I am reading the exif data from only 3 jpeg files per second on win 8.1 with a core-i7 laptop and using the local hard drive.  It looks like exiftool is being restarted for every single image.  Michael's code does not return a process handle or other simple means of checking what is happening.  The code also closes and reopens the pipes after each file is read but does not close the exiftool process so it is hard to tell if exiftool is being closed then reopened for each new jpg file. 

I changed Michaels code to read the exif from a whole folder with about 70 photos and exiftool provides the results in under a second.  I suspect that piping will be faster than writing and reading json files.   The data in the string variable for just 70 photos and all tags was 500 KB so recursively reading tags from a large folder structure might lead to running out of memory.  Currently Michael's code gets exiftool to read all tags, then it makes an array of tags and values, then finds and returns the tag that you specified.  The code could be refactored to find only specific tags of interest. 

In short, getting data from exiftool using piping works well for reading tags from a single file or a directory.   I do not have a way of issuing a new command via piping to the stay-open instance of exiftool.  For example I want to read the tags from anther jpeg file using the instance of exiftool that was previously opened.  Can a new command be sent to an open instance of exiftool using a pipe?
If it can, then piping would be better than shelling.

[Phil Harvey]

Can a new command be sent to an open instance of exiftool using a pipe?

Yes.  That is the whole point of the -stay_open option.

- Phil

[Skippy]

Apparently piping with VBA is a bit flakey and your code can hang if the expected response is not forthcoming from the other application.  Here is more info on how to do it if you want to try.  http://stackoverflow.com/questions/12374592/using-vba-to-control-another-program-entirely.  I am sticking with shelling.  Shellwait is used for tasks that need to finish before I proceed and just shell for tasks that can grind on for ages and for which I don't need instant feedback.  The former are unusually reading from exif and the latter writing to exif (i.e. synchronous vs asynchronous processes).  Shellwait is a VBA procedure not a built in function. 

[harrys]

I am also looking at alternatives ... VBA Excel

When listing file properties using headings of ClassName and PropertyName  These are stored in a sorted list when the file is given
so that there is less  time  when Various properties are called across the columns for the same Class  (  WIA Namespace DSO FindFirstFile Folder3)

Is there a piping  alternate like below

'''''''   the Stdout.Readall  does work Shell VBA

Property Let SWSoList(V)
   Set wSoList = Nothing
   Set wSoList = CreateObject("System.Collections.SortedList")
   Set wSoList = V

End Property

'wSoList now contains the  keys of the needed property name
'
Sub GetNeededInfo()
   Dim RI&, SnSA$(), Exiftool$, PropNa$
   
   For RI = 0 To wSoList.Count - 1
      PropNa = wSoList.getkey(RI)
      NeededInfo = NeededInfo & " -" & PropNa
      wSoList.Item(PropNa) = ""
   Next RI
   
   Exiftool = "C:\exifc\exiftool.exe -s -m -stay_open "
   ShCmd = "cmd /c " & Exiftool & NeededInfo & " " & Chr(34) & FileNa & Chr(34)
   SnSA = Split(CreateObject("wscript.shell").exec(ShCmd).StdOut.ReadAll, vbCrLf)
   
   ' returns values like
'Directory                       : U:/Apr7Store/bas
'FileAccessDate                  : 2019:04:07 05:56:11+10:00
'FileCreateDate                  : 2019:04:07 05:56:11+10:00
'FileModifyDate                  : 2019:04:06 14:33:39+10:00

'
' put in the items for the Keys '
'
   For RI = 0 To UBound(SnSA)
      If Left(SnSA(RI), 4) <> "====" Then
         PropNa = Trim(Left(SnSA(RI), 31))
         With wSoList
            If .contains(PropNa) Then
               .Item(PropNa) = Mid(SnSA(RI), 35)
            End If
         End With
      End If
   Next RI
End Sub

' Have in my younger  ( age <70) days use some piping like

'exiftool '-Comment<$Comment Stuff added to comment' dst.jpg
'exiftool -gpslatitude -gpslongitude -csv -r DIR > out.csv
'     ExifTool = "C:\exifc\exiftool.exe"
Private Function ExecuteCommand(mCommand As String)
    Dim start As STARTUPINFO
    Dim SA As SECURITY_ATTRIBUTES
    If Len(mCommand) = 0 Then Exit Function
    SA.nLength = Len(SA)
    SA.bInheritHandle = 1&
    SA.lpSecurityDescriptor = 0&
    If CreatePipe(hReadPipe, hWritePipe, SA, 0) = 0 Then Exit Function
    start.cb = Len(start)
    start.dwFlags = &H100& Or &H1
    start.hStdOutput = hWritePipe
    start.hStdError = hWritePipe
    If CreateProcessA(0&, mCommand, SA, SA, 1&, &H20&, 0&, 0&, start, proc) <> 1 Then Exit Function
    CloseHandle (hWritePipe)
End Function