Progress Output Question/Request

[daviss92]

First off, I'd like to say that I love the ExifTool. I'm by no means a developer, but I love to fool around with programming in my spare time (trying to teach myself Python, know some AppleScript/Bash). My day job also benefits from using these, without going into too much detail, I'm responsible for the integrity of 1000's of image files.

I've searched all over for some details on full reporting of the exiftools output. I came across "-progress" which spits out all files touched. This is great! But when I'm working with 1000's of files at a time it can be a pain if exiftool has edited 100's of files, while 100's more are untouched... and going through the output (via Excel, or any other program) can be tedious. I could even be missing something and everything below this could be useless.. But here we go!

I created a rough Applescript (which I welcome anyone to use if you have a similar issue), but my question is would there be anymore updating to the "-progress" command in the future? My Applescript works now, but this wouldn't work for everyone because of their current OS X version, deprecated commands, etc. (Works for me on 10.10.5).

To Summarize the Code:

• Strips the "=====" and "[134/572]" stuff
• Sorts into lists and spits out Files Touched/Files Untouched
• Saves text file on users desktop with FileName.EXT + "_ExifTool_Results.txt"

Applescript:
-- Pull User Picked File and Read / Convert to List
set theFile to choose file with prompt "Please Select File:"
tell application "Finder" to set fileName to (name of theFile)
set fileHandler to open for access theFile
set theFileRead to paragraphs of (read fileHandler)
close access fileHandler

-- Fixing Up Path if Needed
set pathChar to "\\" & " "
set newPathContent to {}
repeat with nextPathLine in theFileRead
   if nextPathLine contains pathChar then
      set stringPathToFind to pathChar
      set stringPathToReplace to "\\_"
      set {oldTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringPathToFind}
      set oldPath to every text item of nextPathLine
      set AppleScript's text item delimiters to stringPathToReplace
      set newPath to oldPath as string
      set AppleScript's text item delimiters to oldTID
      copy newPath to the end of newPathContent
   else
      copy nextPathLine to the end of newPathContent
   end if
end repeat

-- Determining Variables
set overviewFound to {}
set notTouchedFound to {}
set touchedFound to {}
set finalTouchedFiles to {}
set counterUnChanged to 0
set counterChanged to 0

-- Parse List One-by-One
repeat with nextLine in newPathContent
   -- Copying Directories Scanned, Image Files Updated, and Image Files Unchanged to New Variable
   if (nextLine contains "directories scanned") or (nextLine contains "image files updated") or (nextLine contains "image files unchanged") then
      copy nextLine to the end of overviewFound
   end if
   -- Copying All Items with "========" at Beginning to New Variable While Removing "========"
   if nextLine contains "======== " then
      set stringToFind to "======== "
      set stringToReplace to ""
      set theContent to nextLine
      set {oldTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringToFind}
      set ti to every text item of theContent
      set AppleScript's text item delimiters to stringToReplace
      set newContent to ti as string
      set AppleScript's text item delimiters to oldTID
      set trimText to newContent
      --This Line Below Removes Anything after File (e.g. [124/514])
      set trimTextFinal to text 1 thru ((offset of " [" in trimText) - 1) of trimText
      copy trimTextFinal to the end of touchedFound
   end if
   -- Copying All Items with "Nothing change in " at Beginning to New Variable While Removing "Nothing change in "
   if nextLine contains "Nothing changed in " then
      set stringToFind to "Nothing changed in "
      set stringToReplace to ""
      set theContent to nextLine
      set {oldTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringToFind}
      set ti to every text item of theContent
      set AppleScript's text item delimiters to stringToReplace
      set newContent to ti as string
      set AppleScript's text item delimiters to oldTID
      set trimText2 to newContent
      --This Line Below Removes Anything after File (e.g. [124/514])
      set trimTextFinal2 to text 1 thru ((offset of " [" in trimText2) - 1) of trimText2
      copy trimTextFinal2 to the end of notTouchedFound
   end if
end repeat

-- Go Through both Variables "touchedFound" and "notTouchedFound". Creates New Variable with Final Outcome
repeat with a from 1 to count of items of touchedFound
   set counterChanged to counterChanged + 1
   set n to item a of touchedFound
   if n is not in notTouchedFound then
      set end of finalTouchedFiles to "[" & counterChanged & "] " & n
   end if
end repeat

-- Begin Process to Write to File
set finalFile to (path to desktop as Unicode text) & fileName & "_ExifTool_Results.txt"
set myFile to open for access file finalFile with write permission
write "Results for ExifTool Command:" & return to myFile
repeat with z in overviewFound
   write z & return to myFile
end repeat
write return & "Image Files Updated" & return & "----------------------------------------------" & return to myFile
repeat with y in finalTouchedFiles
   write y & return to myFile
end repeat
write return & "Image Files Unchanged" & return & "----------------------------------------------" & return to myFile
repeat with x in notTouchedFound
   set counterUnChanged to counterUnChanged + 1
   write "[" & counterUnChanged & "] " & x & return to myFile
end repeat
close access myFile

-- Finish File Process If Needed
set stringFind to "\\_"
set stringReplace to pathChar
set theFileFinal to finalFile
set stringContent to read alias theFileFinal as «class utf8»
if stringContent contains "\\_" then
   set {oldFinalTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringFind}
   set si to every text item of stringContent
   set AppleScript's text item delimiters to stringReplace
   set newFinalContent to si as string
   set AppleScript's text item delimiters to oldFinalTID
   
   set sd to open for access theFileFinal with write permission
   set eof of sd to 0
   write newFinalContent to sd as «class utf8»
   close access sd
end if

Applescript Output File:

Code Select Expand

Results for ExifTool Command:
    2 directories scanned
   30 image files updated
   27 image files unchanged

Image Files Updated
----------------------------------------------
[1] /Users/daviss/Desktop/exiftest/ACTIVIASRBP12X100G.EPS
[2] /Users/daviss/Desktop/exiftest/APVLYNSABLUEBPIE620G.EPS
[3] /Users/daviss/Desktop/exiftest/ARIZONABLUEBERRYDIET.EPS
[4] /Users/daviss/Desktop/exiftest/ARTHURSORGANICSBLUEBTE.EPS
[5] /Users/daviss/Desktop/exiftest/ARTHUSBLUEBERRYTEA325M.EPS
[6] /Users/daviss/Desktop/exiftest/ASSTFRUITCUP1.EPS
[7] /Users/daviss/Desktop/exiftest/ASTROBBANTXWLDBB650G.EPS
[8] /Users/daviss/Desktop/exiftest/ASTROBIOBESTACAIBLUBR.EPS
[9] /Users/daviss/Desktop/exiftest/ASTROJEUNNESEYOGOURT.EPS
[10] /Users/daviss/Desktop/exiftest/ASTROZEROBB650G.EPS
[11] /Users/daviss/Desktop/exiftest/BBPLNSTEROLS8X100G.EPS
[12] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT1.EPS
[13] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT2.EPS
[14] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT3.EPS
[15] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT4.EPS
[16] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT5.EPS
[17] /Users/daviss/Desktop/exiftest/BEAUTYFRUIT6.EPS
[18] /Users/daviss/Desktop/exiftest/BERRYSMOOTHIE.EPS
[19] /Users/daviss/Desktop/exiftest/BIGBLUESBLUEBERRIES12.EPS
[20] /Users/daviss/Desktop/exiftest/BLEBRYSCNSPLT.EPS
[21] /Users/daviss/Desktop/exiftest/BLUBERRYLOAF.EPS
[22] /Users/daviss/Desktop/exiftest/BLUBRYSCNSPK.EPS
[23] /Users/daviss/Desktop/exiftest/BLUEBERRIES.EPS
[24] /Users/daviss/Desktop/exiftest/BLUEBERRIES.PSD
[25] /Users/daviss/Desktop/exiftest/BLUEBERRIES_FRSH_CLIP.EPS
[26] /Users/daviss/Desktop/exiftest/BLUEBERRIES_PNT_ONT.EPS
[27] /Users/daviss/Desktop/exiftest/BLUEBERRIESBEAUTY.EPS
[28] /Users/daviss/Desktop/exiftest/BLUEBERRIESBOWL.EPS
[29] /Users/daviss/Desktop/exiftest/BLUEBERRIESBOXTOPOFF.EPS
[30] /Users/daviss/Desktop/exiftest/BLUEBERRIESBSFIELD.EPS

Image Files Unchanged
----------------------------------------------
[1] /Users/daviss/Desktop/exiftest/test_subfolder2/ARIZONABLUEBERRYDIET.EPS
[2] /Users/daviss/Desktop/exiftest/test_subfolder2/ARTHURSORGANICSBLUEBTE.EPS
[3] /Users/daviss/Desktop/exiftest/test_subfolder2/ARTHUSBLUEBERRYTEA325M.EPS
[4] /Users/daviss/Desktop/exiftest/test_subfolder2/ASSTFRUITCUP1.EPS
[5] /Users/daviss/Desktop/exiftest/test_subfolder2/ASTROBBANTXWLDBB650G.EPS
[6] /Users/daviss/Desktop/exiftest/test_subfolder2/ASTROBIOBESTACAIBLUBR.EPS
[7] /Users/daviss/Desktop/exiftest/test_subfolder2/ASTROJEUNNESEYOGOURT.EPS
[8] /Users/daviss/Desktop/exiftest/test_subfolder2/ASTROZEROBB650G.EPS
[9] /Users/daviss/Desktop/exiftest/test_subfolder2/BBPLNSTEROLS8X100G.EPS
[10] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT1.EPS
[11] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT2.EPS
[12] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT3.EPS
[13] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT4.EPS
[14] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT5.EPS
[15] /Users/daviss/Desktop/exiftest/test_subfolder2/BEAUTYFRUIT6.EPS
[16] /Users/daviss/Desktop/exiftest/test_subfolder2/BERRYSMOOTHIE.EPS
[17] /Users/daviss/Desktop/exiftest/test_subfolder2/BIGBLUESBLUEBERRIES12.EPS
[18] /Users/daviss/Desktop/exiftest/test_subfolder2/BLEBRYSCNSPLT.EPS
[19] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUBERRYLOAF.EPS
[20] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUBRYSCNSPK.EPS
[21] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIES.EPS
[22] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIES.PSD
[23] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIES_FRSH_CLIP.EPS
[24] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIES_PNT_ONT.EPS
[25] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIESBEAUTY.EPS
[26] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIESBOWL.EPS
[27] /Users/daviss/Desktop/exiftest/test_subfolder2/BLUEBERRIESBOXTOPOFF.EPS


I hope all this makes sense! Not sure if this is even in the right place.

Thanks for taking the time to read this!

[Phil Harvey]

The output of the -progress option will not change in the future.  Many other scripts rely on the exact syntax of this output.

But it sounds like you don't want to know the progress (since you're tossing the "[134/572]" stuff), and if so you shouldn't be using the -progress option since it involves an extra pass through the directory structure to pre-determine the files to be processed.  Instead, you should use -v0 to just list the name of each file as it is processed.  The -v0 output is the same as -progress, but without the  "[134/572]" stuff.

- Phil

[daviss92]

Thanks for the quick response.

I tested my Applescript with the -v0 and it works the same. I could omit that section of code that strips the "[134/572]" stuff but I'll leave it for now. It's re-assuring that if anyone uses -v0 or -progress with the applescript I created, it will run the same.

My original question was more/less asking if in the future there would be a command that spits out results in a similar format. It's not the end of the world if this is a no, but I figured I'd ask anyways.

Cheers.

[Phil Harvey]

Ah, I see.  I don't foresee this as being a feature that I would be adding to ExifTool.  (But you never know, I suppose.)

- Phil

[daviss92]

Actually, related to this.. Is it possible to include any "Error: Message Here - /Path/To/File.ext" in the output?

Current Output:

Code Select Expand


======== /Users/daviss/Desktop/exif test/test_subfolder2/ARIZONABLUEBERRYDIET.EPS [31/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ARIZONABLUEBERRYDIET.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ARTHURSORGANICSBLUEBTE.EPS [32/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ARTHURSORGANICSBLUEBTE.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ARTHUSBLUEBERRYTEA325M.EPS [33/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ARTHUSBLUEBERRYTEA325M.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ASSTFRUITCUP1.EPS [34/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ASSTFRUITCUP1.EPS


Something similar like this:

Code Select Expand


======== /Users/daviss/Desktop/exif test/test_subfolder2/ARIZONABLUEBERRYDIET.EPS [31/57]
Error: Not a valid EPS (looks more like a PSD) - /Users/daviss/Desktop/exif test/test_subfolder2/ARIZONABLUEBERRYDIET.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ARTHURSORGANICSBLUEBTE.EPS [32/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ARTHURSORGANICSBLUEBTE.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ARTHUSBLUEBERRYTEA325M.EPS [33/57]
Nothing changed in /Users/daviss/Desktop/exif test/test_subfolder2/ARTHUSBLUEBERRYTEA325M.EPS
======== /Users/daviss/Desktop/exif test/test_subfolder2/ASSTFRUITCUP1.EPS [34/57]
Error: Not a valid EPS (looks more like a PSD) - /Users/daviss/Desktop/exif test/test_subfolder2/ASSTFRUITCUP1.EPS


I've noticed if there is an issue, it'll still show in "-progress" or "-v0", but without any indicator of it being an issue.

Thanks!

[Phil Harvey]

The error messages like this should go to stderr.  You must also capture stderr to see them.  Perhaps adding "2>&1" to your command will do this for you.

- Phil

[daviss92]

I've included 2>&1 into my command but it does not seem to output the error message.

exiftool -r -v0 -overwrite_original -XMP:all= /Users/daviss/Desktop/test\ 3 2>&1 > exiftool_test.txt

E.g. Still shows "1 files weren't updated due to errors", but no recorded Error in the text file.

Code Select Expand


...
======== /Users/daviss/Desktop/test 3/P/PROD_test_SEN_PESTO_142G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SNACKMIX_525G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SNACKMIX_PARTYB225.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SPIN_BABY_142G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SPIN_BABYORG_142G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SPIN_ORG_312.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_STIRFRY_VEG_340G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_SUNFLOWERSEED450G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_COCKTAIL454G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_COCKTAIL_ORN454.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_COCKTAIL_YEL454.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_MEDLEY_340G.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_ROMA_VINE500.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOM_SUNSETCAMPARI450.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TOMATO_GRAPE_551ML.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_TRAILMIX_COL_BULK_500.EPS
======== /Users/daviss/Desktop/test 3/P/PROD_test_VEGMEDLEY_340G.EPS
    7 directories scanned
  420 image files updated
    1 files weren't updated due to errors


[Phil Harvey]

I think that "2>&1" must come after ">exiftool_test.txt".  Google for bourne shell redirection for help with this.

- Phil

[daviss92]

Thanks Phil!

This does the trick.

exiftool -r -v0 -overwrite_original -XMP-xmpMM:all= -XMP-xmpRights:all= -XMP-mb:all= /Users/daviss/Desktop/test\ 4  > tester123456789.txt 2>&1

Code Select Expand


======== /Users/daviss/Desktop/Alpha 4/B/SMINITARTS.EPS
======== /Users/daviss/Desktop/Alpha 4/B/SNAANBREAD340G.EPS
======== /Users/daviss/Desktop/Alpha 4/B/SPIES1KG.EPS
======== /Users/daviss/Desktop/Alpha 4/B/WGCOOKIEOATM.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BASKETCOMPSEN2011.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPBLUEBRRYBAG.EPS
Error: Not a valid EPS (looks more like a PSD) - /Users/daviss/Desktop/Alpha 4/B/BTYCOMPBLUEBRRYBAG.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPBRUSSELSSPRTBAG.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPCHEESEBALL227G.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPCHICKENTAGINE.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPCUPCAKES.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPGMBEEFBOURG.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPGMBEEFBROC.EPS
======== /Users/daviss/Desktop/Alpha 4/B/BTYCOMPGMCHICCACCIATORE.EPS


[daviss92]

For anyone who is interested. Here is an updated code, including Error reporting:

Removed some unnecessary stuff, simplified the code as well.

Applescript:

Code Select Expand


-- Pull User Picked File and Read / Convert to List
set theFile to choose file with prompt "Please Select File:"
tell application "Finder" to set fileName to (name of theFile)
set fileHandler to open for access theFile
set theFileRead to paragraphs of (read fileHandler)
close access fileHandler

-- Determining Main Variables
set overviewFound to {}
set notTouchedFound to {}
set touchedFound to {}
set finalTouchedFiles to {}
set errorFound to {}
set errorFoundTrim to {}
-- Counter Variables
set counterUnChanged to 0
set counterChanged to 0
set counterError to 0

-- Parse List One-by-One
repeat with nextLine in theFileRead
-- Copying Directories Scanned, Image Files Updated, and Image Files Unchanged to New Variable
if (nextLine contains "directories scanned") or (nextLine contains "image files updated") or (nextLine contains "image files unchanged") or (nextLine contains "files weren't updated due to errors") then
set overviewTrimmer to (do shell script "echo " & quoted form of nextLine & " | sed 's|^\\ *||'")
copy overviewTrimmer to the end of overviewFound
-- Copying All Items with "===" at Beginning to New Variable While Removing "==="
else if nextLine contains "======== " then
set stringToFind to "======== "
set stringToReplace to ""
set theContent to nextLine
set {oldTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringToFind}
set ti to every text item of theContent
set AppleScript's text item delimiters to stringToReplace
set newContent to ti as string
set AppleScript's text item delimiters to oldTID
set trimText to newContent
--This Line Below Removes Anything after File (e.g. [124/514])
if nextLine contains " [" then
set trimTextFinal to text 1 thru ((offset of " [" in trimText) - 1) of trimText
copy trimTextFinal to the end of touchedFound
else
copy trimText to the end of touchedFound
end if
-- Copying All Items with "Nothing change in " at Beginning to New Variable While Removing "Nothing change in "
else if nextLine contains "Nothing changed in " then
set stringToFind to "Nothing changed in "
set stringToReplace to ""
set theContent to nextLine
set {oldTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, stringToFind}
set ti to every text item of theContent
set AppleScript's text item delimiters to stringToReplace
set newContent to ti as string
set AppleScript's text item delimiters to oldTID
set trimText2 to newContent
copy trimText2 to the end of notTouchedFound
else if nextLine contains "Error:" then
copy nextLine to the end of errorFound
set trimError to text ((offset of "- /" in nextLine) + 2) thru -1 of nextLine
copy trimError to the end of errorFoundTrim
end if
end repeat

-- Finish Parsing Variables, Last Steps before Write.
repeat with x from 1 to count of items of touchedFound
set n to item x of touchedFound
if n is not in notTouchedFound and n is not in errorFoundTrim then
set counterChanged to counterChanged + 1
set end of finalTouchedFiles to "[" & counterChanged & "] " & n
end if
end repeat

-- Begin Process to Write to File
set finalFile to (path to desktop as Unicode text) & fileName & "_ExifTool_Results.txt"
set myFile to open for access file finalFile with write permission
write "Results for ExifTool Command:" & return & "------------------------" & return to myFile
repeat with z in overviewFound
write z & return to myFile
end repeat
if (count of finalTouchedFiles) is greater than 1 then
write return & "Image Files Updated:" & return & "------------------------" & return to myFile
repeat with y in finalTouchedFiles
write y & return to myFile
end repeat
end if
if (count of notTouchedFound) is greater than 1 then
write return & "Image Files Unchanged:" & return & "------------------------" & return to myFile
repeat with x in notTouchedFound
set counterUnChanged to counterUnChanged + 1
write "[" & counterUnChanged & "] " & x & return to myFile
end repeat
end if
if (count of errorFound) is greater than 1 then
write return & "Errors Found:" & return & "------------------------" & return to myFile
repeat with z in errorFound
set counterError to counterError + 1
write "[" & counterError & "] " & z & return to myFile
end repeat
end if
close access myFile