PayPal Hire - Need an Image Sorting Script

[BJ1200]

Hi All,

After cleaning up my image metadata, several hundred had errors and warning messages.

I would be willing to pay someone to write a script that would go through all images in a folder and move those that trigger a warning or error message to another folder. The script should check each image, I don't have all the log error reports. If help comes from this forum, a donation will be made to ExifTool using that spot on bottom of the homepage.

I batch added face coordinates as XMP metadata to all my images. If possible, the script should also move those that don't have XMP face coordinates to another folder. The facial recognition didn't find some faces so there were no coordinates to assign to that image. This will help me find those images so I can reprocess using more aggressive facial-rec settings.

The goal is to move out all problem images and those without XMP face coordinates to other folders so I can fix them. There are about 60 folders, some with up to 18,000 images. There's just no way to manually find and move all these images.

I attached some example files and a PDF with an example. It also shows the warning/error messages the JPG files generated.

[Phil Harvey]

To move images that generate exiftool warnings to another folder:

exiftool -if "$warning" -directory="SOME FOLDER NAME" DIR

Where DIR is the directory containing the images.  Add -r to also process images in sub-directories.  Use single quotes instead of double if you are on Mac or Linux.

You can easily add a check for face recognition tags in the -if logic if you know what tag names to check.

- Phil

[ryerman]

Here's a suggestion:

Record the original directory to make it easier to return the image to its original location.
Add -SOME_TAG_NAME<directory to your command.

After fixing, execute a command that includes -directory<SOME_TAG_NAME to return the images to their original folders.

Jim

[BJ1200]

Thanks guys. Dang....I knew this would happen. I am not a developer, not familiar with CMD prompt stuff. This is why I use the ExifGUI. I'm not even sure where to enter this line of code so I will have to use the GUI ExifTool Direct area. I believe I drop the exiftool prefix. 

If Images are in:   C:\Users\BJ12\Volunteer Images
An bad images go here:   C:\Users\BJ12\Volunteer Images\Repaired

The correct code entered in the GUI would be:
-if "$warning" -directory="Repaired" C:\Users\BJ12\Volunteer Images


There is a field called "FaceInfoFaceRectangle" under the XMP-AnVoItPicasa category of XMP data. If this field exists, and there are no error messages, this image is good to go. Not sure how to incorporate a check for this field. (Screenshot is in attached PDF file.

I have already processed images using -r but eventually chickened out. I went back and used the GUI, manually checked the box for metadata removal, each folder one by one.

[BJ1200]

Record the original directory to make it easier to return the image to its original location.
Add -SOME_TAG_NAME<directory to your command.

Jim

Thanks Jim. For some reason I'm scared to death of using CMD stuff. I've had a few bad experiences. I will probably use cut/paste. If I can't see it happen I tend to get paranoid it dldn't...or won't...or can't...or wrong place...or...  Get the personality profile picture here?

[BJ1200]

To move images that generate exiftool warnings to another folder.....:
- Phil

Thanks Phil - 1st donation already sent.
Confirmation number: 5WW73197U8593773S

[BogdanH]

Hi,

Some had bad experience when using ExifToolGUI on very large amount of files (don't ask me why, because I don't have that many files to tryout). So, it is a good advice, you first tryout on smaller amount of files -so you can see/confirm, that the command does what you want. But, no matter what, if single folder contains several thousand of files (!?), then I would process such folder(s) individually; without -r option. In this case, if something goes wrong, it is esier to determine the reason (where and why) and repeat process if necessary.

Bogdan

[BJ1200]

Thanks everyone. I will need to set up a buffet with open bar to payback everyone.

All Images are in:   C:\Users\BJ12\Volunteer Images
Bad images should be moved to here:   C:\Users\BJ12\Volunteer Images\Repaired

Is this the correct code to enter in EXiftTool Direct area of the GUI:
-if "$warning" -directory="Repaired" C:\Users\BJ12\Volunteer Images
(The above code did not work. It didn't move any bad files. It said 0 file read)

What is the code to check tag named "FaceInfoFaceRectangle" exists in the metadata. If this field name does not exist, the code must move the image to another folder. Phil suggested this was possible but well over my head. Below shows the tag location. 

[Phil Harvey]

The command to move the non-FaceInfosFaceRectangle (note the "s" you missed in your discription, but that appears in your screen cap) images is:

exiftool -if "not $FaceInfosFaceRectangle" -directory="SOME OTHER FOLDER" DIR

I'll let Bogdan handle the GUI questions.

- Phil

[BogdanH]

Hi,

No, no... don run away Phil; I'm afraid, your knowledge is required here  I'm not that familiar with commands for sorting/moving files with ExifTool and I just hope it will work via ExifTool direct as expected.
In this case, there are only two things needed to keep in mind:
1. Just commands/options must be written (without "exiftool" at the beginning).
2. If working on folders/directories and -r option is not used, then no file should be selected in Filelist panel. If -r option is used, then selected folder in Browse panel is automatically used for DIR (source directory) -means: DIR shouldn't be specified in this case.

Bogdan

[Phil Harvey]

I'm not running away.

I was just hoping you would describe how the directory selection process works in the GUI, which you did.  Thanks.

What about the quoting though?  Is this what should be entered in the GUI?:

-if "$warning" -directory="Repaired" "C:\Users\BJ12\Volunteer Images"

Also, in the GUI, I believe that the working directory is set to the currently selected folder when running exiftool direct commands.  So directory names in the command ("Repaired") will be relative to this directory, right?

- Phil

[BogdanH]

Hi Phil,

..in the GUI, I believe that the working directory is set to the currently selected folder when running exiftool direct commands.

Yes. For example, if currently selected folder is "c:\Photos" and executing ExifTool direct, this is the same as "being" in that folder (in console window)
c:\Photos>_
..and executing ExifTool.
And if no file is selected in Filelist panel, then GUI expect, files (path) will be specified at the end of command/options -just like if exiftool is executed in console.

Bogdan

[BJ1200]

What about the quoting though?  Is this what should be entered in the GUI?:
-if "$warning" -directory="Repaired" "C:\Users\BJ12\Volunteer Images"
- Phil

I tried the above, even tried using the full path after-directory. Nothing happened but it said said "0 image files read" at the bottom of the GUI. No bad images were moved, no log was produced. So I tried adding exiftool to the beginning. Nothing was moved, but it does generate a log below (Using a small test folder of images for now).

Also, shouldn't it be "$error or $warning" or something like that so it can find images with either problem?

1 directories scanned
    9 files failed condition
    0 image files read
    1 files could not be read
File not found: exiftool
<-END-


THE XMP DATA WORKED! Moved images that don't have the FaceInfosFaceRectangle field. I had to enter the full directory path in both locations. When I didn't enter the full path after -directory=, it would delete all images (or it moved all of them to a place I have yet to find).
-if "not $FaceInfosFaceRectangle" -directory="C:\Users\BJ12\Volunteer Images\Repaired" "C:\Users\BJ12\Volunteer Images"

[BogdanH]

Parameter -directory is the directory to where files should be moved. If you use -directory= only, without specifying it's path/name, then (I assume) particular files in source directoy (the last specification) will be deleted.
If directory names contain spaces (i.e. ..Volunteer images) then whole parameter must be in double quotes. There are few examples here: https://exiftool.org/gui/#p_etdirect

Bogdan

[Phil Harvey]

I tried the above, even tried using the full path after-directory. Nothing happened but it said said "0 image files read" at the bottom of the GUI. No bad images were moved, no log was produced. So I tried adding exiftool to the beginning. Nothing was moved, but it does generate a log below (Using a small test folder of images for now).

Also, shouldn't it be "$error or $warning" or something like that so it can find images with either problem?

1 directories scanned
    9 files failed condition
    0 image files read
    1 files could not be read
File not found: exiftool
<-END-

I don't know why you didn't get a log before you added "exiftool" to the parameters (which shouldn't be there btw).  Perhaps bogdan can answer this.

After adding exiftool, the log shows that 9 files failed the condition.  So if the condition was -if "$warning", then the files didn't have a warning.  You can test for errors too if you want, exactly as you specified above.

When I didn't enter the full path after -directory=, it would delete all images (or it moved all of them to a place I have yet to find).

Setting -directory= should do nothing.  There must be something else going on here.

If you use -directory= only, without specifying it's path/name, then (I assume) particular files in source directoy (the last specification) will be deleted.

No.  I'm not that cruel.

In fact, there should be no way to get exiftool to delete a file.

- Phil

[BJ1200]

Hi Bogdan and Phil,

My bad, those 9 files only contain errors (below). I was under the assumption using $warning was a generic term for any problematic file. I will add known warning files and test again.

   3 image files updated
    2 image files unchanged
    4 files weren't updated due to errors
Error: JPEG EOI marker not found - anthony batarse.jpg
Error: Not a valid JPEG (looks more like a PNG) - billy lockwood.jpg
Error: Not a valid JPEG (looks more like a BMP) - Chris Menges.jpg
Error: Not a valid JPEG (looks more like a XMP) - jennifer ellison.jpg

[BJ1200]

Well this is frustrating. Awhile back I used the GUI to clean metadata and when it was done with a folder it created a log report, similar to the below:

Warning: [minor] Ignored APP1 XMP segment with non-standard header - Matt Stone.jpg
Warning: Multiple Photoshop records - Melody Noel.jpg
Error: Not a valid JPEG (looks more like a GIF) - Mike Bailey.jpg
Warning: Multiple Photoshop records - Mike Rembis.jpg
Warning: [minor] Ignored APP1 XMP segment with non-standard header - Miles Chalmers.jpg

I have manually extracted many of the images in the log that were listed as "Warnings" to use as test images. Using the code to find $warning images does not detect them. So I processed these images again that produced the original log - using the GUI to remove all metadata. To my surprise, none show up on the log as "warning" images. I suspect cleaning the metadata the first time fixes some of them, but I can't find one with a warning message.

[BJ1200]

I'm not able to get this to work, to scan for images with warning or errors, then move the. It moves all images, not just those with errors or warnings. Here's the log:

======== –if
======== $error or $warning
    1 directories scanned
   22 image files updated
    2 files weren't updated due to errors
Error: File not found - â€"if
Error: File not found - $error or $warning
<-END-

[Phil Harvey]

In hour last post it looks to me as if you are using some funny Unicode character instead of a simple dash in "-if".  Try typing simple dash.

- Phil

Edit: I meant "your", not "hour".

[BJ1200]

In hour last post it looks to me as if you are using some funny Unicode character instead of a simple dash in "-if".  Try typing simple dash.
- Phil

Hmm, good catch. However, that dash works with the code for moving images without XMP fields. I create the long string of code in Word and found the " signs are problamatic. When pasted into ExifTool Direct I have to delete those " and type them again. Then it works fine. I will try and change the dash.

[BJ1200]

Hi Phil,

These must be two distinctly different evaluations for errors and warnings. Using the GUI to remove all metadata picks up (logs) substantially more errors than using -if "$error".

Using the GUI to remove all metadata (Log results):
0 image files updated
15221 image files unchanged
   15 files weren't updated due to errors
Error: Not a valid JPEG (looks more like a PNG) - Adam McKay.jpg
Error: JPEG EOI marker not found - Alicia Leigh Willis.jpg
Error: Not a valid JPEG (looks more like a PNG) - Cliff De Young.jpg
Error: JPEG EOI marker not found - Colleen Kelly.jpg
Error: Not a valid JPEG (looks more like a PNG) - Ed Gale.jpg
Error: JPEG EOI marker not found - George Fisher.jpg
Error: Not a valid JPEG (looks more like a GIF) - Gert Fröbe.jpg
Error: JPEG EOI marker not found - Hannah Rebecca Telle.jpg
Error: JPEG EOI marker not found - Marisa Petroro.jpg
Error: Not a valid JPEG (looks more like a GIF) - Michael Goodliffe.jpg
Error: Not a valid JPEG (looks more like a GIF) - nolan hemmings.jpg
Error: JPEG EOI marker not found - Patrick Stevenson.jpg
Error: JPEG EOI marker not found - Sandra Nelson.jpg
Error: Not a valid JPEG (looks more like a GIF) - susan vidler.jpg
Error: Not a valid JPEG (looks more like a BMP) - Thomas Gottschalk.jpg
<-END-

Now the same folder using the -if "$error" method in ExifDirect:
  1 directories scanned
15236 files failed condition
    0 image files read
    1 files could not be read
File not found: C:/Users/Media/J??rgen Prochnow.jpg
<-END-

[Phil Harvey]

I should have mentioned this.  I didn't realize the errors you were getting were while writing.

The -if "$error" will be true only for files that give errors when reading, which is very different than when writing.  Many errors when writing are only warnings when reading, and you will get some warnings when writing that you don't get when reading.  This is because the conditions to successfully write an image are more strict than for reading.

It is more difficult if you want to identify all of the images which give write errors.  Also, some write errors won't show up unless you write the specific type of metadata which generates the error.  For example, a JPEG image with corrupted EXIF won't generate errors if you write XMP only.  So the whole thing is much more complicated if this is what you are asking, and in this case I don't know of a good way to do what you want.

- Phil