Author Topic: Sorting out photos with content verification (md5 / sha256)  (Read 136 times)

lumiere

  • Newbie
  • *
  • Posts: 3
Hello all,

I have several family pictures scattered across many, many folders often with clones around.
Some of them were taken with burst mode (many pictures in the same second) so YYYY/MM/DD/HH/MM/SS is not the right pattern.
I would like to copy them over to final folder destination sorting them by:
YYYY/MM/DD/HH-MM-SS.[ext]
But if there is already a picture with particular name in the target folder then I would like to compare md5/sha256 of their content to really make sure they are the same.
If they are the same then skip it. If they are not then create new file with [name]-1.jpg, [name]-2.jpg etc
Is there any way I can achieve this with exiftool ?
Also is there any way I can check the md5/sha256 in parallel ? I have quite good CPU with many threads and can burn it to speed up the check.

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 17043
    • ExifTool Home Page
Re: Sorting out photos with content verification (md5 / sha256)
« Reply #1 on: July 13, 2020, 02:44:31 PM »
ExifTool doesn't have the ability to do a checksum of the entire file.  I would suggest using an md5 utility to generate checksums for all of your files, then removing the files from the list that have duplicate checksums, then sending the remaining file names to ExifTool with a command like this:

exiftool "-filename<createdate" -d %Y/%m/%d/%H-%M-%S%%-c.%%e -ext jpg -r DIR

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

lumiere

  • Newbie
  • *
  • Posts: 3
Re: Sorting out photos with content verification (md5 / sha256)
« Reply #2 on: July 13, 2020, 06:17:18 PM »
Thanks @Phil !
Could it be more generic command working not only with jpg but all jpg/png/gif at once ?
Also can it be sorted based on exif data stored inside each file ?

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 17043
    • ExifTool Home Page
Re: Sorting out photos with content verification (md5 / sha256)
« Reply #3 on: July 13, 2020, 08:20:15 PM »
Yes.  Just add more -ext options with whatever files you want to move.  Or leave out the -ext option entirely to move anything in the exiftool -listwf extension list (provided it has a CreateDate tag).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

lumiere

  • Newbie
  • *
  • Posts: 3
Re: Sorting out photos with content verification (md5 / sha256)
« Reply #4 on: July 14, 2020, 01:51:37 PM »
Thanks Phil again :-)
WIll it sort based on exif data stored inside those files or based on ... filename ?
I am not sure I understand what the syntax "-filename<createdate" means ...

Phil Harvey

  • ExifTool Author
  • Administrator
  • ExifTool Freak
  • *****
  • Posts: 17043
    • ExifTool Home Page
Re: Sorting out photos with content verification (md5 / sha256)
« Reply #5 on: July 15, 2020, 06:27:25 AM »
Read here for more details about setting the FileName.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).