ExifTool Forum

Hi 4 things to start

1.) Hi All
2.) You're Registration Questions for excluding computers were the best I have ever seen
3.) Thanks for all the hard work, I can't believe I didn't find this tool years ago!
4.) Need a little help :)

I have about 20K photos I have taken over the years, and for the most part, its JPG's from phones and old camera's, but I have about 10K that are RAW (Nikon) from DSLR and Mirrorless (Nikon). Now that I have decided to organize this hot mess due, mostly to the "stay at home" order.

Of those DSLR/Mirrorless images, Lightroom has XMP files for a bunch of them. I have used this command to reorganize my photos

exiftool -progress "-FileName<FileModifyDate" "-FileName<DateCreated" "-FileName<CreateDate" -d "/mnt/user/Photography-Archive/exifdata/VIDEO/%Y/%m/%d/%Y-%m-%d-%H-%M-%S-0%%c.%%le" -r -v "/mnt/user/Photography-Archive/ToProcess/" -ext mov -ext MOV -ext mp4 -ext MP4

# MOVES ALL Pictures
exiftool -progress "-FileName<FileModifyDate" "-FileName<DateCreated" "-FileName<CreateDate" -d "/mnt/user/Photography-Archive/exifdata/%Y/%m/%d/%Y-%m-%d-%H-%M-%S-0%%c.%%le" -r -v "/mnt/user/Photography-Archive/ToProcess/"

The first line moves all movies into the Base Directory under "VIDEO"
The second moves pictures into "Base Directory/YEAR/"

I've run it on directories that I know include no XMP files and it works fine. but now I need to start on the RAW Directories.
Q: Will it keep XMP files with the Orginal? I have read the XMP Page, but couldn't figure out how to make the XMP's move with the Originals. I am not confident that Lightroom puts "%Y/%m/%d/%Y-%m-%d-%H-%M-%S-0%%c.%%le" info into the XMP's. If I knew it did, it wouldn't be a problem.

What do people do here?

Is there a way to use ExifTool to check for duplicates? maybe adding a MD5 Hash as a keyword? and checking to see if that exist?

I did find
exiftool FILE -imageuniqueid=`exiftool FILE -all= -o - | md5`

As a means to hash images, I'll give that a shot, anyone using it to actually find duplicates?

Thanks!

Quote from: wesley on April 21, 2020, 04:42:50 PM
Q: Will it keep XMP files with the Orginal? I have read the XMP Page,

No, exiftool only operates on one file at a time and doesn't match one file to another.

The basic idea is to use the -TagsFromFile option (https://exiftool.org/exiftool_pod.html#tagsFromFile-SRCFILE-or-FMT) in combination with the -FileOrder option (https://exiftool.org/exiftool_pod.html#fileOrder-NUM---TAG) to make sure one file is processed before the other.

To rename based upon the data in the XMP file, you would do something like
exiftool -fileorder Filename -TagsFromFile %d%f.xmp "-FileName<DateCreated" "-FileName<CreateDate" -d "/mnt/user/Photography-Archive/exifdata/%Y/%m/%d/%Y-%m-%d-%H-%M-%S-0%%c.%%le" <FileOrDir>

To do the reverse, you would need to specify the RAW extension.  Here's an example using NEF
exiftool -fileorder -Filename -TagsFromFile %d%f.NEF"-FileName<FileModifyDate" "-FileName<DateCreated" "-FileName<CreateDate" -d "/mnt/user/Photography-Archive/exifdata/%Y/%m/%d/%Y-%m-%d-%H-%M-%S-0%%c.%%le" <FileOrDir>

In the first example, exiftool makes sure that the files are in alphabetical order based upon filename (-fileorder Filename).  It then takes the data from the associated XMP file to do the renaming.  I've removed the FileModifyDate rename part because it's unlikely that the XMP file will have the same timestamp as the RAW file.  Because the only file extension that exiftool will processes that comes after .xmp is .zip, this means that the command will rename all other files with the same filename before it renames the XMP files.  Obviously, if you have a XMP file matching a .zip file, it will fail at that point.

In the second example, in order to process the XMP files first before the RAW files, the file order is reversed, indicated by the dash in front of filename (-fileorder -Filename).

I leave it up to you to test (use Testname instead of Filename) to see which is the correct option for you.

QuoteIs there a way to use ExifTool to check for duplicates?

Exiftool has no ability to check for duplicates.  There are several free tools out available out on the web.  For example, Dupguru (https://dupeguru.voltaicideas.net/) is cross platform and has an option for image searches.

Quotemaybe adding a MD5 Hash as a keyword?

The biggest problem with this is that if there is even one byte difference in the metadata, you have a completely different hash.  Additionally, the same picture can have the exact same graphical data, but be binarily different.  An example of this is a progressive encoded jpg vs. a standard encoded jpg.  Or a PNG with a compression level of 0 (no compression) vs level 9 (best compression).

There are a few posts on this forum about dealing with an MD5 hash.