ExifTool Forum

ExifTool => Developers => Topic started by: Joanna Carter on August 25, 2018, 04:50:54 AM

Title: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 25, 2018, 04:50:54 AM
Hi there

I am "sorting" a source directory hierarchy into a destination date-ordered hierarchy but, as can happen, the source hierarchy contains duplicate files. Consequently, when these files are copied, I get an (expected) error to the console, but the command continues anyway.

Error: './f:32.jpg' already exists - /Volumes/Joanna C/DCIM/Cours/Exposition/f:32.jpg

Is there any way to inhibit this console message?
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 25, 2018, 07:28:30 AM
Error messages can not be silenced.  You need to redirect the STDOUT and STDERR in your app to capture these.

Take a look at my C++ wrapper (http://owl.phy.queensu.ca/~phil/cpp_exiftool/) for an example of how this is done in C++.

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 25, 2018, 01:33:32 PM
Aaarrgghhhhh!!!!

I haven't touched C++ in anger for over 25 years  ;D

Nevertheless, I did find the Swift equivalent, thanks to your code.

The only problem I have now is trying to get STDOUT and STDERR to respond after I've finished intercepting them.

Although it is quite nice to have an empty console display  ::)
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 26, 2018, 01:14:52 PM
Rather than suppress the error messages, is there a switch that will automatically add a version number to the file name?

I tried /Users/joannacarter/Pictures/%Y/%m/%d/%%f%%c in exiftool -r -ext JPG -o . '-Directory<DateTimeOriginal' -d /Users/joannacarter/Pictures/%Y/%m/%d /Volumes/Joanna\ C/DCIM/ but that really messed things up  :o

??? ???
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 26, 2018, 09:07:07 PM
The %c is what you want.  You add it to the format string when setting the FileName.

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: StarGeek on August 26, 2018, 09:50:47 PM
To be clear, you can't use it with -Directory to work properly, you have to use -Filename and set the full filename, including the extension .%e or %E
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 04:25:24 AM
Hmmm. I still seem to be missing something. I am still getting errors like:

Error: './JNA_0001.jpg' already exists - /Volumes/Joanna C/DCIM/Projections/2018/juin/JNA_0001.jpg

Here is the command line I am creating:

exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . -FileName<DateTimeOriginal -d /Users/joannacarter/Pictures/%Y/%m/%d/%%f%%-c.%%e /Volumes/Joanna C/DCIM/
And there is only ever one file with the original name

Might this have something to do with this from the -w docs?
QuoteThis same FMT syntax is used with the -o and -TagsFromFile options, although %c is only valid for output file names
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 07:34:42 AM
You need to add quotes around "-FileName<DateTimeOriginal".

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 09:16:36 AM
???

Unless I start playing around with %c, everything else works fine, even without the quotes. Don't forget this is not being typed into the terminal.

exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . -FileName<DateTimeOriginal -d /Users/joannacarter/Pictures/%Y/%m/%d/%%f.%%le /Volumes/Joanna C/DCIM/ works fine, apart from kicking up:

Error: './f:32.jpg' already exists - /Volumes/Joanna C/DCIM/Cours/Exposition/f:32.jpg

However, if I change the command in the hope of avoiding the "already exists" errors by appending a "copy number":

exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . -FileName<DateTimeOriginal -d /Users/joannacarter/Pictures/%Y/%m/%d/%%f%%-c.%%le /Volumes/Joanna C/DCIM/

... nothing changes. The file names are all exactly the same as without the %%-c and the errors are still produced.
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 09:30:51 AM
Try it from the command line.  I did and it worked as expected.  The error is somehow in your running of the command from Swift.

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 10:13:12 AM
Sorry Phil but I'm not getting the same results here.

Copying the command from my code to Terminal, surrounding -FileName<DateTimeOriginal with single quotes and escaping the space in the source directory gives me:

exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . '-FileName<DateTimeOriginal' -d /Users/joannacarter/Pictures/%Y/%m/%d/%%f%%-c.%%le /Volumes/Joanna\ C/DCIM/

Running that gives me exactly the same "already exists" errors and lack of copy numbers as running it from my app  :-[

I must clarify that, apart from that, the command does exactly what I want it to do, whether from the Terminal or from my app.

When you are trying it, does your source hierarchy contain duplicate files in different sub-directories?
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 10:37:04 AM
> exiftool a.jpg -datetimeoriginal                                                       
Date/Time Original              : 2018:08:27 07:31:40
> exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . '-FileName<DateTimeOriginal' -d tmp/Pictures/%Y/%m/%d/%%f%%-c.%%le -v a.jpg
======== a.jpg
Setting new values from a.jpg
'a.jpg' --> 'tmp/Pictures/2018/08/27/a.jpg'
Created directory tmp/Pictures
Created directory tmp/Pictures/2018
Created directory tmp/Pictures/2018/08
Created directory tmp/Pictures/2018/08/27
Rewriting a.jpg...
Nothing changed in a.jpg
> exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . '-FileName<DateTimeOriginal' -d tmp/Pictures/%Y/%m/%d/%%f%%-c.%%le -v a.jpg
======== a.jpg
Setting new values from a.jpg
'a.jpg' --> 'tmp/Pictures/2018/08/27/a-1.jpg'
Rewriting a.jpg...
Nothing changed in a.jpg


Note that the second time I ran the command it added "-1" to the file name.

Can you replicate this?
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 11:46:09 AM
If I cd to the source directory, and try your first command

/Users/joannacarter/Développement/PhotoBrowser/PhotoBrowser/Resources/exif/exiftool a.jpg -datetimeoriginal

I get absolutely nothing, not even an error.

If I then copy your "full" command and simply change the destination directory:

/Users/joannacarter/Développement/PhotoBrowser/PhotoBrowser/Resources/exif/exiftool -preserve -extension jpg -recurse -ignoreMinorErrors -q -q -extension jpg -out . '-FileName<DateTimeOriginal' -d /Users/joannacarter/Pictures/destination/%Y/%m/%d/%%f%%-c.%%le -v a.jpg

... I get the following output:


======== a.jpg
Setting new values from a.jpg
Error: './a.jpg' already exists - a.jpg


Directory structure is:

/Users/
-joannacarter/
--Pictures/
---source/ (cd to here before running command
---destination/

Anything to do with not having ExifTool installed in its default location, only in my app's project?
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 11:48:05 AM
Quote from: Joanna Carter on August 27, 2018, 11:46:09 AM
If I cd to the source directory, and try your first command
I get absolutely nothing, not even an error.

OK.  That's the problem.  The %c only applies to the DateTimeOriginal value.  If there is no DateTimeOriginal, then  '-FileName<DateTimeOriginal' does nothing and the %c in the date/time string isn't used.  What did you intend to happen to files without a DateTimeOriginal?

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 12:07:23 PM
I suppose I expected all files to have a DateTimeOriginal  :-\
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 01:25:02 PM
OK. Now I get it  ::)

When I try "the" command on files that do have a DateTimeOriginal, everything is wonderful - the world is a lovely place  8)

So, obviously (?), if a file doesn't have DateTimeOriginal, it doesn't get processed at all and I get that file exists error.

The question is, having examined the EXIF in a few of those files and found they have been essentially stripped bare, is there anything I can do to conditionally fall back to the Finder date created?

Or do I just say that only files with DateTimeOriginal can be processed, or can I do a single command to detect that and set it to the Finder date/time?

Questions, questions, questions - why do I always end up doing stuff that  pushes the boundaries?  :P ::)

Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 01:47:39 PM
And the answer is, I can use the FileCreateDate tag!!!

This seems to be the same as DateTimeOriginal for RAW files, which is the primary target of this command for what I want to do.

Finally, I think we can mark this issue as solved. Many thanks for all your help.
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 02:01:55 PM
Great.  But missing dates are a common problem, and FileCreateDate isn't always reliable, so I would suggest using it only as a fallback.  ie) doing this:

exiftool ... '-FileName<FileCreateDate' '-FileName<DateTimeOriginal' ...

Then DateTimeOriginal will get used if it exists, otherwise it will use FileCreateDate.

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 04:51:12 PM
Interesting. So, bearing in mind the "arrow" between the date and the "directory/fileName" is from right to left, therefore the "priority" flows from right to left?

I think I'm finally getting the hang of this  8)
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Joanna Carter on August 27, 2018, 05:50:57 PM
So, finally, here's how to create that command in my Swift wrapper:


  public static func copyAndOrganiseFilesByDate(from sourceURL: URL, to destinationURL: URL) throws
  {
    let arguments: [Argument] = [.preserveFileModificationDateTime,
                                 .recurse,
                                 .ignoreMinorErrors,
                                 .quietWarnings,
                                 .extension("jpg", exclude: false),
                                 .outputFlag,
                                 .currentDirectory,
                                 .fileNameFromTag(System.fileCreateDate),
                                 .fileNameFromTag(Exif.dateTimeOriginal),
                                 .dateFormatFlag,
                                 .fileNameWithFormat(url: destinationURL, dateFormatSpecifiers: [.year, .month, .day], fileNameFormatSpecifiers: [.fileName, .copyNumber, .lowercaseExtension]),
                                 .directory(sourceURL)]
   
    do
    {
      try execute(with: arguments)
    }
    catch
    {
      throw Error.couldNotCopyAndOrganiseFiles
    }
  }


With seems like a lot of typing but, with code completion, it's barely more than the command line  ;D
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on August 27, 2018, 07:09:54 PM
Quote from: Joanna Carter on August 27, 2018, 04:51:12 PM
Interesting. So, bearing in mind the "arrow" between the date and the "directory/fileName" is from right to left, therefore the "priority" flows from right to left?

Not quite.  In general, arguments are evaluated left to right (https://exiftool.org/faq.html#Q22), which does give the right argument priority, but the "arrow" may go either way: "-FileName<FileCreateDate" is the same as "-FileCreateDate>FileName"

- Phil
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: lnjustin on February 11, 2023, 03:41:36 PM
When I get an 'already exists' error, is there a way to move the duplicate file to a specified directory?
Title: Re: Avoid errors when copying files that already exist in the destination
Post by: Phil Harvey on February 11, 2023, 04:41:39 PM
No.  The thing to do is add some form of "%c" when writing the file name so that the duplicate gets a number added to it (I like "%-c" personally, to put a "-" before the number).

Then all the files will get written and you can figure out what to do with the duplicates afterwards.

- Phil