ExifTool Forum

ExifTool => Newbies => Topic started by: ceej on February 07, 2023, 09:48:00 PM

Title: -fileOrder and -r : are files sorted within or across directories?
Post by: ceej on February 07, 2023, 09:48:00 PM
In the -fileOrder description this appears last: Note that files are sorted across directory boundaries if multiple input directories are specified.

1. is the FileName the default fileOrder?
2. is "multiple input directories" the same as "-r"?

That is, if I have directory structure
  1980s
    1981
      01, 02, 03, ...
    1982
      01, 02, 03, ...
    etc..
then -
3. what would be the processing difference between these two commands:
a. exiftool -XMP:title<filename -r 1980s/.
b. exiftool -XMP:title<filename 1980s/1981/01/. 1980s/1982/04/. 1980s/1987/06/.

In (a) would files be processed in filename order within each lowest directory?
What about in (b)?

4. what if each command above included: -fileOrder DateTimeOriginal
Same two questions.
In (b), would all files be extracted and sorted by DateTimeOriginal without regard to the directory structure, then processed?

Please excuse if commands are poorly formed / incorrect.
I hope that you can decipher the question I am trying to ask and the understanding I am trying to get.
Title: Re: -fileOrder and -r : are files sorted within or across directories?
Post by: StarGeek on February 08, 2023, 01:23:07 AM
Quote from: ceej on February 07, 2023, 09:48:00 PM1. is the FileName the default fileOrder?

The default order is the order in which the file system passes the list back to exiftool. Exiftool will not sort the files without the -FileOrder option (https://exiftool.org/exiftool_pod.html#fileOrder-NUM---TAG). In many cases, this is likely to be alphabetical by directory but, depending upon the file system, it may not be.  For example, I use Stablebit Drive pool to group several drives under a single drive name.  The file will never be in alphabetical order because the drive pool passes back the lists based upon the underlying drive.  So if I have drives D:, E:, and F: in a pool, the drive pool will pass back all the D: files, all the E: files, and all the F: files in groups.

Quote2. is "multiple input directories" the same as "-r"?

3. what would be the processing difference between these two commands:

Yes, the would be functionally the same.  Exiftool will order all the files in all the directories specified, recursing if that option is set.

As an example, I have 6 files, each in a separate directory.  I randomize the DateTimeOriginal value for each file, then use -fileorder to sort by that tag.  First run uses -r and the second lists each directory separately.  The output order is the same
C:\>exiftool -P -overwrite_original -d %s -all=  "-DateTimeOriginal<${filename;$_=int(rand(1636531200))}" -r Y:\!temp\dd
    7 directories scanned
    6 image files updated

C:\>exiftool -G1 -a -s -fileorder DateTimeOriginal -DateTimeOriginal -r Y:\!temp\dd
======== Y:/!temp/dd/a/a.jpg
[ExifIFD]      DateTimeOriginal                : 1978:03:31 16:01:33
======== Y:/!temp/dd/f/f.JPG
[ExifIFD]      DateTimeOriginal                : 1990:07:02 11:24:27
======== Y:/!temp/dd/d/d.jpg
[ExifIFD]      DateTimeOriginal                : 1991:03:31 18:21:09
======== Y:/!temp/dd/c/c.jpg
[ExifIFD]      DateTimeOriginal                : 1994:07:08 02:19:13
======== Y:/!temp/dd/b/b.jpg
[ExifIFD]      DateTimeOriginal                : 2004:03:31 06:11:04
======== Y:/!temp/dd/e/e.jpg
[ExifIFD]      DateTimeOriginal                : 2014:08:12 07:58:32
    7 directories scanned
    6 image files read

C:\>exiftool -G1 -a -s -fileorder DateTimeOriginal -DateTimeOriginal -r Y:\!temp\dd\f Y:\!temp\dd\a Y:\!temp\dd\e Y:\!temp\dd\b Y:\!temp\dd\d Y:\!temp\dd\c
======== Y:/!temp/dd/a/a.jpg
[ExifIFD]      DateTimeOriginal                : 1978:03:31 16:01:33
======== Y:/!temp/dd/f/f.JPG
[ExifIFD]      DateTimeOriginal                : 1990:07:02 11:24:27
======== Y:/!temp/dd/d/d.jpg
[ExifIFD]      DateTimeOriginal                : 1991:03:31 18:21:09
======== Y:/!temp/dd/c/c.jpg
[ExifIFD]      DateTimeOriginal                : 1994:07:08 02:19:13
======== Y:/!temp/dd/b/b.jpg
[ExifIFD]      DateTimeOriginal                : 2004:03:31 06:11:04
======== Y:/!temp/dd/e/e.jpg
[ExifIFD]      DateTimeOriginal                : 2014:08:12 07:58:32
    6 directories scanned
    6 image files read

Without the file order, they would be processed according to how the files were passed back to exiftool.  Using the same files without -fileorder, you can see how they differently, especially with my drive pool
C:\>exiftool -G1 -a -s -DateTimeOriginal -r Y:\!temp\dd\f Y:\!temp\dd\a Y:\!temp\dd\e Y:\!temp\dd\b Y:\!temp\dd\d Y:\!temp\dd\c
======== Y:/!temp/dd/f/f.JPG
[ExifIFD]       DateTimeOriginal                : 1990:07:02 11:24:27
======== Y:/!temp/dd/a/a.jpg
[ExifIFD]       DateTimeOriginal                : 1978:03:31 16:01:33
======== Y:/!temp/dd/e/e.jpg
[ExifIFD]       DateTimeOriginal                : 2014:08:12 07:58:32
======== Y:/!temp/dd/b/b.jpg
[ExifIFD]       DateTimeOriginal                : 2004:03:31 06:11:04
======== Y:/!temp/dd/d/d.jpg
[ExifIFD]       DateTimeOriginal                : 1991:03:31 18:21:09
======== Y:/!temp/dd/c/c.jpg
[ExifIFD]       DateTimeOriginal                : 1994:07:08 02:19:13
    6 directories scanned
    6 image files read

After making a copy of the files onto an non-pooled drive, running the same commands are as follows
C:\Programs\My_Stuff>exiftool -G1 -a -s -DateTimeOriginal -r D:\!test
======== D:/!test/dd/a/a.jpg
[ExifIFD]       DateTimeOriginal                : 1978:03:31 16:01:33
======== D:/!test/dd/b/b.jpg
[ExifIFD]       DateTimeOriginal                : 2004:03:31 06:11:04
======== D:/!test/dd/c/c.jpg
[ExifIFD]       DateTimeOriginal                : 1994:07:08 02:19:13
======== D:/!test/dd/d/d.jpg
[ExifIFD]       DateTimeOriginal                : 1991:03:31 18:21:09
======== D:/!test/dd/e/e.jpg
[ExifIFD]       DateTimeOriginal                : 2014:08:12 07:58:32
======== D:/!test/dd/f/f.JPG
[ExifIFD]       DateTimeOriginal                : 1990:07:02 11:24:27
    8 directories scanned
    6 image files read

C:\Programs\My_Stuff>exiftool -G1 -a -s -DateTimeOriginal -r D:\!test\dd\f D:\!test\dd\a D:\!test\dd\e D:\!test\dd\b D:\!test\dd\d D:\!test\dd\c
======== D:/!test/dd/f/f.JPG
[ExifIFD]       DateTimeOriginal                : 1990:07:02 11:24:27
======== D:/!test/dd/a/a.jpg
[ExifIFD]       DateTimeOriginal                : 1978:03:31 16:01:33
======== D:/!test/dd/e/e.jpg
[ExifIFD]       DateTimeOriginal                : 2014:08:12 07:58:32
======== D:/!test/dd/b/b.jpg
[ExifIFD]       DateTimeOriginal                : 2004:03:31 06:11:04
======== D:/!test/dd/d/d.jpg
[ExifIFD]       DateTimeOriginal                : 1991:03:31 18:21:09
======== D:/!test/dd/c/c.jpg
[ExifIFD]       DateTimeOriginal                : 1994:07:08 02:19:13
    6 directories scanned
    6 image files read

Notice how with recurse, the files are passed back in directory order, but the second test passes them back in the order of directories specified.

Finally, remember that the order on the command line will be different from the desktop when it comes to numbers.  The desktop will do a "smart ordering" of numbers while the command line is reading them as strings.

command line
C:\>exiftool -G1 -a -s -DateTimeOriginal Y:\!temp\dd
======== Y:/!temp/dd/011.jpg
[ExifIFD]       DateTimeOriginal                : 1991:03:31 18:21:09
======== Y:/!temp/dd/02.jpg
[ExifIFD]       DateTimeOriginal                : 2004:03:31 06:11:04
======== Y:/!temp/dd/1.jpg
[ExifIFD]       DateTimeOriginal                : 1978:03:31 16:01:33
======== Y:/!temp/dd/10.jpg
[ExifIFD]       DateTimeOriginal                : 1994:07:08 02:19:13
======== Y:/!temp/dd/12.jpg
[ExifIFD]       DateTimeOriginal                : 2014:08:12 07:58:32
======== Y:/!temp/dd/5.JPG
[ExifIFD]       DateTimeOriginal                : 1990:07:02 11:24:27
    1 directories scanned
    6 image files read

Desktop
Title: Re: -fileOrder and -r : are files sorted within or across directories?
Post by: ceej on February 08, 2023, 01:55:06 PM
Thank You.
Let me try to repeat to be certain I understand. At a high level, the process is:
1. create a list of files based upon FILE - one or more source file names, directory names, or - for the standard input
2. as each fileinfo is returned by the OS, apply any -if tests and filters (e.g. -ext) to decide if the file goes into the list
3. if -r is present, continue reading files from all subdirectories, recursively
4. when all files-to-be-processed are in the list, sort the list by -fileOrder if present
5. go through list and do all other command options on each file

And if no -fileOrder is present, the files in the list are in whatever order the OS provided them, and in whatever order the directories were given on the command line or encountered via the -r

Finally, regarding smart-ordering of strings-with-numbers, that applies to "alphabetical by directory" names as well, correct?

I really appreciate your help and that of others on this forum.
For me, at least, having a basic understanding of the process, as well as a clear understanding of the meaning/definition of options, makes it easier for me to use the program, even if I never have to use/depend upon any of those particulars!

  -ceej

PS
Hmmm..
It looks to me as if #2 is not done until after #3 or #4.
When processing ~10,000 files w/no sort, -if testing Title, then printing directory, filename, title, I see a long pause, then bursts of output, separated by pauses of short to long duration. 
Title: Re: -fileOrder and -r : are files sorted within or across directories?
Post by: StarGeek on February 08, 2023, 03:49:13 PM
When the -if option (https://exiftool.org/exiftool_pod.html#if-NUM-EXPR) has to read the embedded data, like the Title tag, then the file is already on the list for processing.  The -if option can have a number following it (see the -fast option (https://exiftool.org/exiftool_pod.html#fast-NUM) for the meanings of the numbers) which might speed up the processing depending upon the data being compared. For example, if the -if option is comparing only file system tags, such as the Filename or FileModifyDate, then you could add -if3 and exiftool won't read the file's embedded data, only the system tags.

You probably want to look at the Under the Hood page (https://exiftool.org/under.html) for a more in depth look at how exiftool works.
Title: Re: -fileOrder and -r : are files sorted within or across directories?
Post by: ceej on February 09, 2023, 12:08:56 AM
Thanks again!