A number of failing cases using the -@ operator -- LONG

[calberga]

I hope that I'm running into a real problem, and not just failing to understand the program.

I'm trying to tag (IPTC) a large number of files.  (I posted a question recently and found that I had missed the "do not quote" warning on -@ files.)  I am currently have in serious problems with a large -@ file, with three types of error (at last count).

1)  I've run into this before, but there is a odd inconsistency.  A number of my file and directories have ANSI characters, mostly, if not all, accented letters.  They are the names of locations in various countries.  For these files I get "File not found" errors.  One of the strange things is that the same directory/filenames HAVE worked in a previous -@ file in which I set the ReferenceNumber tag. 

2) In at least one case exif seems to insert a new-line within a line from the .args file, resulting in a value to be set being interpreted as a filename.

3) In the third case, text seems to be added spontaneously to certain filenames.

A workaround for the first problem, which I have used in another case, would be to temporarily rename the offending files.  I would rather not have to do this, but have a couple of ideas that I can try, either within exiftool, or with an separate CLISP program.  I am at a total loss as to the other two cases.

I apologize for posting such a long screed, but I felt the the examples needed to be presented in as complete a state as possible.

No files have been renamed between the creation of the .args file and the following examples, and the directory and filenames are taken directly from output files created by exiftool, such as:

exiftool -T -r -directory -filename -source -ReferenceNumber "D:/From-Netbook/Photographs/Boxed Slides" > "D://From-Netbook/Photographs/Sx1.txt"

The exiftool command

exiftool -@ "D:/From-Netbook/Photographs/All-tagged.args" -common_args -L -P -v0 -overwrite_original > D:/From-Netbook/Photographs/Sourceset.txt

Following are triplets, the entry (or in one case, entries) in All-tagged.args followed by the error message displayed in the command window, and then the corresponding item from Sourceset.txt  Note that the text from the Command window is wrapped at the width of the window (100 characters), ignoring spaces etc.,   I have left it as-is.

Note last character of "Perú"

-Source=DP(C)-01623
D:/From-Netbook/Photographs/Digital Photos (C)/2005-09-21 to 2005-10-13 Perú (C)/2005-10-10 Manu Tented Camp to Manu Wildlife Center (C)/2005-10-10 042 Treetop Tower (C).jpg
-execute

Code Select Expand

Error: File not found - D:/From-Netbook/Photographs/Digital Photos (C)/2005-09-21 to 2005-10-13 Per£
(C)/2005-10-10 Manu Tented Camp to Manu Wildlife Center (C)/2005-10-10 042 Treetop Tower (C).jpg

Code Select Expand

======== D:/From-Netbook/Photographs/Digital Photos (C)/2005-09-21 to 2005-10-13 Perœ (C)/2005-10-10 Manu Tented Camp to Manu Wildlife Center (C)/2005-10-10 042 Treetop Tower (C).jpg
    0 image files updated
    1 files weren't updated due to errors

These sort of errors continue, with the snipped examples, being shown (again, the input, the error in the command window, the error in the output file).

Perú : Per£ : Perœ
Chalon sur Sâone : Chalon sur SŸone : Chalon sur Sƒone
Marsala & Érice : Marsala & #rice : Marsala & #rice
Iglésias & Gúspini : Igl#sias & G£spini : Igl#sias & Gœspini
Sa Ena 'e Thomes & Ispinigòla : Sa Ena 'e Thomes & Ispinig#la : Sa Ena 'e Thomes & Ispinig#la
Île Ste.-Marie : Ile Ste.-Marie : Ile Ste.-Marie

Next we have the case where extra characters seem to be inserted into the text from the .args file.  The result is that the parsing fails to recognize the next tag assignment and instead treats the value as a file name.  The four items in the .args file that threw the following errors are:

-Source=VS-01
D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01.jpg
-execute
-Source=VS-02
D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01b.jpg
-execute
-Source=VS-03
D:/From-Netbook/Photographs/Large Prints/2011 Formal.jpg
-execute
-Source=VS-04
D:/From-Netbook/Photographs/Misc Pictures/16259.jpg
-execute

The error messages in the command window are:

Code Select Expand

Error: File not found - D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01/2116
9
Error: File not found - VS-02
Error: File not found - D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01b/211
70
Error: File not found - VS-03
Error: File not found - D:/From-Netbook/Photographs/Large Prints/2011 Formal/51339
Error: File not found - VS-04
Error: File not found - D:/From-Netbook/Photographs/Misc Pictures/16259/52031

The items in the .txt file are:

Code Select Expand

======== D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01/21169
======== VS-02
======== D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1947 Misc - 01b/21170
======== VS-03
======== D:/From-Netbook/Photographs/Large Prints/2011 Formal/51339
======== VS-04
======== D:/From-Netbook/Photographs/Misc Pictures/16259/52031

At the end of this section (there are 23 such entries) is a final "file not found" for a "filename" that seems to come out of nowhere.  I haven't figured out how to even display this!

There are a lot of file with names of the form "nn-nn-nn (D).jpg".  In many (possibly all, I haven't fully check) of these the space before the "(D)" has been replaced by a hyphen.

-Source=31-02-01
D:/From-Netbook/Photographs/Boxed Slides/1975-1979 (D)/31-02-01 (D).jpg
-execute

Code Select Expand

Error: File not found - D:/From-Netbook/Photographs/Boxed Slides/1975-1979-(D)/31-02-01-(D).jpg

Code Select Expand

======== D:/From-Netbook/Photographs/Boxed Slides/1975-1979-(D)/31-02-01-(D).jpg
    0 image files updated
    1 files weren't updated due to errors

Another set of file, as in the following example, don't appear to have any error, the file names are identical as far as I can see in the .args file and in both sets of output from exiftool.  One might suspect the tilde (~), but that character occurs is very many file names that give no problems.

-Source=33mm-neg-D-06
D:/From-Netbook/Photographs/Boxed Photographs Sorted/1948~49 Elaine - RT Store - Wedding in RT/1948~49 RT D-06.jpg
-execute

Code Select Expand

Error: File not found - D:/From-Netbook/Photographs/Boxed Photographs Sorted/1948~49 Elaine - RT Sto
re - Wedding in RT/1948~49 RT D-06.jpg

Code Select Expand

======== D:/From-Netbook/Photographs/Boxed Photographs Sorted/1948~49 Elaine - RT Store - Wedding in RT/1948~49 RT D-06.jpg
    0 image files updated
    1 files weren't updated due to errors


Ah a fourth set of errors: these "minor" errors keep being reported.  Is there a way to "clean" the files?

Code Select Expand

Warning: [minor] Ignored empty rdf:Alt list for dc:description - D:/From-Netbook/Photographs/Digital
Photos (S)/2012-01~02 Australia (S)/2012-01-24 Australian Open (S)/DSC_0139.jpg

Code Select Expand

Warning: [minor] Ignored empty rdf:Bag list for dc:subject - D:/From-Netbook/Photographs/Recent Fami
ly/Katie & Emma/2011/2011-01-29 Winter Camping 00.jpg

Code Select Expand

Warning: [minor] Ignored empty rdf:Bag list for dc:subject - D:/From-Netbook/Photographs/Recent Fami
ly/Katie & Emma/2011/2011-07 Caribbean-13.jpg

[Phil Harvey]

1) Unfortunately your first class of error (special characters in the file/directory names) is a problem in Windows with no easy solution because the standard C libraries in Windows don't support Unicode characters in file names.

2) I have no idea about the problem where part of -Source=VS-30 is somehow taken as a file name.  Could you post the exact .arg file you used?  If I can reproduce the problem I can figure it out.

3) This seems similar to 2 except that spaces are replaced by hyphens instead of equals signs being replaced by spaces.  Very odd.  Have you tried a different version of ExifTool just in case the current installation is corrupt somehow?

4) The minor XMP errors may be fixed by simply writing any XMP.  If not, rebuilding the XMP (with -xmp:all= -tagsfromfile @ -xmp:all) will fix them.

- Phil

[calberga]

Thanks for the reply.  The .args file I'm using has about 54500 entries, and would be a major pain.  I'm going to try snippets of it to see if the same errors manifest.  If they do I'll send them on, if they don't, then I'll just do things in stages instead of trying a "start it and go read a book" approach.

[calberga]

A PS:

One very confusing bit, however, is that I have another .args file, which differs from the one in question only in that it is setting the ReferenceNumber tag, which contains all the same filenames, but which worked without any error messages!  I'm going to isolate five or six examples from it, run them again, and see what happens.

[Phil Harvey]

Note that problem number 1 may work sometimes depending on the exact characters in the file names and your windows environment settings.

- Phil

[calberga]

Well, that was interesting.  Here is the command and the test .args file, each entry one which failed in the total file.

exiftool -@ "D:/From-Netbook/Photographs/Test1.args" -common_args -L -P -v0 -overwrite_original > D:/From-Netbook/Photographs/Test1trace.txt

Code Select Expand

-ReferenceNumber=22508
D:/From-Netbook/Photographs/Digital Photos (C)/2005-09-21 to 2005-10-13 Perú (C)/2005-09-29 Cuzco (C)/2005-09-29 023 Public Market (C).jpg
-execute
-ReferenceNumber=21171
D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1948 MISC-020.jpg
-execute
-ReferenceNumber=9726
D:/From-Netbook/Photographs/Boxed Slides/1975-1979 (D)/31-02-01 (D).jpg
-execute
-ReferenceNumber=2457
D:/From-Netbook/Photographs/Boxed Photographs Sorted/1948~49 Elaine - RT Store - Wedding in RT/1948~49 RT D P 06.jpg
-execute

There were NO errors, and the output file is:

Code Select Expand

======== D:/From-Netbook/Photographs/Digital Photos (C)/2005-09-21 to 2005-10-13 Perú (C)/2005-09-29 Cuzco (C)/2005-09-29 023 Public Market (C).jpg
    1 image files updated
======== D:/From-Netbook/Photographs/Cairo Class of 1952 Pictures/1948 MISC-020.jpg
    1 image files updated
======== D:/From-Netbook/Photographs/Boxed Slides/1975-1979 (D)/31-02-01 (D).jpg
    1 image files updated
======== D:/From-Netbook/Photographs/Boxed Photographs Sorted/1948~49 Elaine - RT Store - Wedding in RT/1948~49 RT D P 06.jpg
    1 image files updated

Throws hands in the air.

Could there be some effect of having such long .args files?  Of course the one setting the ReferenceNumber tags is just as long and seems to work on all of the cases!

Next?  I guess I have to re-write my LISP program which renames all the files to $nnnnn.jpg, creates an .args file, invokes exiftools and then renames the files back to the original names.  Bah!

[Phil Harvey]

How are you creating the .args file?  It sounds like you are using different character encodings for different versions of this file.  If you are using an editor to change this file, check your editor settings.

- Phil

[calberga]

I have had a few wrong character encoding, but I think that I've eliminated them from the example I've given.  In many cases I have been using exiftool to write a tab-delimited file which I import into Paradox, rearrange fields and add sequence numbers, then export.  The encoding errors have occurred when I accidentally exported using the OEM option rather than ANSI.  Once I get the (correct) export I use a simple text editor, EditPad (the "classic", not the souped up version) to convert to the .args file.  I haven't found another text editor that doesn't muck up the encodings someway or other, or add line breaks.

I have used LISP to create some of the .args files, but I also did file renaming in those cases.

I'm about to try another mass tag setting, like the one that worked previously with -ReferenceNumber, and will report back, or else start coding somemore LISP!

[calberga]

PS

I don't know if it is relevant, but I have my keyboard set to "United States-International".