non US characters (iso latin 1) recursive tag writing issue

[artofit]

OS: Win 7 64bits
Partition:ntfs
ExifTool 9.92

Command:

Code Select Expand

exiftool -P -r -ext jpg -ext jpeg -overwrite_original -charset latin1 "-Comment<${filename}-${directory}" f:/art

Issues:
When either some sub-directory or image file name contains a non US char - such as éçàñ - exiftool can't process it and gives a warning on the console output:

Error renaming temporary file to <filename with full path>

How to fix that?

Thanks

[Phil Harvey]

Darn.  Looks like I have to drag out the PC yet again.   I hope I can reproduce this with Win XP.

Give me a day or two to look into this.

- Phil

Edit: You can help me by telling me what are the permissions on the original file?  I can see this happening if it is read only because it seems that the Windows "DeleteFileW" won't delete a read-only file, while Perl "unlink" will (and the Windows routine is used if the file has any special characters in the name).  Also, does the temporary file still exist after exiftool exits?

[Phil Harvey]

I've tested this out.  My guess was correct.  This is an unfortunate difference between the Windows and Perl file i/o routines.  It affects just read-only files.

It would be a pain to patch this, so I think the thing to do is document this and call it a "feature".

- Phil

[Phil Harvey]

This is what I plan to add to the documentation:

WRITING READ-ONLY FILES
       In general, ExifTool may be used to write metadata to read-only files
       provided that the user has write permission in the directory.  However,
       there are two cases where file write permission is also required:

       1) When using the -overwrite_original_in_place option.

       2) On Windows if the file has Unicode characters in its name, and a)
       the -overwrite_original option is used, or b) the "_original" backup
       already exists.

- Phil

[artofit]

I'm not sure to follow.
I'm running as admin level
I put all images as non read-only

Which now feels odd as:
20131117_152455.JPG is not processed
but
éêèïî.jpg is well processed.

here's my console outpt:

F:\dbtest>C:\PortableApps\MM\ExifToolGUI\exiftool -P -r -ext jpg -ext jpeg -overwrite_original -charset latin1 "-
Comment<${filename}#${directory}" f:/dbtest/test
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_152455.JPG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_153514.JPEG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_162439.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/CZK_├║┼»├¢┼¥├í─ì─Å├®─ø├¡┼ê├│┼Ö┼í┼Ñ
.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/HU_őű.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/PL(check accent)_─à─ç─Ö┼é┼ä┼ø┼║┼╝.
jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RO_ăâîşș.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RU┬½ÐæÐÅÐêðÁÐÇÐéÐïÐâð©ð¥ð┐ÐÄÐëÐìÐè
ðÂÐîð╗ð║ð╣Ðçð│Ðäð┤Ðüð░ðÀÐàÐåð▓ð▒ð¢ð╝┬╗.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/pas autoris├® ├®criture.jpeg
Warning: Non-standard header for APP1 XMP segment - f:/dbtest/test/West/ES_áñ.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├®├¬├¿├»├«.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├╣┬Á├▒.Jpg
    3 directories scanned
   14 image files updated

F:\dbtest>dir *.j* /B /S /O-G
F:\dbtest\documents\err recovery\err_Color_Chart.jpg
F:\dbtest\mm_photo_codec\JPEG_Color_Chart.jpg
F:\dbtest\mm_photo_codec\jxr_treefrog.jxr
F:\dbtest\mm_photo_codec\JPEG2000_180157.jp2
F:\dbtest\test\20131117_153514.JPEG
F:\dbtest\test\20131117_162439.jpg
F:\dbtest\test\pas autorisé écriture.jpeg
F:\dbtest\test\éêèïî.jpg
F:\dbtest\test\ùµñ.Jpg
F:\dbtest\test\20131117_152455.JPG
F:\dbtest\test\East\HU_őű.jpg
F:\dbtest\test\East\NOK_åæø.jpg
F:\dbtest\test\East\PL(check accent)_ąćęłńśźż.jpg
F:\dbtest\test\East\RO_ăâîşș.jpg
F:\dbtest\test\East\RU«ёяшертыуиопющэъжьлкйчгфдсазхцвбнм».jpg
F:\dbtest\test\East\CZK_úůýžáčďéěíňóřšť.jpg
F:\dbtest\test\West\test.jpg
F:\dbtest\test\West\ES_áñ.jpg

F:\dbtest>

[Phil Harvey]

This is not the problem you originally posted, which was an "Error renaming temporary file" error.

- Phil

[artofit]

Thanks for noticing I was unprecise, and mistaken in the bug reproduction, so here again.

I run exiftool 9.92 under a win7 64bits multi-users OS with an user account having admin priviledges.

According to the fact that command terminal is explicitly runned with admin level or not the behaviour is the same for the following:

I put the read-only protection from folder "test" and all-subfolders.

Console output, being:

Code Select Expand

C:\Users\Angel>C:\PortableApps\MM\ExifToolGUI\exiftool -P -r -ext jpg -ext jpeg -overwrite_original -charset latin1 "-Comment<${filename}#${directory}" f:/dbtest/test
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_152455.JPG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_153514.JPEG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_162439.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/CZK_├║┼»├¢┼¥├í─ì─Å├®─ø├¡┼ê├│┼Ö┼í┼Ñ.jpg
Error renaming temporary file to f:/dbtest/test/East/CZK_├║┼»├¢┼¥├í─ì─Å├®─ø├¡┼ê├│┼Ö┼í┼Ñ.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/HU_őű.jpg
Error renaming temporary file to f:/dbtest/test/East/HU_őű.jpg
Error renaming temporary file to f:/dbtest/test/East/NOK_├Ñ├ª├©.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/PL(check accent)_─à─ç─Ö┼é┼ä┼ø┼║┼╝.jpg
Error renaming temporary file to f:/dbtest/test/East/PL(check accent)_─à─ç─Ö┼é┼ä┼ø┼║┼╝.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RO_ăâîşș.jpg
Error renaming temporary file to f:/dbtest/test/East/RO_ăâîşș.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RU┬½ÐæÐÅÐêðÁÐÇÐéÐïÐâð©ð¥ð┐ÐÄÐëÐìÐèðÂÐîð╗ð║ð╣
Ðçð│Ðäð┤Ðüð░ðÀÐàÐåð▓ð▒ð¢ð╝┬╗.jpg
Error renaming temporary file to f:/dbtest/test/East/RU┬½ÐæÐÅÐêðÁÐÇÐéÐïÐâð©ð¥ð┐ÐÄÐëÐìÐèðÂÐîð╗ð║ð╣Ðçð│Ðäð┤Ðüð░ðÀÐàÐåð▓ð▒ð¢ð╝
┬╗.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/pas autoris├® ├®criture.jpeg
Error renaming temporary file to f:/dbtest/test/pas autoris├® ├®criture.jpeg
Warning: Non-standard header for APP1 XMP segment - f:/dbtest/test/West/ES_áñ.jpg
Error renaming temporary file to f:/dbtest/test/West/ES_áñ.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├®├¬├¿├»├«.jpg
Error renaming temporary file to f:/dbtest/test/├®├¬├¿├»├«.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├╣┬Á├▒.Jpg
Error renaming temporary file to f:/dbtest/test/├╣┬Á├▒.Jpg
    3 directories scanned
    4 image files updated
   10 files weren't updated due to errors

C:\Users\Angel>


If I remove the read-only protection on the folder "test" and its subfolders, console output is:

Code Select Expand

F:\dbtest\test>C:\PortableApps\MM\ExifToolGUI\exiftool -P -r -ext jpg -ext jpeg -overwrite_original -charset latin1 "-Comment<${filename}#${directory}" f:/dbtest/test
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_152455.JPG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_153514.JPEG
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_162439.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/CZK_├║┼»├¢┼¥├í─ì─Å├®─ø├¡┼ê├│┼Ö┼í┼Ñ.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/HU_őű.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/PL(check accent)_─à─ç─Ö┼é┼ä┼ø┼║┼╝.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RO_ăâîşș.jpg
Warning: Some character(s) could not be encoded in Latin - f:/dbtest/test/East/RU┬½ÐæÐÅÐêðÁÐÇÐéÐïÐâð©ð¥ð┐ÐÄÐëÐìÐèðÂÐîð╗ð║ð╣Ðçð│Ðäð┤Ðüð░ðÀÐàÐåð▓ð▒ð¢ð╝┬╗.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/pas autoris├® ├®criture.jpeg
Warning: Non-standard header for APP1 XMP segment - f:/dbtest/test/West/ES_áñ.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├®├¬├¿├»├«.jpg
Warning: Invalid EXIF text encoding - f:/dbtest/test/├╣┬Á├▒.Jpg
    3 directories scanned
   14 image files updated

F:\dbtest\test>

[Phil Harvey]

Great.  Problem solved.

Now if you want to learn more about your other problem (the encoding problems), read FAQ 10.

- Phil

[artofit]

Yes, I just still wonder for the warnings, e.g.

Why do I get:

Code Select Expand

Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_152455.JPG

While, I for image ES_áñ.jpg, I get

Code Select Expand

Warning: Non-standard header for APP1 XMP segment - f:/dbtest/test/West/ES_áñ.jpg

[Phil Harvey]

If you use the -v3 option, you can see the exact data responsible for generating these warnings.  The invalid EXIF text is due to an incorrect encoding, probably for EXIF:UserComment.  The Non-standard APP1 header is an incorrect XMP ID string at the start of the XMP APP1 segment.

Both problems are indicative of buggy software that wrote this metadata.

- Phil

[artofit]

Both problems are indicative of buggy software that wrote this metadata.

Sometimes warning are important, sometimes not.
1/In general is this important, and so does it needs/makes sence to fix it?
2/ If yes, why exiftool 9.93 (thanks) doesn't fix it, as it overwrites the tag?
If no, isn't it better to add -q -q to cli so as to ignore the warnings and focus on the possible errors?

Thanks

Code Select Expand

Setting new values from f:/dbtest/test/20131117_152455.JPG
Writing Ducky:Comment if tag exists
Writing PNG:Comment
Writing MIE-Doc:Comment
Writing Comment:Comment
Warning: Invalid EXIF text encoding - f:/dbtest/test/20131117_152455.JPG
Rewriting f:/dbtest/test/20131117_152455.JPG...
  Editing tags in: APP12 COM Comment Ducky MIE-Doc PNG
  Creating tags in: COM Comment MIE-Doc PNG
JPEG APP1 (13118 bytes):
    0000: 45 78 69 66 00 00 49 49 2a 00 08 00 00 00 0c 00 [Exif..II*.......]
    0010: 00 01 04 00 01 00 00 00 c0 0c 00 00 01 01 04 00 [................]
    0020: 01 00 00 00 90 09 00 00 0f 01 02 00 08 00 00 00 [................]
    0030: 9e 00 00 00 10 01 02 00 09 00 00 00 a6 00 00 00 [................]
    0040: 12 01 03 00 01 00 00 00 06 00 00 00 1a 01 05 00 [................]
    0050: 01 00 00 00 d0 00 00 00 1b 01 05 00 01 00 00 00 [................]
    0060: d8 00 00 00 28 01 03 00 01 00 00 00 02 00 00 00 [....(...........]
    [snip 13006 bytes]
JPEG APP13 (58 bytes):
    0000: 50 68 6f 74 6f 73 68 6f 70 20 33 2e 30 00 38 42 [Photoshop 3.0.8B]
    0010: 49 4d 04 04 00 00 00 00 00 1f 1c 02 78 00 13 32 [IM..........x..2]
    0020: 30 31 33 31 31 31 37 5f 31 35 32 34 35 35 2e 6a [0131117_152455.j]
    0030: 70 67 1c 02 00 00 02 00 04 00                   [pg........]
JPEG COM (34 bytes):
    - Comment = '20131117_152455.JPG#f:/dbtest/test'
    + Comment = '20131117_152455.JPG#f:/dbtest/test'
JPEG DQT (130 bytes):
JPEG SOF0:
JPEG DHT (416 bytes):
JPEG SOS

Code Select Expand

[Phil Harvey]

1/In general is this important, and so does it needs/makes sence to fix it?

How important it is depends on a number of factors.  If the software you are using can't read the value correctly, and you rely on this value, then it is important.

2/ If yes, why exiftool 9.93 (thanks) doesn't fix it, as it overwrites the tag?

But ExifTool does fix this if you write the problem tag (UserComment as I mentioned).  You are writing Comment (which is not an EXIF tag), not UserComment.

- Phil