ExifTool Forum

ExifTool => Newbies => Topic started by: MB on September 11, 2022, 02:17:00 PM

Title: Encoding of a csv-file (charset not defined in the command line)
Post by: MB on September 11, 2022, 02:17:00 PM
Why in Windows the csv-export-file is not detected as UTF8-encoded (Exiftool default), as soon as the "GPSPosition"-tag is included in the tag-list in the command line?

1. export of "ImageDescription" only
-csv "-ImageDescription" "H:\Exiftool\2022-08" > H:\ExiftoolExport\2022-08-NurBildunterschrift.csv -c "%d° %d' %.2f"\" -r

The german Umlaut "ü" is encoded as hex C3 BC (UTF8).
According to Windows-Editor "Notepad" the csv-File is encoded in UTF8 (Exiftool default) and
the german Umlaut "ü" is shown as "ü".
According to the Hex-Editor "wxMEdit" the csv-File is encoded in UTF8 (Exiftool default) and
the german Umlaut "ü" is shown as "ü".

2. export of  "GPSPosition" only
-csv "-GPSPosition" "H:\Exiftool\2022-08" > H:\ExiftoolExport\2022-08-NurGPS.csv -c "%d° %d' %.2f"\" -r

According to the Windows-Editor "Notepad" the csv-File is encoded in ANSI.
According to the Hex-Editor "wxMEdit" the csv-File is encoded in ISO-88592-2/Latin-2.


3. export of "GPSPosition" and "ImageDescription"
-csv "-GPSPosition" "-ImageDescription" "H:\Exiftool\2022-08" > H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv -c "%d° %d' %.2f"\" -r

The german Umlaut "ü" is encoded as hex C3 BC (UTF8).
According to the Windows-Editor "Notepad" the csv-File is encoded in ANSI and
the german Umlaut "ü" is shown as "ü" (C3+BC in ISO-88592-1/Latin-1)
According to the Hex-Editor "wxMEdit" the csv-File is encoded in ISO-88592-2/Latin-2
and the german Umlaut "ü" is shown as "Ăź" (C3+BC in ISO-88592-2/Latin-2).

Thanks for your help
Martin
Title: Re: Encoding of a csv-file (charset not defined in the command line)
Post by: Phil Harvey on September 11, 2022, 03:30:36 PM
Interesting.  What happens if you output ImageDescription first?:

csv "-ImageDescription" "-GPSPosition" "H:\Exiftool\2022-08" > H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv -c "%d° %d' %.2f"\" -r
... but what we are learning here is how Notepad and wxMEdit recognize character encoding of text files, and isn't really related to ExifTool at all.

However, one way to mark a text file as UTF-8 is to start with the UTF-8 BOM (0xef 0xbb 0xbf).   You could do it with the following commands and the attached file:

1. copy utf8bom.txt H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv

2. exiftool -csv "-GPSPosition" "-ImageDescription" "H:\Exiftool\2022-08" -c "%d° %d' %.2f"\" -r  >> H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv

(note that you need to put all ExifTool options before the redirection ">" -- you had this wrong in your commands.  In my command I am using ">>" to append to the existing file since I want to add the ExifTool output after the BOM.)

- Phil
Title: Re: Encoding of a csv-file (charset not defined in the command line)
Post by: MB on September 13, 2022, 11:07:06 AM
Hello Phil,
many thanks for answering my question, although it isn't related to ExifTool.
1. The tag-sequence has no influence on the result of the csv-export.
2. Position of the Exiftool-options
Exiftool-options before the redirection ">":
exiftool -csv "-GPSPosition" "-ImageDescription" "H:\Exiftool\2022-08" -c "%d° %d' %.2f"\" -r  > H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv
In this case the generated records of the csv-file are written to the command line and there is no csv-file in the defined directory.
Exiftool-options after redirection ">":
exiftool -csv "-ImageDescription" "-GPSPosition" "H:\Exiftool\2022-08" > H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv -c "%d° %d' %.2f"\" -r
In this case the generated records of the csv-file are written to the defined csv-file.
3. Encoding of the csv-file
When coding the csv-file as UTF8 with BOM and when using an argument-file, coded as UTF8 with bom as well, then there are no problems with the csv-export as far as charsets are concerned. 

Kind regards
Martin
Title: Re: Encoding of a csv-file (charset not defined in the command line)
Post by: Phil Harvey on September 13, 2022, 12:04:03 PM
Hi Martin,

Quote from: MB on September 13, 2022, 11:07:06 AM2. Position of the Exiftool-options
Exiftool-options before the redirection ">":
exiftool -csv "-GPSPosition" "-ImageDescription" "H:\Exiftool\2022-08" -c "%d° %d' %.2f"\" -r  > H:\ExiftoolExport\2022-08-GPSBildunterschrift.csv
In this case the generated records of the csv-file are written to the command line and there is no csv-file in the defined directory.

Interesting.  It does seem that options work after the redirection.  ...learn something every day.  But my command should have worked unless perhaps the backslash-quote is causing problems somehow, but that would depend on the shell you are using.

- Phil