Special Characters are written as ?

[scw2wi]

I found this 11 years old thread
HOW TO WRITE SPECIAL CHARACTERS (unicode) with exiftool
and I'm having a very similar problem.

Here the steps I'm doing in Windows 11 console.

Code Select Expand

chcp 65001 - this should switch to UTF8

I can now read all my XMP metadata in UTF8, my IPTC metadata in Latin and UTF8 correct with all specialchars like "äöü" or other Latin chars like "âãåæ" or non Latin chars like "ăą"

But when I'm writing metadata to UTF8 XMP and IPTC fields with this command:

Code Select Expand

exiftool -xmp:credit="ä ö ü" -iptc:credit="ä ö ü" "Filename.jpg"
and read the same again with this command:

Code Select Expand

exiftool -G1 -xmp:credit -iptc:credit -iptc:coded* "Filename.jpg"
then I'm getting this output

Code Select Expand

[XMP-photoshop] Credit              : ? ? ?
[IPTC]          Credit              : � � �
[IPTC]          Coded Character Set : UTF8
Since also other SW is showing this output, I'm sure the error happens at writing the metadata.
The dark questionmark looks like the string is saved in Latin but read in UTF8.

I also tried the same with -charset exiftool=UTF8 -charset IPTC=UTF8, but no change.
The question is, what shall I change in my workflow to write UTF8 strings with specialchars.

[StarGeek]

I found this 11 years old thread

Reading FAQ #18, problems with special characters on the Windows command line would be the most up-to-date info.

The only thing that ever worked for me with Windows command line was setting UTF8 system-wide, as done in the linked StackOverflow answer.

[scw2wi]

Thanks a lot for this hint, it helped me one step further.
There are different solutions described, e.g. changing the default charset of Windows (which is a beta feature and not recommended).
My preferred solutions would be to write the metadata tag in an UTF-8 textfile and to read this textfile via exiftool.

So I created an UTF-8 textfile with the following content (just 1 line):

Code Select Expand

-iptc:credit=ä ö üand I checked the binary of this textfile if it's really UTF-8:
EF BB BF 2D 69 70 74 63 3A 63 72 65 64 69 74 3D C3 A4 20 C3 B6 20 C3 BC 0D 0A

When I now call exiftool via Windows command window like this:

Code Select Expand

exiftool -@ exiftool_utf8Args.txt "Filename.jpg"everything is working fine, the IPTC data is written in UTF-8.

But when I'm doing exactly the same via PowerShell, it's not working.
It shows the same wrong char as before: � � �
How can that be?

I checked different textfile encodings: UTF-8, Unicode (UTF-16), Latin (Default)
and it always shows exactly the same result.
So the wrong chars are not depending on the encoding of the textfile.

This is the function I'm using to call exiftool from PowerShell:

Code Select Expand

Function ExifTool($Arguments)
{
    $processinfo = New-Object System.Diagnostics.ProcessStartInfo
    $processinfo.FileName               = $exiftoolPath
    $processinfo.Arguments              = $Arguments
    $processinfo.UseShellExecute        = $false
    $processinfo.CreateNoWindow         = $true
    $processinfo.RedirectStandardOutput = $true
    $processinfo.RedirectStandardError  = $true
    $processinfo.StandardOutputEncoding = [System.Text.Encoding]::UTF8
    $processinfo.StandardErrorEncoding  = [System.Text.Encoding]::UTF8
    $process = New-Object System.Diagnostics.Process
    $process.StartInfo = $processinfo
    [void]$process.Start()
    $stdout=$process.StandardOutput.ReadToEnd()
    return $stdout
}
I also tried to add
$processinfo.StandardInputEncoding = [System.Text.Encoding]::UTF8
but StandardInputEncoding is not valid and gives an error.

What can be the reason that it's working fine via the Command Window, but not via PowerShell?

[StarGeek]

I cannot help with Powershell. I've banged my head against PS's idiosyncrasies too many times and it refuses to act like every other command line. I can only advise using CMD.

[FrankB]

In ExifToolGui, you can show the ExifTool commands that have been executed, AND create a powershell script to replay those commands.
This is a powershell script generated bu GUI. For my JPG's it works. Maybe it helps...

Code Select Expand

#
# When you receive a message ".... cannot be loaded because running scripts is disabled on this system",
# Copy, paste the next line in a new window:
# Set-ExecutionPolicy bypass -Scope Process
# Then uncomment and execute.
#
# To permanently disable start ISE as admin and execute:
# Set-ExecutionPolicy bypass -Scope LocalMachine
#
# It is recommended to run this script with WMF 5.1.
# You can check your version by issueing the command: $PSVersionTable.PSVersion
#
try {
  [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
}
catch {
# The first time in ISE we get an "invalid handle". Write something with -VER and try again.
  exiftool -VER
  [Console]::OutputEncoding = [System.Text.Encoding]::UTF8
}
# Set Working directory.
Set-Location "<Your dir>"

# New-TemporaryFile may not be known.
try {
    $args = New-TemporaryFile
}
catch {
    $args = Get-Item ([System.IO.Path]::GetTempFilename())
}
Set-Content -Encoding UTF8 -Path $args -Value "#### Generated by ExifToolGui #####"
Add-Content -Path $args -Value "-echo4"
Add-Content -Path $args -Value "{ready12}"
Add-Content -Path $args -Value "-CHARSET"
Add-Content -Path $args -Value "FILENAME=UTF8"
Add-Content -Path $args -Value "-v0"
Add-Content -Path $args -Value "-overwrite_original"
Add-Content -Path $args -Value "-sep"
Add-Content -Path $args -Value "*"
Add-Content -Path $args -Value "-m"
Add-Content -Path $args -Value "-c"
Add-Content -Path $args -Value "%.6f°"
Add-Content -Path $args -Value "-API"
Add-Content -Path $args -Value "WindowsWideFile=1"
Add-Content -Path $args -Value "-xmp:credit=ä ö ü"
Add-Content -Path $args -Value "-iptc:credit=ä ö ü"
Add-Content -Path $args -Value "<your file, or files>"
Add-Content -Path $args -Value "-execute12"
#echo4 triggers NativeCommand exception. ==>> 2>&1 | %{"$_"} <<= redirects Stderr to Stdout
exiftool  -@ $args 2>&1 | %{"$_"}
Remove-Item -Path $args
pause


[scw2wi]

Maybe it helps...

Thanks for this code, it brings in some new aspects.
Do you have any function called exiftool in your PS?

When I run this code in PS-ISE it shows the following error:

Code Select Expand

ObjectNotFound: (exiftool:String) [], CommandNotFoundException

[StarGeek]

Do you have any function called exiftool in your PS?

Exiftool is run at the line third from the bottom.

When I run this code in PS-ISE it shows the following error:

That is the error that indicates exiftool isn't found. It's not located in a directory covered by the PATH env variable.

You might try installing exiftool with Oliver Betz' exiftool installer. That will make exiftool available system wide

[FrankB]

You might try installing exiftool with Oliver Betz' exiftool installer. That will make exiftool available system wide

Oliver's installer works very well. But dont forget to check 'Add to Path' when you run the installer.

If you know where exiftool.exe is located on your computer. You could change the powershell script, so the 'exiftool' line contains the full path.

Prefix it with an & and enclose in double quotes. EG:

Code Select Expand

& "c:\program files\exiftool\exiftool.exe"  -@ $args 2>&1 | %{"$_"}

[StarGeek]

Oliver's installer works very well. But dont forget to check 'Add to Path' when you run the installer.

Good to know. I thought it did that automatically.

[scw2wi]

Thanks a lot for your great support, it's working now - with correct special chars!

I was always calling exiftool (like all other exe files) via the more complex System.Diagnostics.Process
because I learned it as the "better" way and forgot the much simpler way.

Now I know that the "simpler" way is the better way, at least in combination with utf-8.