Replacing special characters in tags

[Jossie]

Hello,

to replace an underscore in a tag with a minus-sign I used the command e.g.
-PersonInImage <${PersonInImage;tr/_/-/}
which works.

Now I have special characters inserted by geosetter, which do not show properly in the tags, e.g. the German letter "ß" shows as ß.

Using the same code as above, i.e.
-IPTC:Province-state < ${province-state;tr/ß/ß/}
does not work. It does replace the "ß", but it also inserts "??" in addition, so I get ß?? instead of just ß in replacement of ß.

What am I doing wrong?

Hermann-Josef

[StarGeek]

What program are you using to view the data?  If you are listing the output in CMD using exiftool, the problem is the character set of the CMD window.  See FAQ #18.

The ß is probably written correctly, (GeoSetter, in my experience, writes things correctly) it's just not displayed correctly because CMD has problems with characters beyond the original ascii set.  It will display ß as ß unless told otherwise. If you add the -L (latin) option to your exiftool command, you should see the proper output.  If you view the fields in some other program such as Adobe Bridge or Irfanview, it will probably show up correctly.

Also, a note on tr///.  It does replacement on single characters only.  It would have replaced à with ß and Ÿ with nothing.  Because ß is a UTC character and requires two bytes to display, those bytes being ß, doing that replacement you would end up with ßÃ, which ends up as unprintable characters.  To replace multiple characters with a single character you would need to use regex substitution s/ß/ß/g

[Jossie]

Thanks a lot for your reply.

There were two issues: one is the character set and one is the PERL syntax.

The display in geosetter was not correct. Also the export to a text file with EXIFtoolGUI did not reproduce the characters correctly. Thus I have written an args-file to correct this, but it did not work as I expected. Now I will replace the tr-command by the s-command and see what I will get.

Hermann-Josef

PS: I think I will by a book to learn the basics of PERL :-)

[Jossie]

Good evening,

the s-command works nicely, if I issue the command in the GUI from the ExifTool direct command line.
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ß/ß/g}
However, the very same command in an args-file together with others, does not work:
# ß
-PersonInImage<${PersonInImage;s/ß/ß/g}
-Subject<${Subject;s/ß/ß/g}
-IPTC:Keywords<${Keywords;s/ß/ß/g}
-IPTC:City<${City;s/ß/ß/g}
-IPTC:Sub-location<${sub-location;s/ß/ß/g}
-IPTC:Province-state<${province-state;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCity<${LocationCreatedCity;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCountryName<${LocationCreatedCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedProvinceState<${LocationCreatedProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedSubLocation<${LocationCreatedSubLocation;s/ß/ß/g}
-XMP-iptcExt:LocationShownCity<${LocationShownCity;s/ß/ß/g}
-XMP-iptcExt:LocationShownCountryName<${LocationShownCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationShownProvinceState<${LocationShownProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ß/ß/g}

So I am puzzled.

One explanation would be that the args-file stops executing, if one of the earlier commands do not find a match. But the log file does not show any errors...

Hermann-Josef

[StarGeek]

Good evening,

Good morning (still morning here)

One explanation would be that the args-file stops executing, if one of the earlier commands do not find a match. But the log file does not show any errors...

Exiftool will issue Warning: No writable tags set if it can't copy a tag, but it does not stop copying the other tags.

the s-command works nicely, if I issue the command in the GUI from the ExifTool direct command line.
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ß/ß/g}
However, the very same command in an args-file together with others, does not work:

In exactly what way does it not work?  What are you using to look at the data?

I copy/pasted your arg file with all the items set to just ß and all the tags were changed to ß.  But, there are tags that GeoSetter should have set that aren't included in your arg file, those being XMP:State, XMP:Location, and XMP:City.

Here's the results I got using your arg file (the MyKeywords and MyLoc tags are my shortcut tags for Keywords/Subject and XMP Location/City/State and their IPTC counterparts)

Code Select Expand

C:\>exiftool -G1 -a -s -L -PersonInImage -MyKeywords -myLoc -LocationCreated* -LocationShown* y:\!temp\Test4.jpg
[XMP-iptcExt]   PersonInImage                   : ß
[XMP-dc]        Subject                         : ß
[IPTC]          Keywords                        : ß
[IPTC]          Province-State                  : ß
[XMP-photoshop] State                           : ß
[IPTC]          City                            : ß
[XMP-photoshop] City                            : ß
[IPTC]          Sub-location                    : ß
[XMP-iptcCore]  Location                        : ß
[XMP-iptcExt]   LocationCreatedCity             : ß
[XMP-iptcExt]   LocationCreatedCountryName      : ß
[XMP-iptcExt]   LocationCreatedProvinceState    : ß
[XMP-iptcExt]   LocationCreatedSublocation      : ß
[XMP-iptcExt]   LocationShownCity               : ß
[XMP-iptcExt]   LocationShownCountryName        : ß
[XMP-iptcExt]   LocationShownProvinceState      : ß
[XMP-iptcExt]   LocationShownSublocation        : ß

C:\>exiftool -P -overwrite_original -@ temp.txt y:\!temp\Test4.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -L -PersonInImage -MyKeywords -myLoc -LocationCreated* -LocationShown* y:\!temp\Test4.jpg
[XMP-iptcExt]   PersonInImage                   : ß
[XMP-dc]        Subject                         : ß
[IPTC]          Keywords                        : ß
[IPTC]          Province-State                  : ß
[XMP-photoshop] State                           : ß
[IPTC]          City                            : ß
[XMP-photoshop] City                            : ß
[IPTC]          Sub-location                    : ß
[XMP-iptcCore]  Location                        : ß
[XMP-iptcExt]   LocationCreatedCity             : ß
[XMP-iptcExt]   LocationCreatedCountryName      : ß
[XMP-iptcExt]   LocationCreatedProvinceState    : ß
[XMP-iptcExt]   LocationCreatedSublocation      : ß
[XMP-iptcExt]   LocationShownCity               : ß
[XMP-iptcExt]   LocationShownCountryName        : ß
[XMP-iptcExt]   LocationShownProvinceState      : ß
[XMP-iptcExt]   LocationShownSublocation        : ß

Using an arg file works around the problem with character code sets in CMD.  If after running your arg file and you don't see a difference, then the problem is elsewhere.

I just remembered something.  What version of GeoSetter are you using?  The Change Log says:
   Requesting location data from GeoNames didn't work correctly with special characters, for example German umlauts

[Jossie]

Many thanks for looking so thoroughly into this problem!

In exactly what way does it not work?  What are you using to look at the data?

I always use EXIFtoolGUI and there EXIFtool direct instead of the command line (cmd-window). After executing the args file, the tags that were supposed to be changed are not. If I just copy the relevant line from the args file and paste it into the command line, it does work, however. Thus I am puzzled.

I have made some more tests and found out the following:

If in my args file, I only have the block of commands, which replace the ß by ß in all sorts of tags, the args-file works as expected:

# ß
-PersonInImage<${PersonInImage;s/ß/ß/g}
-Subject<${Subject;s/ß/ß/g}
-IPTC:Keywords<${Keywords;s/ß/ß/g}
-IPTC:City<${City;s/ß/ß/g}
-IPTC:Sub-location<${sub-location;s/ß/ß/g}

-IPTC:Province-state<${province-state;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCity<${LocationCreatedCity;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCountryName<${LocationCreatedCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedProvinceState<${LocationCreatedProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedSubLocation<${LocationCreatedSubLocation;s/ß/ß/g}
-XMP-iptcExt:LocationShownCity<${LocationShownCity;s/ß/ß/g}
-XMP-iptcExt:LocationShownCountryName<${LocationShownCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationShownProvinceState<${LocationShownProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ß/ß/g}

However, if I add additional blocks to also replace other characters like ä, the args-file does not work.

# ß
-PersonInImage<${PersonInImage;s/ß/ß/g}
-Subject<${Subject;s/ß/ß/g}
-IPTC:Keywords<${Keywords;s/ß/ß/g}
-IPTC:City<${City;s/ß/ß/g}
-IPTC:Sub-location<${sub-location;s/ß/ß/g}

-IPTC:Province-state<${province-state;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCity<${LocationCreatedCity;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedCountryName<${LocationCreatedCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedProvinceState<${LocationCreatedProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationCreatedSubLocation<${LocationCreatedSubLocation;s/ß/ß/g}
-XMP-iptcExt:LocationShownCity<${LocationShownCity;s/ß/ß/g}
-XMP-iptcExt:LocationShownCountryName<${LocationShownCountryName;s/ß/ß/g}
-XMP-iptcExt:LocationShownProvinceState<${LocationShownProvinceState;s/ß/ß/g}
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ß/ß/g}
# ä
-PersonInImage<${PersonInImage;s/ä/ä/g}
-Subject<${Subject;s/ä/ä/g}
-IPTC:Keywords<${Keywords;s/ä/ä/g}
-IPTC:City<${City;s/ä/ä/g}
-IPTC:Sub-location<${sub-location;s/ä/ä/g}
-IPTC:Province-state<${province-state;s/ä/ä/g}
-XMP-iptcExt:LocationCreatedCity<${LocationCreatedCity;s/ä/ä/g}
-XMP-iptcExt:LocationCreatedCountryName<${LocationCreatedCountryName;s/ä/ä/g}
-XMP-iptcExt:LocationCreatedProvinceState<${LocationCreatedProvinceState;s/ä/ä/g}
-XMP-iptcExt:LocationCreatedSubLocation<${LocationCreatedSubLocation;s/ä/ä/g}
-XMP-iptcExt:LocationShownCity<${LocationShownCity;s/ä/ä/g}
-XMP-iptcExt:LocationShownCountryName<${LocationShownCountryName;s/ä/ä/g}
-XMP-iptcExt:LocationShownProvinceState<${LocationShownProvinceState;s/ä/ä/g}
-XMP-iptcExt:LocationShownSubLocation<${LocationShownSubLocation;s/ä/ä/g}


So somehow, the second block does interfer with the first block in some way. Perhaps this is a hint to a solution?

Hermann-Josef

[StarGeek]

Ah, it finally hit me.  Smacking my head for not seeing it earlier

An arg file works the same way as the command line.  The important part of the docs is this line
   1) Many tag values may be assigned in a single command. If two assignments affect the same tag, the latter takes precedence (except for list-type tags, for which both values are written).

The later group of tags is written because it overrides the earlier tags.

Try this instead.  Just add any other substitutions to the filter line, separated by a semicolon

Code Select Expand

-api
Filter=s/ß/ß/g;s/ä/ä/g
-TagsFromFile
@
-PersonInImage
-Subject
-IPTC:Keywords
-IPTC:City
-IPTC:Sub-location
-IPTC:Province-state
-XMP-iptcExt:LocationCreatedCity
-XMP-iptcExt:LocationCreatedCountryName
-XMP-iptcExt:LocationCreatedProvinceState
-XMP-iptcExt:LocationCreatedSubLocation
-XMP-iptcExt:LocationShownCity
-XMP-iptcExt:LocationShownCountryName
-XMP-iptcExt:LocationShownProvinceState
-XMP-iptcExt:LocationShownSubLocation

[Jossie]

Good morning,

many thanks indeed for you advice! The args-file now works nicely and all characters are displayed correctly in the EXIFToolGUI's window. I would never have found this solution, which is far beyond my knowledge horizon  .

However, there is still a problem with the characters. Although they are now ALL displayed correctly in the EXIFToolGUI's window, ALL are still kryptic if I export the tags with EXIFToolGUI to a txt-file. More strange, SOME tags are not correctly displayed in geosetter's ImageInfo. The latter is especially strange, since geosetter uses EXIFtool. How come that only some tags are displayed here correctly?

Best regards

Hermann-Josef

[StarGeek]

I wouldn't be able to tell without being able to look at an example image in detail.  It could still be a character coding issue of some sort.  ExiftoolGui crashes for me within seconds of starting it up, so I can't comment on it.

[Jossie]

Good evening (or good morning  ) and again thanks for looking into this!

You find an example file here. Characters are all displayed correctly in EXIFtoolGUI (as well as in geosetter), but not when using EXIFtool in the commandline. Here I get something like this: "Location: Freiherr-vom-Stein-Stra├ƒe 125", instead of "Location: Freiherr-vom-Stein-Straße 125".

Please note that the "ß" is now represented by "├ƒ" instead of what I had previously: ß

If I export the tags in EXIFtoolGUI as a txt file, I get: "Location: Freiherr-vom-Stein-Straße 125", which is different from what I get in the commandline.

I hope this helps in diagnosis.

Best wishes

Hermann-Josef

[StarGeek]

As I previously mentioned, this is FAQ #18.  The characters are absolutely correct in the file. This is a problem with Windows CMD.  Do the three steps mentioned there.

Code Select Expand

C:\>exiftool -g1 -a -s -location Y:\!temp\ccc\FN_0028.jpg
---- XMP-iptcCore ----
Location                        : Freiherr-vom-Stein-Stra├ƒe 125

C:\>chcp 65001
Active code page: 65001

C:\>exiftool -g1 -a -s -location Y:\!temp\ccc\FN_0028.jpg
---- XMP-iptcCore ----
Location                        : Freiherr-vom-Stein-Straße 125

What program are you using to view the text file?  While I can't speak as to exactly what ExifToolGui is doing, it seems most likely that whatever you're using to view the text file isn't able to detect UTF-8 character set.

[Jossie]

Okay, thanks for verifying that the characters in the file are correct. So the problem must be in the tools I use to view the content.

I used the Windows editor and it says "ANSI", which produces the wrong representation. I see no way to change this with this program.

However, in WinEDT I have the option of switching to UTF8. When done, most characters I am dealing with are then correct, except for the degree-sign.

So finally I think I have to sort this out with my tools available.

Thank you for reassuring me that the data in the file are correct. The rest I have to find out locally.

Best wishes

Hermann-Josef