Using the new function geolocation in Exiftool 12.78 in ExifToolGui

[FrankB]

@Phil,

I really like this addition to ExifTool!
In GUI I have added similar functions a few months ago, but this approach clearly has advantages.
- It doesn't require an external webservice, that tend to change their conditions in time. Requiring users to get apikey's, programmers to change their code etc.
- Obviously it's faster, compared to a webservice.

I think there are enough GUI users that would prefer the new geolocation function over the existing webservice functions. So I have looked into how it can be added to GUI. And I have some remarks/questions;

Options available in the used webservices, not available in the geolocation function:
- You can only search for a location by its coordinates. The reverse is not possible; Get the coordinates of  a city. It is used in GUI to position the map.
- Only English names are available. I can understand that, the datafile would grow considerably. (E.g. München => Munich, Bayern => Bavaria)
To be clear: I'm not requesting features, they're just observations.

- I noticed that the geolocation function writes to '-XMP-photoshop:Country, -XMP-photoshop:State and -XMP-photoshop:City'.
In the past Bogdan has decided to use the fields: '-xmp:LocationShownCountryName, -xmp:LocationShownProvinceState, -xmp:LocationShownCity' in ExifToolGui. I'm sure he had his reasons for doing that. I found this url: https://exiftool.org/gui/articles/where_what.html. Now I dont want to start a discussion which fields to use, all I want is a working solution for GUI users that have already used the 'xmp:Location...' fields.
Options I can think of that would not impact ExifToolGui to much:

- Add an option to the geolocation function to also write to the 'xmp:Location...' fields. That would require a change in ExifTool.
- In Gui write always to both set of fields. That is easy for me to do.

Which do you think is best, if any?

Frank

[Phil Harvey]

Hi Frank,

Options available in the used webservices, not available in the geolocation function:
- You can only search for a location by its coordinates. The reverse is not possible;

A reverse feature would be possible.  I was thinking about this but didn't know how it would be used and so I couldn't imagine what interface would work best.

- Only English names are available. I can understand that, the datafile would grow considerably. (E.g. München => Munich, Bayern => Bavaria)

I thought about this too.  As you said, the file size penalty is prohibitive.

- I noticed that the geolocation function writes to '-XMP-photoshop:Country, -XMP-photoshop:State and -XMP-photoshop:City'.
In the past Bogdan has decided to use the fields: '-xmp:LocationShownCountryName, -xmp:LocationShownProvinceState, -xmp:LocationShownCity' in ExifToolGui.

I thought about this too, and it would make most sense to set these tags from LocationShownGPSLatitude/Longitude (which I could do as well I suppose).  But it didn't make sense to me to set these from GPSLatitude/Longitude because that is the camera location.  However, you can set whatever you want from any tag by using the API Geolocation option and copying the information yourself (eg. "-LocationShownCity<geolocationcity") instead of writing the Geolocate tag.

- Phil

Edit: Ah.  I thought of a simple way to optionally write LocationShown tags instead.  I could do this if you wrote to XMP-iptcExt:Geolocate.  I'll add this feature in 12.79.

[FrankB]

A reverse feature would be possible.  I was thinking about this but didn't know how it would be used and so I couldn't imagine what interface would work best.

Just an idea. What I do in GUI, I check for valid coordinates. If they're not valid I will start the search for a city.

exiftool "-geolocate=45.6429,-72.9374" test.jpg Updates Country, State City in the jpg
exiftool "-geolocate=Madrid" test.jpg Updates the gpscoordinates in the jpg

if you dont update a file but use the '-o -.xmp' variant you can return the coordinates in the XMP xml.

Or introduce -gpslocate?

Only English names are available. I can understand that, the datafile would grow considerably. (E.g. München => Munich, Bayern => Bavaria)

I thought about this too.  As you said, the file size penalty is prohibitive.

No problem. It just means I will keep the webservices.

Edit: Ah.  I thought of a simple way to optionally write LocationShown tags instead.  I could do this if you wrote to XMP-iptcExt:Geolocate.  I'll add this feature in 12.79.

You can do this, but I dont mind to do something like this in GUI: -geolocate<gpsposition -xmp:LocationShownCity<XMP-photoshop:City etc..

[FrankB]

if you dont update a file but use the '-o -.xmp' variant you can return the coordinates in the XMP xml.

I mixed some things up. This would be better:

Search on cordinates:
exiftool -api geolocation=44.234,-75.445

Search on city: (This form can return multiple results)
exiftool -api geolocation="Gouverneur"

Or within a country:
exiftool -api geolocation="GeolocationCity=Gouverneur, GeolocationCountryCode=US"

[Phil Harvey]

exiftool "-geolocate=Madrid" test.jpg Updates the gpscoordinates in the jpg

I wish it were that simple.  The first problem is: What happens when there is more than 1 "Madrid" in the database?  For example, the database has 22 "Kingston" cities, 13 in the U.S. alone.

You can do this, but I dont mind to do something like this in GUI: -geolocate<gpsposition -xmp:LocationShownCity<XMP-photoshop:City etc..

That won't work because you can't currently access tags that are queued for writing on the command line.  But this does work:

exiftool -api geolocation "-xmp:locationshowncity<geolocationcity" ...

- Phil

[Phil Harvey]

Or within a country:
exiftool -api geolocation="GeolocationCity=Gouverneur, GeolocationCountryCode=US"

Bulky, but possible.  Maybe just a flexible format like this:

1. -api geolocation=city - returns matching city with largest population
2. -api geolocation=city,field1 - returns largest matching city with region, country code or country equal to "field1"
3. -api geolocation=city,field1,field2 - returns with region, country code or country matching both fields.

- Phil

[Phil Harvey]

I'm testing a version that increases GPS precision to 20 bits (about 20 m), and adds the ability to do the reverse lookup.  Here is how that is working at the moment.  If multiple cities are possible, the one with the largest population is selected:

Code Select Expand

> exiftool -api geolocation=Kingston
Geolocation City                : Kingston
Geolocation Region              : Kingston
Geolocation Country Code        : JM
Geolocation Country             : Jamaica
Geolocation Time Zone           : America/Jamaica
Geolocation Population          : 900000
Geolocation Position            : 17.9970, -76.7937
Geolocation Warning             : Multiple matches were possible
> exiftool -api geolocation=US,Kingston
Geolocation City                : Kingston
Geolocation Region              : New York
Geolocation Country Code        : US
Geolocation Country             : United States
Geolocation Time Zone           : America/New_York
Geolocation Population          : 20000
Geolocation Position            : 41.9271, -73.9974
Geolocation Warning             : Multiple matches were possible
> exiftool -api geolocation=Kingston,US,Washington
Geolocation City                : Kingston
Geolocation Region              : Washington
Geolocation Country Code        : US
Geolocation Country             : United States
Geolocation Time Zone           : America/Los_Angeles
Geolocation Population          : 2000
Geolocation Position            : 47.7984, -122.4979

[FrankB]

I'm testing a version that increases GPS precision to 20 bits (about 20 m), and adds the ability to do the reverse lookup.  Here is how that is working at the moment.  If multiple cities are possible, the one with the largest population is selected:

I'm a few posts behind, so I'll only reply to this one.

I agree with your remarks. I will be happy to test something in GUI.

Frank

[FrankB]

I'm testing a version that increases GPS precision to 20 bits (about 20 m)

Hi Phil,

I'm adding the functionality in GUI. I'm testing in a well-known area to me, and I have a suggestion.

- Add a new field called GeolocationVillage and it should be filled with the value that is now in GeoLocationCity.
- GeoLocationCity should be filled with the 'Municipality'. See below:

Let me explain with a scenario.

The coordinates I used for my scenario: lat: 51.568199, lon: 5.7144468
To give you an idea, this is how it looks on the map:



Output from ExifTool 12.78:

Code Select Expand

C:\Users\tdb>exiftool -short -api geolocation=51.568199,5.714446
GeolocationCity                 : Doonheide
GeolocationRegion               : North Brabant
GeolocationCountryCode          : NL
GeolocationCountry              : The Netherlands
GeolocationTimeZone             : Europe/Amsterdam
GeolocationPopulation           : 1000
GeolocationDistance             : 1.1 km
GeolocationBearing              : 270
GeolocationPosition             : 51.567, 5.696
Exiftool returns Doonheide as City. I think the answer is correct, if you look in Cities500.txt, it is the place closest to the coordinates. The precision 20mtr or 300mtr doesn't make much of difference I think.

But there's something else. Doonheide is just a 'hamlet', or call it a 'neighbourhood'. People from my area wouldn't call it a 'city' or 'Municipality'.
To find the Municipality have a look at cities500, these are the records with 'NL 06 1652'. They all belong to the same Municipality. (The same Townhall)

2748049 De Rips De Rips  51.55 5.80972 P PPL NL  06 1652   875  29 Europe/Amsterdam 2017-10-17
2750600 De Mortel De Mortel Mortel 51.54 5.70833 P PPL NL  06 1652   905  19 Europe/Amsterdam 2017-10-17
2750803 Milheeze Milheeze  51.50167 5.77917 P PPL NL  06 1652   1400  28 Europe/Amsterdam 2017-10-17
2754897 Handel Handel  51.58 5.70972 P PPL NL  06 1652   1310  20 Europe/Amsterdam 2017-10-17
2755605 Gemert Gemert khymyrt,خيÙ...يرت 51.55583 5.69028 P PPL NL  06 1652   15995  18 Europe/Amsterdam 2018-08-24
2756178 Elsendorp Elsendorp  51.58083 5.76944 P PPL NL  06 1652   560  24 Europe/Amsterdam 2017-10-17
2756688 Doonheide Doonheide  51.56667 5.69444 P PPL NL  06 1652   1030  17 Europe/Amsterdam 2017-03-24
2759511 Bakel Bakel  51.50333 5.74028 P PPL NL  06 1652   4140  23 Europe/Amsterdam 2017-10-17

in https://download.geonames.org/export/dump/admin2Codes.txt
you can find this record:

NL.03.1734   Gemeente Overbetuwe   Gemeente Overbetuwe   6544276
NL.06.1658   Gemeente Heeze-Leende   Gemeente Heeze-Leende   6544277
NL.06.1652   Gemeente Gemert-Bakel   Gemeente Gemert-Bakel   6544279
NL.06.1667   Gemeente Reusel-De Mierden   Gemeente Reusel-De Mierden   6544280

(Municipality translates to Gemeente in Dutch)

[FrankB]

I'm adding the functionality in GUI. I'm testing in a well-known area to me, and I have a suggestion.

- Add a new field called GeolocationVillage and it should be filled with the value that is now in GeoLocationCity.
- GeoLocationCity should be filled with the 'Municipality'. See below:

I just now realize that this does not specify how to handle "-geolocate=lat,lon" <filename>. My personal preference would be to add the Municipality (The value found in 'admin2Codes.txt') in the field 'photoshop:City'

And if it all becomes to much in terms of size, and or complexity, I would understand.

[stoffball]

Hi Frank,

I have a suggestion.

- Add a new field called GeolocationVillage and it should be filled with the value that is now in GeoLocationCity.
- GeoLocationCity should be filled with the 'Municipality'. See below:

My personal preference would be to add the Municipality (The value found in 'admin2Codes.txt') in the field 'photoshop:City'

This would only work for some countries.

E.g. for Germany admin2codes.txt from geonames.org contains only 22 entries:

Code Select Expand

<snip>
DE.02.091 Upper Bavaria Upper Bavaria 2861322
DE.02.092 Lower Bavaria Lower Bavaria 2863622
DE.07.055 Regierungsbezirk Münster Regierungsbezirk Muenster 2867539
DE.02.095 Regierungsbezirk Mittelfranken Regierungsbezirk Mittelfranken 2870736
<snip>

"Regierungsbezirk" means governmental district and is in Germany two level above what I would consider a city, town or municipality. So for Germany and probably some other countries your proposal to use admin2codes for photoshop:city would fail horribly.

[FrankB]

So for Germany and probably some other countries your proposal to use admin2codes for photoshop:city would fail horribly.

You're right. It's not that simple.

@Phil
Forget it!

[Phil Harvey]

I have been absent from posting here because I have been working on this the last few days.  My current thought is to ignore any PPLX  entries in the cities database (section of populated place).  This fixes problems of hamlets within large cities if the cities are also in the cities database, but in your case (with Doonheide) we have a problem that I don't know if we can fix.

- Phil

[FrankB]

we have a problem that I don't know if we can fix.

First of all, I dont even know if it's a problem. Maybe most users would prefer to have the closest location, even if it's not a 'City', but a 'Hamlet'. What triggered me was the name of the XMP field where it is stored 'XMP:City'.

Second. Maybe it's just an error in the data, and the 'algorithm' is correct.

These 2 URL's can be used interactively, for comparing data with the Geonames exports, without an API-KEY. And they take OpenStreetMap data. The data in this case looks good to me: 
Search on city/place:          
https://nominatim.openstreetmap.org/search?format=json&city=doonheide
Search on coordinates:
https://nominatim.openstreetmap.org/reverse?format=json&lat=51.5672357&lon=5.6928729

I did some checks and found some differences, but not a definitive solution on how to use Citiesxxx.txt.

Third:
I also looked at the ZIP export: http://www.geonames.org/export/zip/allCountries.zip

Pro:
The data is taken from 'postal code data'. So that matches the term 'City' better. At least for the checks I did.

Con:
- The same Lat/Lon combination can occur multiple times. In the Netherlands we typically have multiple Post Codes/Zip Codes for 1 city. But even after deleting duplicates a lot more data remains. And some countries dont have Coordinates.

[Phil Harvey]

Hi Frank,

I've just released 12.79 with major improvements to the geolocation feature.

I've added a reverse lookup and Subregion information to this version, as well as increasing the precision of the GPS and population values, and adding alternate-language support.

For the multiple entries at the same GPS coordinates, I keep only the one with the highest population.

- Phil

[FrankB]

Hi Phil,

Already played with it. Looks good!

Expect to have a GUI version ready soon using this feature.

Frank

[Phil Harvey]

Great!

- Phil