Composite tag to geocode lat and lon to city, state, country (config file)

[CWCorrea]

Hello.

The attached config file (geoloc.config) defines the Composite tag GeoLocation that converts GPS coordinates to locality, city, state and country using the bigdatacloud.com free service.

It requires "curl" (https://curl.se) and "jq" (https://jqlang.github.io/jq/) to be installed in your PATH.

exiftool -config geoloc.config -gpslongitude -gpslatitude -geolocation image.jpg

Code Select Expand

GPS Longitude                   : -77.116753
GPS Latitude                    : +38.861903
Geo Location                    : Arlington, Washington, US-VA, US

This config file has been tested with ExifTool 12.77 on macOS and Linux.

Christian W.

[Phil Harvey]

Hi Christian,

Thanks.

What are your thoughts on bigdatacloud's longevity in maintaining this free geolocation service?

This feature is something that could fairly easily be built into ExifTool, but I have 2 reservations:

1. I don't like the idea of ExifTool accessing the internet connection for anything.  It is so refreshing to me to have an app that doesn't do this.

2. I worry about code maintenance with having to find another free service once this one shuts down or starts charging.

- Phil

[StarGeek]

You probably can skip the use of jq by letting exiftool do the parsing.

I totally forgot that had previously uploaded a config that used cURL to get time zone information from Google maps.  You can look at the code I used to do this in this post.

[Hubert]

I was interested in following this up but it appears that free usage of BigDataCloud's geolocation API is only available in certain limited cases:

Web browser:

1. You are allowed to make free and unlimited API calls from a client's web browser using their current coordinates...

Mobile device:

2. You are allowed to make unlimited free API calls from client mobile devices such as iOS, Android or IoT...

And a warning:

"IMPORTANT! Any user detected abusing the service by implementing it on a server-side or a client-side along with coordinates obtained elsewhere will be blacklisted from all of our free API Services."

I suspect that "Coordinates obtained elsewhere" implies that the use of their API with multiple different lat and lon coordinates (for example from image metadata) falls outside their fair usage policy 

[Phil Harvey]

I wonder how large the geolocation database is, and whether there is a freely-available version?  Adding a database lookup would be fairly easy to add.  Then there would be no external dependencies.

- Phil

Edit: I've found a couple of creative commons geolocation databases.  One with all Cities greater than population 500 is only 11 MB (from here).  Maybe I'll look into adding the ability to geolocate the nearest town.  There is also one for geographic features, but this is 300+ MB.

Edit2: I've been playing around with that database.  I can get all of the information I need to geolocate to the nearest city of 500 people or more including full state/province and country name from a database of 4.2 MB.  That would double the download size of ExifTool, so I won't include it by default, but I could think of providing this as an add-on of some type, perhaps with a new -geolocate option or something.

Edit3: Unfortunately this feature would support only english names.

- Phil

[CWCorrea]

Hello Phil,

What are your thoughts on bigdatacloud's longevity in maintaining this free geolocation service?

I believe that as with most online services, the longevity of the service is short.

1. I don't like the idea of ExifTool accessing the internet connection for anything.  It is so refreshing to me to have an app that doesn't do this.

2. I worry about code maintenance with having to find another free service once this one shuts down or starts charging.

I agree with you. ExifTool should not connect to the Internet. Code maintenance will be a unnecessary burden.

I'm exploring other services that allow reverse geocoding, like Open Street Maps, their database is open source.

Christian W.   

[CWCorrea]

You probably can skip the use of jq by letting exiftool do the parsing.

That would be a possibility. I feel more comfortable using jq, so the Composite tag code is simpler.

[CWCorrea]

Hello Hubert.

I suspect that "Coordinates obtained elsewhere" implies that the use of their API with multiple different lat and lon coordinates (for example from image metadata) falls outside their fair usage policy

Thanks for sharing bigdatacloud's fair use policy. I didn't read it when I implemented the config file. It seems that they offer the reverse geocoding service API in exchange of real coordinates so they can map GPS coordinates to IP addresses.

I'm exploring Open Street Maps reverse geocoding. I need to check their policies to see if our use case is allowed.

Christian W.

[CWCorrea]

Adding a database lookup would be fairly easy to add.  Then there would be no external dependencies.

This is a great idea! The database could be separately downloaded by the users that need geolocation.

[CWCorrea]

I've been exploring OpenStreetMap data and found that the JSON returned is not always consistent, for example: with some coordinates the JSON returns the City, State and Country fields and with other coordinates the JSON returns the Town, State and Country fields . This happens even with coordinates within the same country.

It looks like OpenStreetMap's reverse geocoding is a mess and more code is needed to be able to use it in a composite tag.

I'll continue to look into options for doing reverse geocoding online, but I think Phil's idea of using a local database is the best solution.

[Phil Harvey]

I've got a new API Geolocation option working and will include this new feature in version 12.78:

Code Select Expand

> exiftool a.jpg -api geolocation "-gpsL*" "-geolocation*" -e
GPS Latitude Ref                : South
GPS Latitude                    : 44.000000
GPS Longitude Ref               : West
GPS Longitude                   : 75.000000
Geolocation City                : Quellón
Geolocation Region              : Los Lagos Region
Geolocation Country             : Chile
Geolocation Time Zone           : America/Santiago
Geolocation Population          : 30000
Geolocation Distance            : 117.7
Geolocation GPS                 : -43.119,-73.614
Using only cities with population of 1000 or greater, I've got the zipped database size down to 1.7 MB, which I think is small enough to include in the standard ExifTool distribution.

- Phil

[CWCorrea]

I've got a new API Geolocation option working and will include this new feature in version 12.78:

Hello Phil. This is great news!

Will the database be replaceable by the user?

Thanks,

Christian W.

[Phil Harvey]

Hi Christian,

I'm having to use a custom database file to get the size down.  But I would be able to generate alternate versions if the demand arose.

I'm also adding a new write-only Geolocate tag that allows you to do this:

Code Select Expand

> exiftool a.jpg -xmp:all
> exiftool a.jpg -geolocate=44,-72
    1 image files updated
> exiftool a.jpg -xmp:all
XMP Toolkit                     : Image::ExifTool 12.78
Country Code                    : US
City                            : Haverhill
Country                         : United States
State                           : New Hampshire
- Phil

[FrankB]

I have been looking for a 'database file' to use in ExifToolGui. I found these URL's. Maybe they can be an option to use in ExifTool:
http://download.geonames.org/export/dump/cities500.zip
http://download.geonames.org/export/dump/cities1000.zip
http://download.geonames.org/export/dump/cities5000.zip
http://download.geonames.org/export/dump/cities15000.zip
(Where the nbr stands for 'cities with population greater than')

I decided not to use them because they are fairly large, but in stead give the option to use the online api's:
https://geocode.maps.co or https://overpass-api.de/api

But if Exiftool would provide this functionality I would certainly add it to ExifToolGui. Is there an option that I can 'play' with 12.78 to test?

[Phil Harvey]

I've just released version 12.78 with the new Geolocation feature.  Have fun with it!

- Phil