News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Importing GPS data from XMP sidecar files

Started by Alan, July 24, 2011, 10:11:30 PM

Previous topic - Next topic

Alan

I found a very old post from you Bogdan on DPReview that implied you could batch import GPS data from XMP sidecar files produced by GPS software.  Reading the 4.3 doc, it looks like it's a one CR2/JPG/etc to one XMP file import.  Is there a way to batch import multiple like-named file pairs?

I'm a bit new at metadata processing, so forgive my next question.  I noticed that the GPS data from my imported sidecar file ends up in the XMP section and not the EXIF after import.  It appears to be accessible from other tools, but I was curious of why it's in XMP.    Here's what was in the sidecar:

    <rdf:Description about="" xmlns:exif="http://ns.adobe.com/exif/1.0/">
      <exif:GPSVersionID>2.0.0.0</exif:GPSVersionID>
      <exif:GPSLatitude>53,20.71N</exif:GPSLatitude>
      <exif:GPSLongitude>6,15.55W</exif:GPSLongitude>
      <exif:GPSAltitude>24/1</exif:GPSAltitude>
      <exif:GPSAltitudeRef>0</exif:GPSAltitudeRef>
      <exif:GPSTimeStamp>2011-06-13T10:48:04Z</exif:GPSTimeStamp>
      <exif:GPSStatus>A</exif:GPSStatus>
      <exif:GPSMapDatum>WGS-84</exif:GPSMapDatum>
    </rdf:Description>

BogdanH

Hi Alan,

I'm a bit hurry right now, so I'll be short (hoping that Phil will jump in to help on this). I assume, you used GUI to geotag your photos? I ask, because from what I've tried, geo-data should be written into Exif section.
And, please, don't forget to read GUI manual (https://exiftool.org/gui/#m_expimp).

Bogdan

Phil Harvey

The exiftool command line to do this would be:

exiftool -tagsfromfile %d/%f.xmp -@ xmp2gps.args DIR

where DIR is the name of the directory containing the image/sidecar pairs.  If you are running this as an exiftool-direct command from inside ExifToolGUI, you leave out the DIR specification.  You could also add a -r option to recurse into subdirectories.

The "xmp2gps.args" file is one of the standard argument files available in the full exiftool distribution.  It takes GPS stored in XMP and writes it to standard EXIF GPS.

Your GPS is stored in XMP format because XMP is a common format for sidecar metadata files and is often used when the software doesn't want to modify the original image.  My command translates this to EXIF, which is a better location for the GPS when stored in an image.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Alan

Bogdan, Thanks for the quick reply.  And yes, I did read the manual first.   :) 

I was using GUI 4.15 with the following steps and results:

       
  • Selected my photo.cr2 file
  • Clicked Import/Export and Import metadata into a single file
  • Read the Information dialogue box (which gets a little annoying after a few times, but I'm sure you had a good reason for doing it that way) and clicked OK
  • Selected the corresponding photo.xmp file which was produced by GiSteq geo tracker software
  • In the Import into selected file dialogue box, clicked on GPS (which checked EXIF) then Execute with the following response:
Warning - No writable tags found - path/photo.xmp
                        Nothing changed in photo.cr2
                        0 Image files updated
                        1 Image files unchanged

Then I just happened to try selecting XMP instead of GPS.  That worked, but as I said, it copied the GPS tags into the XMP section instead of EXIF and it only let me do one file.  I have hundreds to do.  Perhaps it's something to consider as enhancements for the next release.

Phil, thank you for your quick response also.   I'll try your command because I prefer the GPS data be in the "better" place that most software will look for it. But I may need help with your nomenclature (Novice alert).  You said this requires the .args file from the Full version?  I found that, but which do I need - the Strawberry or Active state PERL library for WIN32?

BTW, both your utilities are excellent and much appreciated.  Not very common these days to find guys working so hard for something they love to do without the commerce.  Just wanted to let you know that!   8)

Alan

Phil Harvey

Hi Alan,

You don't need to install Perl.  The .args files work fine with the Windows version of exiftool.  So just grab them from the full distribution but use the Windows exe version of ExifTool.  I don't include the args files in all distributions just to keep the Windows/Mac distributions as simple as possible.

And thanks, it's nice to be appreciated. :)

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

BogdanH

Hi Alan,
Now it's all clear. Inbetween, you have probably found out, that when "importing" metadata,  no "conversion" is done: (part of) metadata is just copied from one file to another file, into the same "location" (xmp, in your case).
Believe me, if anything, then I for sure agree, that (meta)data should always be in "better" place. And after seeing your case, I'll try to make something in this regard.
If I understood correctly, you have hundreds of file pairs (=image files with sidecars):
photo1.jpg - photo1.xmp
photo2.jpg - photo2.xmp
... etc

and you wish to transfer gps data from each xmp into Exif section of equal named jpg file.

I can't say how much time I'll need, so try with solution Phil gave.

Bogdan

Alan

Bogdan,

You understood perfectly (except I'm dealing with CR2s instead of JPGs).  I've been using this GiSteq GPS logger for about 4 years and it works pretty good, but never cared for how the software created sidecar files during tagging to avoid modifying RAW file metadata.  Now I have hundreds (actually thousands) of pix and sidecars in the pairs like your example below and am trying to maintain the RAW versions as my primary copy. 

I'll let you know how Phil's solution goes and even try out the Import GPS function from an NMEA file that I can export out of the logger software.  Maybe that will work better so I can bypass using GiSteq's tool completely (and eliminate the sidecar creation/import).  It uses GoogleMap also.

Thanks again. 

Alan

BogdanH

My logger also produces NMEA files and I can confirm, that using them with ExifTool is the simplest way.

Bogdan

Alan

Gentlemen,

Sorry for the delay, but I wanted to try a number of photo/gps mergers to see how well it worked.

Bogdan, you were right.  The easiest way was to export my trip logs into a common format (I chose GPX) and then use GUI to import them right in batch.  That method worked even better than the GPS software that came with the logger, finding many more matches, even ones that were a bit out of time sync (like ones taken indoors). 

I even tried manual tagging for photos that I didn't have any close logs, and that was extremely easy, as the doc suggests.  One issue I ran into is when I wanted to manually locate a position, I had to write down the coordinates displayed by hovering over the red current position, then switch back over to a regular GoogleMap window to enable "hunting" with Satellite and Street Views.  Is there a way to copy the coordinates from the EXIFtoolGUI GoogleMap window?

I also ran into an issue with a few of the jpg-s, both ones saved from CR2s and ones that camera created, not mapping but containing what appears to be valid coordinates. I don't see any pattern between the ones that work OK and others that don't. I've tried to attach a couple, but am getting a security error on one small jpg and a server error on another 3 MB file.   Not sure why, but let me know how to let you look at the metadata.

Lastly, I have a suggestion for improvement of the GPS import data window.  A way to recall or retain the last referenced path of the GPS log file(s) would be great, rather than having to navigate to it every time.  Mine are stored in a folder separate from the photos folders.

Again, I'm impressed with both the ease of use and the accuracy of your utilities.  Keep up the great work!

Alan

BogdanH

Hi Alan,
Quote
I wanted to manually locate a position, I had to write down the coordinates displayed by hovering over the red current position, then switch back over to a regular GoogleMap window to enable "hunting" with Satellite and Street Views.  Is there a way to copy the coordinates from the EXIFtoolGUI GoogleMap window?
We're talking about GoogleMap shown by GUI, right? When you locate the place you want on the map, you "mark" the location by clicking on that location, so "marker" becomes visible on (moves to) that location.
Now, if not previously done, you select file(s) into which you wish to save that geo-position. And finally, you click on <<Geotag selected file(s) button on top of GoogleMap window. That is, you don't need to write down coordinates.
To check geo-position saved in image, you select the image and click on Map>> button in GUI window, and marker on map should appear (move to) appropriate position on the map.
I hope I didn't misunderstood your question...

QuoteI also ran into an issue with a few of the jpg-s, both ones saved from CR2s and ones that camera created, not mapping but containing what appears to be valid coordinates....
I'm lost on this... let's hope Phil will jump in...

About GUI should "remember" last folder of where log files are stored.. Thank you for pointing that out -it will be in next update.

Bogdan

Phil Harvey

Quote from: BogdanH on July 29, 2011, 03:41:29 AM
QuoteI also ran into an issue with a few of the jpg-s, both ones saved from CR2s and ones that camera created, not mapping but containing what appears to be valid coordinates....
I'm lost on this... let's hope Phil will jump in...

Let me understand:  So exiftool extracts valid coordinates but it isn't displayed in the google map?

Bogdan may be able to figure this out if you provide a sample image with this problem.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Alan

I see the problem with some of my jpg-s.  They have GPS data in both the exif section (that I imported with EXIFtoolGUI) and in the XMP section (that was created by my previous merge using my logger tool which created the sidecar files for CR2s but updated the XMP section in jpg-s).  Once I deleted the XMP:exif:All, it mapped correctly.

But others that have only GPS data in the exif section that don't map to a location in the GUI (i.e., marker turns red).  I've been unable to attach any of these (various errors from the forum), but in looking at the dump, and I see multiple sets of GPS data values.  There's no GPS data in my original jpg backup, so I must have introduced the bad data somewhere along the way.  I restored the backup, re-imported the data from the GPX logs and it works fine.  Sorry - I've gotta take more frequent backups when I overwrite files like this.

I'll explain my manual tagging issue better in a subsequent note.

Alan

Alan

Sorry for the confusion on the manual tagging, I was using two techniques.

1) As you described, identify the location on the GoogleMap window using either the data from a previous image and clicking the GUI Map button, or just clicking on the GoogleMap window location close to where I think the image was taken then clicking <<Geotag selected file(s) button on top of GoogleMap window.  Hovering over the green marker, GoogleMap displays the decimal coordinates, but right clicking on the marker doesn't provide a copy function, which is what I was looking for for the 2nd technique.

2) If the image was taken in a city, but I was unable to get close enough to a location of where the picture was taken,  then I used this method:
- position the GUI GoogleMap in the vicinity mapping a previous image that has valid data
- hover over the marker and write down the pop-up coordinates
- open a new browser full GoogleMap window, since the GUI GoogleMap tool doesn't seem to support StreetView
- manually enter in the coordinates in the search field
- zoom into StreetView
- navigate to the location of the where image was taken
- zoom out of StreetView
- right click "What's Here" and copy coordinates
- switch back to the GUI GoogleMap window
- paste the coordinates in the Find Place input box and click enter
- Then click <<Geotag selected fields to tag the image

I was looking for a way to (a) get close to the location using previous images) and (b) avoid writing down the approximate coordinates to key them back into the new GoogleMap window.  If this is a GoogleMap API limitation, I understand.  I was just hoping there was a way (a new button perhaps) to populate the Find place input box with the coordinates of the marker's current location similar to how the Set position as home function works.

Hopefully that's clearer.  Or am I making tagging more complex than it needs to be?  Getting to exact locations will help provide me with StreetView images or GoogleMap identified names of the places on the hundreds of images I took on a holiday, but didn't write down when I took them.  Esp. when I go back and use this workflow on pix I took 2-4 years ago.

Alan

BogdanH

Thank you for explaining. What to say? In my honest opinion.. by trying to be absolute accurate, you're complicating geo-tagging too much. For example: even one can pinpoint some location on GoogleMap with accuracy of say 1 meter (3 feet), no commercial gps device has such accuracy. In cities, because of buildings which block sattelites, accuracy is (my quess) within 15 meters (50 feet).  That is, there's no device, which would take someone back to that "exact" position -unless, that person works for some intelligence service  :) And finally, GoogleMap coordinates are calculated values with limited accuracy.
And if speaking about GUI's geotagging (GoogleMap) capability, I don't plan to make further options right now. It's supposed to be simple.. making fun.. where 10 feets more or less doesn't really matter. In that sense, I believe Phil had similar attitude when making geotagging with log files (by interpolating missing locations).
I hope you're not disappointed too much in this regard...

Bogdan

Alan

Bogdan,

Thanks for your response.  Let me explain my thoughts. 

I agree with your evaluation of auto-tagging from GPS devices and matching by photo time.  If you can get 95%+ of your images tagged with a minimum of effort, then job complete.  With the GPS import function of EXIFtool and ETGUI, you have created just that and have done a fantastic job with making it easy.

For the 5% that I didn't have GPS data accurate even within 50' because my device wasn't syncing, wasn't even turned on or some other reason, then manual tagging is needed.  My sole impetus behind my suggestion was to make my workflow simpler and quicker.  The fun part was using the power of GoogleMap StreetView to help figure out where I was, even down to 50', to determine a description of the image that I took other than "cathedral" or "river" or "street corner".  Your tools have done an excellent job of reducing that effort down to a few clicks, zooms and scrolls.  I was just being lazy and trying to avoid picking up a pen to right down a coordinate and then key it back in.   ;)

I've only taken a short look at the Web based storage and geo-tagging tools and I wasn't impressed. But I have not found a better software-based tool  than yours to let me see where I took a picture from imbedded data, esp. in native RAW images.   So I will continue to use EXIFtool and GUI for both auto and manual tagging for all my photos with/without any enhancements at all.  No disaapointment here!  You've already improved my workflow immensely but not making me manually enter all that GPS data or even use the obviously flawed software that came with this 4-yr-old logger!!!   8) 8) 8) 8) 8)

Alan