News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Urgent help needed with XMP data and Maps Made Easy

Started by quigley595, February 11, 2019, 10:35:18 AM

Previous topic - Next topic

quigley595

    Hi folks, complete newbie here, so apologies if this is a stupid one.
    I am processing images with MapsMadeEasy, but I had to fix the altitude figures in the EXIF:GPS   and XMP:XMP-drone-dji   areas
    When I try to use MME for the map, I get a message "Relative Altitude  XMP not present. Unknown."
    The RelativeAltitude field IS in the XMP data, and it looks good....  If I use the original images in MME they all load OK (but I stopped the job because I know the altitude is wrong)...   so it is something I am doing incorrectly with exiftools, or maybe exiftools is stripping something out?

    This is what I did:
    • My images are all in the directory:  C:\Maps\Bad_Data
    • I exported the reqd data to a CSV file by:  exiftool -G -csv -GPS:EXIF:GPSAltitude -XMP-drone-dji:XMP:AbsoluteAltitude -XMP-drone-dji:XMP:RelativeAltitude  "C:\Maps\Bad_Data" > The_GPS_Info_To_Be_Fixed.csv
    • I changed the CSV data (making sure to include the '+' signs where these were in the original data, AbsoluteAltitude  and RelativeAltitude  )
    • I uploaded the CSV to replace the required data with the changed data using:   exiftool -csv=The_GPS_Info_To_Be_Fixed.csv "C:\Maps\Bad_Data"

    At this point everything looks good in ALL the images, and ALL the relevant fields I changed....
    When I bring it into MapsMadeEasy, it tells me that there is NO XMP data in the images  (it is all actually there, and correct as far as I can see)

    Looking at the changed images vs the originals, there is an extra group in the XMP fields....  this is   'XMP-x'   and the order of some of the fields is different from the originals ,but I wouldnt think this is an issue.

    I deleted the 'XMP-x group (which contains 'XMPToolkit     Image::ExifTool 11.17'   by using the command:  exiftool -XMPToolkit-= "C:\Maps\Bad_Data", but still got the same result in MapsMadeEasy....

    Any ideas on this????   Most likely it is MY ignorance!  I can post a couple of the original images if needed.
    thanks,
    Mike




StarGeek

The XMPToolkit shouldn't be a problem unless MME has a really poor XMP parser.

Are you using a config file of some sort?  I ask because there's suddenly a bunch of DJI drone questions coming to the forum (a good thing) and I get the feeling that an earlier thread where there was some work on getting some DJI drone tags writable might have had some unintended consequences.

Run this command on a file that works and a file that has been edited by exiftool that doesn't work and post the results (use the code button help format it).
exiftool -XMP -b File.jpg

That will let us see if there's something wrong with the data or if it's written a different way.  There's a couple of options to tell exiftool to write XMP differently and it's possible that MME doesn't have a robust XMP parser.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Phil Harvey

Are you looking at all the tags?  Run this command on a file that works and compare with one that doesn't:

exiftool -a -G1 FILE

I would be surprised if MapsMadeEasy read the DJI-specific XMP (unless it is DJI-specific software).  Could there be some other same-named tags that it is reading?

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

quigley595

Quote from: StarGeek on February 11, 2019, 11:34:54 AM
The XMPToolkit shouldn't be a problem unless MME has a really poor XMP parser....... <snip  snip>


Thanks StarGeek.....   
I have run your code against the same image, (one the original and other the exiftool changed one)...
I have also run the code from Phil on the same files.

Using your code, there DOES seem to be a difference..  the  <rdf:Description rdf:about='DJI Meta Data' stuff seems to be duplicated and all over the place.
There is also an unprintable character (using your code, but NOT using Phils), but I think that will be OK, as it is in one of the first lines, and is in both the 'Good' and the 'Bad' files.

Unfortunately, I haven't the experience to really analyse the output from your and Phils line of code.... 
I have attached a zip file containing all outputs (from your code, AND Phils code), and also an outline of the process I took to make the changes originally.
If needs be, I can send two of the original images, but they are 6MB each, so I haven't attached them here.

Can you suggest a way forward for me???

thanks,
Mike

quigley595

Quote from: Phil Harvey on February 11, 2019, 02:37:43 PM
Are you looking at all the tags?  Run this command on a file that works and compare with one that doesn't:..  <snip  snip>

Thanks Phil for your reply......
I have run your code and attached all the results in my reply to StarGeeks post above.

thanks a million for this!.....   I am at my wits end here!  :-) 

Mike

StarGeek

The results from Phil's command look pretty much the same except for the order.  It is notable that the GPS tags are plain Latitude and Longitude instead of what was being dealt with in the other DJI drone threads.

The XMP output is definatly different.
   drone-dji:Latitude="-26.7465950"
   drone-dji:Longitude="+153.1291321"

as the original code instead of
  <drone-dji:Latitude>-26.7465950</drone-dji:Latitude>
  <drone-dji:Longitude>+153.1291321</drone-dji:Longitude>

which is what exiftool writes.

It's not a subject I have experience with, but it looks similar to the differences mentioned here.  So try adding -api xmpshorthand to the command to see if that helps.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Phil Harvey

Using shorthand XMP should not make a difference unless the XMP reader is really pathetic.  But try it out as StarGeek suggested because I can't see any other obvious differences.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

quigley595

Quote from: StarGeek on February 11, 2019, 08:24:20 PM
The results from Phil's command look pretty much the same except for the order.  It is notable that the GPS tags are plain Latitude and Longitude instead of what was being dealt with in the other DJI drone threads.   <smip  snip  snip>

Thanks again StarGeek.... 


  • I tried adding  -api xmpshorthand  to the CSV extract code, and NOT to the CSV upload  (exiftool -G -csv -api xmpshorthand -GPS:EXIF:GPSAltitude -XMP-drone-dji:XMP:AbsoluteAltitude -XMP-drone-dji:XMP:RelativeAltitude  DIR CSV_FILENAME)    ......  NO GO
  • I then tried adding  -api xmpshorthand  to the CSV upload code  (exiftool -api xmpshorthand -csv=CSV_FILENAME DIR) ....  STILL NO GO
I'm truly stuck!   I trust exiftool, so I'm sure it is something I am doing wrong...  I just don't know what!

Mike

StarGeek

The only thing I can suggest, since I don't have access to the program to test myself, would be to test one tag at a time.

Start with a test image that you know that works.  Take the first tag from your csv file and update it manually.  Check to see if it works.  Continue to next image.  Tedious, but it might be the only way as I don't see anything from your posts that seems to be a problem.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

quigley595

Quote from: StarGeek on February 12, 2019, 12:37:06 AM
The only thing I can suggest, since I don't have access to the program to test myself, would be to test one tag at a time.  <snip...  snip.... snip>.

StarGeek...  and again thanks!
Legend!! 

That is exactly what I will do....  one tag at a time
Will post later to let you know how it went

Mike

quigley595

NO...  still not working......   I can change the GPSAltitude tag, but as soon as I try the XMP:AbsoluteAltitude, it all blows up.

I agree that the data looks OK....   must be MMEs parsing of it.....   I will contact them and send them a good image, and a bad one..... lets see what they say..

If I get a positive response, I will post it here...  I think this is important enough for people to know.

Mike

Phil Harvey

Exactly how are you writing XMP:AbsoluteAltitude?  Does it "blow up" if you write the same value with a command like this?:

exiftool "-xmp:absolutealtitude=ORIGINAL_VALUE" FILE

where "ORIGINAL_VALUE" is the value that already existed in the file?  This will test to see if MapsMadeEasy was "blowing up" due to possibly contradictory metadata written to different locations.

If this "blows up", then try extracting the XMP and editing it directly:

exiftool -xmp -b FILE > out.xmp

Then edit "out.xmp" with a text editor to change the value of AbsoluteAltitude, and use this command to put it back:

exiftool "-xmp<=out.xmp" FILE

If this works, then the problem is that MapsMadeEasy is very sensitive to the XMP format.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

quigley595

Quote from: Phil Harvey on February 12, 2019, 06:48:27 AM
Exactly how are you writing XMP:AbsoluteAltitude?  Does it "blow up" if you write the same value with a command like this?:

exiftool "-xmp:absolutealtitude=ORIGINAL_VALUE" FILE   <snip   snip   snip>

Thanks Phil.....    I will run a test using your suggestion.....  but before that...  I have a response from Maps Made Easy support, (I sent them 2 images, both the same image, one the 'original' and the other the 'exiftool changed' version (I can send these to you, but they are 6 MB each)...   Here is the MME reply:

Mike,
It appears that you are changing the text of the XMP tags and that is messing things up. If you were just changing the values in place it would likely be fine.

The Good file has a chunk of text that says the following to describe the altitudes:
drone-dji:AbsoluteAltitude="+1.20"
drone-dji:RelativeAltitude="+38.40"

The Bad file has new tag attributes and looks like this:
<drone-dji:RelativeAltitude>+38.4</drone-dji:RelativeAltitude>

They are also in a different order. Our GSD checking code is looking for the specific way that DJI writes it and changing will prevent that from happening

So the bottom line is that I have 2 issues:
1.   The new tag attributes / text
2.   The tags being in a different order

rgds
Mike

quigley595

Quote from: Phil Harvey on February 12, 2019, 06:48:27 AM
Exactly how are you writing XMP:AbsoluteAltitude?  Does it "blow up" if you write the same value with a command like this?:

exiftool "-xmp:absolutealtitude=ORIGINAL_VALUE" FILE

Hi again Phil....
I cant thank you enough for your time on this, but unfortunately, the code didn't work --- the extract was OK, but the rewrite couldn't update with the changed data.

I have attached 3 files.... 

  • 'Code_and_Results.txt'   which shows the code I used, and the results (output messages) of both the Extract and the Re-write code (obviously run separately of course!  :-) :-)
  • 'out-ORIGINAL.xmp'  the original extracted data file
  • 'out-CHANGED.xmp'  the changed data file to be re-written
I used NotePad to make the change to the data (I initially tried 'TextPad', but it reported a warning: "WARNING: "out-ORIGINAL.xmp" contains characters that do not exist in code page 125 (ANSI Latin l). They will be converted to the system default character, if you click OK."    ---  I of course cancelled, and edited the file with NotePad.

So...  what do you think??? 
I hate asking this because you have already done so much, but if we can get this to work....  rather than extracting every field, can I extract only the 3 fields needed (EXIF:GPSAltitude,  XMP_AbsoluteAltitude  and  XMP:RelativeAltitude),  and do this for ALL the files in the directory, then make the changes and rewrite these 3 fields back  (actually only EXIF:GPSAltitude,  XMP_AbsoluteAltitude need to be written back, since XMP:RelativeAltitude is correct - I use that to determine the others by adding the correct known starting elevation?

rgds
Mike

quigley595

Phil,
I tried this....  (it still didn't work, but it may give some insight)

Started off with  copy of the good (unchanged file) original file
1.  exiftool -XMP:AbsoluteAltitude -b DJI_0874.JPG > out-tags.xmp        then I changed the value, and re-wrote using:
2.  exiftool "-XMP:AbsoluteAltitude<=out-tags.xmp" DJI_0874.JPG
Value changed OK, but MME failed with no XMP data recognised

Reinstated a copy of the good (unchanged file) original file
1.  exiftool -EXIF:GPSAltitude -b DJI_0874.JPG > out-tags.xmp        then I changed the value, and re-wrote using:
2.  exiftool "-EXIF:GPSAltitude<=out-tags.xmp" DJI_0874.JPG
Value changed OK, and MME processed correctly  (obviously, because the XMP data hadn't changed.

I don't know if the above is of any help, but clearly, the writing back of the XMP data is causing a blip in the cosmic forces..  maybe a temporal displacement due to flux capacitor jigawatts being out of sync with the voltron arma-giga-thingy...    or something  :-)  :-) :-)

rgds
Mike