Adding heading, pitch, roll from log file.

[lifeofdave]

I'm experimenting with geotagging including orientation information using exiftool 8.76 and can't seem to get exiftool to add the heading, pitch, roll tags.
The plan is to use an android app to log lat, long, heading, pitch and roll data so I'm doing some tests to see how to structure the data from the app.

It seems logical to use a similar format to the Solmeta Geotagger Pro, especially as exiftool now supports PTNTHPR sentences.
I've copied the log data below from someone else's post and modified the time and date to match the test photo I'm working with.

solmetastyletest.log looks like this:

$GPRMC,121352.01,A,3342.0580,S,15105.1157,E,0.04,210.87,150711,,,A*7F
$PTNTHPR,265.2,N,-7.5,N,0.0,N,A*56

I've saved .ExifTool_config file in my home directory which should add the GPSPitch and GPSRoll exif tags. It's pretty much the same as the exiftool example config and seems to load as I can manually add GPSPitch using:
$ exiftool -GPSPitch=35 IMG_3562.JPG

Then in terminal typing the following:
$ exiftool -geotag solmetastyletest.log IMG_3692.JPG
returns:
"1 image files updated"
Ok so far.
Then typing:
$ exiftool IMG_3692
gives this GPS info:
GPS Version ID                  : 2.3.0.0
GPS Latitude Ref                : South
GPS Latitude                    : 33.700967
GPS Longitude Ref               : East
GPS Longitude                   : 151.085262
GPS Time Stamp                  : 12:13:52
GPS Date Stamp                  : 2011:07:15

GPS Date/Time                   : 2011:07:15 12:13:52Z
GPS Latitude                    : 33.700967 S
GPS Longitude                   : 151.085262 E
GPS Position                    : 33.700967 S, 151.085262 E

So standard GPS info is added successfully but not the heading, pitch and roll.

Have I missed something out/failed to change something important in the PTNTHPR sentence?

Do I need more entries to the log file?

I've had a good google but there isn't much info on adding heading, pitch and roll using exiftool as it's such a new feature.

Any pointers really appreciated.

Thanks
Dave

[Phil Harvey]

Hi Dave,

Interesting.  I can't see how you can add the GPSPitch but you can't read it back again.

Could you post the output from this command?:

exiftool -v2 -GPSPitch=35 IMG_3562.JPG

Then try this command afterward:

exiftool -GPSPitch IMG_3562.JPG

Also, it may be useful if you could send me the GPS log and image file so I can test it here (philharvey66 at gmail.com).

- Phil

[lifeofdave]

Hi Phil

Sorry I think made the last post a little confusing. To test that the config file was being read I manually added GPSPitch to one photo and could successfully read it.

However when I tried to use -geotag to automatically add GPSPitch to another photo it didn't read the PTNTHPR sentence and apply the GPSPitch or GPSRoll tags.

The log file just contains the two lines I posted previously, I'll email it over with the image.

Commands and outputs you asked for:

exiftool -v2 -GPSPitch=35 IMG_3692.JPG
Writing XMP-exif:GPSPitch if tag exists
Writing GPS:GPSPitch
======== IMG_3692.JPG
Rewriting IMG_3692.JPG...
  Editing tags in: APP0 APP1 GPS IFD0 JFIF XMP
  Creating tags in: APP1 GPS IFD0
JPEG APP1 (24862 bytes):
  Rewriting IFD0
  ExifByteOrder = II
  Rewriting ExifIFD
  Rewriting MakerNoteCanon
  Rewriting CanonCameraSettings
  Rewriting CanonFocalLength
  Rewriting CanonShotInfo
  Rewriting CanonCameraInfo550D
  Rewriting CanonFileInfo
  Rewriting CropInfo
  Rewriting CustomFunctions2
  Rewriting CanonCustom2
  Rewriting ProcessingInfo
  Rewriting MeasuredColor
  Rewriting ColorData4
  Rewriting VignettingCorr
  Rewriting VignettingCorr2
  Rewriting LightingOpt
  Rewriting LensInfo
  Rewriting InteropIFD
  Rewriting GPS
    + GPS:GPSPitch = '35'
  Rewriting IFD0
JPEG DQT (130 bytes):
JPEG SOF0:
JPEG DHT (416 bytes):
JPEG SOS
    1 image files updated


and

exiftool -GPSPitch IMG_3562.JPG
GPS Pitch                       : 35


Thanks a lot for your help.

[Phil Harvey]

Thanks for the files.  This looks like a bug in ExifTool.  I'll get back to you when I have a solution, but it is likely that it will require a new release to fix this.  I am currently planning a release for Saturday.

- Phil

[Phil Harvey]

OK.  I found the problem and have implemented a fix.  I'll see about bumping up the release to tomorrow instead of Saturday for you.

Thank you very much for reporting this bug.  I'm embarrassed to say that I think this feature never worked properly because it looks like I broke it with an edit to the code just after I did all of my testing.  I hadn't added a test of this feature to the standard test suite , so I missed this problem when I did my release.  I'll see about adding a test for this to the test suite too.

- Phil

[lifeofdave]

Cheers Phil, I'll test it with the next release..

[G8DHE]

Phil, up till now I had tested the Heading Pitch and Roll data using previous (small) data sets and all had worked well ...... 
Yesterday, was the first live shoot where I was intending to use the data straight from the camera and log files ....... nothing from the $PTNTHPR sentences came out on the first few attempts using the the -geotag option 
So I cut down the file to just the records needed and all worked OK .... 
After a few further attempts it appears that the NMEA sentence order is of importance, in the files that do work they all start;

Code Select Expand

$GPRMC,135249.000,A,5051.3701,N,00035.6736,E,0.18,6.89,200212,,,A*6D
$PTNTHPR,114.7,N,0.4,N,-5.6,N,A*3C
$GPGGA,135255.000,5051.3701,N,00035.6737,E,1,07,1.4,6.7,M,47.0,M,,0000*5E
$GPRMC,135254.000,A,5051.3701,N,00035.6737,E,0.08,353.06,200212,,,A*65
$PTNTHPR,108.4,N,0.9,N,-3.0,N,A*3F
$GPGGA,135300.000,5051.3702,N,00035.6737,E,1,07,1.4,6.5,M,47.0,M,,0000*5E

However it seems if the file starts with a $GPGGA;

Code Select Expand

$GPGGA,134437.006,5051.2651,N,00035.7633,E,1,05,4.2,-35.3,M,47.0,M,,0000*42
$GPRMC,134437.006,A,5051.2651,N,00035.7633,E,0.22,155.52,200212,,,A*6D
$PTNTHPR,245.1,N,-88.2,N,74.3,N,A*39
$GPGGA,134442.000,5051.2725,N,00035.7617,E,1,05,4.2,-24.0,M,47.0,M,,0000*41
$GPRMC,134441.000,A,5051.2719,N,00035.7618,E,0.07,81.61,200212,,,A*51
$PTNTHPR,240.2,N,-82.7,N,74.6,N,A*35

Then whilst the normal GPS fields are picked up OK, the $PTNTHPR sentences seem to be ignored, is this expected behaviour ?
I have also noticed that within the NMEA files (from a Solmeta Pro GPS unit) sometimes sentence types appear to repeat so two $GPRMC records follow each other etc. or possibly another sentence has not been recorded I've not spotted this causing a problem, but given the problem above I wonder if this might also cause problems ?  Example below;

Code Select Expand

$GPRMC,134736.000,A,5051.3277,N,00035.7674,E,2.11,26.24,200212,,,D*58
$PTNTHPR,305.5,N,-35.8,N,-2.8,N,A*23
$GPGGA,134741.000,5051.3304,N,00035.7697,E,1,09,0.9,7.9,M,47.0,M,,*59
$GPRMC,134741.000,A,5051.3304,N,00035.7697,E,2.03,36.67,200212,,,D*55
$GPRMC,134746.000,A,5051.3327,N,00035.7721,E,2.16,34.29,200212,,,D*53
$PTNTHPR,53.8,N,61.3,N,-19.3,N,A*09
$GPGGA,134751.000,5051.3355,N,00035.7733,E,1,09,0.9,10.6,M,47.0,M,,*6A
$GPRMC,134751.000,A,5051.3355,N,00035.7733,E,2.27,19.75,200212,,,D*57
$PTNTHPR,40.3,N,64.4,N,-9.1,N,A*31
$GPGGA,134756.000,5051.3379,N,00035.7761,E,1,09,0.9,11.7,M,47.0,M,,*64
$GPRMC,134756.000,A,5051.3379,N,00035.7761,E,2.03,34.04,200212,,,D*56
$PTNTHPR,54.3,N,51.5,N,-2.9,N,A*30
$GPGGA,134801.000,5051.3401,N,00035.7782,E,1,09,0.9,12.7,M,47.0,M,,*6F
$GPRMC,134801.000,A,5051.3401,N,00035.7782,E,1.83,34.26,200212,,,D*55
$PTNTHPR,48.3,N,36.3,N,23.9,N,A*24
$GPGGA,134806.000,5051.3424,N,00035.7800,E,1,09,0.9,13.1,M,47.0,M,,*6D
$GPRMC,134806.000,A,5051.3424,N,00035.7800,E,2.11,53.56,200212,,,D*5E
$PTNTHPR,25.2,N,77.7,N,7.2,N,A*11
$GPGGA,134811.000,5051.3442,N,00035.7848,E,1,09,0.9,13.3,M,47.0,M,,*65
$GPRMC,134811.000,A,5051.3442,N,00035.7848,E,2.54,60.19,200212,,,D*5E
$PTNTHPR,43.3,N,69.5,N,5.7,N,A*19
$GPGGA,134816.000,5051.3454,N,00035.7895,E,1,09,0.9,12.8,M,47.0,M,,*6F
$GPRMC,134816.000,A,5051.3454,N,00035.7895,E,2.44,70.63,200212,,,D*53
$GPGGA,134821.000,5051.3473,N,00035.7921,E,1,10,0.9,13.5,M,47.0,M,,*64
$GPRMC,134821.000,A,5051.3473,N,00035.7921,E,1.90,12.18,200212,,,D*5E
$PTNTHPR,39.5,N,57.8,N,-6.8,N,A*33
$GPGGA,134826.000,5051.3496,N,00035.7924,E,1,09,0.9,15.8,M,47.0,M,,*6E
$GPRMC,134826.000,A,5051.3496,N,00035.7924,E,1.96,9.63,200212,,,D*67
$PTNTHPR,18.3,N,58.8,N,17.5,N,A*2A
$GPGGA,134831.000,5051.3517,N,00035.7938,E,1,09,0.9,19.6,M,47.0,M,,*6F
$GPRMC,134831.000,A,5051.3517,N,00035.7938,E,1.91,351.89,200212,,,D*69
$PTNTHPR,27.3,N,56.0,N,-5.3,N,A*38
$GPGGA,134836.000,5051.3546,N,00035.7933,E,1,09,0.9,21.9,M,47.0,M,,*63
$GPRMC,134836.000,A,5051.3546,N,00035.7933,E,2.10,353.38,200212,,,D*63
$PTNTHPR,358.3,N,71.1,N,17.3,N,A*1B
$GPGGA,134841.000,5051.3578,N,00035.7931,E,1,09,0.9,22.8,M,47.0,M,,*6E
$GPRMC,134841.000,A,5051.3578,N,00035.7931,E,1.89,4.77,200212,,,D*65
$GPGGA,134846.000,5051.3607,N,00035.7940,E,1,09,0.9,23.0,M,47.0,M,,*6D


[Phil Harvey]

Hi Geoff,

What version of ExifTool are you using?  There were problems like this that were fixed in version 8.77.

But the order is important.  Since there is no "end of fix" mark in NMEA, ExifTool assumes that the starts with the first sentence containing a timestamp, and ends when it encounters this same sentence again.  If there are missing PTNTHPR sentences it will cause missing orientation information to be missing for some fixes.

I tried your first 2 examples, and exiftool loads the orientation information properly.  Use the -v4 option to see the information that ExifTool extracts for each fix.

- Phil

[G8DHE]

OK I'm using 8.79, I'll run some checks with a higher debug level and see what gives.  Thank for the fast response!

[Phil Harvey]

Hi Geoff,

In case it helps, here is an example console output from a test using your 2nd log file example.  It should work on any image since I set the geotime manually:

Code Select Expand

> exiftool -geotag t2.log -v4 -geotime="2012:02:20 13:44:40Z" a.jpg
Loaded 1 points from GPS track log file 't2.log'
Ignored 2 points due to GeoMaxHDOP cut
  GPS track start: 2012:02:20 13:44:41 UTC
    2012:02:20 13:44:41 UTC - dir=240.2 first=1 isDate=1 lat=50.8545316666667 lon=0.59603 pitch=-82.7 roll=74.6 speed=0.07 track=81.61
  GPS track end:   2012:02:20 13:44:41 UTC
Writing File:Geotag
  Geotime value:   2012:02:20 13:44:40 UTC
Writing MIE-GPS:GPSLatitude
Writing XMP-exif:GPSLatitude if tag exists
Writing GPS:GPSLatitude
Writing MIE-GPS:GPSLongitude
Writing XMP-exif:GPSLongitude if tag exists
Writing GPS:GPSLongitude
Deleting MIE-GPS:GPSAltitude
Deleting XMP-exif:GPSAltitude
Deleting GPS:GPSAltitude
Deleting XMP-exif:GPSAltitudeRef
Deleting GPS:GPSAltitudeRef
Writing MIE-GPS:GPSTrack
Writing XMP-exif:GPSTrack if tag exists
Writing GPS:GPSTrack
Writing XMP-exif:GPSTrackRef if tag exists
Writing GPS:GPSTrackRef
Writing MIE-GPS:GPSSpeed
Writing XMP-exif:GPSSpeed if tag exists
Writing GPS:GPSSpeed
Writing XMP-exif:GPSSpeedRef if tag exists
Writing GPS:GPSSpeedRef
Writing XMP-exif:GPSImgDirection if tag exists
Writing GPS:GPSImgDirection
Writing XMP-exif:GPSImgDirectionRef if tag exists
Writing GPS:GPSImgDirectionRef
Writing XMP-exif:GPSPitch if tag exists
Writing GPS:GPSPitch
Writing XMP-exif:GPSRoll if tag exists
Writing GPS:GPSRoll
Writing GPS:GPSLatitudeRef
Writing GPS:GPSLongitudeRef
Writing GPS:GPSDateStamp
Writing GPS:GPSTimeStamp
Writing XMP-exif:GPSDateTime if tag exists
======== a.jpg
Rewriting a.jpg...
  Editing tags in: APP0 APP1 File GPS IFD0 JFIF MIE-GPS XMP
  Creating tags in: APP1 File GPS IFD0 MIE-GPS
Creating APP1:
  Creating IFD0
  ExifByteOrder = MM
    + IFD0:XResolution = '72' (mandatory)
    + IFD0:YResolution = '72' (mandatory)
    + IFD0:ResolutionUnit = '2' (mandatory)
    + IFD0:YCbCrPositioning = '1' (mandatory)
  Creating GPS
    + GPS:GPSVersionID = '2 3 0 0' (mandatory)
    + GPS:GPSLatitudeRef = 'N'
    + GPS:GPSLatitude = '50 51 16.3139999999994'
    + GPS:GPSLongitudeRef = 'E'
    + GPS:GPSLongitude = '0 35 45.7080000000001'
    + GPS:GPSTimeStamp = '13 44 40'
    + GPS:GPSSpeedRef = 'N'
    + GPS:GPSSpeed = '0.07'
    + GPS:GPSTrackRef = 'T'
    + GPS:GPSTrack = '81.61'
    + GPS:GPSImgDirectionRef = 'T'
    + GPS:GPSImgDirection = '240.2'
    + GPS:GPSDateStamp = '2012:02:20'
    + GPS:GPSPitch = '-82.7'
    + GPS:GPSRoll = '74.6'
JPEG DQT (65 bytes):
JPEG DQT (65 bytes):
JPEG SOF0:
JPEG DHT (29 bytes):
JPEG DHT (179 bytes):
JPEG DHT (29 bytes):
JPEG DHT (179 bytes):
JPEG SOS
    1 image files updated

Remember that you need the custom tags defined in your config file to be able to write GPSPitch and GPSRoll.

- Phil

[G8DHE]

Hi Phil, OK your results noted attached is a zip'ed set of files I used with the Solmeta-xxx.nmea as downloaded complete (starts with $GPGGA sentence), and then with records pruned off from the start, the results of running;
exiftool -v4 -geosync=-16 -geotag *.nmea dsc_0506.nef
the resulting console log file 0506-v4-GPGGA.txt
removing the first  $GPGGA record so .nmea file starts with $GPRMC sentence, results in 0506-v4-GPRMC.txt
and finally removing the first $GPRMC sentence .nmea file starts with $PTNTHPR sentence results in 0506-v4-PTNTHPR.txt
as you will see when the file starts with a GPGGA record no $PTNTHPR sentence data is written, when it starts with anything else it works as expected ... ?
What am I missing ?

[Phil Harvey]

Wow, what a mess.

1) The NMEA is not in sequence.  The time jumps backward by 1 second between the GPGGA and GPRMC sentences.  The ExifTool logic  assumeshat the sentences are always in the correct temporal sequence (and in fact, that the GPGGA and GPRMC should both correspond to the same time fix).

2) The PTNTHPR sentence is sometimes missing, and sometimes corrupted.  The corrupted ones will cause problems because ExifTool doesn't currently test the checksums.

My suggestion is to do some pre-processing of the NMEA log:

a) Remove the GPGGA sentences entirely (you will only lose the altitude information)

b) Remove any corrupted PTNTHPR sentences.

c) Remove any GPRMC which is followed by anything other than a PTNTHPR.

But if the altitude is a problem, then this solution isn't optimal.  Also, the pre-processing isn't optimal.  I'll think about this to see if there is anything I can do to improve this for future versions, but for now you would be stuck doing some pre-processing.

- Phil

[Phil Harvey]

OK.  I've figured out a way to handle missing and out-of-sequence NMEA information like this.

I still need to do some work before I can say for sure, but I think I should be able to include this improvement in the next release (ExifTool 8.80).

I'll let you know how it goes.

Thanks for this report, and for the sample NMEA file.

- Phil

[G8DHE]

Hi Phil, Yes its not that nice is it :-( asking around there are some suggestions as to why, like;
Time to process different sentences when EGNOS/WAAS/MSAS is used,
Pushing data onto a stack before Pulling it off and hence in reverse time order etc.
Trouble is without knowing the underlying code its difficult to be sure of the cause. 
As not all sentences contain timestamps its not as simple either as simply resorting it either !

[Phil Harvey]

Hi Geoff,

I've got a solution implemented now.  I just need to do some more testing, but it looks good so far.  If all goes well, I'll release 8.80 this weekend with this update.

Supporting the yaw/pitch/roll is turning out to be more work than I had anticipated.   However, on the plus side, the NMEA reader should be more robust now for anyone else with erratic NMEA like this.

- Phil