Help modifying geotiff information

Started by salleman, March 04, 2020, 10:36:46 AM

Previous topic - Next topic

salleman

I've read through a few threads and tried copying the geotiff exif data from a known good file to one with an incorrect projection.

I used the following command:

exiftool -tagsfromfile SC_02_correctEPSG.tif  -GeoTiffDirectory -GeoTiffDoubleParams -GeoTiffAsciiParams SC_02.tif

that I found in a thread in this forum:

https://exiftool.org/forum/index.php?topic=10685.msg56716#msg56716

The problem is that exif happily reads the updated file and shows me exactly what I want to see... but if I open up SC_02.tif in a hex editor or GlobalMapper, it appears that the exif data is corrupted.

Here's the exif data from the original SC_02.tif:

ExifTool Version Number         : 11.90
File Name                       : SC_02.tif_original
Directory                       : .
File Size                       : 67 MB
File Modification Date/Time     : 2020:03:02 13:19:21-06:00
File Access Date/Time           : 2020:03:02 13:19:21-06:00
File Inode Change Date/Time     : 2020:03:04 07:31:13-06:00
File Permissions                : rwxrwxrwx
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 4190
Image Height                    : 4190
Bits Per Sample                 : 8 8 8 8
Compression                     : Uncompressed
Photometric Interpretation      : RGB
Strip Offsets                   : (Binary data 37042 bytes, use -b option to extract)
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 4
Rows Per Strip                  : 1
Strip Byte Counts               : (Binary data 25139 bytes, use -b option to extract)
Planar Configuration            : Chunky
Extra Samples                   : Unassociated Alpha
Pixel Scale                     : 0.0128746620000136 0.0120559770000436 1
Model Tie Point                 : 0 0 0 531697.611622669 4269956.09045799 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_North_American_1983
Geographic Type                 : NAD83
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : NAD83 UTM zone 18N
PCS Citation                    : NAD_1983_UTM_Zone_18N
Proj Linear Units               : Linear Meter
Image Size                      : 4190x4190
Megapixels                      : 17.6


Here is the update exif info:

ExifTool Version Number         : 11.90
File Name                       : SC_02.tif
Directory                       : .
File Size                       : 67 MB
File Modification Date/Time     : 2020:03:04 09:19:40-06:00
File Access Date/Time           : 2020:03:04 09:19:40-06:00
File Inode Change Date/Time     : 2020:03:04 09:19:41-06:00
File Permissions                : rwxrwxrwx
File Type                       : TIFF
File Type Extension             : tif
MIME Type                       : image/tiff
Exif Byte Order                 : Little-endian (Intel, II)
Image Width                     : 4190
Image Height                    : 4190
Bits Per Sample                 : 8 8 8 8
Compression                     : Uncompressed
Photometric Interpretation      : RGB
Strip Offsets                   : (Binary data 37052 bytes, use -b option to extract)
Orientation                     : Horizontal (normal)
Samples Per Pixel               : 4
Rows Per Strip                  : 1
Strip Byte Counts               : (Binary data 25139 bytes, use -b option to extract)
Planar Configuration            : Chunky
Extra Samples                   : Unassociated Alpha
Pixel Scale                     : 0.0128746620000136 0.0120559770000436 1
Model Tie Point                 : 0 0 0 531697.611622669 4269956.09045799 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_NAD_1983_2011
Geographic Type                 : Unknown (6318)
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : Unknown (6347)
PCS Citation                    : NAD83_2011_UTM_zone_18N
Proj Linear Units               : Linear Meter
Image Size                      : 4190x4190
Megapixels                      : 17.6

I'm attaching pictures of before and after views of that file in a hex editor...




Phil Harvey

Try looking at the exiftool -htmldump output in a web browser instead of using the hex editor:

exiftool -htmldump FILE > out.html

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

It looks fine in html... but GlobalMapper is not recognizing the projection data... Any idea why that might be? Also, the projection info went from the end of the file to near the beginning...

salleman

Here's a diff between the old and new files:

< Directory                       : .
---
> Directory                       : test
5,7c5,7
< File Modification Date/Time     : 2020:03:02 13:21:06-06:00
< File Access Date/Time           : 2020:03:02 13:21:06-06:00
< File Inode Change Date/Time     : 2020:03:03 18:01:23-06:00
---
> File Modification Date/Time     : 2020:03:04 07:36:26-06:00
> File Access Date/Time           : 2020:03:04 07:36:26-06:00
> File Inode Change Date/Time     : 2020:03:04 07:36:41-06:00
18c18
< Strip Offsets                   : (Binary data 36400 bytes, use -b option to extract)
---
> Strip Offsets                   : (Binary data 36410 bytes, use -b option to extract)
30,31c30,31
< GT Citation                     : GCS_North_American_1983
< Geographic Type                 : NAD83
---
> GT Citation                     : GCS_NAD_1983_2011
> Geographic Type                 : Unknown (6318)
37,38c37,38
< Projected CS Type               : NAD83 UTM zone 18N
< PCS Citation                    : NAD_1983_UTM_Zone_18N
---
> Projected CS Type               : Unknown (6347)
> PCS Citation                    : NAD83_2011_UTM_zone_18N

Phil Harvey

Could you email me SC_02_correctEPSG.tif so I can try this myself?  philharvey66 at gmail.com

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).


Phil Harvey

Everything works fine when I use your command (and ExifTool 11.90) to copy the geotiff tags from your SC_02.tif.bak file:

> exiftool tmp/* -geotiff:all
======== tmp/SC_02.tif
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_NAD_1983_2011
Geographic Type                 : Unknown (6318)
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : Unknown (6347)
PCS Citation                    : NAD83_2011_UTM_zone_18N
Proj Linear Units               : Linear Meter
======== tmp/SC_02.tif.bak
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_North_American_1983
Geographic Type                 : NAD83
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : NAD83 UTM zone 18N
PCS Citation                    : NAD_1983_UTM_Zone_18N
Proj Linear Units               : Linear Meter
======== tmp/SC_02_correctEPSG.tif
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_NAD_1983_2011
Geographic Type                 : Unknown (6318)
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : Unknown (6347)
PCS Citation                    : NAD83_2011_UTM_zone_18N
Proj Linear Units               : Linear Meter
    3 image files read



> exiftool -tagsfromfile tmp/SC_02.tif.bak -GeoTiffDirectory -GeoTiffDoubleParams -GeoTiffAsciiParams tmp/SC_02.tif
    1 image files updated



>exiftool tmp/* -geotiff:all
======== tmp/SC_02.tif
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_North_American_1983
Geographic Type                 : NAD83
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : NAD83 UTM zone 18N
PCS Citation                    : NAD_1983_UTM_Zone_18N
Proj Linear Units               : Linear Meter
======== tmp/SC_02.tif.bak
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_North_American_1983
Geographic Type                 : NAD83
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : NAD83 UTM zone 18N
PCS Citation                    : NAD_1983_UTM_Zone_18N
Proj Linear Units               : Linear Meter
======== tmp/SC_02.tif_original
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_NAD_1983_2011
Geographic Type                 : Unknown (6318)
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : Unknown (6347)
PCS Citation                    : NAD83_2011_UTM_zone_18N
Proj Linear Units               : Linear Meter
======== tmp/SC_02_correctEPSG.tif
TEST 0 0
Geo Tiff Version                : 1.1.0
GT Model Type                   : Projected
GT Raster Type                  : Pixel Is Area
GT Citation                     : GCS_NAD_1983_2011
Geographic Type                 : Unknown (6318)
Geog Angular Units              : Angular Degree
Geog Ellipsoid                  : GRS 1980
Geog Semi Major Axis            : 6378137
Geog Semi Minor Axis            : 6356752.31414036
Geog Inv Flattening             : 298.257222101
Projected CS Type               : Unknown (6347)
PCS Citation                    : NAD83_2011_UTM_zone_18N
Proj Linear Units               : Linear Meter
    4 image files read


- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

Right, it looks fine in the exif output. I'm not sure what's happening, but GlobalMapper can open the SC_02_correctEPSG.tiff and knows where in the world that data is, but when it opens SC_02.tiff after copying the metadata to it from SC_02_correctEPSG.tiff, it no longer can identify where the data should be. Is it possibly because there's a size mismatch between the two projection strings? Is that perhaps causing some sort of corruption?

Phil Harvey

I don't know why GlobalMapper has problems with the ExifTool-copied file, but as far as I can tell
GeoTiff information looks good.  However, there are a few values in SC_02_correctEPSG.tiff that aren't valid according to the GeoTiff specification that ExifTool follows (the Unknown values).

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

For the past few minutes, I've been trying to use listgeo and geotifcp... I'm not having any luck copying the data that I need, but here's the output of listgeo from SC_02.tif and SC_02_correctEPSG.tif immediately after running:
exiftool -tagsfromfile SC_02_correctEPSG.tif  -GeoTiffDirectory -GeoTiffDoubleParams -GeoTiffAsciiParams SC_020.tif

PS G:\alleman\SSS_Geotif> G:\alleman\SSS_Geotif\listgeo.exe .\SC_02.tif
.\SC_02.tif: Warning, wrong data type 7 for "GeoKeyDirectory"; tag ignored.
.\SC_02.tif: Warning, wrong data type 7 for "GeoDoubleParams"; tag ignored.
Geotiff_Information:
   Version: 1
   Key_Revision: 1.0
   Tagged_Information:
      ModelTiepointTag (2,3):
         0                0                0
         531697.612       4269956.09       0
      ModelPixelScaleTag (1,3):
         0.012874662      0.012055977      1
      End_Of_Tags.
   Keyed_Information:
      End_Of_Keys.
   End_Of_Geotiff.


Corner Coordinates:
Upper Left    ( 531697.612,4269956.090)
Lower Left    ( 531697.612,4269905.576)
Upper Right   ( 531751.556,4269956.090)
Lower Right   ( 531751.556,4269905.576)
Center        ( 531724.584,4269930.833)

-----------------------
PS G:\alleman\SSS_Geotif> G:\alleman\SSS_Geotif\listgeo.exe .\SC_02_correctEPSG.tif
Geotiff_Information:
   Version: 1
   Key_Revision: 1.0
   Tagged_Information:
      ModelTiepointTag (2,3):
         0                0                0
         531697.612       4269956.09       0
      ModelPixelScaleTag (1,3):
         0.012055977      0.012055977      1
      End_Of_Tags.
   Keyed_Information:
      GTModelTypeGeoKey (Short,1): ModelTypeProjected
      GTRasterTypeGeoKey (Short,1): RasterPixelIsArea
      GTCitationGeoKey (Ascii,18): "GCS_NAD_1983_2011"
      GeographicTypeGeoKey (Short,1): Unknown-6318
      GeogAngularUnitsGeoKey (Short,1): Angular_Degree
      GeogEllipsoidGeoKey (Short,1): Ellipse_GRS_1980
      GeogSemiMajorAxisGeoKey (Double,1): 6378137
      GeogSemiMinorAxisGeoKey (Double,1): 6356752.31
      GeogInvFlatteningGeoKey (Double,1): 298.257222
      ProjectedCSTypeGeoKey (Short,1): Unknown-6347
      PCSCitationGeoKey (Ascii,24): "NAD83_2011_UTM_zone_18N"
      ProjLinearUnitsGeoKey (Short,1): Linear_Meter
      End_Of_Keys.
   End_Of_Geotiff.

PCS = 6347 (name unknown)
GCS: 6318/(unknown)
Ellipsoid: 7019/GRS 1980 (6378137.00,6356752.31)
Projection Linear Units: 9001/metre (1.000000m)

Corner Coordinates:
Upper Left    ( 531697.612,4269956.090)
Lower Left    ( 531697.612,4269905.576)
Upper Right   ( 531751.562,4269956.090)
Lower Right   ( 531751.562,4269905.576)
Center        ( 531724.587,4269930.833)

salleman

I also just noticed that it's copying the coordinates of the four corners... That is a problem. Essentially, I just need to get the correct projection from SC_02_correctEPSG.tif and put it into SC_02.tif without changing anything else.

Phil Harvey

OK.  There is another difference that your output revealed:

Your SC_02.tif_original has the wrong EXIF format type for these tags (undef instead of double, etc):

> exiftool SC_02.tif_original -validate -warning
Validate                        : 5 Warnings
Warning                         : Non-standard format (undef) for IFD0 0x87af GeoTiffDirectory
Warning                         : Non-standard format (undef) for IFD0 0x87b0 GeoTiffDoubleParams
Warning                         : Missing required TIFF IFD0 tag 0x011a XResolution
Warning                         : Missing required TIFF IFD0 tag 0x011b YResolution
Warning                         : Missing required TIFF IFD0 tag 0x0128 ResolutionUnit


When you use ExifTool to copy to this file, the format isn't changed, so it remains incorrect.  But if you first delete the geotiff tags before copying them, like this:

exiftool -GeoTiffDirectory= -GeoTiffDoubleParams= -GeoTiffAsciiParams= SC_02.tif

the formats will be set correctly when the tags are subsequently copied using ExifTool.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

Great! Is there a commandline argument to tell exiftool to not copy the corner coordinates? Or to extract those coordinates and put them back in after the copy?

Phil Harvey

Ah.  Those are determined by the PixelScale, ModelTiePoint and ModelTransform tags, which you weren't copying.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman


salleman

I'm still not able to get what I need...

Is there a way to extract the projection information in ascii format, modify it to suit my needs, and re-inject it back into the tif file?

Also, can exiftool read projection information in from an ascii file created by listgeo?

salleman

And does this error message from geotifcp mean anything to you?

test> G:\alleman\SSS_Geotif\geotifcp.exe -g SC_03.md SC_03.tif SC_03_fixed.tif                                                                                           
SC_03.tif: Warning, wrong data type 7 for "GeoKeyDirectory"; tag ignored.                                                                       
SC_03.tif: Warning, wrong data type 7 for "GeoDoubleParams"; tag ignored.                                                                       
Failure in GTIFImport                                                                                       

Phil Harvey

Quote from: salleman on March 04, 2020, 01:45:12 PM
Is there a way to extract the projection information in ascii format, modify it to suit my needs, and re-inject it back into the tif file?

Yes.  Just read/write those tags normally. ie)

exiftool -pixelscale="0.0128746620000136 0.0120559770000436 1" -modeltiepoint="0 0 0 531697.611622669 4269956.09045799 0" FILE

QuoteAlso, can exiftool read projection information in from an ascii file created by listgeo?

Only if it is in a supported format (ie. json or xml)

Quote from: salleman on March 04, 2020, 01:48:27 PM
And does this error message from geotifcp mean anything to you?

test> G:\alleman\SSS_Geotif\geotifcp.exe -g SC_03.md SC_03.tif SC_03_fixed.tif                                                                                           
SC_03.tif: Warning, wrong data type 7 for "GeoKeyDirectory"; tag ignored.                                                                       
SC_03.tif: Warning, wrong data type 7 for "GeoDoubleParams"; tag ignored.                                                                       
Failure in GTIFImport                                                                                       

Yes.  See my post starting with "OK.  There is another difference that your output revealed:".

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

I think I misunderstood something.

Was I supposed to delete the geotiff tags from the file that I was fixing or the one that was more or less correct?

salleman

so, I ran the command that you gave me to delete the tags, but I'm still getting the same errors...

I apologize for being a pest, but I need to get this data fixed as soon as possible.

Phil Harvey

You were meant to delete the tags from the target file before copying.

You can use this command to see if the tags have the wrong format type:

exiftool -validate -warning -a FILE

But we still don't know if this is why GlobalMapper isn't recognizing the GeoTiff information.  Are you also setting the PixelScale and ModelTiePoint tags properly?

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

That's a good question, and not one I can really answer. The image files in question were generated by Chesapeake Technology SonarWiz... well, the ones that needed to be fixed were. I believe the one that I have with the correct projection information was generated by Blue Marble GlobalMapper.  I believe that file was pulled into GlobalMapper and re-exported with the correct projection information.

Deleting the tags from the target file didn't seem to do anything. When I copied the geo information from the good file, it ended up with the same results as before.

I'll run the validation right now. The first file "SC_04.tif" is after copying the geo data from SC_02_correctEPSG. The second file is SC_02_correctEPSG.tif.

/s/.../19031638_Orsted_SJW19_ENT/.../test $ exiftool -validate -warning -a SC_04.tif

Validate                        : 7 Warnings (5 minor)
Warning                         : [minor] Non-standard IFD0 tag 0x830e PixelScale
Warning                         : [minor] Non-standard IFD0 tag 0x8482 ModelTiePoint
Warning                         : [minor] Non-standard IFD0 tag 0x87af GeoTiffDirectory
Warning                         : [minor] Non-standard IFD0 tag 0x87b0 GeoTiffDoubleParams
Warning                         : [minor] Non-standard IFD0 tag 0x87b1 GeoTiffAsciiParams
Warning                         : Missing required IFD0 tag 0x011a XResolution
Warning                         : Missing required IFD0 tag 0x011b YResolution


/s/.../19031638_Orsted_SJW19_ENT/.../test $ exiftool -validate -warning -a SC_02_correctEPSG.tif

Validate                        : 5 Warnings (2 minor)
Warning                         : [minor] Non-standard IFD0 tag 0x830e PixelScale
Warning                         : [minor] Non-standard IFD0 tag 0x8482 ModelTiePoint
Warning                         : Missing required IFD0 tag 0x011a XResolution
Warning                         : Missing required IFD0 tag 0x011b YResolution
Warning                         : Invalid count for IFD0 tag 0x0140 ColorMap

Phil Harvey

One obvious problem here... What version of ExifTool are you using?  You should update to the current version (11.90)

- Phil

Edit: Hmmm.  The output in your first post is from 11.90, but the output in your last post is from a much older version
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

I appear to be running 10.8... I'll download the source and compile 11.9.

salleman

It looks like the Ubuntu repository version is 10.8? Anyway, here's updated output.

/s/.../19031638_Orsted_SJW19_ENT/.../test $ /home/sja/Image-ExifTool-11.90/exiftool -validate -warning -a ./SC_02_correc
tEPSG.tif

Validate                        : 3 Warnings
Warning                         : Missing required TIFF IFD0 tag 0x011a XResolution
Warning                         : Missing required TIFF IFD0 tag 0x011b YResolution
Warning                         : Missing required TIFF IFD0 tag 0x0128 ResolutionUnit

/s/.../19031638_Orsted_SJW19_ENT/.../test $ /home/sja/Image-ExifTool-11.90/exiftool -validate -warning -a ./SC_02.tif

Validate                        : 5 Warnings
Warning                         : Non-standard format (undef) for IFD0 0x87af GeoTiffDirectory
Warning                         : Non-standard format (undef) for IFD0 0x87b0 GeoTiffDoubleParams
Warning                         : Missing required TIFF IFD0 tag 0x011a XResolution
Warning                         : Missing required TIFF IFD0 tag 0x011b YResolution
Warning                         : Missing required TIFF IFD0 tag 0x0128 ResolutionUnit

Phil Harvey

Right, Wow, that's old.

OK, so now you have to repeat all your tests writing the GeoTiff tags using the current version.  That old version may not have been writing things correctly.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

salleman

YAY!!!! So the whole problem was that I was using an outdated version of exiftool? Because I reran all the same commands and everything looks perfect! 8) 8) 8) 8) 8) 8) 8) 8) 8)

Phil Harvey

Yes, that was likely the whole problem.  I do remember some fixes involving writing GeoTiff information, but that was a long while back and I can't recall the details.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).