enhancement: extract binary data from FLIR radiometric jpg

Started by tomas123, March 20, 2013, 12:49:46 PM

Previous topic - Next topic

Phil Harvey

Wow.  Good find.  I googled a bit too, but didn't find this.

It describes the binary format of a ".IMG" file.  Too bad it wasn't ".FFF".  However, the similarities in the data structures may be useful, and the documentation looks helpful.

I'm just about to release ExifTool 9.54, which will decode the 5 new parameters.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

jskala

Hi, I can confirm the values of alpha, beta, and X are constant. With one exception - the SC7650 camera. I have an AFF file from it and it has the following values:

d5 cf 9b bb (alfa1)
a8 c6 4b 3d (alfa2)
86 e3 79 3b (beta1)
c9 01 bb 3b (beta2)
9a 99 19 3f (X)

Which is approx. -0.0048; 0.0498; 0.0038; 0.0057; 0.6. For the parameters 20C, 50%, 10m, the tau results in 0.91. Perhaps the camera works on a different wavelengths and the attenuation is different?

Phil Harvey

I don't have an SC7650 sample image.  Would it be possible for you to post one?

I have only one .seq file from a SC4000 with an AFF header, but we are not yet able to decode it.  I'm assuming that this is the same format as your SC7650 AFF file.  It would be useful if you could provide any information about the format of this file.

Thanks.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

jskala

I'm able to decode two types of AFF format from different cameras/software (they differ slightly). Most of my work is based on your research of the FFF format. I thought you were not interested in AFF. I will make sure I can publish the .seq files I'm working with and I'll get back here soon.

jskala

#124
So here are my findings about the AFF format. The information about FFF in this forum helped me tremendously, thank You guys. I also got some hints from the book Infrared thermography: Errors and uncertainties (preview available at Google books). There is always a file header at the beginning of the file, and then a frame header at the beginning of each frame.

The file header is as follows (the same as FFF):

0x00: (text) AFF
0x04: (text) file creator, 16 bytes
0x14: (uint32) format (100)
0x18: (uint32) records address (0x40)
0x1c: (uint32) record count (1)
0x20: (uint32) next free record ID (2)
0x24: (uint16) swap pattern? (0)
0x26: (7 * uint16) spares (0)
0x34: (2 * uint32) reserved (0)
0x3c: (uint32) checksum (can be zero)


Apparently, the frames differ quite a bit by version. I can decode two. Please note that the file can contain data from a sensor say 327x245, and a 320x240 portion is specified to be actually used. You will see that at the addresses 0x62-0x71.

FLIR SC2000

0x40: (uint16) type (1)
0x42: (uint16) subtype (3)
0x44: (uint32) version (103)
0x48: (uint32) record ID (1)
0x4c: (uint32) record data offset (0x60)
0x50: (uint32) record length in bytes
0x54: unknown (0)
0x58: (uint32) object ID? (65)
0x5c: unknown (0)

0x60: (uint16) byte order (2)
0x62: (uint16) sensor width in pixels (327)
0x64: (uint16) sensor height in pixels (245)
0x66-0x69: unknown
0x6a: (uint16) first "proper" pixel on a scan line (4)
0x6c: (uint16) last "proper" pixel on a scan line (323)
0x6e: (uint16) first "proper" scan line (2)
0x70: (uint16) last proper scan line (241)
0x72-0x8b: unknown

0x8c: (single) emissivity
0x90: (single) distance
0x94: (single) reflected temperature (also used as window temperature?)
0x98: unknown (0)
0x9c: (single) atmospheric temperature
0xa0: (single) relative humidity

0xa4: (single) R
0xa8: (single) B
0xac: (single) F

0xb0: (single) alpha 1
0xb4: (single) alpha 2
0xb8: (single) beta 1
0xbc: (single) beta 2
0xc0: (single) X

0xc4: (single) camera range max
0xc8: (single) camera range min
... camera name, serial nr., lens, filter, calibration date
0x016c: (int32) offset (aka O)
0x0170: (single) gain (aka R2)
... unknown
0x01d8: (text) palette name, 16 bytes
0x01e8: (uint32) UNIX timestamp
0x01ec: (uint16) milliseconds
... unknown
0x0298: (uint16) start of raw data


PH Edit: fixed typo in offset

jskala

A615

0x40: (uint16) type (5)
0x42: (uint16) subtype (3)
0x44: (uint32) version (103)
0x48: (uint32) record ID (1)
0x4c: (uint32) record data offset (0x60)
0x50: (uint32) record length in bytes
0x54: unknown (0)
0x58: (uint32) object ID?
0x5c: unknown (0)

0x60: (text) FF2
... unknown (mostly zeros)
0x84: (uint16) byte order (2)
0x86: (uint16) sensor width in pixels (640)
0x88: (uint16) sensor height in pixels (485)
0x8a-0x8d: unknown
0x8e: (uint16) first "proper" pixel on a scan line (0)
0x90: (uint16) last "proper" pixel on a scan line (639)
0x92: (uint16) first "proper" scan line (0)
0x94: (uint16) last "proper" scan line (479)
... unknown (mostly zeros)

0xb4: (single) emissivity
0xb8: (single) reflected temperature
0xbc: (single) atmospheric temperature
0xc0: (single) distance
0xc4: (single) relative humidity
0xc8: (single) tau
0xcc: (single) window temperature
0xd0: (single) window transmissivity
... unknown (mostly zeros)

0xec: (single) R
0xf0: (single) B
0xf4: (single) F
... unknown (mostly zeros)
0x015c: (single) alpha 1
0x0160: (single) alpha 2
0x164: (single) beta 1
0x168: (single) beta 2
0x16c: (single) X
... unknown (zeros)
0x0188: (single) camera range max
... unknown temperatures
0x0198: (single) camera range min
... unknown
0x02ce: (single) gain (aka R2)
0x0302: (single) -offset (aka -O) negative! stored as float!
... unknown (zeros)
0x036a: (uint16) frame data min
0x036e: (uint16) frame data range
... unknown (zeros)
0x0386: (uint16) image width
0x0388: (uint16) image height
... unknown (zeros)
0x03ce: (uint64) Windows NT timestamp
... unknown (mostly zeros)
0x04ac: (uint16) start of raw data
after the raw data, there is an FFF style footer

Phil Harvey

#126
Excellent, thanks!

Would it be possible to email me a couple of samples? (philharvey66 at gmail.com)  That would give me something to play with when I get a chance.

- Phil

Edit: I see you already sent me a private message with some files, thanks!!
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Videogamer555

Where in an FFF file are the O, B, R1, R2, F constants stored? Can you tell me what the offsets are within the data info header? I know that in an FFF file there's the main header, then an index, and then various chunks (headers and other data) pointed to by the index. One of these is a data info chunk that has metadata about the thermal data (metadata = info about data). Some of this metadata is stuff like camera name, but other parts are the calibration constants (O, B, R1, R2, F, and possibly others) that are very important in converting the ADC data (which are values between 0x0000 and 0xFFFF) into actual usable Kelvin temperature units. I need to know within the metadata chunk offsets (relative to the start of the metadata chunk) where I will find these important constants. Please let me know.

Phil Harvey

Did you see the ExifTool FFF tags documentation?  The constants you want are in the CameraInfo data, and the offsets are listed in the documentation.  The only trick is parsing the FFF format to find the starts of the various data records.  If you want example code, see the ProcessFLIR subroutine in lib/Image/ExifTool/FLIR.pm of the full ExifTool distribution.  It is well commented, and should be self-explanatory.

Also see this document on the FFF format, which is referenced from FLIR.pm.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Videogamer555

#129
Thanks for the info, but what do the table headers Index1 and Index2 mean on all those tables on the page you linked me to?

I would like to point out that your RawData documentation https://exiftool.org/TagNames/FLIR.html#RawData is NOT accurate.

By my own reverse engineering of the FFF format, I find that there are more than 3 header fields in a RawData chunk. There's also 4 fields in the RawData chunk's header for defining a cropping box (tells FLIR software to display only the portion of the image that is inside this region), and there's also a zoom field for telling FLIR software how much to magnify the image when it displays it.

Also, there is a field at the very start of the RawData chunk that is I think is a 16 bit integer version number, but is used to tell the endianness of the 16 bit image data which is particularly useful for PNG compressed data, as PNG does not inherently have a was of showing endianess of the image data. Also there's a field for telling the data type of the image (whether it's a 16bit integer or something else). Although the stuff in this paragraph I'm a bit less sure of, and I'd have to go through my notes to be sure about this.

Also I would like to point out something else that is missing from your documentation on the CameraInfo section https://exiftool.org/TagNames/FLIR.html#CameraInfo that is that at the very beginning of the CameraInfo section, there is a duplicate of the header from the RawData section (same image width, height, cropping, etc). In fact, if my memory serves, FLIR software will ignore the header of the RawData chunk if there is a CameraInfo section in the FFF file (I found changing values in only one of these copies of the image header actually had any effect when viewed in FLIR software).

Also, out of curiosity, just what is usSwapPattern field in the main FFF header used for? I've never figured that out.

tomas123

Quote from: Videogamer555 on April 06, 2014, 12:49:22 PM
By my own reverse engineering of the FFF format, I find that there are more than 3 header fields in a RawData chunk. There's also 4 fields in the RawData chunk's header for defining a cropping box (tells FLIR software to display only the portion of the image that is inside this region), and there's also a zoom field for telling FLIR software how much to magnify the image when it displays it.

Thanks for your advice.
I know this (and another unpublished) tags but I think it's not worth for Phil to decode all Flir informations. (For PiP calculation you can use the tag 0x002a "FLIR PiP Tags")
My intention is a work of proof for decoding the importend tags (RAW values and constants for temperature calculations).
The Flir segment in Exiftool is a hack and not a Flir documentation  ;)

see my post here with samples for using the known tags:
http://www.eevblog.com/forum/testgear/flir-e4-thermal-imaging-camera-teardown/msg342072/#msg342072



If you need a special value you can simple extend exiftool with a config file.
Post the config file here then I can verify the informations.

see my config samples:
https://exiftool.org/forum/index.php/topic,4898.msg27532.html#msg27532
or
https://exiftool.org/forum/index.php/topic,4898.msg24156.html#msg24156

Quote from: Videogamer555 on April 06, 2014, 12:49:22 PM
Also, out of curiosity, just what is usSwapPattern field in the main FFF header used for? I've never figured that out.
see sample image data base is a good entry point
https://exiftool.org/sample_images.html

Phil Harvey

#131
The Index1/Index2 meanings are explained on the main TagNames documentation page.

You say that the RawData documentation is not accurate.  Is any of the decoded information not correct?  I know that the decoding is not complete, but realistically there is no way that it ever could be.  However, if there are any useful tags that ExifTool is not decoding, please give me the details and I will add them.  ExifTool does use the first 2 bytes of the RawData record to determine the byte order, but this isn't extracted as a tag because it is a formatting detail and isn't useful information for the end user.

- Phil

Edit: Tomas points out that this is not FLIR documentation, which is true, but please let me know if there is any better documentation out there.
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

tomas123

Quote from: Phil Harvey on April 07, 2014, 07:25:54 AM
ExifTool does use the first 2 bytes of the RawData record to determine the byte order, but this isn't extracted as a tag because it is a formatting detail and isn't useful information for the end user.

Videogamer555 mean this tag in the header from the linked FFF document
Quote from: Phil Harvey on April 05, 2014, 04:52:37 PM
Also see this document on the FFF format, which is referenced from FLIR.pm.

Quotetypedef struct tagFLIRFILEHEAD
{
   char szFormatID[4]; /* Fileformat ID 'FFF\0' 4 4 */
   char szOrigin[16]; /* File origin 16 20 */
   unsigned long dwVersion; /* File format version 4 24 */
   unsigned long dwIndexOff; /* Pointer to indexes 4 28 */
   unsigned long dwNumUsedIndex; /* Number of indexes 4 32 */
   unsigned long dwNextID; /* Next free index ID 2 36 */
   unsigned short usSwapPattern;/* Swap pattern 2 38 */
   unsigned short Spare[7]; /* Spare 14 52 */
   unsigned long reserved[2]; /* reserved 8 60 */
   unsigned long dwChecksum; /* Head & index checksum 4 64 bytes */
} FLIRFILEHEAD;

and
http://cpansearch.perl.org/src/EXIFTOOL/Image-ExifTool-9.53/lib/Image/ExifTool/FLIR.pm
Quote# Process FLIR FFF record (ref PH/1/3)
# Inputs: 0) ExifTool ref, 1) dirInfo ref, 2) tag table ref
# Returns: 1 if this was a valid FFF record
sub ProcessFLIR($$;$)
{
    my ($exifTool, $dirInfo, $tagTablePtr) = @_;
    my $raf = $$dirInfo{RAF} || new File::RandomAccess($$dirInfo{DataPt});
    my $verbose = $exifTool->Options('Verbose');
    my $out = $exifTool->Options('TextOut');
    my ($i, $buff, $rec);

    # read and verify FFF header
    $raf->Read($buff, 0x40) == 0x40 and $buff =~ /^FFF\0/ or return 0;

    # set file type if reading from FFF file ($tagTablePtr will not be defined)
    $exifTool->SetFileType() unless $tagTablePtr;

    # FLIR file header (ref 3)
    # 0x00 - string[4] file format ID = "FFF\0"
    # 0x04 - string[16] file origin: seen "\0","MTX IR\0","CAMCTRL\0"
    # 0x14 - int32u file format version = 100
    # 0x18 - int32u offset to record directory
    # 0x1c - int32u number of entries in record directory
    # 0x20 - int32u next free index ID = 2
    # 0x24 - int16u swap pattern = 0 (?)

Phil Harvey

Hi Thomas,

Yes, there is that, but I was referring to this comment:

Quote from: Videogamer555 on April 06, 2014, 12:49:22 PM
there is a field at the very start of the RawData chunk that is I think is a 16 bit integer version number

in which he clearly indicates the first field of the RawData chunk.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

tomas123

ok, let us wait for some interesting new tags  :)

for instance:
until now we can't define a color scale which is 100% identical to Flir (Tools)
Flir stretchs the color scale while no uniform temperature distribution