enhancement: extract binary data from FLIR radiometric jpg

[tomas123]

There's no rule without an exception

I found here
http://forums.mtbr.com/lights-night-riding/modding-p60-dropin-style-torch-better-heat-dissipation-674339.html
some pics from a ThermaCAM E65

Code Select Expand

$ exiftool IR_0627.jpg
Make                            : FLIR Systems AB
Camera Model Name               : ThermaCAM E65
...
FLIR Image                      : (Binary data 20065 bytes, use -b option to extract)
FLIR Image Type                 : PNG


there is the byte order in the PNG correct
this gives the correct image

Code Select Expand

exiftool -b -FlirImage IR_0627.jpg > IR_0627.png

but if you open this picture with Flir Software Quickreport (see above) and save it then you also change the byte order (reverse PNG)

The ThermaCAM E65 is from year 2006.
You should not invest too much power in this old versions from Flir.
Newer Cam don't use embedded PNG or saves PNG with reverse byte order.


it has the appearance that the reverse byte order is a global switch

you can see, that the Flir Software changed all byte to little endian (header+data)
old native file from cam

Code Select Expand

exiftool -v3 IR_0627.jpg
  FLIR Record 0x01, offset 0x01c0, length 0x4e3c
    0000: 02 00 a0 00 78 00 00 00 00 00 00 00 3f 01 00 00 [....x.......?...]
    0010: ef 00 0f 00 02 00 01 00 6c 0e 06 00 00 00 00 00 [........l.......]
    0020: 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 [.PNG........IHDR]
    0030: 00 00 00 a0 00 00 00 78 10 00 00 00 00 00 59 7a [.......x......Yz]
    0040: f0 00 00 20 00 49 44 41 54 78 01 45 dd 09 94 5f [... .IDATx.E..._]
    0050: 59 55 2e f0 73 6b 48 aa 92 4a a5 32 a7 d3 69 e8 [YU..skH..J.2..i.]
   
same file after editing with Flir Software

Code Select Expand

exiftool -v3 IR_0627_quickreport.jpg
  FLIR Record 0x01, offset 0x0200, length 0x4e81
    0000: 00 02 00 a0 00 78 00 00 00 00 00 00 01 3f 00 00 [.....x.......?..]
    0010: 00 ef 00 0f 00 02 00 02 0e 6c 00 06 00 00 00 00 [.........l......]
    0020: 89 50 4e 47 0d 0a 1a 0a 00 00 00 0d 49 48 44 52 [.PNG........IHDR]
    0030: 00 00 00 a0 00 00 00 78 10 00 00 00 00 00 59 7a [.......x......Yz]
    0040: f0 00 00 20 00 49 44 41 54 78 01 45 dd 09 74 67 [... .IDATx.E..tg]
    0050: d9 55 1e fa 73 a4 52 95 54 52 a9 54 73 75 75 d9 [.U..s.R.TR.Tsuu.]

[tomas123]

off topic but nice:
imagemagick can change byte order in raw files

use our cat: http://www.nuage.ch/site/flir-i7-some-analysis/

Code Select Expand

$ exiftool -b -FlirImage IR_0248.jpg > cat1.png
$ convert cat1.png gray:- | convert -depth 16 -endian msb -size 120x120 gray:- -auto-level cat2.png

[Phil Harvey]

Hi Thomas,

This is interesting, but I don't plan to modify the raw data.  I just return the PNG as is, or tack a TIFF header onto the raw data, but that's as far as I'll go.  (ExifTool doesn't do image manipulations -- that's for other software.)

- Phil

[tomas123]

I am of the same opinion and was surprised that you have ever extract the funny PNG images.
You can write a note about the reverse byte order of some Flir PNG in your history...

[Phil Harvey]

Hi Tomas, (sorry if I spelled your name wrong before)

I've really done a LOT of work on this, and have managed to accumulate a collection of images from 31 different FLIR camera models (now available in my metadata repository).

I have decoded more potentially useful information:

Code Select Expand

> exiftool ../testpics/FLIR/Man_in_water_-_IR_image.jpg -a -flir:all
Image Maximum Temperature       : 283
Image Minimum Temperature       : 276
Emissivity                      : 0.95
Raw Thermal Image Width         : 640
Raw Thermal Image Height        : 480
Raw Thermal Image Type          : TIFF
Raw Thermal Image               : (Binary data 614604 bytes, use -b option to extract)
Emissivity                      : 0.95
Distance                        : 1.00 m
Reflected Apparent Temperature  : 20.0 C
Atmospheric Temperature         : 20.0 C
IR Window Temperature           : 20.0 C
IR Window Transmission          : 1.00
Relative Humidity               : 50.0 %
Camera Maximum Temperature      : 120.0 C
Camera Minimum Temperature      : -40.0 C
Camera                          : FLIR P640
Camera Serial 1                 : 40402-1100X1
Camera Serial 2                 : 404002646
Camera Firmware                 : 15.0.14
Lens                            : FOL38
Lens Serial 1                   : T197089
Lens Serial Number              : 407802518
Palette Colors                  : 224
Above Color                     : 170 128 128
Below Color                     : 50 128 128
Overflow Color                  : 67 216 98
Underflow Color                 : 41 110 240
Isotherm 1 Color                : 100 128 128
Isotherm 2 Color                : 100 110 240
Palette Method                  : 0
Palette Stretch                 : 2
Palette File Name               : \FlashFS\system\iron.pal
Palette Name                    : Iron
Palette                         : (Binary data 672 bytes, use -b option to extract)

I plan to release the official version in a few days, but I have updated the pre-release in case you get a chance to test it out before then.

- Phil

[tomas123]

great and hard work!

You have not changed the dirty hack
if (length $val == $w * $h * 2) 
I have also seen the color palette...

some hints:
Camera Serial 1 = Part Number
Camera Firmware = App Core ??
(I changed my Camera Firmware from 2.10.7 to 2.19.10 but the exiftool-value is unchanged 20.0.0)

Code Select Expand

$exiftool IR_0088.jpg -a -flir:all
Camera                          : FLIR E40
Camera Serial 1                 : 49001-2001
Camera Serial 2                 : 4903xxxx
Camera Firmware                 : 20.0.0
Lens                            : FOL18
Lens Serial 1                   :
Lens Serial Number              :


with web frontend of camera I get:

Code Select Expand

http://192.168.64.1/SysInfo.asp

Camera
Name: FLIR E40
Part # 49001-2001
Serial # 4903xxxx

Kits
Name Version Date
SW combination  2.10.7
appkit 3.0.8 4-Sep-2012
userconf E40 0.10  28-Apr-2011 
extfontkit 1.1 29-Nov-2011 
SLCO OS image  15.4.4 2012-06-25
prodkit 3.0.0.7 28-Aug-2012 


Firmware
Name Version Date From
MIRA_FPGA 5.3.1.18 - FLIR

Software
Name Version Date From
AppCore 20.0.0.1 03-Sep-2012  upalmer@SE-BRYGG5/ALPHA_1.6 
AppServices 20.0.0.1 03-Sep-2012  upalmer@SE-BRYGG5/ALPHA_1.6 
Bootloader 15.0.10.0 - FLIR
ProdApp 20.0.0.1 28-Aug-2012  upalmer@SE-BRYGG5/ALPHA_1.6 
RTP 20.0.0.1 04-Sep-2012  upalmer@SE-BRYGG5/ALPHA_1.6 
ResMon 20.0.0.1 04-Sep-2012  upalmer@SE-BRYGG5/ALPHA_1.6 
WinCE 6.0.0.0 2005 Microsoft
appcore_dll 1.6.1.1 03-Sep-2012  upalmer@SE-BRYGG5
common_dll 1.6.1.1 03-Sep-2012  upalmer@SE-BRYGG5
fvd 15.0.28.0 - -
ui 20.0.0.1 04-Sep-2012  upalmer@SE-BRYGG5/ALPHA_1.6 

[Phil Harvey]

Great, thanks!

I'll make your suggested changes, but I'm going to mark the AppCoreVersion as Unknown because that one is a bit funny.

And yes, the "$w * $h * 2" test has been very reliable!

Please let me know if you notice anything else that should be changed.

- Phil

[Phil Harvey]

I think I'll go with "CameraSoftware" instead of "AppCoreVersion", and remove the Unknown flag.  Actually, this is equivalent to the original "CameraFirmware" guess, except that that it doesn't imply the software is persistent in memory.

Also, I'm changing a few other names so your output will change to this:

Code Select Expand

Camera Model                    : FLIR E40
Camera Part Number              : 49001-2001
Camera Serial Number            : 4903xxxx
Camera Software                 : 20.0.0
Lens Model                      : FOL18
Lens Part Number                :
Lens Serial Number              :

One question:  When you use the -u option you will see a number of unknown temperatures.  Do you have any idea about the significance of these?  For the "Cat" image for example, the Unknown information is:

Code Select Expand

[MakerNotes]    Unknown Temperature             : 0
[MakerNotes]    Unknown Maximum Temperature     : 37
[MakerNotes]    Unknown Minimum Temperature     : 22
[MakerNotes]    FLIR 0x0007                     :
[MakerNotes]    FLIR 0x0008                     :
[MakerNotes]    FLIR 0x0009                     :
[APP1]          Unknown Temperature 2           : 150.0 C
[APP1]          Unknown Temperature 3           : -60.0 C
[APP1]          Unknown Temperature 4           : 120.0 C
[APP1]          Unknown Temperature 5           : -40.0 C
[APP1]          Unknown Temperature 6           : 150.0 C
[APP1]          Unknown Temperature 7           : -60.0 C

I'm talking about the 6 unknown temperatures from APP1.  (We already discussed the ones in the MakerNotes.)

- Phil

[tomas123]

@Phil
I think decoding of this unknown values make no sense. Flir has no common usages of this free variables.


I'm sorry for this off-topic posting, but I will give some hints for other readers for using Flir raw images with some scripting tools. (google hit this page)

use:
Exiftool V 9.25
sample image from attachment
Imagemagick convert Q16 (16 Bit)

for panorama stitching:
- 16 Bit Panorama Software
- Unix Tools: awk, hexdump


(sample 1) Extract Color Table and Raw Image from Flir radiometric JPG and generate a new Image

the embedded Flir Color Palette uses Y Cr Cb (you must change the channel Cb with Cr ->  YCbCr )

first check numbers of colors of current table

Code Select Expand

$ exiftool IR_1546.jpg | grep Palette
Palette Colors                  : 224
Palette Method                  : 0
Palette Stretch                 : 1
Palette Name                    : Iron
Palette                         : (Binary data 672 bytes, use -b option to extract)

Size is here 224x1 Pixel (=672/3)

Code Select Expand

$ exiftool IR_1546.jpg -b -Palette | convert -size 224X1 -depth 8 YCbCr:- -separate -swap 1,2 -set colorspace YCbCr -combine -colorspace RGB pal-iron.png

hint: this is a  ugly [16,235] video pal color table ( see http://avisynth.org/mediawiki/ColorBars_theory )
for better color tables [0,255] use color tables from attachment

now extract the Flir Raw Image (a 16 Bit Tiff from -40 Grad to +120 Grad)

Code Select Expand

$ exiftool -b -RawThermalImage IR_1546.jpg > IR_1546.tiff
( for multiple images - use exiftool -b -RawThermalImage IR*.jpg -w %f.tif )

transform the Color Lookup Table to the grayscale Image
you must expand your Tiff to suitable 8 Bit Values (suggest: use contrast-stretch 2%,2%)

Code Select Expand

$ convert IR_1546.tiff  -contrast-stretch 2%,2% -depth 8 -resize 200\% pal-iron.png -clut entrance-iron.png

the temperature values can be calculated only with the Flir Software ( a little bit non linear )
if you compare your own maked colors with a flir tool, then you can write the known temperature to many images  (batch)

check your sensor values

Code Select Expand

$  identify -verbose IR_1546.tiff | grep -E "min:|max:"
      min: 12417 (0.189471)
      max: 15410 (0.235142)

use absolute contrast level for all images in your serie, try some values
I choose min=13400 / max=14500
now generate an image with a temperature scale
use: pal-iron.png and IR_1546.tiff from above

Code Select Expand

$ convert -size 20x256 gradient: pal-iron.png -clut -mattecolor white -frame 5x5 gradient.png
$ convert gradient.png -background white -font ArialB -pointsize 15 label:'+4.0' +swap -gravity Center -append  label:'-5.9' -append gradient-tmp.png
$ convert IR_1546.tiff -level 13400,14500 -depth 8 -resize 300\% pal-iron.png -clut -background white -flatten +matte gradient-tmp.png -gravity East +append raw2color.png

see result raw2color.png in attachments
for other color tables see also attachment




(sample 2) make a panorama from raw images
here I use a Flir E40 with 160x120 Pixel for generating a 800x600 Pixel panorama image

extract RAW images

Code Select Expand

$ exiftool -b -RawThermalImage IR*.jpg -w %f.tif

find max / min sensor values and expand Tiff for better recognition of panorama software

Code Select Expand

min=$(identify -verbose IR*.tif | grep -n3 statist | grep min | sort -k 3 | head -n 1 | cut -d' '  -f8)
max=$(identify -verbose IR*.tif | grep -n3 statist | grep max | sort -k 3 | tail -n 1 | cut -d' '  -f8)
echo $min $max
for i in IR*.tif; do convert $i -level $min,$max -resize 200\% _$i;done

now you have some 16-Bit-Grayscale-Tiff for stitching (don't forget  $min $max for later correction, see below)

I'm using two panorame softwares for stitching 16 Bit Images

• free tool: Microsoft Research Image Composite Editor (ICE) (save panorama as Tiff = 16 Bit)
• Photoshop Photomerge

stitch your 16 Bit _IR*.tif images and save result to a 16 Bit Tif

now comes the hardest part:

Variant 1: you can colorize your panorama with imagemagick (see above) -> you're done

Variant 2: generate a new Flir Radiometric Panaroma JPG for measuring with Flir Software

- now you need a Flir Image with enough space for injecting your new Raw Values in the space of a dummy Flir Picture
- you can generate a large Flir Dummy Image with the very bad Panorama function of Software Flir Tools  ( use 30 day Trial)
- alternative use an other large Flir Images from Internet ( see my attachment for a 800x600 Pixel sample)
- or Phil has an idea how to write a new JPEG APP1 (Flir) segment

in this sample I create a 800x600 Panorama

(1) with your Panorama Software create a exact 800x600 Pixel Grayscale 16 Bit Image panorama.tif (same size like dummy-jpg!!)
(2) decode a 800x600 radiometric dummy image in decimal values (with hexdump) for using with awk (a flir800x600.jpg dummy you can use my attachment)
(3) inject your 800x600 Raw Panorama Values into the 800x600 dummy Image (don't forget reusing $min $max from above)

Code Select Expand

$ hexdump -v -e '1/1 "%u" " "' flir800x600.jpg > flir800x600.dec
$ convert panorama.tif -colorspace Gray +level $min,$max -background black -flatten gray:- | hexdump -v -e '1/1 "%u\n"' | awk -f pano_dec.txt - flir800x600.dec > Pano800x600.jpg

injecting awk script pano_dec.txt -> see attachment

open Pano800x600.jpg with Flir Software (this generates a new jpg from the embedded raw values)
select you color table and set some measuring points

-> see the result Pano800x600.jpg with two measuring points as attachment

[Phil Harvey]

This is very useful, thanks.

I have changed my notes to indicate Y Cr Cb colors instead of Y Cb Cr.

You say that only FLIR software can calculate the temperatures, isn't the formula given in "the measurement formula" reference you gave?  Additionally, I think that ExifTool now extracts all of the necessary parameters to apply this formula.  The only thing I am not sure about is scaling the digital values to a voltage.

- Phil

[tomas123]

you are right
With some reference values and the measurement formula you can calculate beetwen sensor voltage (=RAW??) and object temperature
- but I wanted to give to other users a simple guide 

In Flir Quicktool you can export a (Excel) *.csv table with temperature of every pixel.
With the known raw values then you have enough reference values (Voltage to Temperature) for further studies about the measurement formula
see my message: https://exiftool.org/forum/index.php/topic,4898.msg23663.html#msg23663

PS: The injection of panorama raw values to another large Flir JPG (same size) is a dirty hack.
Can Exiftool with minor adjustments write the JPEG APP1 (Flir) segments with a RawThermalImage of another size?
There are only two places where the image size H/W and (H-1)/(W-1) are entered.

[Phil Harvey]

I am only decoding the raw image size from the RawData record, but this information is also duplicated in the CameraInfo record, and possibly other places.  So changing the image size may be more difficult than you think.  Plus, it would require a significant hack to ExifTool because there would still be a lot of work to do to rewrite the FLIR APP1 structure.

- Phil

[tomas123]

ok, no problem

only for your information:

I found here a nice support document for the Flir FFF Header (google is my friend )
http://www.workswell.cz/manuals/flir/hardware/A3xx_and_A6xx_models/Streaming_format_ThermoVision.pdf

with this information we can decode all unknown Bytes off Flir Header:

Code Select Expand

$ exiftool -v5 IR_1135.jpg

JPEG APP1 (42312 bytes):
    0f0c: 46 4c 49 52 00 01 00 00                         [FLIR....        ]
----------------------tagFLIRFILEHEAD---------------------------------------
                                  46 46 46 00             [        FFF.....] char szFormatID[4]; /* Fileformat ID 'FFF\0'       4  4
                                              00 00 00 00 [           .....] char szOrigin[16]; /* File origin                 16 20
    0f1c: 00 00 00 00 00 00 00 00 00 00 00 00
                                              00 00 00 64 [...............d] unsigned long dwVersion; /* File format version    4 24 
    0f2c: 00 00 00 40                                                        unsigned long dwIndexOff; /* Pointer to indexes    4 28
                      00 00 00 0e                                            unsigned long dwNumUsedIndex; /* Number of indexes 4 32
                                  00 00 00 02                                unsigned long dwNextID; /* Next free index ID      4 36
                                              00 00                          unsigned short usSwapPattern;/* endian order(!!)   2 38
                                                    00 00 [...@............] unsigned short Spare[7]; /* Spare                 14 52
    0f3c: 00 00 00 00 00 00 00 00 00 00 00 00
                                              00 00 a5 40 [...............@] unsigned long reserved[2]; /* reserved             8 60
    0f4c: 00 00 00 00                                                       
                      56 52 ac e1                                            unsigned long dwChecksum; /* Head & index checksum 4 64 bytes */
---------------------- 1. tagFLIRFILEINDEX-----------------------------------------
                                  00 01                                      TAG_MAIN_T wMainType; /* Main type of index       2  2
                                        00 02                                unsigned short wSubType; /* Sub type of index     2  4
                                              00 00 00 65 [....VR.........e] unsigned long dwVersion; /* Version for data      4  8
    0f5c: 00 00 00 01                                                        unsigned long dwIndexID; /* Index ID              4 12
                      00 00 02 00                                            unsigned long dwDataPtr; /* Pointer to data       4 16
                                  00 00 96 20                                unsigned long dwDataSize; /* Size of data         4 20
                                              00 00 00 00 [........... ....] unsigned long dwParent; /* Parentnr               4 24
    0f6c: 00 00 00 00                                                        unsigned long dwObjectNr; /* This object nr       4 28
                      a2 95 6b 65                                            unsigned long dwChecksum; /* Data checksum        4 32 bytes */
---------------------- 2. tagFLIRFILEINDEX-----------------------------------------
                                  00 20                                      TAG_MAIN_T wMainType; /* Main type of index       2  2
                                        00 01                                unsigned short wSubType; /* Sub type of index     2  4
                                              00 00 00 71 [......ke. .....q] unsigned long dwVersion; /* Version for data      4  8
    0f7c: 00 00 00 01 00 00 98 20 00 00 09 ac 00 00 00 00 [....... ........]
    0f8c: 00 00 00 00 fc e2 f6 68
---------------------- 3. tagFLIRFILEINDEX-----------------------------------------
                                  00 22 00 01 00 00 00 68 [.......h.".....h]
    0f9c: 00 00 00 01 00 00 a1 cc 00 00 03 10 00 00 00 00 [................]
    0fac: 00 00 00 00 9b 9f 81 a2
---------------------- 4. tagFLIRFILEINDEX-----------------------------------------
                                  00 21 00 01 00 00 01 05 [.........!......]
    0fbc: 00 00 00 01 00 00 a4 dc 00 00 00 64 00 00 00 00 [...........d....]
    0fcc: 00 00 00 00 6d 9d 1a a1
----------------------END-----------------------------------------
                                  00 00 00 00 00 00 00 00 [....m...........]
    0fdc: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]


with your great work the "wMainType; /* Main type of index" decoded to

Code Select Expand

0x01 => {        Name => 'RawData',
0x20 => {        Name => 'CameraInfo',
0x21 - ToolInfo (spot tool, line tool, area tool)
0x22 => {        Name => 'PaletteInfo',
0x23 => {        Name => 'TextInfo',
0x24 => {        Name => 'EmbeddedAudioFile',
# 0x27: 01 00 08 00 10 00 00 00
0x2b => {        Name => 'GPSInfo',
0x2e => {        Name => 'ParamInfo',

they say:  if wMainType = 01 =  FFF_TAGID_Pixels
then are the next 2 Bytes wSubType

Code Select Expand

/* Sub Tags for FFF_TAGID_Pixels */
enum {
FFF_Pixels_BE = 1, /* Big endian pixel data block */
FFF_Pixels_LE = 2, /* Little endian pixel data block */
FFF_Pixels_PNG = 3 /* PNG compressed pixel data block
*/
but I think, that 1 and 2 apply for all tags ( endian order)

[Phil Harvey]

Great find!  Thanks for the link!

I'll study this and see if I missed anything useful.  Just one point though...

they say:  if wMainType = 01 =  FFF_TAGID_Pixels
then are the next 2 Bytes wSubType

Code Select Expand

/* Sub Tags for FFF_TAGID_Pixels */
enum {
FFF_Pixels_BE = 1, /* Big endian pixel data block */
FFF_Pixels_LE = 2, /* Little endian pixel data block */
FFF_Pixels_PNG = 3 /* PNG compressed pixel data block
*/
but I think, that 1 and 2 apply for all tags ( endian order)

I already thought of this, and had compared the "subtype" for the raw data records, but I must have been comparing the wrong values because I found exceptions and so discounted this correlation.  But trying again now I find the relationship 2=TIFF, 3=PNG to hold true.  To date, I haven't seen a big-endian TIFF.   For all other record types, the "subtype" is 1 in all my samples, regardless of whether record fields are big or little endian.

- Phil

P.S. From this documentation it looks like FLIR is a stand-alone file format (not just in JPEG APP1).  Have you ever come across any samples of this?

Edit: I found one sample here

[Phil Harvey]

I have gone over fff.h carefully, but haven't found anything that makes me want to change my current decoding.  It did confirm a number of my suspicions though, so this was useful.  Also, I updated the comments to be more complete.  The header file is a bit out of date though, and doesn't include a number of additional records that I have decoded.

Too bad I didn't know about this earlier, it would have saved a lot of work.  The same thing happened when I decoded the CRW format -- I found the format document after I had already reverse-engineered the format myself.

- Phil