enhancement: extract binary data from FLIR radiometric jpg

[tomas123]

but I want to think about this for a while and study your notes

for better reading a short summary of this post
https://exiftool.org/forum/index.php/topic,4898.msg23944.html#msg23944
without the physical part



Variant A: Emissivity of object = 1,0

boundary condition:
     object distance = 0
     external optics transmission = 1.0

T = B / ln(R1/(R2*(S+O))+F)

T = object temperature in Kelvins
S = 16 Bit RAW value
R1 Planck R1 constant
R2     Planck R2 constant
B    Planck B constant. Value range 1300 - 1600.
F     Planck F constant. Value range 0.5 - 2.
O    Planck O (offset) constant. Its a negative value.

ln() natural logarithm


Variant B: Emissivity of object < 1,0

boundary condition:
    object distance = 0
    external optics transmission = 1.0

now me must calculate the amount of radiance from reflected objects
for that, we need two auxiliary calculation

RAW_refl=R1/(R2*(e^(B/T_refl)-F))-O
T_refl = reflected temperature in Kelvins
RAW_refl is linear to amount of radiance of the reflected objects
e  Euler's number

RAW_obj=(S-(1-Emissivity)*RAW_refl)/Emissivity
RAW_obj is linear to amount of radiance of the measured object
  Emissivity = Emissivity of object
  S = 16 Bit FLIR RAW value

now we use the formula from variant A and replace the 16-Bit-Image-Value "S" with the calculated RAW_obj
T_obj= B / ln(R1/(R2*(RAW_obj+O))+F)
  T_obj = object temperature in Kelvins
  R1 Planck R1 constant
  R2     Planck R2 constant
  B    Planck B constant. Value range 1300 - 1600.
  F     Planck F constant. Value range 0.5 - 2.
  O    Planck O (offset) constant. Its a negative value.

[Phil Harvey]

Great, thanks.  And I'll reference this post from the ExifTool FLIR CameraInfo tags documentation.

- Phil

[tomas123]

I'm looking in your pre release Image-ExifTool-9.28p.tar.gz

you can't unfortunately print a tag twice in different formats ("Planck B" and  "spectral range" of camera )
a suggest as oneliner for Flir.pm

Code Select Expand

Line 252:     0x5C => { Name => 'PlanckB',  Format => 'float', PrintConv => 'sprintf("%.8g [%.1f µm]",$val,14387.6515/$val)' }, #1

Output

Code Select Expand

Planck B                        : 1395.7 [10.3 µm]

more accurate: Wavelength by max infrared sensitivity

[Phil Harvey]

Thanks for this suggestion.

Perhaps a separate Composite tag named SpectralRange would be better?:

Code Select Expand

======== ../testpics/FLIR/Cat.jpg
Planck B                        : 1375
Spectral Range                  : 10.5 um


... or may be "CameraSpectralRange"?

(note that I use a "u" instead of "µ" to avoid complications due to the special character sets.)

- Phil

[tomas123]

range for a single value is not a good description
let's use "sensor wavelength"
http://en.wikipedia.org/wiki/Infrared
or
"thermal wavelength"

All Flir "Thermal" Image Cameras with the capability to save a radiometric jpg are specified for a spectral range 7.5 to 13 μm.
But the internal calibrated value for calculating the temperature with Planck's Law is this single wavelength.
The saved values for Planck R1, R2, B, F, O are for every camera fixed values (for a preset temp range).
These values are a fingerprint and changed only after a calibration from FLIR service.

[Phil Harvey]

This makes sense, thanks.  Or how about CalibratedWavelength?

- Phil

Edit: ... or perhaps PeakSpectralSensitivity?

[tomas123]

PeakSpectralSensitivity is nice

I was looking for a way with all the included tags to create a new image with the same temperature range as the original FLIR image.
For this we need additional information about the temperature scale in Flir.jpg

I found the necessary  informationen in this two new tags:

Code Select Expand

   
# FLIR camera record (ref PH)
%Image::ExifTool::FLIR::CameraInfo = (
...
    0x338 => { Name => 'RAW_center',     Format => 'int16u', PrintConv => 'sprintf("%i",$val)' },
    0x33c => { Name => 'RAW_max-min',     Format => 'int16u', PrintConv => 'sprintf("%i",$val)' },

it's crazy, but Flir save the median [=(max+min)/2] 16-bit-AD-range of the calculated jpg-image and the (16 Bit) difference between max/min

With this two tags I wrote this short shell script, which calculates a new jpg only from Flir tags and embedded binary palette and RAW values. Only with exiftool and convert.

The function between 16-Bit-RAW to temperature is not linear. Therefore I used the imagemagick convert fx operator for calculating every pixel with Plancks Law and therefore with the natural logarithm  (fx is a great feature).

flir2jpg.sh

Code Select Expand

#!/bin/bash
echo "usage $0 flirSource.jpg Destination.jpg"

# test for Tiff or PNG
Flir=$(exiftool -Flir:all "$1")
Type=$(echo "$Flir" | grep "Raw Thermal Image Type" | cut -d: -f2)
if [ "$Type" != " TIFF" ]
    then
        echo "only for RawThermalImage=TIFF"
        exit 1
fi

# extract color table and expand [16,235] video pal color table
PalCol=$(exiftool -flir:all $1 | grep "Palette Colors" | cut -d: -f2)
exiftool $1 -b -Palette | convert -size ${PalCol}X1 -depth 8 YCbCr:- -separate -swap 1,2 -set colorspace YCbCr -combine -colorspace RGB -auto-level Palette.png

# get Flir values for Plancks Law
R1=$(echo "$Flir" | grep "Planck R1" | cut -d: -f2)
R2=$(echo "$Flir" | grep "Planck R2" | cut -d: -f2)
B=$(echo "$Flir" | grep "Planck B" | cut -d: -f2)
O=$(echo "$Flir" | grep "Planck O" | cut -d: -f2)
F=$(echo "$Flir" | grep "Planck F" | cut -d: -f2)

# echo $R1 $R2 $B $O $F

# get refl temp
Temp_refl=$(echo "$Flir" | grep "Reflected Apparent Temperature" | sed 's/[^0-9.-]*//g')
Emissivity=$(echo "$Flir" | grep "Emissivity" | cut -d: -f2)

RAWmedium=$(echo "$Flir" | grep "Raw Value Median" | cut -d: -f2)
RAWdelta=$(echo "$Flir" | grep "Raw Value Range" | cut -d: -f2)

RAWmax=$(echo "scale = 8;$RAWmedium+$RAWdelta/2" | bc -l )
RAWmin=$(echo "scale = 8;$RAWmedium-$RAWdelta/2" | bc -l )

# echo "RAW-Range: $RAWmin $RAWmax"

# calc Temperature Range of measured object with Emissivity < 1
RAWrefl=$(echo "scale = 8;$R1/($R2*(e($B/($Temp_refl+273.15))-$F))-$O" | bc -l )
RAWmaxobj=$(echo "scale = 8;($RAWmax-(1-$Emissivity)*$RAWrefl)/$Emissivity" | bc -l )
RAWminobj=$(echo "scale = 8;($RAWmin-(1-$Emissivity)*$RAWrefl)/$Emissivity" | bc -l )

Temp_max=$(echo "scale = 8;$B/l($R1/($R2*($RAWmaxobj+$O))+$F)-273.15" | bc -l )
Temp_min=$(echo "scale = 8;$B/l($R1/($R2*($RAWminobj+$O))+$F)-273.15" | bc -l )

# convert to "%.1f" for printing
LC_NUMERIC="en_US.UTF-8"
Temp_min=$(printf "%.1f" $Temp_min)
Temp_max=$(printf "%.1f" $Temp_max)

# draw temp scale
convert -size 20x256 gradient: Palette.png -clut -mattecolor white -frame 5x5 gradient.png
convert gradient.png -background white -font ArialB -pointsize 15 label:"$Temp_max °C" +swap -gravity Center -append  label:"$Temp_min" -append gradient.png

# convert every RAW-16-Bit Pixel with Planck's Law to a Temperature Grayscale value and append temp scale
Smax=$(echo "scale = 8;$B/l($R1/($R2*($RAWmax+$O))+$F)" | bc -l )
Smin=$(echo "scale = 8;$B/l($R1/($R2*($RAWmin+$O))+$F)" | bc -l )
Sdelta=$(echo "scale = 8;$Smax-$Smin" | bc -l )
exiftool -b -RawThermalImage $1 | convert - -fx "($B/ln($R1/($R2*(65535*u+$O))+$F)-$Smin)/$Sdelta" -resize 200\% Palette.png -clut -background white -flatten +matte gradient.png -gravity East +append $2

echo "wrote $2 with Temp-Range: $Temp_min / $Temp_max"


sample

Code Select Expand

./flir2jpg.sh IR_1546-rainbow.jpg flir2jpg-rainbow.jpg
You can compare the original image from attachment with this result from script - look at the same temperature range
... in the scale outside of the image;-)



some hints for interested readers:
- see my post here for a collection of nice palettes with fullrange level (0-255) instead of ugly embedded palalettes (16-235)
https://exiftool.org/forum/index.php/topic,4898.msg23763.html#msg23763

- you need the 16 Bit Version of Imagemagick
$ convert -version
Version: ImageMagick 6.7.8-8 2012-08-09 Q16 http://www.imagemagick.org

- convert need the freetype library for writing fonts
$ convert -list configure | grep DELEG
DELEGATES     bzlib freetype jpeg jng jp2 lcms png tiff x11 xml zlib

- with freetype library you get this error: "convert: unable to read font "
-> see http://www.imagemagick.org/discourse-server/viewtopic.php?f=3&t=20529

[tomas123]

where JD) found the names for this tags and what they mean?
Palette Method                  : 0
Palette Stretch                 : 3

I have some different displays of the iron palette and I think, it's a adjusted gamma value.

[Phil Harvey]

PeakSpectralSensitivity it is then.

And wow, two more tags!  Cool! 

I think I'd like to call them RawValueMedian and RawValueRange.

I don't have any more details about the meaning of the Palette tags decoded by Jens Duttke.  He decoded these a while ago and hasn't been actively working on this recently.

I spent a bit of time trying to understand your script.  Very impressive.  You are certainly a power programmer when it comes to shell scripting.

- Phil

[tomas123]

ok, thats fine

I edited the script above, before the server tomorrow closed the post for editing

Code Select Expand

RAWmedium=$(echo "$Flir" | grep "Raw Value Median" | cut -d: -f2)
RAWdelta=$(echo "$Flir" | grep "Raw Value Range" | cut -d: -f2)


[tomas123]

shell scripting is kids scripting compared to regex and perl ( http://en.wikipedia.org/wiki/Obfuscated_Perl_Contest )

convert is the swiss knife 

[Phil Harvey]

shell scripting is kids scripting compared to regex and perl

Funny.  I was thinking how much simpler your script would be if it was written in Perl.   (But one big reason for this is that you could call the API directly and avoid all of the "cut"-ing for each tag.)

- Phil

Edit: I sent and e-mail to Jens to ask about the Palette tags.  Looking back at the information he gave me, it seems as if he may have decoded these by reverse-engineering with the KLIRViewer iPad App.

[tomas123]

I downloaded the new version 9.28 and all works fine.

Question:
(1) How do you delete the images inside the jpg in your Meta Information Repository
( https://exiftool.org/sample_images.html )
I know, how to write thumbnails: $ exiftool 1.jpg '-ThumbnailImage<=thumb.jpg'

(2) I furthermore work at panorama radiometric images.
Could you imagine to enable write access for a complete FFF-File?
The advantage is the possibility to use FLIR Software for settings measuring point etc. in panorama.

I can create a FFF Header for a 16Bit-Tiff-Panorama self,  its simple.
Flir Software needs only the two tags for working (0x0001 RawData and 0x0020 CameraInfo)
Exiftool must "only" extend the  EXIF APP1 frame and fragment the file in 64KByte-Parts...
I tested it painful with a hexeditor - there is no recalculation of addresses required...


Thanks for your great work and have a great holiday!

[tomas123]

Hallo Phil,

I hope, you are full recovered back from holiday 

You have not answered to my last post 

In the meantime I worked on Flir-PiP (Picture in Picture) Images
You find some nice samples here:

Flir B60: http://commons.wikimedia.org/wiki/File:Aqua_Tower_thermal_image.jpg
Flir E40: https://exiftool.org/forum/index.php/topic,4898.msg23838.html#msg23838
extract IR_2523_PiP.jpg from my attachment https://exiftool.org/forum/index.php?PHPSESSID=t6r4dh04hnsdhk4vj1h4ag6ku2&action=dlattach;topic=4898.0;attach=606
Flir T640 http://www.flickr.com/photos/prusajr/6945941592/sizes/o/in/photostream/ (large!)


You can decode some more tags with the config file config.txt from attachment

But there is a bug inside my config file.
I can't decode the reverse byte Order for first tag (float) Real2IR

exiftool.exe -config config.txt -flir:all Aqua_Tower_thermal_image.jpg -v3 | grep -B1 -A11 "Tag 0x002a"
| PIP (SubDirectory) -->
| - Tag 0x002a (64 bytes):
|    52008: f4 ff db 3f 02 00 f6 ff 24 00 8e 00 24 00 8e 00 [...?....$...$...]
|    52018: 35 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 [5...............]
|    52028: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
|    52038: 00 00 00 00 00 00 00 00 00 00 00 00 02 00 00 00 [................]
| + [BinaryData directory, 64 bytes]
| | Real2IR = -1.62168278564919e+032
| | - Tag 0x0000 (4 bytes, float[1]):
| |    52008: f4 ff db 3f                                     [...?]
| | OffsetX = 2
| | - Tag 0x0004 (2 bytes, int16s[1]):
| |    5200c: 02 00                                           [..]

correct is f4 ff db 3f -> 1,7187
you know, how to correct this ;-)

--------------------------------------------------

infos to the new tags

Code Select Expand


$ exiftool.exe -config config.txt -flir:all Aqua_Tower_thermal_image.jpg
...
Embedded Image Width            : 480  //see my sample below
Embedded Image Height           : 480
Embedded Image                  : (Binary data 335334 bytes, use -b option to extract)
Embedded Image Format           : PNG in YCbCr (see sample below)
Real 2 IR                       :  1,7187 (Image Size Relation Real/Radiometric, reverse byte order)
Offset X                        : +2 (X Offset from Center for Insertion Point)
Offset Y                        : -10
Pip X1                          : 36 (crop size position for radiometric image)
Pip X2                          : 142
Pip Y1                          : 36
Pip Y2                          : 142
Focus Distance RAW              : 0  // Flir E40/E50/E60 value proportional to rotation angle of focus ring)
Focus Distance                  : 4.5 m (with focus ring adjusted distance)


SAMPLE
now decode the aqua tower and overlay to a  full size images (ignore Plancks law for better reading)

download http://commons.wikimedia.org/wiki/File:Aqua_Tower_thermal_image.jpg

// extract real image, convert embedded YCbCr-PNG  to RGB

$ exiftool -config config.txt Aqua_Tower_thermal_image.jpg -b -EmbeddedImage | convert -  -set colorspace YCbCr -colorspace RGB Tower.jpg

extract the palette, convert YCrCb-RAW to RGB

$ exiftool Aqua_Tower_thermal_image.jpg -paletteco*
Palette Colors                  : 224
$ exiftool Aqua_Tower_thermal_image.jpg -b -Palette | convert -size 224X1 -depth 8 YCbCr:- -separate -swap 1,2 -set colorspace YCbCr -combine -colorspace RGB palette.png

next step is radiometric image

$ exiftool -config config.txt Aqua_Tower_thermal_image.jpg -rawvalue* -rawtherm*
Raw Value Median                : 13102
Raw Value Range                 : 1995
Raw Thermal Image Width         : 180
Raw Thermal Image Height        : 180
Raw Thermal Image Type          : PNG
Raw Thermal Image               : (Binary data 48946 bytes, use -b option to extract)

calculate level for max/min temperature
// 13102-1995/2=12105
// 13102+1995/2=14100

extract radiometric image and change reverse byte order 16Bit-PNG

$ exiftool Aqua_Tower_thermal_image.jpg -b -RawThermalImage | convert - gray:- | convert -depth 16 -endian msb -size 180x180 gray:- -level 12105,14100 raw.png

now overlay all three images

$ exiftool -config config.txt Aqua_Tower_thermal_image.jpg -real2ir -RawThermalImageWidth -EmbeddedImageWidth -Offset*
real 2 IR                       : 1,7187 // please correct my config file
Raw Thermal Image Width         : 180
Embedded Image Width            : 480
Offset X                        : +2
Offset Y                        : -10

calculate the resize factor
480/1,7187=279 Pixel horizontal for resized raw image

$ convert raw.png palette.png -clut -resize 279x Tower.jpg +swap -gravity Center -geometry +2-10 -compose over -composite TowerPiP.jpg

you can see, we have now the twice size (quad pixel) as original
(73kByte)

we have some little differences in overlay, the photographer only saw the cropped image on display and the focus (move overlay) is not perfect...



for clarification the last tags, decode the original flir image size and compare with original

$ exiftool -config config.txt Aqua_Tower_thermal_image.jpg -pip* -offset* -real2ir
Pip X1                          : 36
Pip X2                          : 142
Pip Y1                          : 36
Pip Y2                          : 142
Offset X                        : +2
Offset Y                        : -10
Real 2 IR                       : 1,7187

with this tags calculate the radiometric image size
  142-36+1= 107px horizontal
  142-36+1= 107px vertical
calculate the resize factor for radiometric image
480px/180px/1.71= 155%

and now: all-in-one (ImageMagick is great)

Code Select Expand

convert raw.png -gravity center -crop 107x107+0+0 palette.png -clut -resize 155% Tower.jpg +swap -gravity Center -geometry +2-10 -compose over -composite -crop 270x270+2-10 -resize 240x Aqua_Tower_thermal_image.jpg -gravity East +append compare.jpg


perfect

[Phil Harvey]

Hi Tomas,

Sorry, I missed this somehow.  I'm still on vacation, but I can answer these questions.

Question:
(1) How do you delete the images inside the jpg in your Meta Information Repository
( https://exiftool.org/sample_images.html )
I know, how to write thumbnails: $ exiftool 1.jpg '-ThumbnailImage<=thumb.jpg'

The command to delete the ThumbnailImage is:  exiftool -thumbnailimage= 1.jpg

(2) I furthermore work at panorama radiometric images.
Could you imagine to enable write access for a complete FFF-File?

Adding the ability to write a format is a lot more work than reading.  I don't think the demand for this feature would be great enough to justify this effort.

Thanks for the details about the PiP information.  I'll study this and post back here if I have any questions.

- Phil