suggest for a enhancement: decrypt binary data in the EXIF-Header from a FLIR radiometric jpg
FLIR Infrared Software stamps every picture with a flir-logo and doesnt support
batch processing.
Newer Flir Thermographie Cameras saves the
16 Bit sensor raw datas in the Exif Header of a normal jpg.
With this 16 Bit raw datas you can generate a own false color pictures (imagemagick etc.) without flir logo.
exiftool can't decrypt the embedded raw values (binary)
exiftool.exe -v5 IR_0193.jpg
...snip
JPEG APP1 (42312 bytes):
104e: 46 4c 49 52 00 01 00 00 46 46 46 00 00 00 00 00 [FLIR....FFF.....] // magic bytes sequence / 8. Byte+1 = numbers of linked blocks (max 65532 Bytes see sample below)
105e: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 64 [...............d]
106e: 00 00 00 40 00 00 00 0e 00 00 00 02 00 00 00 00 [...@............]
107e: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 a5 40 [...............@]
108e: 00 00 00 00 6d df 65 d4 00 20 00 01 00 00 00 71 [....m.e.. .....q]
109e: 00 00 00 01 00 00 02 00 00 00 09 ac 00 00 00 00 [................] // start address 1. segment + length (address is relative to this block - see awk script)
10ae: 00 00 00 00 db 1f ca ed 00 22 00 01 00 00 00 68 [.........".....h]
10be: 00 00 00 01 00 00 0b ac 00 00 03 10 00 00 00 00 [................] // start address 2. segment + length
10ce: 00 00 00 00 9b 9f 81 a2 00 21 00 01 00 00 01 05 [.........!......]
10de: 00 00 00 01 00 00 0e bc 00 00 00 64 00 00 00 00 [...........d....] // start address 3. segment + length
10ee: 00 00 00 00 6d 9d 1a a1 00 01 00 02 00 00 00 65 [....m..........e]
10fe: 00 00 00 01 00 00 0f 20 00 00 96 20 00 00 00 00 [....... ... ....] // start address 4. segment + length (0x0f20+0x104e=0x1f6e)
110e: 00 00 00 00 f9 d7 91 31 00 00 00 00 00 00 00 00 [.......1........]
111e: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
112e: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
...
1f6e: 33 93 92 43 31 00 00 00 02 00 a0 00 78 00 00 00 [3..C1.......x...] // magic bytes 0200 and sensor size in little endian 0x00a0 0x0078
1f7e: 00 00 00 00 9f 00 00 00 77 00 00 00 00 00 00 00 [........w.......]
1f8e: 00 00 00 00 00 00 00 00 f9 2e e7 2e fb 2e ef 2e [................] // here starts the 16 Bit Stream
1f9e: 05 2f f6 2e e6 2e f2 2e 01 2f f7 2e 07 2f ea 2e [./......./.../..]
1fae: f4 2e fb 2e fa 2e 03 2f 01 2f 06 2f f1 2e 07 2f [......./././.../]
...
Warning = Ignored APP1 segment length 42312 (unknown header)
With reverse engineering I decrypted the header.
In the sample above there are a header for 4 segments (4x32Byte Header)
The address of first segments stands (6*16+5)Bytes from begin of JPEG-APP1-block.
3-Byte-Start-Address: 00 02 00, then 3 Byte with the length of the segment 00 09 ac
.... Second segment starts: 00 0b ac / length: 00 03 10
.... Fourth segment starts 00 0f 20 / length: 00 96 20
The problem is that the raw sensor values are always written in different segments (from first to fourth).
If you open the camera-jpg with a flir software (Flir Tools, Quicktools etc.) the software rewrites this block and save the raw values on another position.
As workaround I look in the top of every segment for a magic byte sequence "00 20" followed by the sensor size in little endian (!)
here Flir E40: W/H = 160 /120px = 0x00A0 / 0x0078
With this information over the sensor size you can calculate the the required segment size for saving all sensor raw data:
Segment_Size= sensor_width x sensor_height x 2Byte(16Bit) + 32Byte(Header)
in this sample above from a flir e40 we need: 160px* 120px * 2Byte + 32Byte = 38432 Byte = 0x9620 Bytes
With this information we can check the start address of segments above and the length of every segment
-> we found that the 4. segment has a necessary segment length of "00 09 ac" bytes for saving raw datas
-> result: the sensor raw values saved in the 4. segment
In the 4. segment, after discussed header of 32 Byte (with sensor size etc.), starts the stream of 16 Bit Raw Values (little endian).
I wrote a short awk-script for decoding the raw values and generating a *.pgm 16 Bit picture.
This script only works for newer Flir Cams with embedded 16 Bit RAW Data. I tested it successful with pictures from
Flir E40, Flir E30bx, Flir T400, Flir i60 and Flir P640 downloaded from wikipedia and flickr ;-)
Some older Flir Thermal Imaging Cameras save a PNG in the exif header (B50, B60, P60, i7).
For this Cameras you need another code (see sample
http://www.nuage.ch/site/flir-i7-some-analysis ).
The working awk script generates a picture in portable graymap format (PGM)
http://en.wikipedia.org/wiki/Netpbm_format:
# cat raw.txt
{
for(i=1; i <=NF; i++)
{
# find MagicBytes off JPEG APP1 (Flir) [8. Byte = 00/01/02]
# 46 4c 49 52 00 01 00 ?? 46 46 46 00 00 00 00 00 [FLIR....FFF.....]
if ( $(i)=="46" && $(i)$(i+1)$(i+2)$(i+3)$(i+4)$(i+5)$(i+6)"00"$(i+8)$(i+9)$(i+10) == "464c495200010000464646" )
{
# search first 5 segments of JPEG APP1 (address header of segments start at 0x50, step 0x20)
for (i1=5*16; i1<256; i1=i1+32)
{
# MagicBytes: Block_Size= sensor_width x sensor_height x 2Byte(16Bit) + 32Byte(Header)
# Flir E40: 160x120px -> 120*160*2+20=38432 = 0x9620
# Flir T400: 320x240px -> 00 40 01 f0 00 -> 0140h x 00f0h -> 0x025820 (153632 Byte)
# calculate entry address of segment
a=i+sprintf("%d","0x"$(i+i1+5)$(i+i1+6)$(i+i1+7))+8;
# magic bytes for following sensor size
if ( $(a)$(a+1) == "0200" )
{
# sensor witdh and height in Little-Endian
w=sprintf("%d","0x"$(a+3)$(a+2));
h=sprintf("%d","0x"$(a+5)$(a+4));
# calculatte required segment_size= width x height x 2Byte + 32Byte(Header)
bs_sensor=2*w*h+32;
# read real segments size from header
bs_exif=1*sprintf("%d","0x"$(i+i1+8)$(i+i1+9)$(i+i1+10)$(i+i1+11));
if ( bs_sensor == bs_exif )
{
# skip header
a=a+32;
# !! here you can set own values !!
min=0;
max=65535;
# now write 16 Bit gray scale picture
print "P2", w, h ,max-min;
print "# this is a *.pgm picture";
print "# sensor size: ",w,"x",h;
print "# start address of 16 Bit raw data: "a-1,"d / first 2 bytes: 0x",$(a),"+ 0x100 * 0x",$(a+1);
print "# raw values from range from min ", min, "to max ",max-min;
for (k=0; k < w*h*2; k=k+2)
{
# max Exif segment size 65534 Byte -> go to next segment (add 12 Byte to start address)
if ((a+k-i+4)%65536 == 0 ) a=a+12;
tmp=1*sprintf("%d","0x"$(a+k+1)$(a+k))-min;
if ( tmp >= (max-min) )
print max-min;
else
{ if ( tmp <= 0 )
print "0";
else
print tmp;
}
}
}
}
}
}
}
}
I'm not using gnu awk with binary extension.
For native awk (bsd) I use hexdump for generating Asci-Code from the jpg-binary.
In this way I can feeding awk with ascii code without stop code 0x00.
Therefore you must use the awk-code in a hexdump pipe (alternate save hexdump-ascii).
Save the result as a pgm picture...
# hexdump -v -e '1/1 "%02x" " "' IR_1955.jpg | awk -f raw.txt > 16bitRAW.pgm
ore use imegamagick convert for saving a 16 Bit png
# hexdump -v -e '1/1 "%02x" " "' IR_1955.jpg | awk -f raw.txt | convert - IR_1955.png
finished...
The Script can also decode large
raw datas over multiple Exif-Segments (max size is 65534Byte / segment)
There is a nice sample for testing from a very large 640 x 480 Flir Infrared Camera P640 in wikepdia:
http://commons.wikimedia.org/wiki/File:Man_in_water_-_IR_image.jpgin the picture the sensor raw datas are shared over 10 JPEG-APP1-Blocks
> exiftool.exe -v3 Man_in_water_-_IR_image.jpg
... snip
JPEG APP1 (65532 bytes):
1d7a: 46 4c 49 52 00 01 00 [b]09[/b] 46 46 46 00 00 00 00 00 [FLIR....FFF.....] // 09 => 10 Blocks linked
1d8a: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 64 [...............d]
1d9a: 00 00 00 40 00 00 00 0e 00 00 00 02 00 00 00 00 [...@............]
1daa: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
1dba: 00 00 00 00 93 79 3c e2 00 01 00 02 00 00 00 65 [.....y<........e]
1dca: 00 00 00 01 00 00 02 00 00 09 60 20 00 00 00 00 [..........` ....]
1dda: 00 00 00 00 59 bc 40 05 00 20 00 01 00 00 00 6f [....Y.@.. .....o]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
11d7a: 46 4c 49 52 00 01 01 09 78 3b 5e 3b 68 3b 50 3b [FLIR....x;^;h;P;]
11d8a: 4d 3b 4d 3b 56 3b 61 3b 41 3b 39 3b 3a 3b 38 3b [M;M;V;a;A;9;:;8;]
11d9a: 2a 3b 39 3b 2c 3b 06 3b 03 3b 15 3b 07 3b 0c 3b [*;9;,;.;.;.;.;.;]
11daa: 0e 3b e5 3a 03 3b 1c 3b 4f 3b 2f 3b 08 3b ff 3a [.;.:.;.;O;/;.;.:]
11dba: 11 3b 11 3b 29 3b 29 3b 31 3b 34 3b 2a 3b 53 3b [.;.;););1;4;*;S;]
11dca: 53 3b 55 3b 5a 3b 60 3b 72 3b 6e 3b 5e 3b 76 3b [S;U;Z;`;r;n;^;v;]
11dda: a8 3b bc 3b b3 3b 53 3b 30 3b 52 3b 58 3b 41 3b [.;.;.;S;0;R;X;A;]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
21d7a: 46 4c 49 52 00 01 02 09 f4 3b ee 3b 05 3c 02 3c [FLIR.....;.;.<.<]
21d8a: 02 3c 08 3c fe 3b 08 3c 09 3c 2f 3c 04 3c 16 3c [.<.<.;.<.</<.<.<]
21d9a: f5 3b fe 3b f5 3b f4 3b 05 3c f1 3b fc 3b ec 3b [.;.;.;.;.<.;.;.;]
21daa: ee 3b ee 3b f5 3b f5 3b ed 3b f2 3b f7 3b 05 3c [.;.;.;.;.;.;.;.<]
21dba: e2 39 df 39 eb 39 ec 39 10 3a 57 3a 42 3a 33 3a [.9.9.9.9.:W:B:3:]
21dca: 40 3a 5e 3a 78 3a 82 3a 81 3a a0 3a ac 3a 8b 3a [@:^:x:.:.:.:.:.:]
21dda: b2 3a c3 3a 5d 3a 94 3a 9e 3a e8 3a bb 3a b6 3a [.:.:]:.:.:.:.:.:]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
31d7a: 46 4c 49 52 00 01 03 09 a9 3a a6 3a 9b 3a 9e 3a [FLIR.....:.:.:.:]
31d8a: bb 3a bb 3a bd 3a bc 3a b2 3a e2 3a dd 3a ba 3a [.:.:.:.:.:.:.:.:]
31d9a: b4 3a b9 3a b5 3a b7 3a bf 3a ba 3a c1 3a ce 3a [.:.:.:.:.:.:.:.:]
31daa: cb 3a d7 3a d0 3a c7 3a 74 3a 27 3a 3b 3a 15 3a [.:.:.:.:t:':;:.:]
31dba: 2b 3a 23 3a 23 3a 12 3a 10 3a 39 3a 39 3a 63 3a [+:#:#:.:.:9:9:c:]
31dca: a8 3a c0 3a be 3a 72 3a 08 3a e9 39 e7 39 01 3a [.:.:.:r:.:.9.9.:]
31dda: 0c 3a 61 3a 2f 3a e6 39 a7 39 a3 39 a3 39 a2 39 [.:a:/:.9.9.9.9.9]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
41d7a: 46 4c 49 52 00 01 04 09 4e 3b 5c 3b 61 3b 21 3b [FLIR....N;\;a;!;]
41d8a: 09 3b 3a 3b d6 3a 64 3a 23 3b a4 3b ae 3b ac 3b [.;:;.:d:#;.;.;.;]
41d9a: d3 3b c9 3b de 3b c4 3b b0 3b c6 3b c1 3b d2 3b [.;.;.;.;.;.;.;.;]
41daa: ce 3b d4 3b cb 3b c4 3b cb 3b c8 3b 81 3b 7e 3b [.;.;.;.;.;.;.;~;]
41dba: 85 3b a0 3b 7c 3b 74 3b 6d 3b 6b 3b 62 3b 69 3b [.;.;|;t;m;k;b;i;]
41dca: 7f 3b 7f 3b 72 3b 5d 3b 58 3b 2d 3b 0b 3b 73 3b [.;.;r;];X;-;.;s;]
41dda: 0e 3b 2a 3b 22 3b 2f 3b da 3a c1 3a cd 3a a8 3a [.;*;";/;.:.:.:.:]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
51d7a: 46 4c 49 52 00 01 05 09 85 3a 84 3a 84 3a a1 3a [FLIR.....:.:.:.:]
51d8a: a3 3a c1 3a da 3a cc 3a a6 3a e9 3a d6 3a a5 3a [.:.:.:.:.:.:.:.:]
51d9a: 94 3a df 3a 08 3b 42 3b 97 3b ab 3b ac 3b 6c 3b [.:.:.;B;.;.;.;l;]
51daa: 68 3b 80 3b 89 3b 79 3b 89 3b 78 3b 3c 3b 2b 3b [h;.;.;y;.;x;<;+;]
51dba: 46 3b 41 3b 63 3b 91 3b 94 3b 96 3b 79 3b 77 3b [F;A;c;.;.;.;y;w;]
51dca: 76 3b 99 3b 76 3b d6 3a 17 3b 48 3b 50 3b 5c 3b [v;.;v;.:.;H;P;\;]
51dda: 6d 3b 6a 3b 6f 3b 78 3b 64 3b 93 3b 7c 3b 42 3b [m;j;o;x;d;.;|;B;]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
61d7a: 46 4c 49 52 00 01 06 09 10 3b 06 3b f6 3a 09 3b [FLIR.....;.;.:.;]
61d8a: 0b 3b 46 3b 79 3b cc 3b 9c 3b 8d 3b 6b 3b 94 3b [.;F;y;.;.;.;k;.;]
61d9a: 85 3b 66 3b 5d 3b 6c 3b 93 3b 9a 3b aa 3b e4 3b [.;f;];l;.;.;.;.;]
61daa: 92 3b b8 3b c8 3b b9 3b 8d 3b 70 3b 92 3b 9a 3b [.;.;.;.;.;p;.;.;]
61dba: b2 3b 66 3b 5e 3b 69 3b 6c 3b 45 3b 45 3b 35 3b [.;f;^;i;l;E;E;5;]
61dca: 2e 3b 67 3b dc 3b de 3b af 3b c6 3b cc 3b c9 3b [.;g;.;.;.;.;.;.;]
61dda: 85 3b 32 3b 25 3b 87 3b b7 3b b0 3b 8c 3b 22 3b [.;2;%;.;.;.;.;";]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
71d7a: 46 4c 49 52 00 01 07 09 98 3b 71 3b 5f 3b 47 3b [FLIR.....;q;_;G;]
71d8a: 6d 3b c7 3b cb 3b cd 3b b0 3b a7 3b a1 3b c5 3b [m;.;.;.;.;.;.;.;]
71d9a: 80 3b 87 3b 70 3b 6d 3b 92 3b ad 3b 86 3b c5 3b [.;.;p;m;.;.;.;.;]
71daa: b4 3b af 3b c6 3b d6 3b d0 3b ac 3b b4 3b 8e 3b [.;.;.;.;.;.;.;.;]
71dba: 91 3b bb 3b b1 3b b2 3b c4 3b c6 3b c9 3b a9 3b [.;.;.;.;.;.;.;.;]
71dca: bb 3b ba 3b be 3b cb 3b b3 3b 9d 3b 9e 3b a6 3b [.;.;.;.;.;.;.;.;]
71dda: a7 3b a3 3b a6 3b ae 3b 9b 3b 94 3b bb 3b bf 3b [.;.;.;.;.;.;.;.;]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (65532 bytes):
81d7a: 46 4c 49 52 00 01 08 09 40 3b 5f 3b 45 3b 30 3b [FLIR....@;_;E;0;]
81d8a: 27 3b 0c 3b 28 3b 51 3b 3d 3b 2a 3b 8e 3b 9f 3b [';.;(;Q;=;*;.;.;]
81d9a: 51 3b 0e 3b 1d 3b 0c 3b 28 3b 35 3b 45 3b 06 3b [Q;.;.;.;(;5;E;.;]
81daa: 07 3b 05 3b 08 3b 0f 3b f0 3a e8 3a f3 3a 06 3b [.;.;.;.;.:.:.:.;]
81dba: 0f 3b 13 3b 13 3b fc 3a f5 3a f8 3a fc 3a d1 3a [.;.;.;.:.:.:.:.:]
81dca: d0 3a c0 3a ef 3a d9 3a e2 3a 37 3b 4b 3b 4a 3b [.:.:.:.:.:7;K;J;]
81dda: eb 3a ef 3a bb 3a bc 3a d6 3a e5 3a bc 3a 83 3a [.:.:.:.:.:.:.:.:]
[snip 65420 bytes]
Warning = Ignored APP1 segment length 65532 (unknown header)
JPEG APP1 (28644 bytes):
91d7a: 46 4c 49 52 00 01 09 09 e4 3a fd 3a e8 3a f9 3a [FLIR.....:.:.:.:]
91d8a: fe 3a 07 3b f8 3a fc 3a f4 3a fe 3a dc 3a e7 3a [.:.;.:.:.:.:.:.:]
91d9a: d9 3a c3 3a c1 3a bd 3a ab 3a c5 3a c8 3a b2 3a [.:.:.:.:.:.:.:.:]
91daa: d3 3a c7 3a a1 3a af 3a 94 3a 93 3a 92 3a 74 3a [.:.:.:.:.:.:.:t:]
91dba: 7f 3a 86 3a ae 3a a4 3a b1 3a 97 3a b7 3a 9b 3a [.:.:.:.:.:.:.:.:]
91dca: 9d 3a 91 3a 8a 3a 9e 3a 8f 3a 96 3a 93 3a a5 3a [.:.:.:.:.:.:.:.:]
91dda: b3 3a b5 3a 96 3a b7 3a 7a 3a 96 3a 94 3a af 3a [.:.:.:.:z:.:.:.:]
[snip 28532 bytes]
Warning = Ignored APP1 segment length 28644 (unknown header)
as attachments the converted picture from wikipedia-site overlayed with 2 different palettes
# convert p640.png rain256.png -clut p640-rainbow.jpg
# convert p640.png Midgreen256.png -clut p640-Midgreen.jpg