DJI H20T thermal radiometric jpeg file

[Jonathan McGillivray]

DJI have released a new camera with an integrated 640 x 512 radiometric thermal camera. The thermal camera does not seem to be based on a Flir core.

I have been trying to get the raw temperature information from the radiometric jpeg but they seem to be storing the temperature information in a different format to the Flir R-JPEG images. Their spec sheet just says that the image format is R-JPEG (16 bit).

Any help or guidance with how to extract the temperature information using ExifTool would be greatly appreciated. I have attached a sample image.

[Phil Harvey]

There are a number of new APP segments in this file.  There is 650 kB of data in APP3 segments that could be the raw thermal data.  There is another 32 kB in APP5 that could potentially be something like this too.  APP4 is smaller, and possibly contains some setting or calibration information.  APP6 contains this text:

Code Select Expand

DTAT
{
"points":[
{"x":292,"y":228,"id":0},
{"x":399,"y":178,"id":1},
{"x":489,"y":276,"id":2},
{"x":290,"y":388,"id":3},
{"x":106,"y":341,"id":4}],
"rects":[
{"x":66,"y":75,"width":169,"height":179,"id":0},
{"x":465,"y":56,"width":104,"height":164,"id":1}],
"code":0
}

Lots of fun stuff to try to decode.  You can see it yourself with the -v3 option or with -htmldump

- Phil

[Jonathan McGillivray]

Thanks for the quick response.

You are correct. The APP3 segment seems to be the raw sensor values. I've had a friend look at the image in python and he's managed to convert the APP3 segments from byte arrays into 16-bit unsigned integer pixels. See attached .tiff image generated from the APP3 segment data; However, values range from 18073 - 21222. I'm guessing the calibration data is required to convert the raw sensor values to actual temperature readings.

Can't seem to find out what encoding to use to decode the APP4 segment. Any ideas?

APP6 contains information stored from adding 'features' to the image in their software. For example, users can add points or rectangles to get temperature measurements from the image within their software. This information is subsequently stored in the image file under this segment.

[Phil Harvey]

Some of the APP4 data may be 2-byte integers.  Other parts look like maybe 4-byte floats, but I don't have time to look into this in any depth right now.

- Phil

[mangulo]

Hi all,  ... I would like to seek help from the university to decoding APP4 segment.

[Jonathan McGillivray]

This is the code in python. Not sure how one could do it using ExifTools yet

Code Select Expand


from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
import tifffile as tiff

im = Image.open("DJI_20200513193136_0004_THRM.JPG")

# concatenate APP3 chunks of data
a = im.applist[3][1]
for i in range(4, 14):
    a += im.applist[i][1]

# create image from bytes
img = Image.frombytes('I;16L', (640, 512), a)

temps = np.array(img)

# still need to apply formula to image to convert raw sensor values to temps

tiff.imsave('DJI_20200513193136_0004_THRM.tiff', temps)
np.savetxt('temps.csv', temps, delimiter=',')

plt.imshow(temps.astype(np.uint), cmap="inferno")
plt.show()


[musteresel]

I'm working on this atm.

When zeroing the APP4 segment and then opening the image with the DJI Thermal Analysis Tool gives the error message

Code Select Expand


>>>> ir_image_temp_info_get
ERROR: magic header adjust failed-7


When zeroing the APP5 segment, then that tool gives this error message:

Code Select Expand


>>>> ir_image_temp_info_get
ERROR: curve lut monotonicity adjust failed-8


So the APP5 segment is apparently also needed for decoding; it's a series of ever increasing values (unsigned 16 bit, with wrap around)

The APP4 segment is (in my images) mostly zeros, but information seemingly spread out at random.  Do you know what kind of format that could be?

[Jonathan McGillivray]

It seems like a compensation is done to the images for Flat Field Corrections to prevent a sudden change in temperature values between FFCs. My guess is that maybe this compensation is stored within the APP5 segment?

I've been trying to compare the APP4 data between two different cameras. See attached spreadsheet showing the APP4 data in hex for 2 different cameras and multiple images from the same camera. It seems like all the images start with the same header. If I zero any of the values within this header no temperature information is displayed within the software. I then started changing other values within the APP4 segment. This caused the temperatures within the software to change which shows that these values are definitely used to convert the raw sensor values to temperature values. I highlighted these values in green within the spreadsheet. I'm not sure how one would go about converting these values from binary/hex to actual decimal or integer values without knowing what you're looking for. Hoping that Phil will be able to add some input once he's got time to look at it.

[musteresel]

I've had some success with reverse engineering the DJI Thermal Analysis Tool:

- It is an electron app
- This means it has an app.asar file; which contains a js file with the code which runs the application
- That code uses node-ffi (and ref, ref-struct, ref-array) libraries to access a dll: dji_ir_measure.dll (which uses another dll which I think contains the actual code)

This allowed me to get out which functions are in that DLL, and also on what kind of data structures they operate.  I was able to assemble a C header file with these structures (member names come from the JS code, structure names had been obfuscated) and typedefs for functions pointers for (most of) the available functions:

Code Select Expand


#include <stdint.h>

#pragma pack(1)
struct A
{
  uint16_t raw_width;
  uint16_t raw_height;
  uint32_t raw_size;
  uint32_t header_size;
  uint32_t curvelut_size;
};

#pragma pack(1)
struct S
{
#pragma pack(1)
  struct correct_params_t
  {
    uint16_t distance;
    uint16_t humidity; //hunidity??
    uint16_t emiss;
    int16_t reflection;
    int16_t ambient_temp;
  } correct_params;
};


#pragma pack(1)
struct N
{
  uint32_t app_version;
  char dll_version[16];
#pragma pack(1)
  struct device_info_t
  {
    char serial_number[32];
    uint64_t timestamp;
#pragma pack(1)
    struct version_t
    {
      char sys[16];
      char ps[16];
      uint32_t pl;
      uint32_t hardware;
    } version;
  } device_info;
  struct A raw_info;
  struct S temp_param;
  char padd[10]; // no idea why this is neccessary? it's not in the JS code...
};



// before using any function with this, probably there needs to be a
// function called which fills a struct A!
#pragma pack(1)
struct L
{
  char * product_name; // "Plug\0" : "IR_CAM\0"
  uint16_t *raw; // pointer to array?
  uint16_t *raw_header; // pointer to fresh allocated array?
  int16_t *curve_lut; // pointer to fresh allocated array?
  uint16_t width; // these are set!
  uint16_t height; // this, too
  uint16_t debug; // set to 0; increase!
};

#pragma pack(1)
struct ColorBar
  {
    uint8_t auto_enable;
    uint8_t temp_unit;
    uint8_t res2;
    uint8_t res3;
    int16_t high;
    int16_t low;
  };


#pragma pack(1)
struct H
{
#pragma pack(1)
  struct image_processing_t
  {
    uint8_t format_input;
    uint8_t format_output;
    uint8_t brightness;
    uint8_t contrast;
    uint8_t sharpness;
    uint8_t pseudo_color;
    uint8_t res1;
    uint8_t res2;
  } image_processing;
#pragma pack(1)
  struct isotherm_t
  {
    uint8_t enable;
    uint8_t res1;
    uint8_t res2;
    uint8_t res3;
    int16_t high;
    int16_t low;
  } isotherm;
#pragma pack(1)
  struct roi_t
  {
    uint8_t roi_mode;
    uint8_t res1;
    uint16_t start_x;
    uint16_t start_y;
    uint16_t size_width;
    uint16_t size_height;
  } roi;
  struct ColorBar color_bar;
};


#pragma pack(1)
struct D
{
  struct ColorBar color_bar;
};

#pragma pack(1)
struct M
{
  uint8_t pseudo_color_r[2560];
  uint8_t pseudo_color_g[2560];
  uint8_t pseudo_color_b[2560];
};


typedef char type_c;
typedef char type_e;
typedef char type_a;


#pragma pack()
struct S_small
{
  int16_t lut_temp_ptr;
  uint16_t lut_y16_ptr;
  uint32_t lut_length;
};


typedef int (*dji_ir_image_product_info_get)
(
char const *,
struct A * // type_c * ??
);

typedef int (*dji_ir_image_temp_info_get)
(
struct L*,
struct N *
);

typedef int (*dji_ir_image_temp_measurement)
(
struct L*, /* "rawInfo" */
struct S* /* nullable, "tempInfo" */,
int16_t * /* fresh allocated; width * height * 2 (bc. 16 bit!), "temperData" */
);

typedef int (*dji_ir_image_isp_process)
(
struct L*,
struct S * /* nullable*/,
struct H *,
struct D *, // type_a *,
uint8_t * /* out buffer */
);

typedef int (*dji_get_pseudo_color_data_rgb)
(
struct M * /* most probably! was type_g */
);

typedef int (*dji_ir_refresh_temp_discrete_lut)
(
struct L*,
struct S* /* nullable*/,
struct S_small * /* nullable*/
);

typedef void (*dji_ir_refresh_temp_discrete_lut_free)(void);

typedef int (*dji_ir_save_data)
(char *, uint32_t, char*, uint32_t);

typedef int (*dji_ir_read_data)
(char *, uint32_t, char*, uint32_t *);


This allows me to link against the DLL and get the temperature data, for example:

Code Select Expand


void temp_measurement_example(dji_ir_image_temp_measurement fn) // supply pointer to function loaded from DLL
{
  int const width = 640;
  int const height = 512;
  char const o[] = "IR_CAM\0";
  char *raw, *raw_header, *curve_lut;
  size_t raw_size, raw_header_size, curve_lut_size;
  ReadJpgFile("file.jpg",
              &raw, &raw_size,
              &raw_header, &raw_header_size,
              &curve_lut, &curve_lut_size);
  struct L h;
  h.product_name = o;
  h.raw = raw;  // this is the APP3 segments concatenated
  h.raw_header = raw_header; // this is the APP4 segment
  h.curve_lut = curve_lut; // this is the APP5 segment
  h.width = width;
  h.height = height;
  h.debug = 0;
  int16_t * data = malloc(2 * width * height);
  int ret = fn(&h, NULL, data); // call to the DLL function

  // now we have the temperature values in data! In (°C/10)


  int16_t max = INT16_MIN;
  int16_t min = INT16_MAX;
  for (int i = 0; i < width * height; ++i)
    {
      if (max < data[i]) max = data[i];
      if (min > data[i]) min = data[i];
    }
  printf("min = %d\nmax = %d\n", min, max);
  printf("TopLeft = %d\nTopRight = %d\nBottomRight = %d\nBottomLeft = %d",
         data[0],
         data[width - 1],
         data[width * height - 1],
         data[width * (height - 1)]
         );
}



It doesn't end here...

Being an electron app, it is possible to INJECT code into the DJI Thermal Tool; which makes it possible to observe exactly how and when functions from the DLL are called:

1. Unpack app.asar

Code Select Expand

npx asar extract app.asar extracted
2. Modify extracted/app/dist/js/app.dc53e99d.js file to include any logging or similar
3. Pack a new app.asar

Code Select Expand

npx asar pack extracted app.asar
4. Run DJI Tool with injected code Perhaps

Code Select Expand

set  ELECTRON_ENABLE_LOGGING=true (as per https://stackoverflow.com/a/40929681/1116364) to be able to use

Code Select Expand

console.log("WOW") in the injected code

What's really nice is that the DLL by default seems to print quite a bit debugging information; here's what I get when opening one of my files:

Code Select Expand


>>>> ir_image_temp_measurement
temp head : [0x55AA][0x0612]
dev  head : [0xAA55][0x0612]
K1          -130.443
K2          26.1648
K3          758.386
K4          0
KF          79.5847
B1          0
B2          -575.543
Distance    5
Humidity    70
Emiss       100
Reflection  230
D1          21
D2          -945
KJ          100
DB          0
KK          261


Searching for the HEX values of Distance, Humidity, ... shows that they are at offsets in the APP4 segment:

- 0x44 for distance
- 0x46 for humidity
- 0x48 for Emiss
- 0x4a for reflection
- 0x4c for D1 (what ever that is)

Code Select Expand


00000030: 0000 0000 5b2b 9f42 0000 0000 bee2 0fc4  ....[+.B........
00000040: 0000 0000 0500 4600 6400 e600 1500 0000  ......F.d.......
00000050: 4ffc ffff 6400 0000 0501 0100 0300 0000  O...d...........


I don't know if these offsets are constant, though.  APP4 does not seem to use IFD tags, or I'm just not used to decoding them yet?

However, the values show that there are quite a few parameters to extract.

*edit*

Looking at your spreadsheet shows where most of these values seem to be. The question remains: Are these offsets constant?
Also the last two bytes of the APP4 segment change relatively "freely", this could be a checksum or similar?

Linking these posts (which are related to FLIR raw values decoding) because the parameters here might be similar, or possibly even the same with the extra parameters to include distance and external optics transmission in the calculation:

- https://exiftool.org/forum/index.php/topic,4898.msg23944.html#msg23944
- https://exiftool.org/forum/index.php/topic,4898.msg23972.html#msg23972

[Phil Harvey]

Wow, that is some excellent detective work!!

The offsets for Distance, etc are fairly low numbers (0x44 is near the start of the segment), which is great.  If they were variable, one would expect to see a size word stored in the segment before 0x44 with a value value of less than or equal to 0x44.  But I don't see anything before 0x44 that looks like size/offset value or a version number.  So my guess is that the offsets are constant for these parameters.

- Phil

[FRANKOOL]

Congratulations to everyone for your effort. Is there a relatively simple way to convert thermal files from dji to tiff with the temperature information compensated by the FFC?
In app4 the raw values ​​of the differentials multiplied by 0.005, which is the thermal sensitivity of the camera, coincide with the values ​​obtained in the DJI software. With this and a value from raw pixel to centigrade we could convert to real degrees, but this step is the one that we do not know how to do. Greetings and thank you very much for the effort!

[hairyape1]

DJI have been very short sighted in effectively encrypting their data so that only their software can be used to view the data. Being able to view the raw data in a more useful format eg tiff would allow so many more possibilities in terms of analysis etc.
Ive been in discussions with DJI daily for the last couple of months to try and resolve this and they are not being very helpful :/
This is all way beyond my level of understanding, but great to see that you are making some progress.
Thank you all for your efforts so far. If DJI every give me any actually useful information Ill post it here.
Cheers
Fingers crossed we can overcome this one way or another.

[hairyape1]

If it helps to determine whether offsets/calibration data is standard from one sensor to another... here are a selection of example images captured with my h20t. All captured on the same day within a few minutes of each other.
Very happy to go and collect more data if anything specific would be of use.

[Jonathan McGillivray]

That's some really good reverse engineering @musteresel! Thanks for sharing all your work. Based on the info you provided I've created a wrapper to use the .dll within python using ctypes. Anyone who's not too comfortable with C (like myself) can use this in the meantime to batch convert the DJI R-JPEG images to 16 bit tiff images.

It would be great to still try and figure out what formula DJI is applying to do the temperature conversion within their .dlls. I tried to take a quick look at the values provided with Electron logging enabled and I couldn't pick up a correlation between Flir's calibration values and the values printed through the logging. My values are as follows:

Code Select Expand


K1          -98.3823
K2          25.5608
K3          655.766
K4          0
KF          80.438
B1          0
B2          -463.6
Distance    5
Humidity    70
Emiss       100
Reflection  230
D1          21
D2          -945
KJ          100
DB          0
KK          204


My guess is that maybe K1, K2, K3, K4, KF, B1, B2 are the values used within the formula. I'll try take a look into it in more detail when I get some more time. If anyone else with experience in calibration of thermal sensors has got any more insight it would be appreciated.

Python Wrapper for DLLs

Code Select Expand


from ctypes import (
    Structure,
    c_uint64,
    c_uint32,
    c_uint16,
    c_int16,
    c_uint8,
    c_int,
    c_char,
    c_char_p,
    POINTER,
    util,
    CDLL
)

import sys

class A(Structure):
    _pack_ = 1
    _fields_ = [
        ("raw_width", c_uint16),
        ("raw_height", c_uint16),
        ("raw_size", c_uint32),
        ("header_size", c_uint32),
        ("curvelut_size", c_uint32)
    ]


class correct_params_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("distance", c_uint16),
        ("humidity", c_uint16),
        ("emiss", c_uint16),
        ("reflection", c_int16),
        ("ambient_temp", c_int16),
    ]


class S(Structure):
    _pack_ = 1
    _fields_ = [
        ("correct_params", correct_params_t),
    ]


class version_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("sys", c_char * 16),
        ("ps", c_char * 16),
        ("pl", c_uint32),
        ("hardware", c_uint32),

    ]


class device_info_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("serial_number", c_char * 32),
        ("timestamp", c_uint64),
        ("version", version_t)
    ]


class N(Structure):
    _pack_ = 1
    _fields_ = [
        ("app_version", c_uint32),
        ("dll_version", c_char * 16),
        ("device_info", device_info_t),
        ("raw_info", A),
        ("temp_param", S)
    ]


class L(Structure):
    _pack_ = 1
    _fields_ = [
        ("product_name", c_char_p), # "Plug\0" : "IR_CAM\0"
        ("raw", POINTER(c_uint16)),
        ("raw_header", POINTER(c_uint16)),
        ("curve_lut", POINTER(c_int16)),
        ("width", c_uint16),
        ("height", c_uint16),
        ("debug", c_uint16) # set to 0
    ]

class ColorBar(Structure):
    _pack_ = 1
    _fields_ = [
        ("auto_enable", c_uint8),
        ("temp_unit", c_uint8),
        ("res2", c_uint8),
        ("res3", c_uint8),
        ("high", c_int16),
        ("low", c_int16)
    ]

class image_processing_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("format_input", c_uint8),
        ("format_output", c_uint8),
        ("brightness", c_uint8),
        ("contrast", c_uint8),
        ("sharpness", c_uint8),
        ("pseudo_color", c_uint8),
        ("res1", c_uint8),
        ("res2", c_uint8)
    ]

class isotherm_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("enable", c_uint8),
        ("res1", c_uint8),
        ("res2", c_uint8),
        ("res3", c_uint8),
        ("high", c_int16),
        ("low", c_int16)
    ]

class roi_t(Structure):
    _pack_ = 1
    _fields_ = [
        ("roi_mode", c_uint8),
        ("res1", c_uint8),
        ("start_x", c_uint16),
        ("start_y", c_uint16),
        ("size_width", c_uint16),
        ("size_height", c_uint16),
    ]

class H(Structure):
    _pack_ = 1
    _fields_ = [
        ("image_processing", image_processing_t),
        ("isotherm", isotherm_t),
        ("roi", roi_t),
        ("color_bar", ColorBar)
    ]

class D(Structure):
    _pack_ = 1
    _fields_ = [
        ("color_bar", ColorBar),
    ]

class M(Structure):
    _pack_ = 1
    _fields_ = [
        ("pseudo_color_r", c_uint8 * 2560),
        ("pseudo_color_g[2560]", c_uint8 * 2560),
        ("pseudo_color_b[2560]", c_uint8 * 2560),
    ]
class S_small(Structure):
    _fields_ = [
        ("lut_temp_ptr", c_int16),
        ("lut_y16_ptr", c_uint16),
        ("lut_length", c_uint32),
    ]

mylib_path = util.find_library("./dji_ir_measure")
if not mylib_path:
    print("Unable to find the specified library.")
    sys.exit()

try:
    mylib = CDLL(mylib_path)
except OSError:
    print("Unable to load the system C library")
    sys.exit()

dji_ir_image_temp_measurement = mylib.dji_ir_image_temp_measurement
dji_ir_image_temp_measurement.argtypes = [POINTER(L), POINTER(S), POINTER(c_int16)]
dji_ir_image_temp_measurement.restype = c_int


You can then use the above code to convert the R-JPEG images as follows:

Code Select Expand


from PIL import Image
from ctypes import (
    c_uint16,
    c_int16,
    c_char_p,
    POINTER,
    cast,
    create_string_buffer,
    byref
)
import numpy as np
import tifffile as tiff

from dji_ir_measure_wrapper import dji_ir_image_temp_measurement, L, S, correct_params_t

def read_image(filepath):
    im = Image.open(filepath)
    a3, a4, a5 = None, None, None
    for app in im.applist:
        if app[0] == 'APP3':
            a3 = a3 + app[1] if a3 else app[1]
        elif app[0] == 'APP4':
            a4 = a4 + app[1] if a4 else app[1]
        elif app[0] == 'APP5':
            a5 = a5 + app[1] if a5 else app[1]
    width, height = im.size
    return {'raw': a3, 'raw_header': a4, 'curve_lut': a5, 'width': width, 'height': height}

def create_L_instance(image):
    instance = L()
    instance.product_name = cast(create_string_buffer(b"IR_CAM"),
                          c_char_p)
    instance.raw = cast(image['raw'], POINTER(c_uint16))
    instance.raw_header = cast(image['raw_header'], POINTER(c_uint16))
    instance.curve_lut = cast(image['curve_lut'], POINTER(c_int16))
    instance.width = image['width']
    instance.height = image['height']
    instance.debug = 0
    return instance

def create_params_instance(settings):
    instance = S()
    params = correct_params_t(**settings)
    instance.correct_params = params
    return instance

image = read_image('DJI_20200513193308_0030_THRM.JPG')
settings = {'distance': 20, 'humidity': 70, 'emiss': 100, 'reflection': 230}

data = np.zeros(image['width'] * image['height'], dtype=np.int16)
data_ptr = data.ctypes.data_as(POINTER(c_int16))

h = create_L_instance(image)
params = create_params_instance(settings)
ret = dji_ir_image_temp_measurement(byref(h), params, data_ptr) # // call to the DLL function

max_temp = np.nanmax(data) / 10
min_temp = np.nanmin(data) / 10
print("Max: {}, Min: {}".format(max_temp, min_temp))

data = np.reshape(data, (image['height'], image['width'], 1))
tiff.imsave('image.tiff', data) #temperature values in data are °C/10


Note that the above script needs to be run from a 32bit python version with libdji_irprocessing.dll and dji_ir_measure.dll in the same directory.

[hairyape1]

Thanks Jonathan,

Just having a go with python to convert some images this end well to be honest not, me but a friend is helping out.... Im good with the actual thermal data but less experienced with python etc. Will report back how we get on.

Are any of the calibration values constant from one sensor to the next? Im wondering if they use some constant values and only vary a few depending on specific sensor variation from one sample to the next. When I was involved in calibrating optris sensors we would have some fixed values and then some sensor specific values and some lens specific.

Cheers

[Chanc3]

Hello all

Following this with great interest, as quite frankly, the DJI software is bad for what I need - batch image editing and a reporting feature! Do you think it would ever be possible to convert a DJI RJEPG to a FLIR format? Would make a great many peoples' lives a lot easier!

Happy to put up a small bounty for this!

[sebastianbarria]

Hi everyone.
I'm having the same problem about extracting thermal information from ZH20T camera.
I used to extract pixel by pixel information from older cameras using Thomas123 php script file, but now, with the new DJI Matrice 300, the JPEG files have the information "hidden".
I can't find a way to extract the planck constants in order to find the pixel by pixel temperature.
Does anyone have an idea on how to do that?

Just in case, I'm attaching a thermal JPEG file captured with the M300 RTK.
Thanks.

[remi_beauvais]

Hi everyone!

I've combined all your incredible findings in a script so you don't have to!
So, in the link below you can find a little tool that can look into a folder with the source JPG and create the tiff with temperature x10 in a subfolder.
It works on windows with a portable version of python.

How to use :
You have to edit the param.py file to link to your folder with the images you want to transform :
Here is the param.py file :
Image_folder = 'D:/Drone/temperature_JPG/'
Output_folder = "out_temperature/"

So, if you want to transform the images in C:\your\path\to\the\DJI\images\ you have to change the following line :
Image_folder = 'D:/Drone/temperature_JPG/'
to
Image_folder = 'C:/your/path/to/the/DJI/images/'


Please notice the type of separator used ("/" instead of "\") for the absolute path.

When the param.py file is configured, you just have to launch the "extract.bat" file and it will create the subfolder in the source image path and you will be granted with a little log in the cmd prompt.


Here to download the thingy : https://drive.google.com/file/d/1ckUXwcGND4UjqpLqqYQV8COt_LzdFzOA

If you find a better solution, I would be really happy to hear from you!

Rémi

[JoseASanchez]

Hi everyone,

I'm working in the same issue. Do you have any advance?.

@musteresel, could you provide the source code of the function ReadJpgFile?.

Thanks

[Phil Harvey]

I'm finally getting around to looking at this, and am adding the ability to read DJI FLIR APP3/APP4/APP5 data.  It does look to me as if the parameters are at constant offsets in APP4.  I am currently decoding the raw thermal and calibration data, as well as all parameters from APP4 listed in the debugging output.  Below is the output for all files posted so far in this thread.  (Note I am decoding "D1" as "AmbientTemperature", taking a cue from the correct_params_t structure which has c_int16 "ambient_temp" immediately following c_int16 "reflection".)

Code Select Expand

> exiftool tmp2 -dji:all -G
======== tmp2/DJI_20200715204138_0144_THRM.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP4]          K1                              : -101.738883972168
[APP4]          K2                              : 11.5029211044312
[APP4]          K3                              : 1047.04809570312
[APP4]          K4                              : 0
[APP4]          KF                              : 81.1580429077148
[APP4]          B1                              : 0
[APP4]          B2                              : -435.885711669922
[APP4]          Object Distance                 : 5
[APP4]          Relative Humidity               : 70
[APP4]          Emissivity                      : 100
[APP4]          Reflection                      : 230
[APP4]          Ambient Temperature             : 21
[APP4]          D2                              : -945
[APP4]          KJ                              : 100
[APP4]          DB                              : 0
[APP4]          KK                              : 191
[APP5]          Thermal Calibration             : (Binary data 32768 bytes, use -b option to extract)
======== tmp2/DJI_20200715204037_0108_THRM.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP4]          K1                              : -101.738883972168
[APP4]          K2                              : 11.5029211044312
[APP4]          K3                              : 1047.04809570312
[APP4]          K4                              : 0
[APP4]          KF                              : 81.1580429077148
[APP4]          B1                              : 0
[APP4]          B2                              : -435.885711669922
[APP4]          Object Distance                 : 5
[APP4]          Relative Humidity               : 70
[APP4]          Emissivity                      : 100
[APP4]          Reflection                      : 230
[APP4]          Ambient Temperature             : 21
[APP4]          D2                              : -945
[APP4]          KJ                              : 100
[APP4]          DB                              : 0
[APP4]          KK                              : 191
[APP5]          Thermal Calibration             : (Binary data 32768 bytes, use -b option to extract)
======== tmp2/DJI_20200513193136_0004_THRM.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP4]          K1                              : -98.3822555541992
[APP4]          K2                              : 25.5608234405518
[APP4]          K3                              : 655.765869140625
[APP4]          K4                              : 0
[APP4]          KF                              : 80.4379577636719
[APP4]          B1                              : 0
[APP4]          B2                              : -463.600006103516
[APP4]          Object Distance                 : 5
[APP4]          Relative Humidity               : 70
[APP4]          Emissivity                      : 100
[APP4]          Reflection                      : 230
[APP4]          Ambient Temperature             : 21
[APP4]          D2                              : -945
[APP4]          KJ                              : 100
[APP4]          DB                              : 0
[APP4]          KK                              : 204
[APP5]          Thermal Calibration             : (Binary data 32768 bytes, use -b option to extract)
======== tmp2/DJI_20200715203807_0023_THRM.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP4]          K1                              : -101.738883972168
[APP4]          K2                              : 11.5029211044312
[APP4]          K3                              : 1047.04809570312
[APP4]          K4                              : 0
[APP4]          KF                              : 81.1580429077148
[APP4]          B1                              : 0
[APP4]          B2                              : -435.885711669922
[APP4]          Object Distance                 : 5
[APP4]          Relative Humidity               : 70
[APP4]          Emissivity                      : 100
[APP4]          Reflection                      : 230
[APP4]          Ambient Temperature             : 21
[APP4]          D2                              : -945
[APP4]          KJ                              : 100
[APP4]          DB                              : 0
[APP4]          KK                              : 191
[APP5]          Thermal Calibration             : (Binary data 32768 bytes, use -b option to extract)
======== tmp2/DJI_20200916170844_0002_THRM.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP4]          K1                              : -116.967163085938
[APP4]          K2                              : 22.0576038360596
[APP4]          K3                              : 784.67138671875
[APP4]          K4                              : 0
[APP4]          KF                              : 80.1084518432617
[APP4]          B1                              : 0
[APP4]          B2                              : -605.314270019531
[APP4]          Object Distance                 : 5
[APP4]          Relative Humidity               : 70
[APP4]          Emissivity                      : 100
[APP4]          Reflection                      : 230
[APP4]          Ambient Temperature             : 21
[APP4]          D2                              : -945
[APP4]          KJ                              : 100
[APP4]          DB                              : 0
[APP4]          KK                              : 269
[APP5]          Thermal Calibration             : (Binary data 32768 bytes, use -b option to extract)
    1 directories scanned
    5 image files read

ExifTool 12.15 should contain this update.  Plus I'll add anything else useful that is discovered.  Up until now ExifTool hasn't ventured into doing temperature calculations, but I would consider adding this feature as well if someone can figure out how to do this based on the information extracted.

- Phil

[kiah09]

Hello,

I am currently deciding on whether to purchase an XT2 or H20T. The main purpose of my purchase is to have accurate temperature data in a programmable file format (.txt, .csc, .tiff). I have little to no experience in determining each image's calibration values in the H20T, and it seems like a major issue in obtaining accurate temperature readings. In addition, it looks like the formula to derive the temperature values is more sound and backed up via testing and verification with the FLIR product (XT2) and NOT with the H20T. Is my understanding of the issues with the H20T data correct? Are these issues resolved? Because from my reading of this forum, it seems as though they are not quite resolved, but I wanted to make sure before making such a large purchase. Thank you!

[daz]

There is an official DJI Thermal SDK with an API and Linux and Windows lib binaries. This may help reverse engineering the thermal calibration.

https://www.dji.com/au/downloads/softwares/dji-thermal-sdk

[Chanc3]

I'm a little out of the loop now. Has anyone managed to get anywhere with this?

[remi_beauvais]

Hi everyone!

I've combined all your incredible findings in a script so you don't have to!
So, in the link below you can find a little tool that can look into a folder with the source JPG and create the tiff with temperature x10 in a subfolder.
It works on windows with a portable version of python.

How to use :
You have to edit the param.py file to link to your folder with the images you want to transform :
Here is the param.py file :
Image_folder = 'D:/Drone/temperature_JPG/'
Output_folder = "out_temperature/"

So, if you want to transform the images in C:\your\path\to\the\DJI\images\ you have to change the following line :
Image_folder = 'D:/Drone/temperature_JPG/'
to
Image_folder = 'C:/your/path/to/the/DJI/images/'


Please notice the type of separator used ("/" instead of "\") for the absolute path.

When the param.py file is configured, you just have to launch the "extract.bat" file and it will create the subfolder in the source image path and you will be granted with a little log in the cmd prompt.


If you find a better solution, I would be really happy to hear from you!

Rémi

The new link since the old one is broken : https://drive.google.com/file/d/1S1-KOr9CjIeYOB3Nn6j6JHNYXwDTziyB

[geo-nerd]

Hi everyone!

I'm looking for some help with this error I'm getting with the package Remi put together from all your coded pieces.

I followed Remi's instructions in changing the imagery input and output folders and then ran the bat file..but I got an error as it couldn't located the .dll file even though both dll files are in the same directory within the same file folder structure that Remi had.


this is my line of code that I tweaked from the wrapper.py script to try to get it to look in the right place:

Code Select Expand

libname_so = 'dji_ir_measure' + so_ext
mylib = CDLL(os.path.join(os.path.dirname("C:/Users/Cami/Downloads/traitement_temperature-1/traitement_temperature"), libname_so))

And this is the error message I got:

Code Select Expand

RESTART: C:\Users\Cami\Downloads\traitement_temperature-1\traitement_temperature\dji_ir_measure_wrapper.py
Traceback (most recent call last):
  File "C:\Users\Cami\Downloads\traitement_temperature-1\traitement_temperature\dji_ir_measure_wrapper.py", line 175, in <module>
    mylib = CDLL(os.path.join(os.path.dirname("C:/Users/Cami/Downloads/traitement_temperature-1/traitement_temperature/dji_ir_measure.dll"), libname_so))
  File "C:\Users\Cami\Downloads\traitement_temperature-1\traitement_temperature\python_portable\WPy32-3850\python-3.8.5\lib\ctypes\__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'C:\Users\Cami\Downloads\traitement_temperature-1\traitement_temperature\dji_ir_measure.dll' (or one of its dependencies). Try using the full path with constructor syntax.

Any help is sooo greatly appreciated!

[OZ]

Hi everyone,

Thank you all for your efforts to extract the raw values from DJI radiometric format!

According to this, did someone manage to extract the value of the real temperature from M2EA radiometric Images (I found that this is not the same format as H20T, why DJI??!!?!)??

I manage to extract the raw data from the image using Exiftool v12.30 but what is the formula to calculate the temp in Celsius?

Thanks a lot!!

[daz]

I use the DJI thermal SDK, it comes with a tool to process a binary of temperature floats from a JPEG. Dockerized here: https://github.com/daz/dji-thermal-sdk-docker

[OZ]

@daz Thanks for the docker, but this version still does not support Mavic 2 enterprise advanced radiometric images.

I was wondering if someone has successfully done so using Exiftool...

[daz]

Nah sorry, the thermal SDK only supports XT S and H20T. Nobody has successfully reversed engineered the proprietary DJI temperature from raw algorithms for any cameras yet. I would suggest pulling apart the Thermal Analysis Tool 2.0 the same way users have here with version 1, seeing if you can expose any APIs in the DLL that relate to the Mavic thermal. Can you upload an example image from the Mavic?

[OZ]

Sure, check out the attached image.

[juan.bueno]

Hello to everyone, I'm new in this forum. I would like to know if there is any news regarding the extraction of the temperature values ​​of the DJI M2EA

[ccg91982]

Tengo el mismo problema, sin experiencia en Python, y con una necesidad extrema de poder transformar las imágenes térmicas de M2EA. Si lo llego a saber no lo compro.

Gracias a todos por esta linea de trabajo, me parecéis excepcionales.

[Fedro]

Hi everyone!

I've combined all your incredible findings in a script so you don't have to!
So, in the link below you can find a little tool that can look into a folder with the source JPG and create the tiff with temperature x10 in a subfolder.
It works on windows with a portable version of python.

Hi Rémi,
really thank you for your solution, I was able to convert the images thanks to your tool.

The thing I noticed next is that the converted tiff images have no longer all the geographic coordinates that the jpg images had.

Is there a way to transcribe the geographic coordinates on the new tiff images? this would be useful for photogrammetric processing with pix4d software or others.

Thank you,
Fedro

[Phil Harvey]

Hi Fedro,

Is there a way to transcribe the geographic coordinates on the new tiff images?

You should be able to do that with ExifTool:

exiftool -tagsfromfile %d%f.jpg "-gps*" -ext tif DIR

(assuming you are using a ".TIF" extension)

- Phil

[Tavana81]

Hello to everyone, I'm new in this forum. I would like to know if there is any news regarding the extraction of the temperature values ​​of the DJI M2EA

Has anyone extracted the formula of thermal calculation of the m2ea .

[sebastianbarria]

The method to get the temperatures from the m2ea is the same as the m300... the thermal information is not inside the metadata (rawThermalImage) but in the blocks.
In this forum there's a code to read those temperatures... or maybe this link could help:
https://github.com/gtatters/Thermimage/blob/master/R/raw2temp.R
(I'm not expert in python, so I can't help you that much)
Good luck!

[rroldan]

Hi everyone. I know this post is old, but related with the topic, we've just released a tool to convert RJPG images to TIFF images with radiometric and all metadata information.
Please, visit https://thermo-converter.com/ to learn more.

[jdavis]

Phil, I'm using exiftool 12.40, I'm trying to read the APP4 data from a DJI thermal image as you did below.

Code Select Expand

jdavis@openactive-desk:~/Downloads/jonny-house/IronBow Thermal Ortho Mavic 3 T 90ft$ exiftool -dji:all -G DJI_20221027130834_0001_T.JPG
[APP3]          Thermal Data                    : (Binary data 655360 bytes, use -b option to extract)
[APP5]          Thermal Calibration            : (Binary data 23818 bytes, use -b option to extract)



Am I doing something wrong? Or did DJI maybe change the file format? Thanks!

[Phil Harvey]

I would say that your model likely doesn't store the information mentioned.  But if you could upload an image I could take a look.

- Phil

[jdavis]

Hey Phil,
 Sorry I forgot to get back with you! Here is an image. Its from a DJI Mavic 3. I have loads of them, so if you need anymore, or from any other dji thermal drone let me know!

[Phil Harvey]

Your file contains an unrecognized APP4 segment which may contain some useful information.  The format is very different form the APP4 which has been decoded for the ZH20T:

Code Select Expand

JPEG APP4 (256 bytes):
   a884f: ff d2 d1 ff 69 69 72 70 00 00 00 00 00 00 00 00 [....iirp........]
   a885f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a886f: 00 00 c8 41 00 00 a0 40 33 33 73 3f 00 00 00 3f [...A...@33s?...?]
   a887f: 00 00 c8 41 00 00 00 00 00 00 00 00 00 00 00 00 [...A............]
   a888f: 2c 01 20 00 80 2c 01 80 00 00 00 00 00 00 00 00 [,. ..,..........]
   a889f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a88af: e8 1f 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a88bf: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a88cf: 01 03 00 00 00 4d 69 6e 69 5f 36 34 30 00 00 00 [.....Mini_640...]
   a88df: 00 00 00 00 00 03 05 01 0a 00 02 00 00 80 02 00 [................]
   a88ef: 02 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a88ff: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a890f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a891f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a892f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
   a893f: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 [................]
Starting at offset 0xa886f (32 bytes from the start of APP4), there are 6 floating-point numbers that look interesting:

Code Select Expand

       25
        5
     0.95
      0.5
       25
Any idea what these are?

We could probably decode this information with more samples having as many known parameters as possible.

- Phil

[jdavis]

Here you can see in the parameters section what those values are:

25 = reflected temp
0.5 = humidity
5 = distance
0.95 = emissivity

Thanks!

[Phil Harvey]

Great!

Do you have any samples of images where these values are different?  Specifically, I would like to know whether the first 25 or the last 25 represents the reflected temperature (and then what the other 25 represents).

- Phil

[Phil Harvey]

BTW, feel free to send me any DJI samples that you have that I don't already have.  Here is a list of models for which I have samples (the "Model" tag):

FC1102
FC2103
FC220
FC2204
FC2403
FC300C
FC300S
FC300X
FC300XW
FC330
FC6310
FC7203
FC7303
FLIR
M3T
MAVIC2-ENTERPRISE-ADVANCED
PHANTOM VISION FC200
XT2
ZH20T

You can email them to philharvey66 at gmail.com

Thanks.

- Phil

[jdavis]

I just checked images from two different cameras, both on an M3T and they have the exact same values. I'm going to get a pilot to change the settings in the flight app and see if they show up in the exif. I'll let you know when I have the information. thanks.

[jdavis]

Looks like I can not change those settings in the M3T. So they are basically just defaults, which are useless really.

https://dl.djicdn.com/downloads/DJI_Mavic_3_Enterprise/20230303/DJI_Mavic_3E_3T_User_Manual_EN.pdf

[Phil Harvey]

OK, thanks for checking.

I tried modifying the file by hand and downloaded the DJI thermal analysis tool to see what changed in the display.  The second 25 is the reflected temperature, and I'm going to assume the first 25 is the ambient temperature (though not displayed by the DJI software).  I've added the code to extract these and the other parameters.  I figure they may be useful at some point since the DJI software goes to the effort to display them.

- Phil

[knussear]

Hi everyone!

I've combined all your incredible findings in a script so you don't have to!
So, in the link below you can find a little tool that can look into a folder with the source JPG and create the tiff with temperature x10 in a subfolder.
It works on windows with a portable version of python.

I'm getting an error when trying this.

Code Select Expand

C:\Users\knussear\Documents\Argentina\traitement_temperature>.\python_portable\WPy32-3850\python-3.8.5\python.exe extract_temp.py
C:/Users/knussear/Documents/Argentina/Flight2Thermal/DJI_0001.jpg
Traceback (most recent call last):
  File "extract_temp.py", line 100, in <module>
    image_raw = read_image(imagepath)
  File "extract_temp.py", line 34, in read_image
    APP4 = im.applist[14][1]
IndexError: list index out of range

C:\Users\knussear\Documents\Argentina\traitement_temperature>pause
Press any key to continue . . .

Any idea how to resolve this?

[rszostak]

I didn't noticed it in this conversation, but i think it can be useful:

H20T ThermalCalibration binary data contains vector of numbers that look like some calibration curve after interpretation as int16 and plotting:

Code Select Expand

b_thermal_calibration = subprocess.check_output(['exiftool', '-b', '-ThermalCalibration', img_path])
thermal_calibration = np.frombuffer(b_thermal_calibration, dtype=np.int16)
plt.plot(thermal_calibration)


[rszostak]

I want to share with you the results of my experiments with H20T camera.

389 images were taken of a homogeneous target with a constant temperature (dark fabric chair seat).
I extracted the temperatures using the DJI TSDK. As the target is homogeneous I averaged the temperature for the whole image.
For the same images, I extracted the raw data from the EXIF (ThermalData). I interpreted them as int16 and also averaged for the whole image.

The comparison of results shows the image below.



You can see that with time as the internal components of the camera warm up, the measured with TSDK temperature converges to a certain equilibrium.
The raw data fluctuates significantly. The rapid changes appear to be related to the FFC correction.

Most of EXIF tags values does not change with time. Tags that have different values among images are:

Code Select Expand

['SourceFile', 'FileName', 'FileSize', 'FileModifyDate', 'FileAccessDate', 'FileCreateDate', 'ModifyDate', 'DateTimeOriginal', 'CreateDate', 'GPSLatitudeRef', 'GPSLongitudeRef', 'ThumbnailLength', 'AbsoluteAltitude', 'RelativeAltitude', 'GimbalPitchDegree', 'FlightRollDegree', 'FlightYawDegree', 'FlightPitchDegree', 'LRFStatus', 'LRFTargetDistance', 'LRFTargetAlt', 'LRFTargetAbsAlt', 'MPImageLength', 'MPImageStart', 'ThermalData', 'ThumbnailImage', 'PreviewImage']None of them are thermal calibration related. Even ThermalCalibration tag have the same binary bits for each photo. Where then the FFC information can be stored (to decode rapid changes in RAW data)? Is it possible that this information is encoded in the file, but outside the EXIF standard, so exiftool does not extract it?

All images can be downloaded here: https://aghedupl-my.sharepoint.com/:u:/g/personal/rszostak_agh_edu_pl/Ecqkog2BQntGhiBQ9miorBQBGX0od16t-AOOdnL2nSV4bQ?e=OSGnUD

[kaido]

Hi,

Have you had any luck with DJI mavic 3T/30T thermal data extraction?

[Phil Harvey]

There is a lot of unknown information in the DJI APP4 segment.  I've found some interesting unknown values, and share them here for your opinion.  One of the parameters seems to correspond with the jumps you see in your raw values -- the "Test8" tag in the attached CSV file.  Here is the command I used to generate this file:

exiftool -config dji_test.config rjpg -fileorder filename -csv -test'*' > out.csv

I've also found 7 other interesting floats that seem to increase monotonically through the images (Test1-Test7).  Maybe you can figure out what these mean.  If you come up with anything, we can give them a name and add them to the production ExifTool.

- Phil

[kaido]

Phil,

First run didn't show anything
output
rjpg/DJI_20230520113922_0575_T.JPG

https://drive.google.com/drive/folders/1PvH_6VQf3Bl7fvbljV-YRo4d6gr5At3F?usp=sharing

I'll do tomorrow some pictures with temperature measurements Fluke FLK-IRR1-SOL hopefully we have some data with what we can compare results.

Kaido

[Phil Harvey]

Code Select Expand

> exiftool -config ~/Desktop/dji_test.config ~/Desktop/ -ext jpg -test"*"
======== /Users/phil/Desktop/H20T_DJI_20200513193136_0004_THRM.JPG
Test 1                          : 26.9140625
Test 2                          : 28.0625
Test 3                          : 42.4279174804688
Test 4                          : 25.578125
Test 5                          : 17.3046875
Test 6                          : 26.9296875
Test 7                          : 27.8515625
Test 8                          : 26.9375
======== /Users/phil/Desktop/DJI_20230520113922_0575_T.JPG
======== /Users/phil/Desktop/Mavic_3T_DJI_20230524111538_0398_T.JPG
======== /Users/phil/Desktop/MAVIC2-ENTERPRISE-ADVANCED_dji__DJI_0068__thermal.jpg
- Phil

[kaido]

Phil,

images here with: air temp ~15C, panel temp ~30C, shot from screen and two IR pictures sky and panel+sky

https://drive.google.com/drive/folders/1PvH_6VQf3Bl7fvbljV-YRo4d6gr5At3F?usp=sharing

temp_test folder


Br,
Kaido

[Phil Harvey]

My last post was to address rszostak's question about the calibration jumps in the DJI H20T images.

- Phil

[kaido]

Phil,

looks like DJI uses InfiRay IR sensor is it possible they use same style data structure?

https://www.infiray.com/products/m600-thermal-hand-scanner.html - it looks like M3T IR sensor spec

[Phil Harvey]

Do you mean DJI using InfiRay structure, or visa versa?

If so, then which DJI model(s)?  There isn't much consistency in the DJI metadata.

Also, I don't have many InfiRay samples.

- Phil

[kaido]

dji uses infiray sensor and data structure. mavic 3t

[Phil Harvey]

ExifTool decodes all of the APP segments in my M3T sample.  APP3 is the raw thermal data just like InfiRay, but the other APP segments are different from InfiRay.

- Phil

[Drone Link]

Hey all,

I know im late to the party, but whats the latest on this issue? It seems to me the prevailing options are:
dji_h20t_rpeg_to_tif-main; by tejakattenborn
or traitement_temperature; by Remi

Is there anything more up to date? Would love to be updated on whats happened.
Thanks