Extracting Depth Image data from Samsung phone

[t0theRANCH]

I was trying to use the information available here on the forums to extract a depth map from a picture I took with my phone (S21 Ultra, picture attached).

exiftool -s -u pic.jpg

Code Select Expand


ExifToolVersion                 : 12.16
FileName                        : pic.jpg
Directory                       : .
FileSize                        : 5.0 MiB
FileModifyDate                  : 2021:07:31 07:58:13-04:00
FileAccessDate                  : 2021:08:01 09:17:22-04:00
FileInodeChangeDate             : 2021:07:31 07:58:29-04:00
FilePermissions                 : rw-rw-r--
FileType                        : JPEG
FileTypeExtension               : jpg
MIMEType                        : image/jpeg
ExifByteOrder                   : Little-endian (Intel, II)
Make                            : samsung
Model                           : SM-G998W
Orientation                     : Rotate 90 CW
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
Software                        : G998WVLU4AUF7
ModifyDate                      : 2021:07:30 22:22:12
YCbCrPositioning                : Centered
ExposureTime                    : 1/9
FNumber                         : 1.8
ExposureProgram                 : Program AE
ISO                             : 640
ExifVersion                     : 0220
DateTimeOriginal                : 2021:07:30 22:22:12
CreateDate                      : 2021:07:30 22:22:12
OffsetTime                      : -04:00
OffsetTimeOriginal              : -04:00
ShutterSpeedValue               : 0.9
ApertureValue                   : 1.8
ExposureCompensation            : 0
MaxApertureValue                : 1.8
MeteringMode                    : Center-weighted average
Flash                           : No Flash
FocalLength                     : 6.7 mm
ColorSpace                      : sRGB
ExifImageWidth                  : 4000
ExifImageHeight                 : 1800
ExposureMode                    : Auto
WhiteBalance                    : Auto
DigitalZoomRatio                : 2
FocalLengthIn35mmFormat         : 24 mm
SceneCaptureType                : Standard
ImageUniqueID                   : XA8XLNF00SM
Warning                         : [minor] Unknown APP4 segment
ImageWidth                      : 4000
ImageHeight                     : 1800
EncodingProcess                 : Baseline DCT, Huffman coding
BitsPerSample                   : 8
ColorComponents                 : 3
YCbCrSubSampling                : YCbCr4:2:0 (2 2)
SamsungTrailer_0x0bd0Name       : Dual_Relighting_Bokeh_Info
SamsungTrailer_0x0bd0           : (Binary data 2104 bytes, use -b option to extract)
SamsungTrailer_0x0ab3Name       : DualShot_Extra_Info
SamsungTrailer_0x0ab4Name       : DualShot_Core_Info
SamsungTrailer_0x0ab4           : (Binary data 312 bytes, use -b option to extract)
DepthMapName                    : DualShot_DepthMap_5
DepthMapData                    : (Binary data 450200 bytes, use -b option to extract)
EmbeddedImageName               : DualShot_2
EmbeddedImage                   : (Binary data 898516 bytes, use -b option to extract)
SamsungTrailer_0x0a01Name       : Image_UTC_Data
TimeStamp                       : 2021:07:30 22:22:12-04:00
SamsungTrailer_0x0aa1Name       : MCC_Data
SamsungTrailer_0x0aa1           : (Binary data 3 bytes, use -b option to extract)
Aperture                        : 1.8
ImageSize                       : 4000x1800
Megapixels                      : 7.2
ScaleFactor35efl                : 3.6
ShutterSpeed                    : 1/9
SubSecDateTimeOriginal          : 2021:07:30 22:22:12-04:00
SubSecModifyDate                : 2021:07:30 22:22:12-04:00
CircleOfConfusion               : 0.008 mm
FOV                             : 73.7 deg
FocalLength35efl                : 6.7 mm (35 mm equivalent: 24.0 mm)
HyperfocalDistance              : 2.97 m
LightValue                      : 2.2


So then I tried:
exiftool -b -DepthMapData pic.jpg > depthmap.jpg

I ended up with an image that I wasn't able to view, so I tried using just about every other file extension I thought might be appropriate to write the output to (png, tiff, pgm, etc), with a similar result.

I'm not quite sure how to proceed here, none of the other solutions I've read have been helpful yet.

[Phil Harvey]

In other Samsung models the depth map data is raw pixel data, and ExifTool will add a tiff header to this in a Composite DepthMapTiff tag if the DepthMapWidth and DepthMapHeight tags are available.  But unfortunately this information isn't available in your file, and the DepthMapData seems to have some type of header.  I'll have to take a look at this to see if I can determine how to generate a valid image from this data.  I'm quite far behind in work like this, but I'll see what I can do and post back here when I know more.

- Phil

[Phil Harvey]

Actually, I figured this out already.

ExifTool 12.30 (to be released within a week or so) will generate a DepthMapTiff image for this file.  There are actually 2 DepthMapData tags in this file, and the first one is the standard raw pixel data.  I have also found the DepthMapWidth/Height values in a new location in the DepthMapExtra data, and use these to generate the TIFF header.

I don't know what the other DepthMapData contains, but it is 200 bytes larger and isn't simply raw pixel data, so this will be renamed to DepthMapData2 in ExifTool 12.30.

- Phil

[t0theRANCH]

Amazing, thanks for the quick reply. I'll have to give it a shot when it comes out

[Nannou]

Hi, I'm just throwing my hat in to this ring because I have a similar need to extract depth data from Samsung phone images. This is the data I get from my image (attached to post).

Code Select Expand

[MakerNotes]    DepthMapName                    : DualShot_DepthMap_1
[MakerNotes]    DepthMapData                    : (Binary data 750000 bytes, use -b option to extract)
[MakerNotes]    EmbeddedImageName               : DualShot_1
[MakerNotes]    EmbeddedImage                   : (Binary data 2298518 bytes, use -b option to extract)

I'm guessing it's a similar situation, where the DepthMapData is being fed out as raw with no header. Perhaps the smaller EmbeddedImage has the other information?

I am hoping that your new update (12.30) will help me in my effort to pull out readable Depth data. I just got one of these cool Looking Glass Portrait displays, which can display holographic and moving imagery, and I'm trying to find a stand-alone way to extract Depth data from my Samsung Note20's camera. I know it's possible because there are online tools that can do the job. But I'd rather not rely on some random server to do the job.

Cheers.

[Phil Harvey]

Thanks for the sample.  ExifTool 12.30 will add a header to this and generate a DepthMapTiff Composite tag.

- Phil