How to embed Google depth map into image?

elmimmo · September 27, 2017, 08:24:11 AM

Having a photograph or illustration in JPEG format and its corresponding depth map as a separate JPEG/PNG grayscale image, how can I embed the latter into the former in a manner that is compatible with Google's Depthmap Metadata specs, which purportedly does so using Adobe XMP metadata?

Google Camera (which is not available to download from Google Play anymore for whatever reason) can creates images like that with its "Lens Blur" mode. Web apps Depthy.me and Depth Map Viewer are compatible with those images and allow to turn them into wigglegrams and 3D meshes, respectively. I'd like to create my own.

Phil Harvey · September 27, 2017, 08:53:52 AM

The attached config file will allow you to write GDepth information. Use the -config option to invoke this configuration.

I'll see about including this in ExifTool 10.62.

- Phil

elmimmo · September 28, 2017, 05:49:27 AM

Excuse me for being somewhat obtuse 😅 but could you provide the actual command to merge depthmap.jpg into image.jpg using that config file?

Phil Harvey · September 28, 2017, 07:11:15 AM

Something like this:

exiftool -config gdepth.config "-xmp-gdepth:data<=depthmap.jpg" -xmp-gdepth:mime=image/jpeg -xmp-gdepth:near=1000 -xmp-gdepth:far=8000 -xmp-gdepth:units=mm -xmp-gdepth:format=RangeLinear -xmp-gdepth:measuretype=OpticalAxis "-xmp-gdepth:imagewidth<imagewidth" "-xmp-gdepth:imageheight<imageheight" image.jpg

- Phil

elmimmo · September 28, 2017, 08:06:05 AM

Neither Depthy.me nor Depth Map Viewer recognize the embedded depth map of the resulting image from running that command with exiftool 10.55.

BTW, I tried with the attached image and depthmap. I do not have a device onto which Google Play will allow me to install Google Camera so cannot provide an image produced by that.

Phil Harvey · September 28, 2017, 08:59:24 AM

I suggest you use ExifTool to compare metadata with a image that works in Depthy.me or Depth Map Viewer. I can't test this myself.

You should be able to use ExifTool to generate a file with the same metadata as one that works.

- Phil

jordi · January 18, 2018, 04:39:38 AM

I have been trying to use the config file above without success also.

what I was trying is also to use a png file as depth map, but again when I try to use them in depthy the app can not recognize the exif.

what I found is an image that works :

https://github.com/spite/android-lens-blur-depth-extractor/blob/master/table.jpg

so from here I can see there are two attributes Image Data and Data

Not sure but would be great to have that working

thanks in advance

Phil Harvey · January 18, 2018, 08:58:52 AM

As I said, I can't test this myself.

Looking at the image you linked, it is the XMP that contains all of the GDepth information:

Code Select

> exiftool ~/Desktop/table.jpg -xmp:all -G1
[XMP-x]         XMP Toolkit                     : Adobe XMP Core 5.1.0-jc003
[XMP-GFocus]    Blur At Infinity                : 0.013350779
[XMP-GFocus]    Focal Distance                  : 16.577824
[XMP-GFocus]    Focal Point X                   : 0.5208333
[XMP-GFocus]    Focal Point Y                   : 0.58125
[XMP-GImage]    Image Mime Type                 : image/jpeg
[XMP-GDepth]    Format                          : RangeInverse
[XMP-GDepth]    Near                            : 12.423587799072266
[XMP-GDepth]    Far                             : 390.539306640625
[XMP-GDepth]    Mime                            : image/png
[XMP-xmpNote]   Has Extended XMP                : E0CC0C923EA0770C77E3E4EF99F538B3
[XMP-GImage]    Image Data                      : (Binary data 189118 bytes, use -b option to extract)
[XMP-GDepth]    Data                            : (Binary data 201371 bytes, use -b option to extract)

It would be informative to know if the GFocus information is required. Try running this command and see if the resulting file is still OK:

exiftool -xmp-gfocus:all= table.jpg

If it is still OK, then we are good because ExifTool may be used to write the XMP-GImage and XMP-GDepth tags, which should give you what you need.

But if it doesn't work without the GFocus tags, then I need to add the ability to write these in ExifTool.

- Phil

jordi · January 18, 2018, 10:22:11 AM

I'm afraid that once removed GFocus, the file does not work anymore, so it seems to be required.

Thanks again

Phil Harvey · January 18, 2018, 10:29:29 AM

OK. I'll add the ability to write GFocus tags.

But until then you can use the attached config file when you want to write them. You use the config file like this:

exiftool -config gfocus.config <tags to write> ...

The first thing to test is to restore the GFocus tags and make sure that it works again:

exiftool -config gfocus.config -tagsfromfile table.jpg_original -xmp-gfocus:all table.jpg

- Phil

jordi · January 18, 2018, 10:44:02 AM

I have removed them and later successfully restored, but unfortunately, once restored is not working even I can proof they are there :

Code Select

[XMP-x]         XMP Toolkit                     : Image::ExifTool 10.75
[XMP-xmpNote]   Has Extended XMP                : B86E6C5331D923084ACA447FC493EB0C
[XMP-GImage]    Image Mime Type                 : image/jpeg
[XMP-GDepth]    Far                             : 390.539306640625
[XMP-GDepth]    Format                          : RangeInverse
[XMP-GDepth]    Mime                            : image/png
[XMP-GDepth]    Near                            : 12.423587799072266
[XMP-GFocus]    Blur At Infinity                : 0.013350779
[XMP-GFocus]    Focal Distance                  : 16.577824
[XMP-GFocus]    Focal Point X                   : 0.5208333
[XMP-GFocus]    Focal Point Y                   : 0.58125
[XMP-GImage]    Image Data                      : (Binary data 189118 bytes, use -b option to extract)
[XMP-GDepth]    Data                            : (Binary data 201371 bytes, use -b option to extract)

jordi · January 18, 2018, 10:51:29 AM

If this can be helpful, depthy.me allows you to upload the image and the depthmap separately and generates the new image which contains the structure correctly working.

I've been looking into the code and found where the image is written :

https://github.com/panrafal/depthy/blob/master/app/scripts/classes/GDepthEncoder.js

Thanks

Phil Harvey · January 18, 2018, 10:55:35 AM

Unfortunately at this point I have to refer you over to the depthy support, because apparently this app has a problem reading valid XMP. As far as I can tell the metadata in the exiftool-edited image is correct. The only significant differences I can see are that ExifTool does not use the shorthand XMP format, and that the order of the XMP properties is different, but neither of these should cause a problem if depthy parses the XMP properly.

You should submit this as a bug report to them along with the image in question.

- Phil

jordi · January 18, 2018, 11:08:06 AM

Ok thanks, I'll report that and try to find what happens.

On the other say just say that I have tried with another depth map viewer and it's also not recognizing the depthmap, while it originally recognize it.

https://www.clicktorelease.com/code/depth-player/

Phil Harvey · January 18, 2018, 11:14:32 AM

I just submitted a bug report to depthy myself:

https://github.com/panrafal/depthy/issues/38

- Phil

News:

How to embed Google depth map into image?