Creating stereoscopic images for Google Cardboard

[YPOC]

Hello,

I'm trying to combine two images to display them via the Google Cardboard Camera app. These images encode the right panoramic view into the left panoramic view, as described on this Google page.
So far I've been able to extract the encoded Image, that is stored as binary base64-encoded value in XMP-GImage:Data and also set ExifTool's config file so that I can (theoretically) write the GImage:Data tag. For this post I will be using this image created with the Cardboard Camera app.

My config file looks like this:

Code Select Expand

%Image::ExifTool::UserDefined::GImage = (
    GROUPS => { 0 => 'XMP', 1 => 'XMP-GImage', 2 => 'Image' },
    NAMESPACE => { 'GImage' => 'http://ns.google.com/photos/1.0/image/' },
    WRITABLE => 'string', # (default to string-type tags)
    # Example 8.  XMP-xxx:NewXMPxxxTag1 (an alternate-language tag)
    # - replace "NewXMPxxxTag1" with your own tag name (eg. "MyTag")
    Data => { Binary => 1, },
    # Example 9.  XMP-xxx:NewXMPxxxTag2 (a string tag in the Author category)
    Mime => { },
);

# The %Image::ExifTool::UserDefined hash defines new tags to be added
# to existing tables.
%Image::ExifTool::UserDefined = (
    # new XMP namespaces (ie. XXX) must be added to the Main XMP table:
    'Image::ExifTool::XMP::Main' => {
        GImage => {
            SubDirectory => {
                TagTable => 'Image::ExifTool::UserDefined::GImage',
            },
        },
    },
);


1;  #end

To extract the encoded image from the left perspective I used .\exiftool.exe -b -XMP-GImage:Data .\MyImage.jpg > .\MyImage_right.jpg.bin (yes, I'm on Windows.) The resulting file starts with /9j/4AAQSk... and ends with MIlGgggD//Z and a new line, so two lines in total.
Then I run certutil.exe -decode .\MyImage_right.jpg.bin .\MyImage_right.jpg to get the right panorama, so far everything works great. (Except that I haven't been able to pipe ExifTool's output into any base64-decoder)

Now I would like to create these stereoscopic photospheres myself, but I haven't had much success. Just trying to rewrite the .bin into the left view results in an incorrect image. My command for this is .\exiftool.exe "-XMP-GImage:Data<=.\MyImage_right.jpg.bin" .\MyImage.jpg The resulting image is now 3.505.127 Bytes, the original only 2.232.764 Bytes. Also, the extracted .bin already is 2.541.070 Bytes (should be 952.899 Bytes, according to the -X output). In fact, when I extract the freshly written XMP-GImage:Data again it's double the size and contains a dot between each character.
Can anybody point me in the right direction? I thought using the -b option is for exactly that case.

[Phil Harvey]

I don't have any time right now to read this in detail, but it seems like you will have to decode the base64 data with something like this in your tag definition:

Code Select Expand

        ValueConv => 'Image::ExifTool::XMP::DecodeBase64($val)',

And remove the "Binary => 1" flag.

If you can't figure it out, post a (small) sample image and I'll try it when I get a chance (in a couple of days).  Some people have trouble posting images, so an alternative is to upload it to a file sharing service somewhere.

- Phil

[YPOC]

Thank you for your reply Phil.
I adjusted my config file just like you said and ran .\exiftool.exe -b -XMP-GImage:Data .\MyImage.vr.jpg > .\MyImage_right.vr.jpg, but it produced a jpg with a file size of 1875 KB (should be roughly half of the original image, so 2181 KB / 2) and it can't be opened by any image viewer.

For this thread I'm using this image created in the Cardboard Camera app, to download click the little arrow pointing down in the upper right corner, otherwise the metadata won't be stored correctly.

So to sum up: At first I'd like to extract the binary XMP-GImage:Data from the left perspective and just as a test rewrite it into the left perspective.

[Phil Harvey]

OK, everything works fine for me with the attached config file.  I can read and write a valid 952899 byte JPEG image. Reading should have worked for you with the addition I suggested, but a ValueConvInv was required to be able to write the Base64 data.

Code Select Expand

% ls -l a.jpg
-rw-r--r--@ 1 phil  staff  2232764 21 May 08:32 a.jpg

% exiftool -config gimage.config -b -xmp-gimage:data a.jpg > b.jpg

% ls -l b.jpg
-rw-r--r--@ 1 phil  staff  952899 21 May 08:25 b.jpg

% exiftool -config gimage.config "-xmp-gimage:data<=b.jpg" a.jpg
    1 image files updated

% exiftool -config gimage.config -data a.jpg
Data                            : (Binary data 952899 bytes, use -b option to extract)

(Since you are in Windows, you would do a "dir" instead of "ls -l" in my above commands.)

- Phil

Edit:  ExifTool 10.54 will support these GImage tags (named ImageData and ImageMimeType).

[YPOC]

Hi Phil, thank you for taking your time to troubleshoot this issue for me.

I have tried the steps you described, and while I get the same results you posted (meaning the encoded image has the same size before and after exporting it) I get different file sizes for the left perspective (before 2.232.764 Bytes, after 2.254.264 Bytes). Therefore the Google Cardboard Camera app fails to display the image and the Google Photos app (which seems to be a bit more robust) displays the left perspective for both eyes.
Where does this difference in file size come from? I have uploaded the image after extracting and re-encoding the right into the left perspective, maybe this helps in figuring out the problem.

[Phil Harvey]

The difference in the file size is unrelated to a problem displaying the image (unless the app isn't parsing the XMP properly).  See FAQ 13.

There must be some other difference.  I'll look at your uploaded file when I get a chance.

- Phil

[Phil Harvey]

OK, I've taken a closer look.  The only notable differences are some change in ordering of the XMP properties (which shouldn't be significant), and the fact that ExifTool inserts linefeeds in the base 64 output (which is perfectly legal, but maybe the camera app can't handle standard base 64).  If the linefeeds are the problem, then changing the ValueConvInv to this should fix it:

Code Select Expand

        ValueConvInv => 'Image::ExifTool::XMP::EncodeBase64($val, 1)',

Please let me know if this does the trick.

- Phil

[YPOC]

Unfortunately it still doesn't work. Windows tells me the new generated file is 2.233.088 Bytes, while the original is 2.232.764 Bytes. You can find the new generated file here.

[Phil Harvey]

As I said, the file size is not significant.

I can see no significant differences in the metadata of these files.  It would be useful if the Google Cardboard Camera app people could tell you why their app isn't loading the file.

- Phil

[jens12345]

I know this is quite an old thread but I ran into the same issue. Images following the Google specification for VR images created with ExifTool can not be processed by the Android Cardboard application ("this is not a VR image"). The issue is not related to the base64 encoding but to the structure of the XML. I  analyzed the files with Unix hexdump and it appears that the XMP part of images created by the Cardboard Camera app look like this (slightly formatted for readability):

Code Select Expand


http://ns.adobe.com/xap/1.0/
<x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="Adobe XMP">
  <rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
    <rdf:Description xmlns:GPano="http://ns.google.com/photos/1.0/panorama/"
     xmlns:GImage="http://ns.google.com/photos/1.0/image/"
     xmlns:xmpNote="http://ns.adobe.com/xmp/note/"
     rdf:about=""
     GPano:CroppedAreaLeftPixels="0"
     GPano:CroppedAreaTopPixels="1620"
     GPano:CroppedAreaImageWidthPixels="9262"
     GPano:CroppedAreaImageHeightPixels="1624"
     GPano:FullPanoWidthPixels="9262"
     GPano:FullPanoHeightPixels="4631"
     GPano:InitialViewHeadingDegrees="180"
     GImage:Mime="image/jpeg"
     xmpNote:HasExtendedXMP="ceb9f6c4a305955504a856dafdfe371f"/>
  </rdf:RDF>
</x:xmpmeta>

If I use ExifTool to create these kind of images the XMP has a different fornat:

Code Select Expand


<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

<rdf:Description rdf:about=''
  xmlns:xmpNote='http://ns.adobe.com/xmp/note/'>
  <xmpNote:HasExtendedXMP>6B88EFA1352935825671FB752D8D067C</xmpNote:HasExtendedXMP>
</rdf:Description>

<rdf:Description rdf:about=''
  xmlns:GImage='http://ns.google.com/photos/1.0/image/'>
  <GImage:Mime>image/jpeg</GImage:Mime>
</rdf:Description>

<rdf:Description rdf:about=''
  xmlns:GPano='http://ns.google.com/photos/1.0/panorama/'>
  <GPano:CroppedAreaImageHeightPixels>1624</GPano:CroppedAreaImageHeightPixels>
  <GPano:CroppedAreaImageWidthPixels>9262</GPano:CroppedAreaImageWidthPixels>
  <GPano:CroppedAreaLeftPixels>0</GPano:CroppedAreaLeftPixels>
  <GPano:CroppedAreaTopPixels>1620</GPano:CroppedAreaTopPixels>
  <GPano:FullPanoHeightPixels>4631</GPano:FullPanoHeightPixels>
  <GPano:FullPanoWidthPixels>9262</GPano:FullPanoWidthPixels>
  <GPano:InitialViewHeadingDegrees>180</GPano:InitialViewHeadingDegrees>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>


Apparently ExifTool writes one rdf:Description tag per namespace. Google Cardboard is confused and seems to use quite an intolerant XML parser.
I assume the problem is on Google's side and not in ExifTool. I tried patching such a file with a hex editor and after removing the additional tags it worked with Cardboard. I posted a bug report on Google VR Developer forum but I do not expect that somebody will react or even fix this: https://plus.google.com/103723281823338770763/posts/ekD3KAgoWMU

If somebody knows a workaround to convince ExifTool to write the compact format please post. Personally I do not need a fix I have written a small Java program using the icafe library on GitHub. This gives me more convenience anyway. I thought it might be useful to post because others may run into the same issue and ExifTool is very popular for these kind of tasks.

[Phil Harvey]

If this is indeed the problem then the fault is definitely in the Android Cardboard Camera app, and some other work-around must be used.  Currently there is no way to get ExifTool to write XMP using the compact format.

- Phil

[koka]

Currently there is no way to get ExifTool to write XMP using the compact format.

I tried to use either of -api Compact = or -z, but editing the metadata with exiftool:

Code Select Expand

exiftool Lenovo.vr.jpg -XMP-GPano:CroppedAreaTopPixels="0" -o exiftool.vr.jpg
breaks compatibility with https://arvr.google.com/vr180/apps/
The same thing - "this is not a VR image"
This is fixed compatibility:

Code Select Expand

exiv2 -M"set Xmp.GPano.CroppedAreaTopPixels 0" exiftool.vr.jpg
Maybe there is some parameter for exiftool -exiv2LikeXMP?
I wanted to try:

Code Select Expand

exiftool exiftool.vr.jpg "-xmp<=exiv2Like.XMP"
but I can't create valid exiv2Like.XMP because of big ImageData.xmp

[Phil Harvey]

Could you attach the Exiv2 version of the file so I can compare the result?

- Phil

[koka]

ok

[Phil Harvey]

The most significant difference is that there is no xpacket wrapper for the XMP in the original file.  ExifTool will write XMP like this without adding a wrapper, but the Exiv2 file is wrapped.

Does this command produce a useable result?:

exiftool Lenovo.vr.jpg -xmp:all= "-all:all<xmp:all" -XMP-GPano:CroppedAreaTopPixels="0" -o exiftool.vr.jpg

This will rebuild the XMP, adding the xpacket wrapper.

- Phil