Discrepancies between tags in Samsung Motion Photos

[kayson]

Hi,

I'm trying to debug this issue with immich, a self-hosted google photos replacement.

When a motion photo is uploaded to immich, the server tries to extract the video. It sums the DirectoryItemLength and DirectoryItemPadding for the item with a DirectoryItemSemantic of "MotionPhoto", starts that many bytes from the end of the file, and reads that into an mp4.

This method worked for Samsung's One UI 5, but stopped working in One UI 6. You can find two example files here.

One UI 6:

Code Select Expand

File Size                       : 8725672 B
Directory Item Mime             : image/jpeg
Directory Item Semantic         : Primary
Directory Item Length           : 5189246
Directory Item Padding          : 111
Directory Item Mime             : video/mp4
Directory Item Semantic         : MotionPhoto
Directory Item Length           : 3536315
Directory Item Padding          : 68
Embedded Video Type             : MotionPhoto_Data
Embedded Video File             : (Binary data 3536247 bytes, use -b option to extract)

One UI 5:

Code Select Expand

File Size                       : 8412785 B
Directory Item Mime             : image/jpeg
Directory Item Semantic         : Primary
Directory Item Length           : 0
Directory Item Padding          : 59
Directory Item Mime             : video/mp4
Directory Item Semantic         : MotionPhoto
Directory Item Length           : 4605679
Directory Item Padding          : 0
Embedded Video Type             : MotionPhoto_Data
Embedded Video File             : (Binary data 4605623 bytes, use -b option to extract)

The issue is that, in the case of One UI 6, there isn't actually any padding at the end of the mp4. In fact, the whole file is the length+padding of the jpeg plus the length of the video item (5189246 + 111 + 3536315 = 8725672). However, the Embedded Video File size is exactly the Directory item's length - padding (3536315 - 68 = 3536247).

In the case of One UI 5, the jpeg item length is just 0, which seems just totally incorrect, and the length of the video item does not match the size of the Embedded Video File.

Any ideas what's going on here?

From Topic #7161, and digging around the exiftool documentation, it seems that the Directory tags are coming from an XMP in the exif, and the Embedded Video tags are coming from the Samsung "trailer".

From a hex dump, it seems to me that the One UI 6 video directory item length is actually correct, and the embedded video file is off (based on the location of the mp4 header to the end of the file).

Some questions:

• The XMP is written by Samsung, correct? So they're doing the math wrong?
• What exactly is the Samsung "trailer"? Just the bytes after the end of the jpeg? Is the format documented somewhere?
• The original tag names are something like SamsungTrailer_0x0a30; where does the hex and its format come from? From a hex dump, I can see the codes just before the associated name/data, but the bytes are flipped so I'm wondering if there's an endian-ness swap?
• Finally, what's the best way to get the video out that also supports motion photos from other platforms? Extract the binary data from the tag?
  

Many thanks for any help!

[Phil Harvey]

It never occurred to me to use the XMP to try to locate metadata in the Samsung trailer -- this seems like the wrong way to do it since the connection between the XMP and the trailer is tenuous at best.

1. The XMP is written by Samsung, correct? So they're doing the math wrong?

No idea.  They can write any garbage they want, and manufacturers aren't good about being consistent between models.

2 What exactly is the Samsung "trailer"? Just the bytes after the end of the jpeg?

Yes.

Is the format documented somewhere?

No, but you can see how it is decoded in the ExifTool ProcessSamsung routine (including some comments that may help explain the format).

3. The original tag names are something like SamsungTrailer_0x0a30; where does the hex and its format come from? From a hex dump, I can see the codes just before the associated name/data, but the bytes are flipped so I'm wondering if there's an endian-ness swap?

See the above-mentioned ProcessSamsung code.

4. Finally, what's the best way to get the video out that also supports motion photos from other platforms? Extract the binary data from the tag?

If you're asking about the best way to do this using ExifTool, then I would suggest something like -b -W %f_%t%-c.%s for all tags that you are interested in.  What do you mean by other platforms?  Other camera makes?  Formats other than JPG?  You would have to look at these on a case-by-case basis.

- Phil

[kayson]

Thanks, Phil. It is strange that they have the info in the XMP, but wrong, and in two different ways depending on the OS version.

Thanks for the pointer to the code! I will take a look.

By other platforms, I mean other phone manufacturers. I'm guessing they're using different formats so the tags wouldn't be the same, though I don't have any other phone to test with.

[StarGeek]

By other platforms, I mean other phone manufacturers. I'm guessing they're using different formats so the tags wouldn't be the same, though I don't have any other phone to test with.

I believe that Apple Live photos actually save them as two separate files, a jpeg and an mp4.  At least that's what I've been given when I've gotten samples.

[kayson]

Any ideas on other android phones? Like pixels or xiaomi?

[kolerus]

Hello

Could you please advise how to extract a "MotionPhoto" video (it's an mp4 file) from jpg files created by OnePlus phones?

I've gone as far, as to extract the entire trailer (exiftool -trailer -b file.jpg > video.mp4)

The video is inside it, but it has some extra 853552 bytes in front - and 190 bytes at the end.
I don't know how to exclude these...

You can find the jpg file and the properly extracted (by my phone's app) video here:

https://drive.google.com/drive/folders/1Z1nE2a_TJcU64FZ4ncCV9h1tu4nmCmGo?usp=sharing

[StarGeek]

Use the first command in this post.

[kolerus]

Use the first command in this post.

Thanks, but it does not seem to work:

Code Select Expand

>exiftool -b -W %d%f_%t%-c.%s -EmbeddedVideoFile folder_with_files
    1 directories scanned
  16 image files read
    0 output files created
I'm using ExifTool Version 13.16

My files from OnePlus are probably in a slightly different format than the Samsung.

[kolerus]

FWIW I have a workaround.

I run exiftool twice

Code Select Expand

> exiftool myfile.jpg > out.txt
> exiftool exiftool -trailer -b myfile.jpg > file.tmp

And then I look in out.txt for the lines with "Video Length" (let's call this value N) and "Directory Item Length" (there are three numbers; A, B, C)

Then I take from file.tmp: N bytes, starting from offset B* - and that's my proper mp4 file.

*) it should probably by A+B, but A is always zero


EDIT:

Code Select Expand

This is the relevant fragment from my out.txt:
Video Length                    : 7280688
Directory Item Mime             : image/jpeg, image/jpeg, video/mp4
Directory Item Semantic         : Primary, GainMap, MotionPhoto
Directory Item Length           : 0, 288587, 7280878


[Phil Harvey]

I don't think I have a sample of this.  Could you send me one so I can add support to ExifTool?  My email is philharvey66 at gmail.com

- Phil

[kolerus]

I don't think I have a sample of this.  Could you send me one so I can add support to ExifTool?  My email is philharvey66 at gmail.com

- Phil

I sent the link to my google drive in my first post:

https://drive.google.com/drive/folders/1Z1nE2a_TJcU64FZ4ncCV9h1tu4nmCmGo?usp=sharing

[Phil Harvey]

Great thanks.  I should be able to add the ability for this to be extracted automatically by ExifTool 12.17 with a tag name of EmbeddedVideoFile.

- Phil

[kolerus]

Awesome. Thank you very much for your quick and kind response.

[Phil Harvey]

I think I'll use the name MotionPhotoVideo instead of EmbeddedVideoFile because this is more descriptive, and is the name used in Samsung HEIC images.

- Phil