Discrepancies between tags in Samsung Motion Photos

[kayson]

Hi,

I'm trying to debug this issue with immich, a self-hosted google photos replacement.

When a motion photo is uploaded to immich, the server tries to extract the video. It sums the DirectoryItemLength and DirectoryItemPadding for the item with a DirectoryItemSemantic of "MotionPhoto", starts that many bytes from the end of the file, and reads that into an mp4.

This method worked for Samsung's One UI 5, but stopped working in One UI 6. You can find two example files here.

One UI 6:

Code Select Expand

File Size                       : 8725672 B
Directory Item Mime             : image/jpeg
Directory Item Semantic         : Primary
Directory Item Length           : 5189246
Directory Item Padding          : 111
Directory Item Mime             : video/mp4
Directory Item Semantic         : MotionPhoto
Directory Item Length           : 3536315
Directory Item Padding          : 68
Embedded Video Type             : MotionPhoto_Data
Embedded Video File             : (Binary data 3536247 bytes, use -b option to extract)

One UI 5:

Code Select Expand

File Size                       : 8412785 B
Directory Item Mime             : image/jpeg
Directory Item Semantic         : Primary
Directory Item Length           : 0
Directory Item Padding          : 59
Directory Item Mime             : video/mp4
Directory Item Semantic         : MotionPhoto
Directory Item Length           : 4605679
Directory Item Padding          : 0
Embedded Video Type             : MotionPhoto_Data
Embedded Video File             : (Binary data 4605623 bytes, use -b option to extract)

The issue is that, in the case of One UI 6, there isn't actually any padding at the end of the mp4. In fact, the whole file is the length+padding of the jpeg plus the length of the video item (5189246 + 111 + 3536315 = 8725672). However, the Embedded Video File size is exactly the Directory item's length - padding (3536315 - 68 = 3536247).

In the case of One UI 5, the jpeg item length is just 0, which seems just totally incorrect, and the length of the video item does not match the size of the Embedded Video File.

Any ideas what's going on here?

From Topic #7161, and digging around the exiftool documentation, it seems that the Directory tags are coming from an XMP in the exif, and the Embedded Video tags are coming from the Samsung "trailer".

From a hex dump, it seems to me that the One UI 6 video directory item length is actually correct, and the embedded video file is off (based on the location of the mp4 header to the end of the file).

Some questions:

• The XMP is written by Samsung, correct? So they're doing the math wrong?
• What exactly is the Samsung "trailer"? Just the bytes after the end of the jpeg? Is the format documented somewhere?
• The original tag names are something like SamsungTrailer_0x0a30; where does the hex and its format come from? From a hex dump, I can see the codes just before the associated name/data, but the bytes are flipped so I'm wondering if there's an endian-ness swap?
• Finally, what's the best way to get the video out that also supports motion photos from other platforms? Extract the binary data from the tag?
  

Many thanks for any help!

[Phil Harvey]

It never occurred to me to use the XMP to try to locate metadata in the Samsung trailer -- this seems like the wrong way to do it since the connection between the XMP and the trailer is tenuous at best.

1. The XMP is written by Samsung, correct? So they're doing the math wrong?

No idea.  They can write any garbage they want, and manufacturers aren't good about being consistent between models.

2 What exactly is the Samsung "trailer"? Just the bytes after the end of the jpeg?

Yes.

Is the format documented somewhere?

No, but you can see how it is decoded in the ExifTool ProcessSamsung routine (including some comments that may help explain the format).

3. The original tag names are something like SamsungTrailer_0x0a30; where does the hex and its format come from? From a hex dump, I can see the codes just before the associated name/data, but the bytes are flipped so I'm wondering if there's an endian-ness swap?

See the above-mentioned ProcessSamsung code.

4. Finally, what's the best way to get the video out that also supports motion photos from other platforms? Extract the binary data from the tag?

If you're asking about the best way to do this using ExifTool, then I would suggest something like -b -W %f_%t%-c.%s for all tags that you are interested in.  What do you mean by other platforms?  Other camera makes?  Formats other than JPG?  You would have to look at these on a case-by-case basis.

- Phil

[kayson]

Thanks, Phil. It is strange that they have the info in the XMP, but wrong, and in two different ways depending on the OS version.

Thanks for the pointer to the code! I will take a look.

By other platforms, I mean other phone manufacturers. I'm guessing they're using different formats so the tags wouldn't be the same, though I don't have any other phone to test with.

[StarGeek]

By other platforms, I mean other phone manufacturers. I'm guessing they're using different formats so the tags wouldn't be the same, though I don't have any other phone to test with.

I believe that Apple Live photos actually save them as two separate files, a jpeg and an mp4.  At least that's what I've been given when I've gotten samples.

[kayson]

Any ideas on other android phones? Like pixels or xiaomi?