Strange bytes in JPEG meta data (Sony Xperia XZ1 Compact)

[joakimk]

Hello again!

I have completed my app which edits meta data in JPEG files, on Android. I've run the following test image through the -validate option, and although it's not "OK" the warnings are all "minor". So I'm assuming the image to be OK.

However, I'm a bit confused by the "Aaaa" and "Aaab" sections in the output. Could you please have a look at the attached photo and see that it's not been damaged?

<snip>

Code Select Expand

Focus Is Lens Moving            : 0
Aaaa                            : 90000000,e0000000
Aaab                            : ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff
Abaa                            : 05000000,5a000000
</snip>

Thanks!

[Phil Harvey]

Hi Joakim,

Most of the warnings are because you aren't following the TIFF specification which says that all offsets should be even numbers (ie. all data values should be on 2-byte boundaries).

The most significant problem is that the maker notes have been corrupted because the offsets have not been updated properly.

The Aaaa, etc tags are XperiaCamera XMP tags.  Add -G1 to the exiftool command to see where the tags are coming from.

- Phil

[joakimk]

Great, thanks!

Well, my app has edited the UserComment, ImageDescription, and IPTC Caption/Abstract. Is it possible for you to say if it is the content in these tags that is causing the errors/warning? I'm hoping the problems are instead caused by the Xperia device ;-)

I'll email another jpeg, taken with the same device but that has not been through my app.

[Phil Harvey]

Hi Joakim,

The content of the tags is fine, and there is no problem in the original image.

The problem is that your app hasn't formatted the metadata properly.

Try using the exiftool -htmldump feature to see what is happening.

- Phil

[joakimk]

Yes. I've had iterations om adjusting these things before, using the htmldump option. I'll do another round!

However, how important is it to get that stuff 100% correct? I don't want my app to cause any problems with the JPEGs.

[Phil Harvey]

Getting rid of the odd offsets should be easy, and no respectable app should have this problem.

Fixing the makernote problem is harder.  Maker notes are important to some people, while others don't even know they exist.

- Phil

[joakimk]

OK, so all tags should be of even length?

I can understand that I need to take care that the tags I edit get even length, and if I don't then this may offset other tags "further down" in the file. But, starting from the ExifIFD "top", I see that already the first tag (ExposureTime) is odd -- and I have not intentionally touched that tag or anything "above" it.

If you look at the UserComment tag, you can see that the length indeed is even (160 bytes). The ImageDescription, which I've also edited, has length 77, so I've added a null-byte pad at the end to make it even. The "original" image validates OK, so apparently my app is affecting the meta data negatively. Since the content lengths are indeed even, I'm just not sure how to approach this problem. Could you perhaps point me in the right direction?

Code Select Expand

ExifIFD entries

ExifIFD-00 ExposureTime (odd)
ExifIFD-01 FNumber (odd)
ExifIFD-02 ISO
ExifIFD-03 ExifVersion
ExifIFD-04 DateTimeOriginal (odd)
ExifIFD-05 CreateDate (odd)
ExifIFD-06 OffsetTime
ExifIFD-07 OffsetTimeOriginal (odd)
ExifIFD-08 OffsetTimeDigitized
ExifIFD-09 ComponentsConfiguration
ExifIFD-10 ShutterSpeedValue (odd)
ExifIFD-11 ExposureCompensation (odd)
ExifIFD-12 MeteringMode
ExifIFD-13 LightSource
ExifIFD-14 Flash
ExifIFD-15 FocalLength (odd)
ExifIFD-16 MakerNotes
ExifIFD-17 UserComment (odd)
ExifIFD-18 SubSecTime (odd)
ExifIFD-19 SubSecTimeOriginal
ExifIFD-20 SubSecTimeDigitized (odd)
ExifIFD-21 FlashpixVersion
ExifIFD-22 ColorSpace
ExifIFD-23 ExifImageWidth
ExifIFD-24 ExifImageHeight
ExifIFD-25 InteropOffset
ExifIFD-26 CustomRendered
ExifIFD-27 ExposureMode
ExifIFD-28 WhiteBalance
ExifIFD-29 DigitalZoomRatio (odd)
ExifIFD-30 SceneCaptureType
ExifIFD-31 SubjectDistanceRange
Next IFD
UserComment value








IFD0 entries
IFD0-00 ImageDescription (odd)
IFD0-01 Make (odd)
IFD0-02 Model (odd)
IFD0-03 Orientation
IFD0-04 XResolution (odd)

[joakimk]

... does it make sense to consider the bytes "top-down", in this manner, when interpreting the -htmldump output?

[Phil Harvey]

The values don't need to be an even number of bytes, but if they are odd there must be a padding byte afterward before the next value so it lies on an even word boundary.

[joakimk]

Yes, I understand, and I will take another look at the tags I'm writing. However, and sorry for asking again, is it "strange" that there are problems with the offset already with the ExposureTime tag?

Does that mean my app must be affecting the bytes above the ExifIfd section?

[Phil Harvey]

I saw no problems at all in the original image, so I don't know what you are asking.

If you are talking about the modified image, then it isn't strange if your software isn't designed to put values on word boundaries.  Wrong, but not strange.  I've seen this before -- not all developers read the TIFF specification.

- Phil

[joakimk]

Referring to the modified image, and the exiftool output pasted previously:

Obviously, my app is introducing some problems, but why are there warnings (odd offset) already at the ExposureTime tag? I only edit the UserComment, ImageDescription, and IPTC Caption/Abstract tags, and they are all "further down" in the file.

Question: can the offset of these three tags affect the offset of the ExposureTime tag? Or is the ExposureTime warning an indication that my app affects metadata not limited to these tags?

Having a very hard time conveying this question, I understand your confusion. Hope this clarifies.

[joakimk]

Aside from ensuring even length through padding in the tags I'm changing, what else can I do to handle this problem?

If other things are getting messed up, perhaps it's due to bugs in the Apache Sanselan library I'm using to actually edit metadata.

[Phil Harvey]

Have you looked at the -htmldump output?  The metadata is completely restructured after your edit.

I'm assuming now that your app must be using some other library since you seem to not understand the details of how things were written.  If so, the bugs are in the library you are using, and I doubt you will be able to get around it without changing libraries.

- Phil

[joakimk]

Yes, I have used the -htmldump output, and your software is a life saver!
Also, yes, as I said, I am relying on an external library to edit meta data: Apache Sanselan v0.97. This library is in "incubator" status, and has not been updated for a long time, but it  is in a stable release. Meanwhile, we're waiting for the Apache Commons Imaging library to supercede it, but it's apparently going to be a while before that gets its first stable release.

So, using Sanselan, I'm following this example to add/change fields:
https://apache.googlesource.com/sanselan/+/e3b28b1c83ed93ce763e59afbbf5d2dba86901fb/src/test/java/org/apache/commons/sanselan/examples/WriteExifMetadataExample.java

Perhaps you might have a quick scroll through that example, and see if you think they are navigating the Exif structure properly?

Regarding my original question; I can try to fix the odd offsets, but if the library is jumbling the meta data, then maybe I simply can not develop such an app at this time? :-( :-(