Orientation tags in .heic photo

[paolobenve]

I received the attacched heic photo from an iphoner.

It has been shot 90º CW, and so exiftool says: tag Orientation.

However, I could verify that the photo itself is already correctly oriented, and all the photo viewer I could use, show it correctly oriented.

I'm wondering: if the orientation tag says that it must be rotated in order to show it correctly, why all the photo softwares show it correctly, without considering the Orientation tag? are they using some other tag?

[StarGeek]

Orientation is a bit messy when it comes to HEIC files.  They're based upon the MP4 format, which is why you will see Quicktime tags in it.

In addition to the EXIF:Orientation tag, you'll find the Composite:Rotation tag.  Composite:Rotation isn't actually in the file, it's a tag that exiftool creates by combining the QuickTime:MatrixStructure and QuickTime:HandlerType, which together indicate how the image (or video, in the case of MP4s) is supposed to be orientated.

However, I could verify that the photo itself is already correctly oriented, and all the photo viewer I could use, show it correctly oriented.

Except it's not.  Photos never actually start in portrait mode, they are always in landscape mode.  They have to be rotated one way or another, either by manipulating the raw image data but more commonly by software using the EXIF:Orientation. But obviously the raw data is rotated before hand.  If you look at the Rotation tag in your image, it says 270, which means the "top" of the image is on the right side in the image's raw orientation. It needs to be rotated 270° to bring the top to the correct orientation and this is happening behind the scenes.

I believe that the codecs used to display the HEIC image are automatically rotating it before it passes on the data using the Quicktime tags that control the rotation.  A similar thing happens with my NEF raw files.  I have any programs that display images set to ignore the EXIF:Orientation so I can see how the images are truly stored and NEF images are always rotated according to EXIF:Orientation no matter what program I use.

If I recall, the fact that the EXIF:Orientation isn't matching the actual image rotation has popped up before and it seems that programs haven't quite figured out how to deal with discrepancies between the two yet.

[paolobenve]

thank you for the answer!

Looking at the image itself, it's clear that it has to be shown in portrait mode.

Now, I'm supposing that the file already carries the physical image in portrait mode: I could verify that in python importing it with pillow and saving it to jpg: I get the image as it must be, in portrait mode.

But there is the Orientation tag that says that you must rotate it 90º CW. Rotating it you get a wrongly rotated image.

What am I wrong in?

[StarGeek]

Looking at the image itself, it's clear that it has to be shown in portrait mode.

Now, I'm supposing that the file already carries the physical image in portrait mode

As I said, it is not.  Camera's always take the picture in landscape mode and the image data is rotated by software when needed.

When you look at the data with exiftool, the width is greater than the height. That's because exiftool is directly reading the data in the file.  Any other program displaying the image has get the results as delivered from whatever HEIC codec decoded it, which, in most cases, will do the rotation automatically.

Code Select Expand

C:\>exiftool -G -a -s -ImageWidth -ImageHeight -ExifImageWidth -ExifImageHeight -RegionAppliedToDimensionsW -RegionAppliedToDimensionsH -MetaImageSize -ImageSize "Y:\!temp\New folder\IMG_4340.original.HEIC"
[File]          ImageWidth                      : 4032
[File]          ImageHeight                     : 3024
[EXIF]          ExifImageWidth                  : 4032
[EXIF]          ExifImageHeight                 : 3024
[XMP]           RegionAppliedToDimensionsW      : 4032
[XMP]           RegionAppliedToDimensionsH      : 3024
[QuickTime]     MetaImageSize                   : 4032x3024
[Composite]     ImageSize                       : 4032x3024

But there is the Orientation tag that says that you must rotate it 90º CW. Rotating it you get a wrongly rotated image.

This is where everything is really confusing and I have no answer.  With this image, I see no way that rotating it 90° clockwise would ever end up with the correct orientation and the Rotation of 270° is obviously correct.  Unfortunately, I don't have access to anything that takes HEIC pictures, so I can't look into it.

[greybeard]

This is where everything is really confusing and I have no answer.  With this image, I see no way that rotating it 90° clockwise would ever end up with the correct orientation and the Rotation of 270° is obviously correct.  Unfortunately, I don't have access to anything that takes HEIC pictures, so I can't look into it.

If you modify both EXIF and QuickTime rotation tags to Horizontal and 0 (to prevent rotation on display) you will see that rotation by 90 degrees clockwise does give you the correct image.

[StarGeek]

If you modify both EXIF and QuickTime rotation tags to Horizontal and 0 (to prevent rotation on display) you will see that rotation by 90 degrees clockwise does give you the correct image.

Ok, this is breaking my brain.  On a Mov video, I start at 0 rotation and change it to 90, 180, and then 270.  At each step it rotates the video by 90° clockwise.

On this HEIC image, I start at 0 rotation and do the same.  Each 90° increment rotates the image counter clockwise.

Ah, I just made an assumption about the Rotation tag.  In the video, it is the Composite:Rotation, but I didn't realize that in the image, it's the Quicktime:Rotation tag, which obviously doesn't work the same as the the Composite one.

[greybeard]

If you modify both EXIF and QuickTime rotation tags to Horizontal and 0 (to prevent rotation on display) you will see that rotation by 90 degrees clockwise does give you the correct image.

Ok, this is breaking my brain.  On a Mov video, I start at 0 rotation and change it to 90, 180, and then 270.  At each step it rotates the video by 90° clockwise.

On this HEIC image, I start at 0 rotation and do the same.  Each 90° increment rotates the image counter clockwise.

Ah, I just made an assumption about the Rotation tag.  In the video, it is the Composite:Rotation, but I didn't realize that in the image, it's the Quicktime:Rotation tag, which obviously doesn't work the same as the the Composite one.

I've not checked it with videos but both my Fujifilm camera and Apple iPhone have the option to create heic files.

What I didn't realise until I tested today is that Mac Preview uses the QuickTime rotation rather than the EXIF orientation to decide how to orient the image - maybe other image viewers do the same.

[paolobenve]

Ok, but the QuickTime rotation is physically there or is "calculated" by exiftool?

[StarGeek]

Quicktime tags are actual embedded tags.  Composite tags are the ones computed by exiftool.

[greybeard]

Well technically the QuickTime Rotation tag is calculated because the raw value is 0, 1, 2 or 3 and Exiftool multiplies that value by 90 for display.

But its not calculated based on multiple tags which is what I think you probably mean so StarGeek is right.

[Phil Harvey]

Actually, Rotation is derived from the rotation angle of the MatrixStructure, which the arctangent of the first 2 elements in the unit matrix.

- Phil

[greybeard]

Actually, Rotation is derived from the rotation angle of the MatrixStructure, which the arctangent of the first 2 elements in the unit matrix.

- Phil

Maybe I've misunderstood - this is from the QuickTime code:

Code Select Expand

        Name => 'Rotation',
        Format => 'int8u',
        Writable => 'int8u',
        Protected => 1,
        ValueConv => '$val * 90',
        ValueConvInv => 'int($val / 90 + 0.5)',
This is from an analysis of the file from the OP using exiftool -v3 :

Code Select Expand

  | | 5)  Rotation = 3
  | |    - Tag 'irot' (1 bytes, int8u):
  | |        08bd: 03                                              [.]

This is the result using exiftool -a -G1 -Rotation ex1.heic:

Code Select Expand

[QuickTime]    Rotation                        : 270

[Phil Harvey]

You are correct.  I should have gone back to the topic to see we were talking about HEIC files.  My response was for MOV/MP4 QuickTime files.

- Phil

[mceachen]

Each 90° increment rotates the image counter clockwise.

I'm seeing this as well, and checked output from iPhone 12, 13, and 15: ExifTool v12.84 returns the correct EXIF:Orientation, and counter-clockwise value for Quicktime:Rotation (and no Composite:Rotation).

From the example above (which is from an iPhone 12), using ExifTool v12.84:

Code Select Expand

$ exiftool -g -Rotation -Orientation IMG_4340.original.HEIC
---- EXIF ----
Orientation                     : Rotate 90 CW
---- QuickTime ----
Rotation                        : 270

Here's what I've found:

EXIF:Orientation  |	Quicktime:Rotation  |	Final Orientation (by Preview & Gnome Image Viewer)
Rotate 90 CW	270	↑ A
null	270	↑ A
Horizontal (normal)	270	↑ A
null	0	← ᗉ
Horizontal (normal)	0	← ᗉ
Rotate 90 CW	0	← ᗉ
null	90	↓ ∀
Horizontal (normal)	90	↓ ∀
Rotate 90 CW	90	↓ ∀

TL;DR:

1. Apple Preview on macOS Sonoma 14.4.1, and Gnome Image Viewer v42.0 render all these examples the same (thank goodness).

2. These applications ignore EXIF:Orientation.

3. Quicktime:Rotation is counter clockwise, as StarGeek stated.

For extra credit, I took a photo from an upside-down iPhone 13 pro:

Code Select Expand

$ exiftool -g -EXIF:Orientation -Quicktime:Rotation IMG_9995.HEIC
---- EXIF ----
Orientation                     : Rotate 270 CW
---- QuickTime ----
Rotation                        : 90

This image orientation behaved similarly to IMG_4340, where EXIF:Orientation was always ignored and  Quicktime:Rotation was interpreted as counter clockwise.

So, question for Phil: is ExifTool actually interpretting the Quicktime:Rotation correctly? If it is, should it convert it so the value can be interpretted in clockwise degrees, like every other image and video format? This would be a breaking change for prior consumers, of course, but I think it could be considered a bugfix, too.

FWIW I'm working around the issue by negating the rotation in HEIC images, but I'm happy to change my code if you see fit to flip Quicktime:Rotation on HEIC.

[Phil Harvey]

I'll have to do some tests myself, but I think you are correct that I should probably change the direction of the Composite QuickTime:Rotation tag.

- Phil