exiftool -G does not report all tags present in .mp4 file

[derMart]

1) Your system type: Ubuntu 20.04

2) The ExifTool version you are using: 12.30

For a mp4 file generated with OpenCamera on Android 10, adding the -G option removes some tags from the output. Details as follows:

Code Select Expand

exiftool file.mp4
results in

Code Select Expand

ExifTool Version Number         : 12.30
File Name                       : VID_20210822_124819.mp4
Directory                       : test
File Size                       : 4.6 MiB
File Modification Date/Time     : 2021:08:22 12:48:23+02:00
File Access Date/Time           : 2021:08:22 17:53:44+02:00
File Inode Change Date/Time     : 2021:08:22 17:53:06+02:00
File Permissions                : -rwxr-----
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 v2 [ISO 14496-14]
Minor Version                   : 0.0.0
Compatible Brands               : isom, mp42
Movie Header Version            : 0
Create Date                     : 2021:08:22 10:48:23
Modify Date                     : 2021:08:22 10:48:23
Time Scale                      : 1000
Duration                        : 2.47 s
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 2
Android Version                 : 10
Android Manufacturer            : Sony
Android Model                   : G8441
Track Header Version            : 0
Track Create Date               : 2021:08:22 10:48:23
Track Modify Date               : 2021:08:22 10:48:23
Track ID                        : 1
Track Duration                  : 2.47 s
Track Layer                     : 0
Track Volume                    : 0.00%
Matrix Structure                : 0 1 0 -1 0 0 0 0 1
Image Width                     : 1280
Image Height                    : 720
Media Header Version            : 0
Media Create Date               : 2021:08:22 10:48:23
Media Modify Date               : 2021:08:22 10:48:23
Media Time Scale                : 90000
Media Duration                  : 2.47 s
Handler Type                    : Video Track
Handler Description             : VideoHandle
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 1280
Source Image Height             : 720
X Resolution                    : 72
Y Resolution                    : 72
Bit Depth                       : 24
Pixel Aspect Ratio              : 65536:65536
Color Representation            : nclx 1 1 1
Video Frame Rate                : 29.93
Media Data Size                 : 4365912
Media Data Offset               : 405268
Image Size                      : 1280x720
Megapixels                      : 0.922
Avg Bitrate                     : 14.1 Mbps
Rotation                        : 90

Code Select Expand

exiftool -X file.mp4
correctly prints all groups in xml fashion like this:

Code Select Expand

<?xml version='1.0' encoding='UTF-8'?>
<rdf:RDF xmlns:rdf='http://www.w3.org/1999/02/22-rdf-syntax-ns#'>

<rdf:Description rdf:about='test/VID_20210822_124819.mp4'
  xmlns:et='http://ns.exiftool.org/1.0/' et:toolkit='Image::ExifTool 12.30'
  xmlns:ExifTool='http://ns.exiftool.org/ExifTool/1.0/'
  xmlns:System='http://ns.exiftool.org/File/System/1.0/'
  xmlns:File='http://ns.exiftool.org/File/1.0/'
  xmlns:QuickTime='http://ns.exiftool.org/QuickTime/QuickTime/1.0/'
  xmlns:Keys='http://ns.exiftool.org/QuickTime/Keys/1.0/'
  xmlns:Track1='http://ns.exiftool.org/QuickTime/Track1/1.0/'
  xmlns:Composite='http://ns.exiftool.org/Composite/1.0/'>
<ExifTool:ExifToolVersion>12.30</ExifTool:ExifToolVersion>
<System:FileName>VID_20210822_124819.mp4</System:FileName>
<System:Directory>test</System:Directory>
<System:FileSize>4.6 MiB</System:FileSize>
<System:FileModifyDate>2021:08:22 12:48:23+02:00</System:FileModifyDate>
<System:FileAccessDate>2021:08:22 17:53:44+02:00</System:FileAccessDate>
<System:FileInodeChangeDate>2021:08:22 17:53:06+02:00</System:FileInodeChangeDate>
<System:FilePermissions>-rwxr-----</System:FilePermissions>
<File:FileType>MP4</File:FileType>
<File:FileTypeExtension>mp4</File:FileTypeExtension>
<File:MIMEType>video/mp4</File:MIMEType>
<QuickTime:MajorBrand>MP4 v2 [ISO 14496-14]</QuickTime:MajorBrand>
<QuickTime:MinorVersion>0.0.0</QuickTime:MinorVersion>
<QuickTime:CompatibleBrands>
  <rdf:Bag>
   <rdf:li>isom</rdf:li>
   <rdf:li>mp42</rdf:li>
  </rdf:Bag>
</QuickTime:CompatibleBrands>
<QuickTime:MovieHeaderVersion>0</QuickTime:MovieHeaderVersion>
<QuickTime:CreateDate>2021:08:22 10:48:23</QuickTime:CreateDate>
<QuickTime:ModifyDate>2021:08:22 10:48:23</QuickTime:ModifyDate>
<QuickTime:TimeScale>1000</QuickTime:TimeScale>
<QuickTime:Duration>2.47 s</QuickTime:Duration>
<QuickTime:PreferredRate>1</QuickTime:PreferredRate>
<QuickTime:PreferredVolume>100.00%</QuickTime:PreferredVolume>
<QuickTime:MatrixStructure>1 0 0 0 1 0 0 0 1</QuickTime:MatrixStructure>
<QuickTime:PreviewTime>0 s</QuickTime:PreviewTime>
<QuickTime:PreviewDuration>0 s</QuickTime:PreviewDuration>
<QuickTime:PosterTime>0 s</QuickTime:PosterTime>
<QuickTime:SelectionTime>0 s</QuickTime:SelectionTime>
<QuickTime:SelectionDuration>0 s</QuickTime:SelectionDuration>
<QuickTime:CurrentTime>0 s</QuickTime:CurrentTime>
<QuickTime:NextTrackID>2</QuickTime:NextTrackID>
<QuickTime:HandlerType>Metadata Tags</QuickTime:HandlerType>
<QuickTime:MediaDataSize>4365912</QuickTime:MediaDataSize>
<QuickTime:MediaDataOffset>405268</QuickTime:MediaDataOffset>
<Keys:AndroidVersion>10</Keys:AndroidVersion>
<Keys:AndroidManufacturer>Sony</Keys:AndroidManufacturer>
<Keys:AndroidModel>G8441</Keys:AndroidModel>
<Track1:TrackHeaderVersion>0</Track1:TrackHeaderVersion>
<Track1:TrackCreateDate>2021:08:22 10:48:23</Track1:TrackCreateDate>
<Track1:TrackModifyDate>2021:08:22 10:48:23</Track1:TrackModifyDate>
<Track1:TrackID>1</Track1:TrackID>
<Track1:TrackDuration>2.47 s</Track1:TrackDuration>
<Track1:TrackLayer>0</Track1:TrackLayer>
<Track1:TrackVolume>0.00%</Track1:TrackVolume>
<Track1:MatrixStructure>0 1 0 -1 0 0 0 0 1</Track1:MatrixStructure>
<Track1:ImageWidth>1280</Track1:ImageWidth>
<Track1:ImageHeight>720</Track1:ImageHeight>
<Track1:MediaHeaderVersion>0</Track1:MediaHeaderVersion>
<Track1:MediaCreateDate>2021:08:22 10:48:23</Track1:MediaCreateDate>
<Track1:MediaModifyDate>2021:08:22 10:48:23</Track1:MediaModifyDate>
<Track1:MediaTimeScale>90000</Track1:MediaTimeScale>
<Track1:MediaDuration>2.47 s</Track1:MediaDuration>
<Track1:HandlerType>Video Track</Track1:HandlerType>
<Track1:HandlerDescription>VideoHandle</Track1:HandlerDescription>
<Track1:GraphicsMode>srcCopy</Track1:GraphicsMode>
<Track1:OpColor>0 0 0</Track1:OpColor>
<Track1:CompressorID>avc1</Track1:CompressorID>
<Track1:SourceImageWidth>1280</Track1:SourceImageWidth>
<Track1:SourceImageHeight>720</Track1:SourceImageHeight>
<Track1:XResolution>72</Track1:XResolution>
<Track1:YResolution>72</Track1:YResolution>
<Track1:BitDepth>24</Track1:BitDepth>
<Track1:PixelAspectRatio>65536:65536</Track1:PixelAspectRatio>
<Track1:ColorRepresentation>nclx 1 1 1</Track1:ColorRepresentation>
<Track1:VideoFrameRate>29.93</Track1:VideoFrameRate>
<Composite:ImageSize>1280x720</Composite:ImageSize>
<Composite:Megapixels>0.922</Composite:Megapixels>
<Composite:AvgBitrate>14.1 Mbps</Composite:AvgBitrate>
<Composite:Rotation>90</Composite:Rotation>
</rdf:Description>
</rdf:RDF>
However,

Code Select Expand

exiftool -G file.mp4
results in

Code Select Expand

[ExifTool]      ExifTool Version Number         : 12.30
[File]          File Name                       : VID_20210822_124819.mp4
[File]          Directory                       : test
[File]          File Size                       : 4.6 MiB
[File]          File Modification Date/Time     : 2021:08:22 12:48:23+02:00
[File]          File Access Date/Time           : 2021:08:22 17:53:44+02:00
[File]          File Inode Change Date/Time     : 2021:08:22 17:53:06+02:00
[File]          File Permissions                : -rwxr-----
[File]          File Type                       : MP4
[File]          File Type Extension             : mp4
[File]          MIME Type                       : video/mp4
[QuickTime]     Major Brand                     : MP4 v2 [ISO 14496-14]
[QuickTime]     Minor Version                   : 0.0.0
[QuickTime]     Compatible Brands               : isom, mp42
[QuickTime]     Movie Header Version            : 0
[QuickTime]     Create Date                     : 2021:08:22 10:48:23
[QuickTime]     Modify Date                     : 2021:08:22 10:48:23
[QuickTime]     Time Scale                      : 1000
[QuickTime]     Duration                        : 2.47 s
[QuickTime]     Preferred Rate                  : 1
[QuickTime]     Preferred Volume                : 100.00%
[QuickTime]     Preview Time                    : 0 s
[QuickTime]     Preview Duration                : 0 s
[QuickTime]     Poster Time                     : 0 s
[QuickTime]     Selection Time                  : 0 s
[QuickTime]     Selection Duration              : 0 s
[QuickTime]     Current Time                    : 0 s
[QuickTime]     Next Track ID                   : 2
[QuickTime]     Android Version                 : 10
[QuickTime]     Android Manufacturer            : Sony
[QuickTime]     Android Model                   : G8441
[QuickTime]     Track Header Version            : 0
[QuickTime]     Track Create Date               : 2021:08:22 10:48:23
[QuickTime]     Track Modify Date               : 2021:08:22 10:48:23
[QuickTime]     Track ID                        : 1
[QuickTime]     Track Duration                  : 2.47 s
[QuickTime]     Track Layer                     : 0
[QuickTime]     Track Volume                    : 0.00%
[QuickTime]     Matrix Structure                : 0 1 0 -1 0 0 0 0 1
[QuickTime]     Image Width                     : 1280
[QuickTime]     Image Height                    : 720
[QuickTime]     Media Header Version            : 0
[QuickTime]     Media Create Date               : 2021:08:22 10:48:23
[QuickTime]     Media Modify Date               : 2021:08:22 10:48:23
[QuickTime]     Media Time Scale                : 90000
[QuickTime]     Media Duration                  : 2.47 s
[QuickTime]     Handler Type                    : Video Track
[QuickTime]     Handler Description             : VideoHandle
[QuickTime]     Graphics Mode                   : srcCopy
[QuickTime]     Op Color                        : 0 0 0
[QuickTime]     Compressor ID                   : avc1
[QuickTime]     Source Image Width              : 1280
[QuickTime]     Source Image Height             : 720
[QuickTime]     X Resolution                    : 72
[QuickTime]     Y Resolution                    : 72
[QuickTime]     Bit Depth                       : 24
[QuickTime]     Pixel Aspect Ratio              : 65536:65536
[QuickTime]     Color Representation            : nclx 1 1 1
[QuickTime]     Video Frame Rate                : 29.93
[QuickTime]     Media Data Size                 : 4365912
[QuickTime]     Media Data Offset               : 405268
[Composite]     Image Size                      : 1280x720
[Composite]     Megapixels                      : 0.922
[Composite]     Avg Bitrate                     : 14.1 Mbps
[Composite]     Rotation                        : 90
One can see, that tags of 'Track1' and 'Keys' groups are removed from the output.
Similarly, this happens when generating JSON output:

Code Select Expand

exiftool -j file.mp4
results in all tags, but without any group information, but

Code Select Expand

exiftool -j -G file.mp4
again results in tags from 'Track1' and 'Keys' groups filtered out. As I am using PyExifTool, I would like to get all tags with their group names in JSON format, which is currently not possible.
Also the documentation of -g or -G options do not mention, that tags are filtered out when using those options, as such I would consider this a bug.

best
Martin

[derMart]

Ah ok, I understand more what is going on. It seems the -X and -G option do not agree upon the same group names.
While AndroidVersion is under group "Keys" in -X output, it is under group "QuickTime" in -G output, similarly for the "Track1" group. That is quite confusing. I would expect the group names to be identical.

[StarGeek]

See FAQ #3

-X (-xmlFormat) option forces the -a (-duplicates) option, which is missing from your other commands.

[derMart]

-X (-xmlFormat) option forces the -a (-duplicates) option, which is missing from your other commands.

so does the -j option, for which the problem also exists. But for the file used, the -a option does not make a difference when used together with -G

Code Select Expand

exiftool -G -a file.mp4 output is identical to

Code Select Expand

exiftool -G file.mp4 in my case.
As stated in my own reply, the problem is not that values are missing, but the group names differ between -X and -G.

[StarGeek]

Ah, ok. 

The xml output is giving the Group 1 names, though I'm not sure why as I can't find it mentioned in the docs.

When you use just -G, that is equivalent to -G0, which gives you the general information type, (XMP, EXIF, Quicktime).  The -G1 option gives the exact location (XMP-dc, IFD0, ItemList).  So if you add -G1 instead of -G to your first command, you'll get the same group names.

Hopefully Phil will chime in on the group names in XML/Json output.

[derMart]

So if you add -G1 instead of -G to your first command, you'll get the same group names.

Ah thank you very much, yes that does the trick. It would be nice, if this could be documented somewhere. E.g in the -X option could be documented that the group names equal to 1 (specific location)
Just some further thoughts:
By default it would be nice if -G and -X would produce the same group names (without being more specific with -G1).
Thinking more about that, it currently is not possible to specify the grouping in the -X option, it is always equivalent to -G1. E.g. combining -X with -G0 is not possible. Maybe it is even possible to refactor that to behave uniformly. E.g. -X respects -G (and the lack of, meaning that -X is also possible to generate no group names)

[Phil Harvey]

Ah thank you very much, yes that does the trick. It would be nice, if this could be documented somewhere. E.g in the -X option could be documented that the group names equal to 1 (specific location)

I'll add something like this to the -X option documentation:

The ExifTool family 1 group names are used as the namespaces in the XML output.

Just some further thoughts:
By default it would be nice if -G and -X would produce the same group names (without being more specific with -G1).
Thinking more about that, it currently is not possible to specify the grouping in the -X option, it is always equivalent to -G1. E.g. combining -X with -G0 is not possible. Maybe it is even possible to refactor that to behave uniformly. E.g. -X respects -G (and the lack of, meaning that -X is also possible to generate no group names)

The -X option output is very specific so that ExifTool can import tags from this option for writing to files.  This would all break if I allowed user-specified namespaces.

- Phil

[StarGeek]

I'll add something like this to the -X option documentation:

The ExifTool family 1 group names are used as the namespaces in the XML output.

Just a bump, as it doesn't look like this has made it into the docs yet.  I just had a question along these lines on StackOverflow.

[Phil Harvey]

I made this change, but the new version hasn't been released yet.  The documentation for version 12.31 will say this:

            Note:  This output is NOT the same as XMP because it uses
            dynamically-generated property names corresponding to the ExifTool
            tag names with ExifTool family 1 group names as namespaces, and
            not the standard XMP properties and namespaces.  To write XMP
            instead, use the -o option with an XMP extension for the output
            file.

- Phil