Copying Sony DSC-RX100M7 MP4 GPS data to normal tags

[hjs]

When creating an MP4 file, Sony DSC-RX100M7 stores the GPS location in embedded XML in the MP4 file and in an XML sidecar file containing the same data.

exiftool can report many of the fields of this XML format (NonRealTimeMeta xmlns="urn:schemas-professionalDisc:nonRealTimeMeta:ver.2.20") both from MP4 and from the sidecar XML file. However, exiftool does not appear to be able to read the ExifGPS data group.

I'd like to copy the GPS data contained in this Sony XML format into fields commonly used for MP4, specifically, the same ones that iOS uses so that the fields would be compatible with Apple Photos.

Is there already a solution for this?

For reference, the data looks like this:

The root element is NonRealTimeMeta in the "urn:schemas-professionalDisc:nonRealTimeMeta:ver.2.20" namespace. This has (among other children) a child element called AcquisitionRecord (in the same, inherited, namespace). This has a child element like this:

      <Group name="ExifGPS">
         <Item name="VersionID" value="2.2.0.0"/>
         <Item name="LatitudeRef" value="N"/>
         <Item name="Latitude" value="66:09:14.920"/>
         <Item name="LongitudeRef" value="E"/>
         <Item name="Longitude" value="27:31:49.382"/>
         <Item name="TimeStamp" value="12:37:52.000"/>
         <Item name="Status" value="A"/>
         <Item name="MeasureMode" value="2"/>
         <Item name="MapDatum" value="WGS-84"/>
         <Item name="DateStamp" value="2021:07:27"/>
         <Item name="Differential" value="0"/>
      </Group>

Notably, there's a sibling element that looks like:

      <Group name="CameraUnitMetadataSet">
         <Item name="CaptureGammaEquation" value="rec709-xvycc"/>
         <Item name="CaptureColorPrimaries" value="rec709"/>
         <Item name="CodingEquations" value="rec709"/>
      </Group>

exiftool knows how to read this sibling group!

[hjs]

Looks like exiftool doesn't really know how to read the sibling group. It appears that there's some generic XML to tags mapping going on, and that mapping autogenerates tags like this:
AcquisitionRecordGroupName      : CameraUnitMetadataSet
AcquisitionRecordGroupItemName  : CaptureGammaEquation
AcquisitionRecordGroupItemValue : rec709-xvycc
AcquisitionRecordChangeTableName: ImagerControlInformation
AcquisitionRecordChangeTableEventStatus: start
AcquisitionRecordChangeTableEventFrameCount: 0

without those tags appearing in the exiftool source code.

FWIW, to normalize other important fields, this worked:
exiftool -tagsfromfile C0024.MP4 '-CreationDateValue>CreationDate' '-DeviceManufacturer>Make' '-DeviceModelName>Model' C0024.MP4

It appears that the same pattern could work for GPS if the XML to tags mapper generated a tag for each of the Item element children of the Group name="ExifGPS" element.

[hjs]

For reference, the full embedded XML looks like this:

<?xml version="1.0" encoding="UTF-8"?>
<NonRealTimeMeta xmlns="urn:schemas-professionalDisc:nonRealTimeMeta:ver.2.20" xmlns:lib="urn:schemas-professionalDisc:lib:ver.2.00" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" lastUpdate="2021-07-27T15:37:55+03:00">
        <TargetMaterial umidRef="060A2B340101010501010D4313000000EA65BDA6229405DE707414FFFE9F377F"/>
        <Duration value="156"/>
        <LtcChangeTable tcFps="25" halfStep="false">
                <LtcChange frameCount="0" value="04560600" status="increment"/>
                <LtcChange frameCount="155" value="09020700" status="end"/>
        </LtcChangeTable>
        <CreationDate value="2021-07-27T15:37:55+03:00"/>
        <VideoFormat>
                <VideoRecPort port="DIRECT"/>
                <VideoFrame videoCodec="AVC_3840_2160_HP@L51" captureFps="25p" formatFps="25p"/>
                <VideoLayout pixel="3840" numOfVerticalLine="2160" aspectRatio="16:9"/>
        </VideoFormat>
        <AudioFormat numOfChannel="2">
                <AudioRecPort port="DIRECT" audioCodec="LPCM16" trackDst="CH1"/>
                <AudioRecPort port="DIRECT" audioCodec="LPCM16" trackDst="CH2"/>
        </AudioFormat>
        <Device manufacturer="Sony" modelName="DSC-RX100M7" serialNo="REDACTED"/>
        <RecordingMode type="normal" cacheRec="false"/>
        <AcquisitionRecord>
                <Group name="CameraUnitMetadataSet">
                        <Item name="CaptureGammaEquation" value="rec709-xvycc"/>
                        <Item name="CaptureColorPrimaries" value="rec709"/>
                        <Item name="CodingEquations" value="rec709"/>
                </Group>
                <Group name="ExifGPS">
                        <Item name="VersionID" value="2.2.0.0"/>
                        <Item name="LatitudeRef" value="N"/>
                        <Item name="Latitude" value="66:09:14.920"/>
                        <Item name="LongitudeRef" value="E"/>
                        <Item name="Longitude" value="27:31:49.382"/>
                        <Item name="TimeStamp" value="12:37:52.000"/>
                        <Item name="Status" value="A"/>
                        <Item name="MeasureMode" value="2"/>
                        <Item name="MapDatum" value="WGS-84"/>
                        <Item name="DateStamp" value="2021:07:27"/>
                        <Item name="Differential" value="0"/>
                </Group>
                <ChangeTable name="ImagerControlInformation">
                        <Event status="start" frameCount="0"/>
                </ChangeTable>
                <ChangeTable name="LensControlInformation">
                        <Event status="start" frameCount="0"/>
                </ChangeTable>
                <ChangeTable name="DistortionCorrection">
                        <Event status="start" frameCount="0"/>
                </ChangeTable>
                <ChangeTable name="Gyroscope">
                        <Event status="start" frameCount="0"/>
                </ChangeTable>
                <ChangeTable name="Accelerometor">
                        <Event status="start" frameCount="0"/>
                </ChangeTable>
        </AcquisitionRecord>
</NonRealTimeMeta>

[StarGeek]

Looks like exiftool doesn't really know how to read the sibling group.

Include the -a (-duplicates) option.  The sibling groups have same name, NonRealTimeMetaAcquisitionRecordGroupItemName and NonRealTimeMetaAcquisitionRecordGroupItemValue.

Because they have duplicate names in exiftool and the data is split into separate tags, a NAME tag and a VALUE tag, I'm not sure how to extract the data separately.

[StarGeek]

Ah, just remembered the join_tags.config file.   Download it from this post and save it in the same directory as exiftool.  You then need to edit it to combine the correct tags like this

Code Select Expand

my $nameTag = 'NonRealTimeMetaAcquisitionRecordGroupItemName';
my $valueTag = 'NonRealTimeMetaAcquisitionRecordGroupItemValue';
Example output:

Code Select Expand

C:\exiftool -config join_tags.config -G1 -a -s -XML:All temp.xml
[XML]           CaptureGammaEquation            : rec709-xvycc
[XML]           CaptureColorPrimaries           : rec709
[XML]           CodingEquations                 : rec709
[XML]           VersionID                       : 2.2.0.0
[XML]           LatitudeRef                     : N
[XML]           Latitude                        : 66:09:14.920
[XML]           LongitudeRef                    : E
[XML]           Longitude                       : 27:31:49.382
[XML]           TimeStamp                       : 12:37:52.000
[XML]           Status                          : A
[XML]           MeasureMode                     : 2
[XML]           MapDatum                        : WGS-84
[XML]           DateStamp                       : 2021:07:27
[XML]           Differential                    : 0

[Phil Harvey]

Could you upload a sample MP4?  I would like to take a look at this.  You can email me the link if you want: philharvey66 at gmail.com

I'm glad StarGeek is here to remember join_tags.config, because I wouldn't have.

- Phil

[hjs]

> Could you upload a sample MP4?

I sent email. Thanks!

> Include the -a (-duplicates) option.

Thank you. This indeed shows the duplicate tagnames. (I did look at the FAQ, but apparently I didn't understand the right things.)

> Ah, just remembered the join_tags.config file.  Download it from this post and save it in the same directory as exiftool.  You then need to edit it to combine the correct tags like this

Thank you. It worked without editing and broke with the edit you suggested.

I used

Code Select Expand

exiftool -config join_tags.config -tagsfromfile C0024.MP4 -G1 -a '-LatitudeRef>GPSLatitudeRef' '-Latitude>GPSLatitude' '-LongitudeRef>GPSLongitudeRef' '-Longitude>GPSLongitude' '-CreationDateValue>CreationDate' '-DeviceManufacturer>Make' '-DeviceModelName>Model' C0024.MP4

I'm using exiftool 12.82 on macOS 14.4.1.

(It turns out that writing to the Sony file the same tag names that I get from reading an Apple-device-generated video file ends up writing the tags into XMP whereas Apple uses mebx, and I see from another thread that mebx writing isn't expected to be supported. I'll have to experiment to see whether the difference matters.)

[hjs]

> It worked without editing and broke with the edit you suggested.

I'm guessing the reason for this is that when exiftool reads the Sony XML from inside MP4 (as opposed to the sidecar), the code that calls into the XMP parser explicitly omits the root element name.

[StarGeek]

> It worked without editing and broke with the edit you suggested.

I'm guessing the reason for this is that when exiftool reads the Sony XML from inside MP4 (as opposed to the sidecar), the code that calls into the XMP parser explicitly omits the root element name.

Since I didn't have the original file, I had to copy your XML data to a file and run exiftool on that.  So that probably did introduce some more tag name levels.

[hjs]

It turns out that writing to the Sony file the same tag names that I get from reading an Apple-device-generated video file ends up writing the tags into XMP whereas Apple uses mebx, and I see from another thread that mebx writing isn't expected to be supported. I'll have to experiment to see whether the difference matters.

Indeed, the command I posted previously wasn't compatible Apple Photos or GraphicConverter.

To get the results I wanted, I needed to do:

Code Select Expand

exiftool -config join_tags.config -tagsfromfile C0024.MP4 -G1 -a '-LatitudeRef>GPSLatitudeRef' '-Latitude>GPSLatitude' '-LongitudeRef>GPSLongitudeRef' '-Longitude>GPSLongitude' '-CreationDateValue>Keys:CreationDate' '-DeviceManufacturer>Keys:Make' '-DeviceModelName>Keys:Model' C0024.MP4
exiftool -tagsfromfile C0024.MP4 '-GPSPosition>Keys:GPSCoordinates' C0024.MP4

That is, all the data I wanted to move needed to go to the Keys group, but the GPS location couldn't go there in one step. Instead, I first needed to move the Sony fields to XMP, and then, once exiftool would read the XMP GPS as a composite, write the composite to Keys:GPSCoordinates.

[StarGeek]

That is, all the data I wanted to move needed to go to the Keys group, but the GPS location couldn't go there in one step. Instead, I first needed to move the Sony fields to XMP, and then, once exiftool would read the XMP GPS as a composite, write the composite to Keys:GPSCoordinates.

Try
exiftool -config join_tags.config -tagsfromfile C0024.MP4 -G1 -a '-Keys:GPSCoordinates<$Latitude $LatitudeRef, $Longitude $LongitudeRef' /path/to/files/

[hjs]

That is, all the data I wanted to move needed to go to the Keys group, but the GPS location couldn't go there in one step. Instead, I first needed to move the Sony fields to XMP, and then, once exiftool would read the XMP GPS as a composite, write the composite to Keys:GPSCoordinates.

Try
exiftool -config join_tags.config -tagsfromfile C0024.MP4 -G1 -a '-Keys:GPSCoordinates<$Latitude $LatitudeRef, $Longitude $LongitudeRef' /path/to/files/

This worked. Thank you!

Additionally, I discovered that to handle files over 2 GB, per https://github.com/photostructure/exiftool-vendored.js/issues/109#issuecomment-1043477394 , I need to add

Code Select Expand

%Image::ExifTool::UserDefined::Options = (
    LargeFileSupport => 1,
);

before the

Code Select Expand

1; #end line in the config file.

[StarGeek]

Additionally, I discovered that to handle files over 2 GB, per https://github.com/photostructure/exiftool-vendored.js/issues/109#issuecomment-1043477394 , I need to add

Yes, or you could add
-api LargeFileSupport
on the command line if you don't want to deal with editing the config file.

Addtional links
Docs on the -api LargeFileSupport option.
Why LargeFileSupport isn't on by default