User defined MIE tags appear to be ignored or skipped when attempting their use

[Earnestly]

• exiftool version 12.60
• perl v5.38.0
• Arch Linux

I'm currently experimenting with using MIE sidecars to store metadata and wish to include some of my own metadata relevant to the data the sidecars are storing.

Here following is the mie-johndee.config I am attempting to use:

Code Select Expand

use Image::ExifTool::MIE;

%Image::ExifTool::UserDefined = (
    'Image::ExifTool::MIE::Meta' => {
        JohnDee => {
            SubDirectory => {
                TagTable => 'Image::ExifTool::UserDefined::JohnDee',
                DirName => 'MIE-JohnDee'
            }
        }
    }
);

%Image::ExifTool::UserDefined::JohnDee = (
    %Image::ExifTool::MIE::tableDefaults,
    GROUPS => {0 => 'MIE', 1 => 'MIE-JohnDee'},
    WRITE_GROUP => 'MIE-JohnDee',

    Origin => {
        Description => 'The original name of the resource being referenced.',
        Writable => 'string'
    },

    CommitDate => {
        Description => 'The time at which the resource was incorporated.',
        Groups => {2 => 'Time'},
        Writable => 'date',
        %dateInfo
    },

    Progress => {
        Description => 'Used to track progression, typically represented as a percentage.',
        Writable => 'double'
    },

    Notes => {
        Description => 'For any comments about the resource.',
        Writable => 'string'
    }
);

1;

While exiftool recognises my MIE-Johndee tag it appears to skip writing or creating them when attempting to use the -mie-johndee:commitdate=now tag:

Code Select Expand

$ TZ=UTC exiftool -config mie-johndee.config -v5 -mie-johndee:commitdate=now -mie-doc:comment='Why are the johndee tags skipped?' -o test.mie
Writing MIE-Johndee:CommitDate
Writing MIE-Doc:Comment
========
'' --> 'test.mie'
Creating MIE file...
  FileType = MIE
  FileTypeExtension = MIE
  MIMEType = application/x-mie
  Editing tags in: MIE MIE-Doc MIE-Johndee MIE-Meta
  Creating tags in: MIE MIE-Doc MIE-Johndee MIE-Meta
Writing MIE1-Main:
Creating MIE1-Meta1:
Creating MIE1-Doc1:
    + MIE1-Doc1:Comment = 'Why are the johndee tags skipped?'
    1 image files created

$ exiftool -d %s -G0:1:2 -mie:all test.mie
[MIE:MIE-Doc:Document] Comment                  : Why are the johndee tags skipped?

What mistake am I making?

[StarGeek]

I don't think that
Writable => 'date',
is an option.

Looking through the source code of MIE.pm and comparing to other date/time tags, they don't give a type, which I think means they default to string

Creating a minimal MIE file with one date/time tag and using the -v (-verbose) option, I see this, which sets ModifyDate as a string

Code Select Expand

  + [MIE-Main directory]
  | Meta (SubDirectory) -->
  | - Tag 'Meta', byte order MM (0 bytes)
  | + [MIE-Meta directory]
  | | Document (SubDirectory) -->
  | | - Tag 'Document', byte order MM (0 bytes)
  | | + [MIE-Doc directory]
  | | | ModifyDate = 2023:08:31 07:41:22-07:00
  | | | - Tag 'ModifyDate' (25 bytes, string):
  | | |     002e: 32 30 32 33 3a 30 38 3a 33 31 20 30 37 3a 34 31 [2023:08:31 07:41]
  | | |     003e: 3a 32 32 2d 30 37 3a 30 30                      [:22-07:00]
Ah, ok, figured it out.

You're referencing the %dateInfo, but there isn't a local copy of that in this file.  I tried copying your line
%Image::ExifTool::MIE::tableDefaults,
changing it to dateInfo, but that didn't work, so I'm not sure that your tableDefaults is doing anything.  Phil will have to answer that, but he's away until mid-September

If I copy/paste the the dateInfo hash into your config and it appears to work.

Code Select Expand

C:\>exiftool -config mie-johndee.config -P -overwrite_original -Origin="hello" -commitdate=now -modifydate=now y:\!temp\Test4.mie
    1 image files updated

C:\>exiftool -config mie-johndee.config -G1 -a -s -e --file:all y:\!temp\Test4.mie
[ExifTool]      ExifToolVersion                 : 12.64
[MIE-Doc]       ModifyDate                      : 2023:08:31 08:01:09-07:00
[MIE-JohnDee]   CommitDate                      : 2023:08:31 08:01:09-07:00
[MIE-JohnDee]   Origin                          : hello

Code Select Expand

use Image::ExifTool::MIE;

my %dateInfo = (
    Shift => 'Time',
    PrintConv => '$self->ConvertDateTime($val)',
    PrintConvInv => '$self->InverseDateTime($val)',
);

%Image::ExifTool::UserDefined = (
    'Image::ExifTool::MIE::Meta' => {
        JohnDee => {
            SubDirectory => {
                TagTable => 'Image::ExifTool::UserDefined::JohnDee',
                DirName => 'MIE-JohnDee'
            }
        }
    }
);

%Image::ExifTool::UserDefined::JohnDee = (
    %Image::ExifTool::MIE::tableDefaults,
    GROUPS => {0 => 'MIE', 1 => 'MIE-JohnDee'},
    WRITE_GROUP => 'MIE-JohnDee',

    Origin => {
        Description => 'The original name of the resource being referenced.',
        Writable => 'string'
    },

    CommitDate => {
        Description => 'The time at which the resource was incorporated.',
        Groups => { 2 => 'Time' },
        %dateInfo
    },

    Progress => {
        Description => 'Used to track progression, typically represented as a percentage.',
        Writable => 'double'
    },

    Notes => {
        Description => 'For any comments about the resource.',
        Writable => 'string'
    }
);

1;

[Earnestly]

I don't think that
Writable => 'date',
is an option.

Good catch, this was something left over from when I was using XMP.

You're referencing the %dateInfo, but there isn't a local copy of that in this file.

I didn't quite know how perl exports module variables but if use vars qw($VERSION %tableDefaults); implies that only %tableDefaults and $VERSION are exported then this makes sense. Thanks for spotting that.

If I copy/paste the the dateInfo hash into your config and it appears to work.

Code Select Expand

C:\>exiftool -config mie-johndee.config -P -overwrite_original -Origin="hello" -commitdate=now -modifydate=now y:\!temp\Test4.mie
    1 image files updated

C:\>exiftool -config mie-johndee.config -G1 -a -s -e --file:all y:\!temp\Test4.mie
[ExifTool]      ExifToolVersion                : 12.64
[MIE-Doc]      ModifyDate                      : 2023:08:31 08:01:09-07:00
[MIE-JohnDee]  CommitDate                      : 2023:08:31 08:01:09-07:00
[MIE-JohnDee]  Origin                          : hello

After applying your suggestions, which I agree are correct, I get some results but they're a bit different and surprising:

Code Select Expand

$ exiftool -config mie-johndee.config -P -overwrite_original -Origin="hello" -commitdate=now -modifydate=now test.mie
    1 image files updated

$ exiftool -d %s -G0:1:2 --file:all test.mie
[ExifTool]      ExifTool Version Number         : 12.60
[MIE:MIE-Doc:Time] Modify Date                  : 1693562053
[MIE:MIE-Unknown:Other] Commit Date             : 2023:09:01 10:54:13+01:00
[MIE:MIE-Unknown:Other] Origin                  : hello

Here the MIE-JohnDee tag (group 1?) appears to be MIE-Unknown and I'm still unable to specify the tags explicitly using -mie-johndee:commitdate, etc. which I would like to do as I don't want other tags being set which happen to share the same name (in general I'd love this behaviour to be disabled completely, but that's by the by).

Also note that Commit Date doesn't seem to benefit from -d %s while the Modify Date did.  I found the same issue when trying to use XMP tags.

I understand that Phil is on holiday at the moment, but thank you for the improvements nevertheless.  For completeness here is the current mie-johndee.config I am using:

Code Select Expand

use Image::ExifTool::MIE;

%Image::ExifTool::UserDefined = (
    'Image::ExifTool::MIE::Meta' => {
        JohnDee => {
            SubDirectory => {
                TagTable => 'Image::ExifTool::UserDefined::JohnDee',
                DirName => 'MIE-JohnDee'
            }
        }
    }
);

%Image::ExifTool::UserDefined::JohnDee = (
    %Image::ExifTool::MIE::tableDefaults,
    GROUPS => {0 => 'MIE', 1 => 'MIE-JohnDee'},
    WRITE_GROUP => 'MIE-JohnDee',

    Origin => {
        Description => 'The original name of the resource being referenced.',
        Writable => 'string'
    },

    CommitDate => {
        Description => 'The time at which the resource was incorporated.',
        Groups => { 2 => 'Time' },
        Shift => 'Time',
        PrintConv => '$self->ConvertDateTime($val)',
        PrintConvInv => '$self->InverseDateTime($val)',
    },

    Progress => {
        Description => 'Used to track progression, typically represented as a percentage.',
        Writable => 'double'
    },

    Notes => {
        Description => 'For any comments about the resource.',
        Writable => 'string'
    }
);

1;

The named hash (dateInfo) was only used once so I have inserted it directly instead.

[StarGeek]

Here the MIE-JohnDee tag (group 1?) appears to be MIE-Unknown

Yes, the group name is not included as part of the MIE file.  You have to use the config file while reading to get that data.  See the second command in my output listing.

and I'm still unable to specify the tags explicitly using -mie-johndee:commitdate, etc. which I would like to do as I don't want other tags being set which happen to share the same name

Hmmm... Neither can I.  I'm not sure what might be wrong.  This part may have to wait for Phil.

Also note that Commit Date doesn't seem to benefit from -d %s while the Modify Date did.  I found the same issue when trying to use XMP tags.

It works correctly here.

Code Select Expand

C:\>exiftool -config mie-johndee2.config -G -a -s -e --file:all  y:\!temp\Test4.mie
[ExifTool]      ExifToolVersion                : 12.64
[MIE]          ModifyDate                      : 2023:09:02 14:24:17-07:00
[MIE]          CommitDate                      : 2023:09:01 12:00:00
[MIE]          Origin                          : hello

C:\>exiftool -config mie-johndee2.config -G -a -s -e --file:all -d %s y:\!temp\Test4.mie
[ExifTool]      ExifToolVersion                : 12.64
[MIE]          ModifyDate                      : 1693689857
[MIE]          CommitDate                      : 1693594800
[MIE]          Origin                          : hello
And it should work for XMP.  Maybe put quotes around it, e.g. -d '%s' Maybe the shell you're using intercepts the percent sign?

But it worked for ModifyDate...  I'm not sure what is happening. I'm on Windows so I can't try and replicate it exactly.

[Earnestly]

Here the MIE-JohnDee tag (group 1?) appears to be MIE-Unknown

Yes, the group name is not included as part of the MIE file.  You have to use the config file while reading to get that data.  See the second command in my output listing.

Ah yes of course, thank you. I was still thinking in terms of XMP where that is not strictly necessary; it might also be why my XMP dates didn't work either. When properly specifying the config the date listing and such works nicely.

I'll wait to see what sense can be made about "direct" tags not seemingly working.

Edit:
Just to add, it is possible to use -mie:commitdate=now which appears to use MIE1-JohnDee1 however attempting to use this numbered tag (-mie-johndee1:commitdate=now) is no good.

Code Select Expand

$ TZ=UTC exiftool -config mie-johndee.config -v5 -mie:commitdate=now -o test.mie
Writing MIE-JohnDee:CommitDate
========
'' --> 'test.mie'
Creating MIE file...
  FileType = MIE
  FileTypeExtension = MIE
  MIMEType = application/x-mie
  Editing tags in: MIE MIE-JohnDee MIE-Meta
  Creating tags in: MIE MIE-JohnDee MIE-Meta
Writing MIE1-Main:
Creating MIE1-Meta1:
Creating MIE1-JohnDee1:
    + MIE1-JohnDee1:CommitDate = '2023:09:02 23:54:17+00:00'
    1 image files created


[StarGeek]

Just to add, it is possible to use -mie:commitdate=now which appears to use MIE1-JohnDee1 however attempting to use this numbered tag (-mie-johndee1:commitdate=now) is no good.

That's interesting.

But CommitDate appears to be a unique name.  Checking the tag names list, it doesn't appear at all and exiftool doesn't write it without your config file.