Using < to re-map CSV properties

[StarGeek]

Took me some time to dig it out (was before I started taking notes w/ links), but here's another shortcut trick.

One user wanted to include up to six tags as part of the filename.  But the problem was that sometimes one or more of the tags didn't exist.  Creating a command to cover all the combinations of possible missing tags would be... difficult to say the least.

So instead, using a shortcut tag with all the tags they wanted to use in the correct order, along with the -m (-ignoreMinorErrors) option and the -sep option, it would take a single command.  Any tags that were missing were replaced by 0 length string because of the -m option and  the -sep option is used as the separator between the tags in the shortcut.

The shortcut tag was
MyShortcut => ['CountryCode','Country','State','City','Location','SupplementalCategories'],

and the command ended up being
exiftool -sep " " -m "-testname<${MyShortcut;}" "X:\!temp\aa\2015-03-31_0043.JPG"

See this post.

[StarGeek]

With regards to editing CSV files, in searching for something else for an idea, I came across Miller, which looks like it's more powerful.

Miller is like awk, sed, cut, join, and sort for data formats such as CSV, TSV, JSON, JSON Lines, and positionally-indexed

[Martin Z]

exiftool -sep " " -m "-testname<${MyShortcut;}" "X:\!temp\aa\2015-03-31_0043.JPG"

Thanks for digging out this from your notes. Unfortunately, if am honest this confused me a little bit...

• I couldn't find any reference to -testname in the post you linked, wouldn't this error?
• Also, I wasn't quite sure why MyShortcut was wrapped in braces - i.e. like ${MyShortcut;}... from my admitted very limited understanding of the 'advanced' syntax commands [see table below], using ${ } makes EXIFtool evaluate the expression before applying the value, right(?), but if so I didn't think you would need to do this if you were re-mapping a tag (with the < syntax)

Disclaimer for anyone else readings -- these are my personal notes/rough understandings only, please do not take the above as definitive.

[Martin Z]

BOLLOCKS!... I refreshed the wrong tab and lost about 30 minutes of typing! 🙈 OK, I have re-typed as much as I can bear to, hopefully all the key points are still there...

Martin: Unknown tags may be extracted even if they aren't defined (but tags must be defined to be written)
StarGeek: It depends.  IPTC and EXIF tags do not include a name as part of the tag. For example, the GPSRoll and GPSPitch tags that are defined in the .Example.config, but without the config file, they could only be extracted with the -u (-unknown) option.  And then the results would be 0xd001 and 0xd000... XMP has the tag name built in, so unknown XMP tags can be extracted by default.  Interestingly, this is how I discovered Victoria's Secret has their own XMP tags (XMP-Victorias_Secret).

So just to clarify, and check I have the right understanding in terms of "unknown tags". I had hoped I could do something like the below...

Code Select Expand

EXIF_Metadata.csv
=================
| SourceFile | MasterDate          | XPSubject | ColumnForAnotherApp1 | ColumnForAnotherApp2 |
|------------|---------------------|-----------|----------------------|----------------------|
| File1.jpg  | 2023:05:30:23:59:59 | Foo       | Ignore this EXIFTool | ... and this too     |
| File2.jpg  | 2023:01:01:12:00:00 | Bar       | Ignore this EXIFTool | ... and this too     |

.EXIFTOOL_CONFIG
================
%Image::ExifTool::UserDefined::Shortcuts = (
    MasterDate => ['FileCreateDate','FileModifyDate','CreateDate','DateTimeOriginal'],
    ColumnForAnotherApp1**,
    ColumnForAnotherApp2**,
);
#------------------------------------------------------------------------------
1;  #end
** i.e. as ColumnForAnotherApp1 and ColumnForAnotherApp2 are defined but not linked to mapped to anything, my hope was that EXIFtool would then be like: "Ah, I am aware of 'ColumnForAnotherApp1' now so I won't throw an error if I see this in a CSV, but there's no mapping / nothing for me to do with this column/tag so I'll skip over it"

AllDates was created way back in the history of exiftool.  Phil has more and more standardized the way he's organized tag since then.

That's cool, TBH the main reason I mentioned it was that I recalled someone asking if AllDates could be changed so it also updated the FileModifyDate and FileCreateDate, and a pretty direct reply from Phil basically saying that there was no chance of AllDates changing, as it would break potentially a huge amount of existing scripts and workflows.

• Which like I say is totally fine, so I just wanted to make the point that I wasn't asking for AllDates to change, and accepted how it currently works 👍🏼

Martin: I could use this to override AllDates too right? #sogood!
StarGeek: Yes, but instead I'd suggest giving your shortcut tag a different name.  The early examples Phil gave for user-defined tags and shortcuts was often "MyTAG", and I've just kept that as my standard.

Again, totally fine! (And TBH I actually agree on the general principle of not overriding standard/ootb functions, and instead creating a similar, alternate copy) -- really I just wanted to check my understanding of the capabilities / make the point for any potential future readers that they could do this if they really wanted.

[StarGeek]

I couldn't find any reference to -testname in the post you linked, wouldn't this error?

From Writing "FileName" and "Directory" tags (also documented on the Extra Tags page)

The write-only TestName tag provides a mechanism for dry-run testing of the rename feature. Writing TestName displays the old and new names without making any changes to the files.

Also, I wasn't quite sure why MyShortcut was wrapped in braces - i.e. like ${MyShortcut;}

Also from the above page

Warning: Writing illegal file names in Windows can have unpredictable results and may result in data loss.
...
Any tag used in generating a file name which may contain these characters must first be filtered to remove or translate these characters. A special feature allows these characters to be removed from a tag value by adding a semicolon inside the braces after a tag name

The filtering is most commonly used with Make and Model, as many people like to put those in a filename and many makes and models include slashes and colons.  It's probably not very likely to pop up with any of the tags used in that shortcut, but the impact of including it would be extremely small.

Note that this only works if there is no other advanced formatting taking place.  In that case you would have to manually add the command to filter the characters.  This is listed  under the Advanced formatting feature

A default expression of tr(/\\?*:|"<>\0)()d is assumed if the expression is empty (ie. ${TAG;}).

[StarGeek]

i.e. as ColumnForAnotherApp1 and ColumnForAnotherApp2 are defined but not linked to mapped to anything, my hope was that EXIFtool would then be like: "Ah, I am aware of 'ColumnForAnotherApp1' now so I won't throw an error if I see this in a CSV, but there's no mapping / nothing for me to do with this column/tag so I'll skip over it"

No need to map anything.  Just don't use column names that are actual tag names.  Exiftool will ignore them when importing from a CSV.

Code Select Expand

C:\>type temp.txt
SourceFile,description,abc,xyz
y:/!temp/Test4.jpg,New description,ignore me,me too

C:\>exiftool -P -overwrite_original -csv=temp.txt y:\!temp\Test4.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -e --file:all y:\!temp\Test4.jpg
[ExifTool]      ExifToolVersion                : 12.63
[XMP-x]        XMPToolkit                      : Image::ExifTool 12.63
[XMP-dc]        Description                    : New description

[Martin Z]

Just don't use column names that are actual tag names.  Exiftool will ignore them when importing from a CSV.

Whaaaaaa!?

OK, I must have missed something way back, as have been literally working with the opposite understanding for MONTHS (i.e. that I could ONLY include valid tag names in the CSV or else EXIFtool would refuse to process the CSV)

Assuming so, that's a major "whooops" on my part!... Urgh! 🙈

➡️ Go to post

Cool, thanks for the explanations!

PS - Wow, t's like peeling an infinite onion (i.e. you un-wrap one layer of info, only to find there are yet more and more)... How does your head not just constantly spin/explode with all this!? 😁