Ingesting image files - creating and moving .xmp files along with image files.

[Skids]

In a post answering a question about creating xmp files and copy/moving image and xmp files Phil posted the following script to do the moving.

Code Select Expand

exiftool.exe -d %Y%m%d_%H%M%S-%-3f -tagsfromfile %d%f.NEF -fileorder -filename "-filename<myname_${subsecdatetimeoriginal;}_%f.%e" -ext nef -ext xmp .

I do not have much experience with the command line or exiftool so decided it would be worth while unpicking the command string to work out what it is doing.  I offer my notes for marking as I have no idea if I am correct.

The command is configured for windows so double quotes need to be replaced with single on MacOS. 

-d %Y%m%d_%H%M%S-%-3f The command starts with a dateformat (-dateformat) string which when decoded looks like this 20250215_102031-034 . Here the 3f refers to sub seconds to 3 decimal places and has nothing to do with %f used outside the date parser where it means base filename of original.

A side note I could note find a definition of -filename in the documents despite finding loads of references to it.

-tagsfromfile %d%f.NEF  "To copy or move metadata, the -tagsFromFile feature is used."  In this example it is reading from the original .nef file with %d refering to the source directory path and %f to the original file base name with the manual addition of the .nef extension.  Meaning that when an xmp file is processed the tags are read from the source nef raw file.

-fileorder -filename  "Sets the processing order"  In this case the processing order is by filename and the "-" prefix means reverse order filemane.  Meaning that in a folder of image files .xmp files are processed before .nef or .jpg files.

"-filename<myname_${subsecdatetimeoriginal;}_%f.%e" I think based on its position in the command i.e. just before the input filters and source folder that this is the portion that sets the output filename.  Here filename is set to "myname_20250215_102031-034_OriginalName.ext . Note that paths may be specified in this string with new folders being created as required e.g. .../myfolder/....

-ext nef -ext xmp Filters the input, only passing .nef and .xmp files to the command.

. Shorthand for source directory or folder - I think windows only.

How did I do?

I swapped references to nef with orf and ran the command against a folder of .orf .jpg .xmp files and as expected the .orf and .xmp were renamed but the .jpg files threw an error because the tag could not be found as the original .orf had a new name.

Based on my understanding and experimentation I conclude that an ingest process that moves, renames and writes metadata is best achieved through a number of calls to exiftool.  The order of actions will make a difference.  For example it appears simpler to rename the raw and jpeg images with the required prefix before the xmp sidecars are created.  This removes the complexity of having to know the raw image file type and ensuring it still exists in the folder.  A minor downside is that the images will be renamed on the SD card unless some intermediate folder is specified in the command.

Any thoughts welcomed.

best wishes

Simon

[greybeard]

The "." is also shorthand for the current directory (i.e. where the command is being run) on a Mac.

exiftool has a really powerful feature that helps in developing and testing this type of command. If you replace the -filename tag with -testname it will display the changes the would have taken place. Nothing is actually updated. Once everything looks good you swap back from -testname to -filename to actually make the changes.

[Skids]

#greybeard Thanks for the tip to use -testname I shall start experimenting with it.

S

[StarGeek]

The command is configured for windows so double quotes need to be replaced with single on MacOS.

While I consider it a best practice to use single quotes on Mac/Linux, the only time you need single quotes is when there is a dollar sign $ as part of the command. Otherwise, double quotes will work as well.

Another Linux/Mac thing to watch for is using wildcards in tag names (see second paragraph under the -TAG option). For example, one thing I often recommend is using a wild card to write GPS coordinates, as this will write both the main GPS tag and the Ref (reference direction) tag
-GPSLatitude*=40.6892 -GPSLongitude*=-74.0445
This should be quoted (double or single) on Mac/Linux because otherwise the asterisk might be replaced by a list of filenames.

-d %Y%m%d_%H%M%S-%-3f The command starts with a dateformat (-dateformat) string which when decoded looks like this 20250215_102031-034 . Here the 3f refers to sub seconds to 3 decimal places and has nothing to do with %f used outside the date parser where it means base filename of original.

See Common Date Format Codes. These codes are taken from the operating system and some of them will not work on Windows.

A side note I could note find a definition of -filename in the documents despite finding loads of references to it.

Filename (and Directory) are tags (called Pseudo-Tags by Phil), so you won't find them on the main documentation page. As tags, they will appear under the Tags Names page, specifically the Extra Tags page, where most (all?) of the file system tags appear. See also the Writing "FileName" and "Directory" tags page, which goes into more details.

"-filename<myname_${subsecdatetimeoriginal;}_%f.%e" I think based on its position in the command i.e. just before the input filters and source folder that this is the portion that sets the output filename.  Here filename is set to "myname_20250215_102031-034_OriginalName.ext . Note that paths may be specified in this string with new folders being created as required e.g. .../myfolder/....

The position of the file being processed on the command line doesn't matter. It can be the first thing in the command, the last thing, or in the middle, as long as it isn't directly after the -TagsFromFile option. Placing it at the end is more of a style guide, as it makes it more clear which is the target file.

I would also say that it's better to call it the target and not the source, as that makes it more clear that it is the file that will be processed. The source would be the file specified by the -TagsFromFile (or the -srcfile option), as that will be the source of any tags that are copied.

-ext nef -ext xmp Filters the input, only passing .nef and .xmp files to the command.

The location of this option doesn't matter. It affects the whole command, not just options that come after it. This applies to most options in the command. For example, using the -if option at the end of a command affects the whole command, not just what comes after it.

. Shorthand for source directory or folder - I think windows only.

This is a standard definition for all modern command lines. Related, two dots .. would be the parent directory, the directory above the current one. If the current directory (single dot .) was Y:\!temp\x\y, then using dotdot would be Y:\!temp\x

I swapped references to nef with orf and ran the command against a folder of .orf .jpg .xmp files and as expected the .orf and .xmp were renamed but the .jpg files threw an error because the tag could not be found as the original .orf had a new name.

Yes, this situation would require two separate commands. First, you have to process the JPEGs, then you could use your command to process the ORF and XMP files.

[Skids]

# Stargeek

Thanks for your very informative and helpful reply.  I will keep this message short as I will be working through your comments for some time to come.

By way of example I have been reading up on various perl commands to process text and after several hours managed to truncate a file name using

Code Select Expand

exiftool -p  '${filename;$_=substr($_,19);}' 'filename.jpg'While this works I'm still trying to understand how and what $ and $_ are.  What threw me with this command is the requirement to use the $_= perlhandler when compared with

Code Select Expand

exiftool -p  '${Make;tr/ /_/;s/__+/_/g;}' 'filename.jpg'Why is $_=tr/ /_/;s/__+/_/g; not required and what exactly is $_ ?  I'm guessing it is in effect a variable or pointer that gets set and passed through the perl function chain.

I would not have got this far without your reply so thanks again.

Simon

[StarGeek]

$_ is what Perl calls the "default variable". This is ChatGPT's explanation of the default variable, which seems pretty accurate to me, no hallucinations.

Any text between the {TAGNAME; ... } construct in exiftool is a bit of Perl code. In this case, the $_ will contain the value of TAGNAME. So ${Make;tr/ /_/;s/__+/_/g;} would be the equivalent of this in Perl

Code Select Expand

$_=$Make;
$_=~tr/ /_/;
$_=~s/__+/_/g;
Note that these use =~, not a simple =. =~ is Perl's Binding Operator. See here and then here.

tr/// and s/// operate on the default variable unless =~ is used to match against a different variable.