Search & replace in tags, bump time for multiple files sequentially

[StarGeek]

Didn't notice until I tried it, but that adds seconds instead of subtracting the total of <numberofpages>-<pagenumber>.

Ahh, I misunderstood you. I was still going off your earlier post which looked like you were just adding seconds (Page 1: 16:59:01 Page 2: 16:59:02). That is more complicated.

Just get things right, the sequence is this?
1. Grab the page number from the file name
2. Grab the total number of pages from XPComment
3. Subtract page number from total number of pages
4. Subtract that result from the time stamp.

Also, trying but failing to adequately reverse engineer your previously suggested code, modified by me as follows:
exiftool "-EXIF:XPComment<${EXIF:XPComment;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;s/\d+(\/\d+$)/$temp$1/}" .
...in order to make it match even when there is additional text after p #/#

Change
s/\d+(\/\d+$)/$temp$1/
into
s/\d+(\/\d+)/$temp$1/

The dollar sign anchored the expression to the end of the string.  Removing it allow the ##/## part to be any where.  But you then need to be careful that the ##/## pattern doesn't appear anywhere else in the string

I don't understand everything going on in your code, such as the \.. In the regex I know, that's an escaped period after the capture group, otherwise known as a literal period.

It's matching the dot in the filename.  Semicolons are how Perl indicates end of commands so take the whole section between the semicolons.
my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./
my $temp creates a temporary variable.  It will be set to the value of $1, the capture group from the following regex if there is a match.  $self->GetValue('FileName') gets the value of the FileName while doing processing inside the XPComment tag.  The regex then matches the last digits before the DotExtension of the filename.

[mazeckenrode]

I was still going off your earlier post which looked like you were just adding seconds (Page 1: 16:59:01 Page 2: 16:59:02).

Well, not really. The 'base' time, from which the others were to be calculated, was 17:00:00.

That is more complicated.

Figured that.  :-/  Though not impossible via ExifTool alone? I'm surprised, but hopeful, if that's the case.

Just get things right, the sequence is this?
1. Grab the page number from the file name
2. Grab the total number of pages from XPComment
3. Subtract page number from total number of pages
4. Subtract that result from the time stamp.

Yes, that's correct.

The dollar sign anchored the expression to the end of the string.

Doh! I should've been able to figure that one out myself. All these dollar signs keep throwing me off, since they don't all mean the same thing.

But you then need to be careful that the ##/## pattern doesn't appear anywhere else in the string

Surely there's a way to make it match the first one only? (Though I don't anticipate needing to have that pattern more than once in a filename.) Pretty sure I did that in Notepad++ regex at some point before. I don't think it's something I have in a script, though, which are all fairly recent.

It's matching the dot in the filename.

Double-doh! I REALLY should have seen that one! I'm really embarrassing myself here, aren't I?

[StarGeek]

Sorry for the delay in getting back to you, life gets in the way sometimes.

Try this, replacing TAG with the tag that has the page number data.  You can repeat the first part for each tag you need to edit
exiftool "-TAG<${TAG;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;s/\d+(\/\d+)/$temp$1/}" "-DateTimeOriginal-<0:0:${TAG;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;m/\d+\/(\d+)/;$_=$1-$temp}" /path/to/files/

Example output.  I used Caption-Abstract as a tag that had extra stuff at the end, including another #/# pattern.  Description has the original format.

Code Select Expand

C:\>exiftool -s -Caption-Abstract -Description -DateTimeOriginal Y:\!temp\bbbb\2020*
======== Y:/!temp/bbbb/2020-05-31 17;00;00 - Document - 01.jpg
Caption-Abstract                : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10 junk junk 5/50 junk
Description                     : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10
DateTimeOriginal                : 2020:05:31 17:00:00
======== Y:/!temp/bbbb/2020-05-31 17;00;00 - Document - 02.jpg
Caption-Abstract                : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10 junk junk 5/50 junk
Description                     : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10
DateTimeOriginal                : 2020:05:31 17:00:00
    2 image files read

C:\>exiftool -P -overwrite_original Y:\!temp\bbbb\2020* "-Description<${Description;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;s/\d+(\/\d+)/$temp$1/}" "-Caption-Abstract<${Caption-Abstract;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;s/\d+(\/\d+)/$temp$1/}" "-DateTimeOriginal-<0:0:${Description;my $temp=$1 if $self->GetValue('FileName')=~m/0*(\d+)\./;m/\d+\/(\d+)/;$_=$1-$temp}"
    2 image files updated

C:\>exiftool -s -Caption-Abstract -Description -DateTimeOriginal Y:\!temp\bbbb\2020*
======== Y:/!temp/bbbb/2020-05-31 17;00;00 - Document - 01.jpg
Caption-Abstract                : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10 junk junk 5/50 junk
Description                     : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 1/10
DateTimeOriginal                : 2020:05:31 16:59:51
======== Y:/!temp/bbbb/2020-05-31 17;00;00 - Document - 02.jpg
Caption-Abstract                : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 2/10 junk junk 5/50 junk
Description                     : Document XYZ, prepared by John Q Public, 31 May 2020, 17:00:00, p 2/10
DateTimeOriginal                : 2020:05:31 16:59:52
    2 image files read

[mazeckenrode]

Sorry for the delay in getting back to you, life gets in the way sometimes.

Hey, no apology necessary. I really appreciate your help.

Try this, replacing TAG with the tag that has the page number data.

That worked like a charm in my test. Thank you! With some time and work on my part, I hope to eventually understand what all the parts of it (the ones I don't already understand) do, so I can apply them on my own.

If I haven't already worn out my welcome, I'd like to bug you (or somebody more knowledgeable than me) for something else related to this thread. I previously learned here that Directory Opus was writing EXIF/XMP:DateTimeOriginal and IPTC:DateCreated in the same operation when applied from date/time values in the DOpus Set Metadata panel's Date taken field, but that IPTC:DateCreated is officially intended for date only, and IPTC:TimeCreated should be used for any time component.

In the interest of maintaining maximum consistency and compatibility between what I have DOpus and ExifTool doing to the same sets of files, and also recognizing that I will sometimes need to amend the date and time I assign to some files, I'd like to be able to parse and capture both components from my filenames (which I would amend first) and use them to update/write all four metadata fields. It seems to be easy enough to do for EXIF/XMP:DateTimeOriginal, but for IPTC:DateCreated and IPTC:TimeCreated, I'm going to have to separate the date and time. On top of that, I need to factor in the fact that I will sometimes be dealing with files (generally those provided by others) using variations of my chosen naming convention ("YYYY-MM-DD HH;MM;SS - <descriptive name> - <page>"). I hope to encompass date/time expressions utilizing any of [-_;., ] (and maybe some I haven't thought of yet), or none of them, in any combination as separators. In Notepad++, I would use this to match and capture:

(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})

If there's a smarter way to do this in ExifTool, I'm open, but what I'm mostly concerned with is how to apply the multiple capture groups from that. I tried searching the forums here for examples of multiple capture groups being used/applied, but have come up empty handed so far (I tried searching for $2, but that only returned a pageful of topic threads with the number 2 in their respective titles). I still don't have my head wrapped around the various uses of $, having now seen $1, $temp=$1, $self, $temp$1, $_, etc.

[StarGeek]

I previously learned here that Directory Opus was writing EXIF/XMP:DateTimeOriginal and IPTC:DateCreated in the same operation when applied from date/time values in the DOpus Set Metadata panel's Date taken field, but that IPTC:DateCreated is officially intended for date only, and IPTC:TimeCreated should be used for any time component.

Can you make an example file available?  I'd like to look it over.

In the interest of maintaining maximum consistency and compatibility between what I have DOpus and ExifTool doing to the same sets of files, and also recognizing that I will sometimes need to amend the date and time I assign to some files, I'd like to be able to parse and capture both components from my filenames (which I would amend first) and use them to update/write all four metadata fields. It seems to be easy enough to do for EXIF/XMP:DateTimeOriginal, but for IPTC:DateCreated and IPTC:TimeCreated, I'm going to have to separate the date and time.

Actually, you don't.  Exiftool will figure it out.  As long as you have the 14 digits in the correct order, that actual format doesn't matter because exiftool is very flexible when it comes to time stamps.  See the 3rd paragraph under FAQ #5
     Having said this, ExifTool is very flexible about the actual format of input date/time values when writing, and will attempt to reformat any values into the standard format unless the -n option is used. Any separators may be used (or in fact, none at all). The first 4 consecutive digits found in the value are interpreted as the year, then next 2 digits are the month, and so on.

So if either the filename or another tag such as DateTimeOriginal is already correct, you can just copy it straight across.  So you could just use
"-IPTC:DateCreated<DateTimeOriginal" "-IPTC:TimeCreated<DateTimeOriginal"

Or if the tags are being set from something else, for example a variable in a batch file
"-IPTC:DateCreated=2020:06:22 12:00:00" "-IPTC:TimeCreated=2020:06:22 12:00:00"

There is one caveat, though.  And that is IPTC:TimeCreated needs a time zone value.  If one is not provided, then exiftool will use the local time zone.

I still don't have my head wrapped around the various uses of $, having now seen $1, $temp=$1, $self, $temp$1, $_, etc.

For the regex capture groups, each capture group gets assigned to a $Number.  It's the same as in Notepad++, though that also allows the use of a \Number

The $temp was just a variable created in the command with the perl my function.  Basically using straight perl at that point.  $temp$1 is just concatenating the temp variable and the regex captured variable.

The $_ is the Perl default variable.  When used inside the braces of a tag, it represents the value of the tag.  If you assign a value to it, then the contents of the tag becomes that value.

Don't bother with the $self variable except for the examples shown.  That's used to directly access the Image::ExifTool Perl Library Module.  In most cases you don't need to bother with it.

[mazeckenrode]

I previously learned here that Directory Opus was writing EXIF/XMP:DateTimeOriginal and IPTC:DateCreated in the same operation when applied from date/time values in the DOpus Set Metadata panel's Date taken field, but that IPTC:DateCreated is officially intended for date only, and IPTC:TimeCreated should be used for any time component.

Can you make an example file available? I'd like to look it over.

Certainly, see attached.

FYI, When I brought up about DOpus writing EXIF, XMP, and IPTC tags under certain circumstances, one of their developers replied: "We don't add IPTC fields unless some are already there." (DOpus forum thread here.) But ExifTool said no metadata prior to my operation, and all three groups after.

I'd like to be able to parse and capture both components from my filenames (which I would amend first) and use them to update/write all four metadata fields. It seems to be easy enough to do for EXIF/XMP:DateTimeOriginal, but for IPTC:DateCreated and IPTC:TimeCreated, I'm going to have to separate the date and time.

Actually, you don't. Exiftool will figure it out.

My experience here is that I'm not getting reliable results. Used this command line with a variety of possible filenames (most of which I wouldn't ordinarily use, but may encounter from other sources):

exiftool "-iptc:datecreated<filename" "-iptc:timecreated<filename" "-exif:datetimeoriginal" "-xmp:datetimeoriginal" .

Filename: "2020-06-22 16;36;58.png"

Command line output:

Code Select Expand

Warning: Invalid time format (use HH:MM:SS[+/-HH:MM]) in IPTC:TimeCreated (ValueConvInv)
Warning: [minor] Creating non-standard IPTC in PNG

From exported JSON:

Code Select Expand

"IPTC:DateCreated": "2020:06:22" (this was the only tag written for this file)

Filename: "2020-06-22 16;36;58Publication-Page19000101.png"

Command line output:

Code Select Expand

Warning: Invalid time format (use HH:MM:SS[+/-HH:MM]) in IPTC:TimeCreated (ValueConvInv)
Warning: [minor] Creating non-standard IPTC in PNG

JSON:

Code Select Expand

"IPTC:DateCreated": "1900:01:01",
"IPTC:ApplicationRecordVersion": 4

Filename: "20200622-163658.png"

Command line output:

Code Select Expand

Warning: [minor] Creating non-standard IPTC in PNG

JSON:

Code Select Expand

"IPTC:DateCreated": "0622:16:36",
"IPTC:TimeCreated": "20:20:06-16:36",
"IPTC:ApplicationRecordVersion": 4

Filenames:
"20200622_163658.png"
"20200622163658.png"

Command line output:

Code Select Expand

Warning: [minor] Creating non-standard IPTC in PNG

JSON:

Code Select Expand

"IPTC:DateCreated": "2020:06:22",
"IPTC:TimeCreated": "20:20:06-04:00",
"IPTC:ApplicationRecordVersion": 4

Filename: "2020062216365812340506123456.png"

Command line output:

Code Select Expand

Warning: [minor] Creating non-standard IPTC in PNG

JSON:

Code Select Expand

"IPTC:DateCreated": "0612:34:56",
"IPTC:TimeCreated": "20:20:06-04:00",
"IPTC:ApplicationRecordVersion": 4

Thanks, too, for the explanation of various $variables. I may have another question or two on that subject at some point, but will have to do some digging for other threads I'd looked at first.

Attached: "2020-10-11 12;13;14 - Document - 01.7z" (1,529)

Contents: "2020-10-11 12;13;14 - Document - 01.png" (4,186) [1 x 1 x 1]

[mazeckenrode]

Used this command line with a variety of possible filenames:

exiftool "-iptc:datecreated<filename" "-iptc:timecreated<filename" "-exif:datetimeoriginal" "-xmp:datetimeoriginal" .

Just noticed that I'd failed to properly complete the EXIF and XMP instructions in the command line above. My bad. Should have been:

exiftool "-iptc:datecreated<filename" "-iptc:timecreated<filename" "-exif:datetimeoriginal<filename" "-xmp:datetimeoriginal<filename" .

Anyway, looks to me like the EXIF and XMP tags are getting the correct date and time, but the IPTC tags are not in any of my test cases, and are not even all consistent in which numbers they end up inheriting from the respective filenames.

[StarGeek]

Can you make an example file available? I'd like to look it over.

Certainly, see attached.

Thanks.  I'll have to look it over in more detail when I get a change but it is being written incorrectly.  If I recall correctly, DO uses exiv2 to write image metadata (yep, it's in the Acknowledgements) so I think I'll see if exiv2 allows it to be written like that.

[StarGeek]

My experience here is that I'm not getting reliable results. Used this command line with a variety of possible filenames (most of which I wouldn't ordinarily use, but may encounter from other sources):

Yeah, you're right, the IPTC:TimeCreated doesn't seem to work well when copying from the filename.  But it does need the time zone.  I've noticed another weirdness regarding it that I'm going to bring up in another thread. edit: which I now can't reproduce, ARRRGG

My suggestion, if you really want to use these IPTC tags, would be to copy the data from one of the EXIF tags like DateTimeOriginal or if you have the EXIF:OffsetTimeOriginal filled out (time zone in the EXIF group), copy from SubSecDateTimeOriginal.  I should also point out that IPTC data in a PNG file is non-standard and about the only things that will read it are exiftool and exiv2 (which, as I mentioned, is what DO uses).

Myself, I tend not to use these tags.  The EXIF and XMP tags are much better tags, though the IPTC ones do get filled out automatically in my workflow if I have the time zone.

[mazeckenrode]

Yeah, you're right, the IPTC:TimeCreated doesn't seem to work well when copying from the filename.

Trying to use these, based on other examples here, to take care of the IPTC tags, but still getting inconsistent and incorrect results:

exiftool "-iptc:datecreated<${filename;m/^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}" .

"-iptc:timecreated<${filename;m/^\d{4}[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}"

In particular, some of my files with more than 14 digits in the filenames are getting date and time values from digits that aren't among the first 14. Is my usage of ^ to anchor at the start of the filename not correct?

I tend not to use these tags. The EXIF and XMP tags are much better tags

I wish to maintain their presence and accuracy primarily because DOpus puts them there. I will be using DOpus, at least on occasion, to modify some tags, and don't want to have to worry about whether there are IPTC tags whose values contradict the corresponding EXIF and XMP tags.

[mazeckenrode]

Whether I use:

exiftool "-iptc:datecreated<${filename;m/^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}" "-iptc:timecreated<${filename;m/^\d{4}[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}" .

...or...

exiftool "-iptc:datecreated<${filename;m/\A(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}" "-iptc:timecreated<${filename;m/\A\d{4}[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1$2$3}" .

...(using ^ as start-of-string anchor in first command line, \A in second) I get the following errors:

Code Select Expand


Warning: syntax error for 'filename' - ./2020-06-22 16;36;58.png
Warning: [minor] Creating non-standard IPTC in PNG - ./2020-06-22 16;36;58.png
Warning: syntax error for 'filename' - ./2020-06-22 16;36;58Publication-Page19000101.png
Warning: [minor] Creating non-standard IPTC in PNG - ./2020-06-22 16;36;58Publication-Page19000101.png
Warning: syntax error for 'filename' - ./20200622-163658.png
Warning: [minor] Creating non-standard IPTC in PNG - ./20200622-163658.png
Warning: syntax error for 'filename' - ./20200622163658.png
Warning: [minor] Creating non-standard IPTC in PNG - ./20200622163658.png
Warning: syntax error for 'filename' - ./2020062216365812340506123456.png
Warning: [minor] Creating non-standard IPTC in PNG - ./2020062216365812340506123456.png
Warning: syntax error for 'filename' - ./20200622_163658.png
Warning: [minor] Creating non-standard IPTC in PNG - ./20200622_163658.png


Actual tagging results for the six files:

For IPTC:DateCreated, (2) correct, (4) incorrect

For IPTC:TimeCreated, (0) correct, (4) incorrect, (2) failed to set

If I use what I'd consider a basically equivalent regex operation in Notepad++:

Find: ^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*\d{2}.*

Replace with: Date: \1:\2:\3

Find: ^\d{4}[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2}).*

Replace with: Time: \1:\2:\3

...they capture and replace exactly what I intended. What's wrong with my ExifTool code?

[StarGeek]

Took me too long to figure it out

You need the concatenation operator to combine the strings.  Or they need to be in double quotes, to concat, though that doesn't work so well on Windows
$_=$1.$2.$3

Code Select Expand

C:\>exiftool -P -overwrite_original  "-iptc:datecreated<${filename;m/^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1.$2.$3}" "-iptc:timecreated<${filename;m/^\d{4}[-_;\., ]*\d{2}[-_;\., ]*\d{2}[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/;$_=$1.$2.$3}"  Y:\!temp\bbbb
    1 directories scanned
    6 image files updated

C:\>exiftool -time:all --system:all -g1 -a -s Y:\!temp\bbbb
======== Y:/!temp/bbbb/2020-06-22 16;36;58.jpg
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
======== Y:/!temp/bbbb/2020-06-22 16;36;58Publication-Page19000101.jpg
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
======== Y:/!temp/bbbb/20200622-163658.png
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
======== Y:/!temp/bbbb/20200622163658.jpg
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
======== Y:/!temp/bbbb/2020062216365812340506123456.jpg
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
======== Y:/!temp/bbbb/20200622_163658.jpg
---- IPTC ----
DateCreated                     : 2020:06:22
TimeCreated                     : 16:36:58-07:00
---- Composite ----
DateTimeCreated                 : 2020:06:22 16:36:58-07:00
DateTimeOriginal                : 2020:06:22 16:36:58-07:00
    1 directories scanned
    6 image files read

In Perl code $_="$1$2$3" would also work, but on the command line it's better to use the above, imo.

[mazeckenrode]

You need the concatenation operator to combine the strings. Or they need to be in double quotes, to concat, though that doesn't work so well on Windows
$_=$1.$2.$3
$_="$1$2$3" would also work, but on the command line it's better to use the above, imo.

Bingo, that works! Thanks for all the help, StarGeek (and Phil).

Pretty sure I saw some command line examples that didn't use any of the above solutions for concatenation in the forum here, which is what I was trying to base my code on, but perhaps those were intended for non-Windows systems.

[StarGeek]

They were probabaly in a regex substitution like
s/^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/$1$2$3/

[mazeckenrode]

They were probabaly in a regex substitution like
s/^(\d{4})[-_;\., ]*(\d{2})[-_;\., ]*(\d{2})/$1$2$3/

Ah, yes, didn't think of distinguishing those two types of regex usage. So I'm guessing the concatenation-operator-or-quotes rule only applies if it follows the regex matching code and /;?