Miscellaneous ExifTool related questions (and information)

[mpegleg]

EDIT: This post content was moved to a new topic in Bug Reports [here], as suggested.

Phil, if it's useful to you, let me know, and I'll pass on any other minor typos that I may find, but overall, your documentation is already extremely well proofread... (So much so, that I think I deserve a prize for spotting that one).

Regards,
-Paul

[Phil Harvey]

Hi Paul,

Great!  I love it when people read the documentation!

Yes, please let me know of any other typos that you find.  Maybe start a new thread in the bug report section for these.

And you get a gold star for every typo you find.  One star for you so far...

- Phil

[mpegleg]

EDIT: Ok, I'm going to scrap this previously TLDR question, because I now understand what Hayo is getting at in reply#8

[Hayo Baan]

Have you looked at the -p and -testname options? These allow you to show results without altering the file.
Or, isn't your problem completely solved with -if? Looks like you simply want a conditional execution, or do I misunderstand your question?

[mpegleg]

Hi Hayo. Yes I looked at the -testname option. It's getting close to what I want, but I think it's possibly a bit limited. I can't see how I can use it to pipe the Regex expression into it and temporarily store it. I think all it does, is display output??

As explained in my last edited paragraph above, this may well be the best option for me, as the SSD will make any temp "throwaway" write operations extremely fast. I'm working on it now. So far so good.

[Hayo Baan]

Re-reading the question, I'm puzzled as to why -if doesn't work; it can have a complete Perl expression as condition, so this should allow you to fully do what you want.
Can you elaborate on the condition you want to express?

[mpegleg]

Or, isn't your problem completely solved with -if? Looks like you simply want a conditional execution, or do I misunderstand your question?

No, I'm already using many combined -if operations with -execute ops.

I think the solution may well be in the temp SSD file using the -srcfile option.

Hayo. It's tricky to explain, without seeing the full code, which I will show you soon if this fails to work, otherwise, everything's ok. The problem is that the -if conditions are all requiring a tag write operation, which I'm trying to avoid, but I might be missing something here, in my not understanding Perl very well.

Stay tuned. I'll test what I'm working on now, and get back to you, and perhaps then, you will be able to give me a better option. Thanks

[mpegleg]

Hayo. The -if operations all work, and work very well, however I can't see a way to make them work without using the "<" operator which will always write to the file.

The majority of my images don't need any tags written to, (and I want them to remain untouched if possible) but in order to manipulate and check the filename using an -if op, I seem to have to write the data to a tag first. Is that not always the case?

eg. -comment < ${filename; $_ =~......


Anyway, I'll keep persevering for the time being and show you the reason soon.

[Hayo Baan]

I don't understand; the if (when the condition fails) will prevent the command from being executed, so there won't be a write (or whatever you do). It furthermore doesn't need to be a write command, anything will work. -filename -if xxx will only display the filename if condition xxx has been met.

[mpegleg]

Ah. I see what you're saying. Let me ponder this some more.

With some more movement of the ol' grey matter... I think that two subsequent -if option statements should do what I want. I'll look into that later. Right now I'm very intrigued by some weird undocumented behaviour of the -srcfile option. Interesting... but also very bewildering. I'll explain later, after much experimentation.

[mpegleg]

Ok. Let's scrap all that previous stuff. lol

Here's the problem code:

Code Select Expand

"-XPComment<${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +/ /g;s/ +$//}. ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g}"

For these following examples, assume the filename has a DateTimeOriginal Year=2018.

These three examples work correctly as expected:

image.jpg  ---> image. 2018
2019-04-22 15-33-14 test1 [PIX123456] abc modx edit1.jpg   ---> abc. 2018
test 123 modx.jpg ---> test 123. 2018


However, if the filename is:

image [PIX123456].jpg ---> .2018   <--- Don't want this file updated. The tag should be left as it is.
2019-04-22 15-33-14 test1 [PIX123456] modx edit1.jpg ---> .2018   <--- and this too.

These should actually be empty because there is no data after [PIX123456], (or before modx) so there is no need to update the tag, and the file should not be touched.


So, I think I will also need an -if statement so that the file won't be updated if the result will be blank (or should be blank) when the filename has nothing after the PIX code. So the complication arises in that normally the tag will be updated, unless like in this case, the filename has nothing after the PIX code, in which case do nothing for this particular file.

ie. If the filename does not end with [PIX......] (including the modx exceptions  ) then do the tag operation.

I'm trying to avoid unnecessary file write operations if possible, as there are thousands of images on my NAS, and the majority won't require tag updating, as they won't be captioned yet. Thus they will still be be of the "image [PIX123456].jpg" or "2019-05-01 12-05-32 [PIX123456].jpg" filename variety.


Note: It doesn't matter how long or complicated the syntax is, because I'm using various arg files to hide store all this code.

[Hayo Baan]

That shouldn't be to hard: I think this -if will do what you need:

Code Select Expand

-if "$filename =~ /(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/ && $1 !~ /^\s*$/"

[Hayo Baan]

Oh, I see, if it didn't contain PIX then you also want it to succeed:

Code Select Expand

-if "$filename !~ /\[PIX\d+\]/ || ($filename =~ /(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/ && $1 !~ /^\s*$/)"

[mpegleg]

Oh, I see, if it didn't contain PIX then you also want it to succeed:

Code Select Expand

-if "$filename !~ /\[PIX\d+\]/ || ($filename =~ /(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/ && $1 !~ /^\s*$/)"

Yes, sorry! I have lots of different scenarios now, with many options depending on various flags that I've set up in another tag.

This will be the major most common one though, as there are no directories required in the output data.

Although it's rare, I still have a few filenames without the PIX code. Mainly very old historical scanned photos, with dates only approximated.



Unfortunately with your code from Reply#12, all filenames seem to be failing the condition??

Try it first with "image.jpg" because that should pass the condition.

I really appreciate your help with this Hayo. It's had me struggling for days.

The old code I was using worked whereby it would come back and clear the tag if it finally resembled ". 2018", but I wasn't happy with this because it meant two updates, and I really want to avoid any writes or updates of the file if the tag is going to end up (or "should" end up) empty.

Also, any updated files always lose their hidden status. *sob*

[Hayo Baan]

Don't know what is going wrong (I guess its the $1 that is not working inside the exiftool -if as in normal Perl it worked), but with a little bit of tinkering I've been able to come up with a slightly simpler one that should work:

Code Select Expand

-if "($filename !~ /\[PIX\d+\]/ || $filename =~ /(?:.*\[PIX\d+\]\s*)?\s*(?:(?!modx)[\S*]+)/)"

[mpegleg]

Seems to be the exact opposite now.

ie. All filenames are passing the condition.

These should fail, but they are currently passing:

2018-08-11 11-43-21 [PIX761807].jpg

2019-04-22 15-33-14 test1 [PIX123456] modx edit1.jpg

image [PIX123456].jpg

[Hayo Baan]

Oops, change [\S*]+ into \S+.

[mpegleg]

Unfortunately that didn't work Hayo... but I'm super excited because I worked it out. This works!

Code Select Expand

-if ${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +/ /g;s/ +$//}. ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g} =~ /^\./

-XPComment<${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +/ /g;s/ +$//}. ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g}



Thanks for your help Hayo. Sorry, it was a hard one to fully explain what I was after,... but all's well that ends well. 

It's using most of your previous code anyway!

My main initial problem was that I couldn't get any Regex code to work with the -if option, so I assumed (wrongly) that it "had" to be done via the "<" operator with tags. Now that it finally works, I feel I've jumped a major hurdle in my understanding. I'm tired, but also very happy. It's true... persistence pays off.

[mpegleg]

Now, to finally finish this all off, I just need a couple of "hopefully" easy Regex fixes please.

They all have some leading spaces that I would rather not have.

They are variants of the other cases, and Windows doesn't show them in the Explorer columns which is why I never even noticed that they were there, until I checked via ExifTool.

eg).

#1:

2009-02-22 13-38-06 Test 11 [PIX749502].jpg

Code Select Expand

${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +//g;s/ +$//} [${directory; use Cwd 'abs_path';$_ = abs_path($_); $_ =~ s/.*\///; $_ =~ s/^[0-9- ]*//;}, ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g}]

======== D:/Photos/2009/2009-02-22 Europe River Cruise/2009-02-22 Amsterdam, Netherlands/2009-02-22 13-38-06 Test 11 [PIX749502].jpg
XP Comment                      :  [Amsterdam, Netherlands, 2009]
Comment                         :  [Amsterdam, Netherlands, 2009]

This results in a single unwanted blank space at the start of the output result "if" the filename has nothing after the [PIX......]



#2:

2009-02-22 13-38-06 Test 12 [PIX749502].jpg

Code Select Expand

${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +/ /g;s/ +$//}    (${DateTimeOriginal;DateFmt('%a. %b %d, %Y');s/\b0+\B//g})    [${directory; use Cwd 'abs_path'; $_ = abs_path($_); $_ =~ s#.*/[0-9-]+\s*([^/]+)/[^/]+$#$1#;}]


======== D:/Photos/2009/2009-02-22 Europe River Cruise/2009-02-22 Amsterdam, Netherlands/2009-02-22 13-38-06 Test 12 [PIX749502].jpg
XP Comment                      :     (Sun. Feb 22, 2009)    [Europe River Cruise]
Comment                         :     (Sun. Feb 22, 2009)    [Europe River Cruise]

...and this one has 4x leading blank spaces that I don't want.

Obviously I am inserting these into the strings because normally if the filename had a text caption string, the padding "would" be needed, but in the cases like these where they aren't needed, I need to chop them off again after they've been processed. Once again, I do want to avoid any unnecessary writes to the tags unless required.

So I'm not sure if that can be easily done with Regex, or whether I'd need to use another -if option command?

Perhaps I need a separate -if condition first, to work out whether to add the spaces or not? Not sure how best to do this.


Regards,
-Paul

ps. I worked out how to search for the leading spaces with say: /\s+/ but I'm not too sure how to then "remove them", nor how to incorporate that code into the whole string.

This is how it all looks with test files after automatic processing by ExifTool. All controlled by the Copyright Flags. Note how Windows Explorer is too smart for it's own good, and doesn't show the leading spaces in the Comments. It obviously doesn't matter here, but on a vertical captioned photo where space is at a premium, it sure does.

[StarGeek]

Trim leading spaces
s/^\s+//

Trim trailing spaces
s(\s+$)()

Trim leading and trailing spaces
s(^\s+|\s+$)()g
Note the g (global) is necessary because otherwise it would only do one or the other.  I used parens instead of slashes in the second two expressions because $/ is interpreted as a new line in a tag redirection (see FAQ #21d).

Just make that your last regex before the closing brace.

[mpegleg]

Thanks StarGeek,

but it didn't seem to work for me, probably because I placed it in the wrong location (more than likely!)

Would you mind modifying one of my codes with it.

Cheers
-Paul

[StarGeek]

Ah, I see the problem.  You have a leading space before the Directory part.  If Filename is empty, that's when what would normally be a space between the two parts becomes a leading space.  So my regex doesn't apply to this.

Try changing the first part to
${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((??!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +//g;s/ +$//;$_.=' '}
And removing the space just before [${directory;

No, that won't work. 
Hmmm...

[mpegleg]

Thanks StarGeek, but would you mind just popping that into a code box, just so I can make sure I get it all correct.

Thanks! 

I can't wait to get this going. I think it's the last bit of code I need.

ps. Awww. Your code looked a little bit sad  ??   

[mpegleg]

No, that won't work. 
Hmmm...

ok. No worries mate. I'll leave it with you to ponder.

Cheers,
Paul 

[StarGeek]

I think it might just be easier to run a trim spaces after the first run

exiftool -api "Filter=s(^\s+|\s+$)()g" -if "$Comment ne $Comment#" -TagsFromFile @ -Comment FileOrDir

[mpegleg]

Hmmm. Thanks again StarGeek. It's interesting to learn of that way of after-processing the result. I might have to delve deeper into these -api options later. They look interesting.

But, and it's a big but... running another exiftool command won't really be practical for me, in the way I've got things set up now. I've also got well over 43000 images on my rather slow NAS, that already takes f-o-r-e-v-e-r to process in just one run, let alone two. (Although it will be considerably faster now that I've cut down the amount of unnecessary tag writing).

So, any chance that there might be a possibility of successfully modding this initial code to strip the spaces?

Code Select Expand

${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +//g;s/ +$//} [${directory; use Cwd 'abs_path';$_ = abs_path($_); $_ =~ s/.*\///; $_ =~ s/^[0-9- ]*//;}, ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g}]

Code Select Expand

${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((?:(?!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +/ /g;s/ +$//}    (${DateTimeOriginal;DateFmt('%a. %b %d, %Y');s/\b0+\B//g})    [${directory; use Cwd 'abs_path'; $_ = abs_path($_); $_ =~ s#.*/[0-9-]+\s*([^/]+)/[^/]+$#$1#;}]

Perhaps Hayo/Phil might also be able to offer some suggestions as well. Here's hoping we can crack this. Thanks.

[StarGeek]

Sorry I couldn't be more helpful, it's just your command line has gotten to complex and too many edge cases for me to pick it apart and figure out what it's doing.

It's interesting to learn of that way of after-processing the result. I might have to delve deeper into these -api options later. They look interesting.

One thing that this command will do is only process files in which there was a change.  The -api filter affects every tag, so it may not be the best use if you are dealing with more than one tag.  But I figured out that doing this puts the regex (and other stuff) in one spot and helps prevent typos when using an IF option and modifying a tag.  Too many times I would end up with one thing in the IF and another thing in the tag.  The "$TAG ne $TAG#" compares the TAG modified by the filter ($TAG) to the original value ($TAG#, see the -n (printConv) option).

[mpegleg]

No need to apologize, as you've been tremendously helpful to me StarGeek, through my whole evolution of this solution to my photo captioning project. Together with both Phil & Hayo, you've all been tremendously accommodating and patient.

I'm so happy I've got this far, yet it does feel like I'm kind of stuck on the cusp of looking for that elusive "final" piece to the 10000 piece jigsaw puzzle (which is usually hiding under the couch, or, the family pooch has eaten it!) .

Yes, the commands certainly are getting complex now, and it does make it very tricky. It may not matter too much with those few extra spaces anyway, and I may just cut them down a few, so they won't be as noticeable. In those few cases where the captions are just a tad too long to fit onto a single line on a vertical photo, I can just manually edit anyway.

I certainly didn't want to underrate your help in any shape or form, yet at the same time I also didn't want Hayo or Phil to not offer their input, by feeling that we've totally nailed the best solution.


Cheers,
-Paul.

[mpegleg]

Wow! The solution turned out to be incredibly simple.

After much testing and brainstorming, I realised that in this case where it will be using the directory names to create the captions, it will ALWAYS require a tag write, so that makes it so much simpler...

It works for both cases of naming using the Parent and/or Current Directory names. It also only updates tags when necessary.

Here's the one for using the Current Dir only (from an arg file)...

-if
substr($Copyright,1,2) eq '0C'  <-- My test in this particular arg file, to see if I want to include Current Directory name.

-xpcomment<${filename; $_ =~ s/^(?:.*\[PIX\d+\]\s*)?((??!modx).)*)(?:\s*modx.*|\....$)/$1/;s/  +//g;s/ +$//} [${directory; use Cwd 'abs_path';$_ = abs_path($_); $_ =~ s/.*\///; $_ =~ s/^[0-9- ]*//;}, ${DateTimeOriginal;DateFmt('%Y');s/\b0+\B//g}]

-execute

-if
substr($Copyright,1,2) eq '0C'

-xpcomment<${xpcomment;s(^\s+|\s+$)()g}  <--- Yes, I did make use of the amazing magical StarGeek Filter code Thank you!


So, it looks like I've pretty much got this all wrapped up now.

Thanks again to everyone for your help,
-Paul

************* Party time *************

[mpegleg]

Hello.

Could somebody please help me with a Regex expression required to achieve the following:

Filename:

2018-11-07 1428 This is a test.jpg


I would like the first 15 characters rearranged as a string, and then entered into the comment tag exactly as follows:

2018:11:07 14:28:00+08:00


Note: I know it might be unusual, but right now while I'm testing, I'd much prefer not to do any "fancy" date manipulation just yet.

For now, I'd just like a straight literal text manipulation from the filename into the tag.


Many Thanks,
-Paul

[Hayo Baan]

Try this regexp: s/^([0-9]{4})-([0-9]{2})-([0-9]{2}) +([0-9]{4}).*/$1:$2:$3 $4:$5:00+08:00/

[mpegleg]

Thanks Hayo. That's close...

Result: 2018:11:07 1428::00+08:00

Just one of those ":" is out of place.

[mpegleg]

...but I "think" I fixed it...

s/^([0-9]{4})-([0-9]{2})-([0-9]{2}) +([0-9]{2})+([0-9]{2}).*/$1:$2:$3 $4:$5$6:00+08:00

Is that right?

Now seems to give the correct result of: 2018:11:07 14:28:00+08:00

Thank you

..and I learnt a lot from studying your answer. Knowing about concatenating those multiple variables like that will be very useful to me.

[Hayo Baan]

Sorry made a smal mistake, use s/^([0-9]{4})-([0-9]{2})-([0-9]{2}) +([0-9]{2})([0-9]{2}).*/$1:$2:$3 $4:$5:00+08:00/

Can you spot the difference

[mpegleg]

Yes. I see what you did. You took away the extra "+" that I added, because then, there is no need for the extra variable $6 that I added.

Thank you

[mpegleg]

OK. The following code now achieves what I want.   ie. It recovers and recreates the UTC CreateDates from the video filenames, when the filename has the format of "YYYY-MM-DD HHMM.mp4".

exiftool "-CreateDate<${filename; s/^([0-9]{4})-([0-9]{2})-([0-9]{2}) +([0-9]{2})([0-9]{2}).*/$1:$2:$3 $4:$5:00/}" -execute -CreateDate-=8 -common_args -r -api largefilesupport=1 -overwrite_original FILE/DIR


However this requires 2x runs, (to subtract 8 hours from my local time for UTC time) which is a bit of a drag, when there are thousands of video files.

Do you think there might be a better, more efficient way, to achieve this, which would only require one loop? ie. Get rid of the -execute. If it's not possible, then that's no major problem... I was just wondering if it may be possible?

[mpegleg]

Aha. I found the solution:

exiftool "-CreateDate<${filename; s/^([0-9]{4})-([0-9]{2})-([0-9]{2}) +([0-9]{2})([0-9]{2}).*/$1:$2:$3 $4:$5:00/}" -api quicktimeutc -r -api largefilesupport=1 -overwrite_original FILE/DIR


the -api quicktimeutc solved the issue

[Hayo Baan]

Excellent! If you're in a different time zone from where the video was taken, you can set the TZ environment variable to the correct value. exiftool will then convert using this time zone. (note: this works on Mac/Linux, not sure about Windows)

[mpegleg]

Hi. My regex ability is still very poor, esp. when converting into ExifTool Perl-like syntax. I've been trying for a little while to do the following...

Hopefully somebody might be able to provide some different examples, with full sample code to achieve the following simple requirement:

I would like to strip the very first character from the filename, and then rename itself to that new name, in the same directory.

In practice, I will be needing to strip a "." character from the beginning of the filename.

ie. abc.jpg becomes bc.jpg   -or-    .abc.jpg -> abc.jpg


Perhaps you could show me some examples using -filename, and/or also using substr, so that I can then choose what I consider the best method for my particular scenario, plus most importantly, learn from your examples.

Many thanks,
-Paul

[mpegleg]

ok. So I worked out this,...

-filename<"${filename;~ s/^.//s}"

...but I couldn't get an equivalent "subst" function working. ??

[Phil Harvey]

You should remove the "~".

Or with substr:

-filename<"${filename;$_ = substr($_, 1)}"

- Phil

[mpegleg]

You should remove the "~".  With substr:

- Phil

ah. ok. Done.

Thanks Phil.

Both are Perfect!