News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

ExifTool for Dummies

Started by Phil Harvey, January 12, 2011, 02:25:05 PM

Previous topic - Next topic

Phil Harvey

I have just started writing an "ExifTool for Dummies" page, and I am looking for some input before I get too far.  I have trouble generating documentation on this subject because often things that are obvious to me can cause problems for some people.

The page is far from complete, and I would be happy to hear general suggestions at this stage about the direction I should be taking here, and/or specific details about what should be included.

TIA

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

BogdanH

Hi Phil,

I think this is a good idea!
Quote..often things that are obvious to me can cause problems for some people.
Just keep in mind, that you're (I assume) writting for those who never seen/used console window before. Thus, when explaining "first steps", even there's only (command/result) text inside console window, having small screenshot of that window explains more than hundred words. As you progress deeper (after first few examples, perhaps), there will be less and less need for screenshots.

One thing came to mind: perhaps one page where it's explained (for dummies) how metadata is organized (block diagram?)... how Exif, Iptc, Xmp, etc. are related... -so user gets a feeling what's that all about. Just an idea..

Wish you patience  :)

Bogdan

Phil Harvey

Hi Bogdan,

Thanks for the suggestions!

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

simonmcnair

Phil,
Phil,
I think the best way to do this would be that whenever you come across an innovative and useful command line, post it with a brief explaination of what it does.  This allows people to look at the command line and understand how it was constructed.  Maybe have an easy, medium and advanced section ? (I love the man pages which have 30 examples or so which cover most of the regular uses of the application).

This would be especially useful when explaining 'escape characters', nesting command lines, etc.

I think you'll find most of the people that need help are, unfortunately, windows ppl like myself who learned DOS batch files only to find out that it's a poor mans substitute for bash scripting or perl.

Keep up the great work.  It's nice to see a developer who cares about documenting their product, who keeps involved with their community and shows an active interest in developing an app which already does practically everything under the sun :-).

cheers
Simon


Phil Harvey

Hi Simon,

Good point.  I definitely need to give a brief summary of shell quoting rules and how to escape characters.

I will definitely give examples, but too many examples may defeat the purpose.  The application documentation already contains lots of examples but I don't think that anyone ever reads them. :(

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

BogdanH

Hi Phil,

Quote from: Phil Harvey on January 13, 2011, 07:20:00 AM
... The application documentation already contains lots of examples but I don't think that anyone ever reads them. :(

Probably true in most cases. The reason for this might be, that examples in documentation are based on "what command/switch does" -thus user must "recognize" particular example is "similar" to what he wish to accomplish.

I believe, what beginner wish to (easy) find is, "how do I make this (fill in) happen". What I mean is, there should be (for example):

Chapter: "Managing keywords"
-saving my first keyword and adding further keywords...
-deleting all or particular keyword..
-how do I show (rename, etc.) all files containing particular (two) keyword(s)...
-I wish to modify some tag, but only if file (doesn't) contain particular keyword(s)...
-etc....
Chapter: "Managing locations"
- ...
Chapter: "Managing Dates and Times"
- ...
That is, user knows what he wish, but doesn't know where to start. I believe, "per chapter" manual makes it easier to find needed example. Of course, there's no need trying to cover all possible combinations -user will get the idea "how stuff works".

Well, that's how I would probably do...

Bogdan

Phil Harvey

Hi Bogdan,

I was hoping to avoid writing a book with various chapters because I don't have the time.  What I want is just a single page of information (or, probably one page per platform), just to help people get started.

I'm not saying that a book wouldn't be useful, just that I don't have the time to write it.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

BogdanH

Time -how right you are! And after reading my post again... I really must reduce coffee :)

Bogdan

Phil Harvey

Quote from: BogdanH on January 13, 2011, 04:14:23 PM
And after reading my post again... I really must reduce coffee :)

...or maybe I should start drinking it.  ;)

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

simonmcnair

Phil,
I don't know if, in my opinion, you should be doing the 'dummies guide' at all.  If people can't get by on the substantial amount of documentation and the faq then there is something wrong.  My case spectacularly describes this point in particular.  I came on to the forums and asked two questions which were already covered by your documentation:

1.) how the date renaming works with escape characters (still can't understand why -d %Y%m%d_%H%M%S%%-c.%%e has the -c escaped; because it applies to the filename not the date formatting ?  sorry to go off topic)
2.) what the tag name is for field x

This means one of two things:
1.) your program has a steep learning curve
2.) your program is too complex (hey, can't complain about added functionality ;-) )
3.) The documentation is too in depth, too quickly

When I first saw the app, I thought 'okay, drop to console and do /?' .  I did this and got about 15 pages of info, the second thing I did was think 'oh shoot this is complex'.  Maybe the app should have, when executed with /? just the examples and then give advanced help on /help or something.  That would stop the newbies getting scared and asking for help straight away.

I'm sure, in your mind you have the top ten queries asked of exiftool, just put them down on the the FAQ at the start titled 'common examples'.

You'll have different kinds of people visiting the site like:
Developers (you shouldn't need to explain much to these ppl)
regular users (They'll ask the questions in the forums and I would expect these to be intelligent questions)
newbies (Like me, who looked at the program and though, oh shoot this is complex, I only want to do x and don't understand the development/unix terminology)

The newbies are just looking for the most common uses of the app, they have 2,00-10,000 digital photos and they want them renamed according to x, or they want the date adjusted in the file as they forgot to set the date on their camera.  All easy and common questions I suspect.  But, as they get used to it they either have one command that they run a few times a year or on each import or their needs get greater and they become an intelligent forum user.

I think worked examples and an explaination of escape characters would do fine.  If you want to provide more than that, try getting some stats out of the forum engine to tell you what the most regularly asked questions are and try to answer them.

:-)

Hope this makes some/any sense.

Simon


simonmcnair

oh it might be worth explaining double quotes,less than, braces and the $ (Sigil, literal string?) too.  Traditionally, I thought quotes were used whenever a filename and/or directory had a space in it, but I can't understand why the quotes are in the example below.


exiftool -d %Y-%m-%d_%H-%M-%S%%-c.%%e "-filename<${datetimeoriginal}" *.jpg

Phil Harvey

Hi Simon,

Thanks for your comments.  It is good to hear from the perspective of a new user.

There are a number of characters that are special in various shells.  I know of 7 different shells  (the Windows cmd shell being only one), and in most of these shells the | > and < characters are used for redirection and piping, so they must be quoted.  Also, in the Unix shells the dollar sign ($) is used for environment variables, so it must be quoted too (with single quotes), but I don't think this is a problem in Windows.

I would have to do some research myself on the various shells to figure out all the quoting rules.  This is one reason I haven't done this yet.

Plus, there are additional rules for Windows batch files.  For some reason the Windows batch command syntax is different from the command line.  But luckily the syntax of the scripts is the same as the command line for all the other shells.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Elle Stone

Hi, all,

"Exiftool for dummies" sounds like a great idea. Though I've used exiftool for several years now, I still feel like an exiftool dummy and thought I'd try to pinpoint how and why I get stumped.

Compliments to Phil:
Exiftool and dcraw are my very most favorite command line programs. In my opinion, Phil and Dave are the heroes of digital photography. Exiftool and dcraw together have saved digital photography from the morass of proprietary formats/software that would require one to choose a camera based on manufacturer-supplied software rather than on the basis of sensor capabilities or available lenses (those being the central "photography" reasons to choose one digital camera over another).

Even Linux users can be exiftool dummies:
I participate in a number of linux users forums. These days more and more linux users (many of whom are windows refugees) are not comfortable with the command line environment. In the "good old days", if you asked a question in a forum the answer was likely to start with "open a terminal and type . . . ". These days the answer is more like to start with "open this or that gui and click . . . ". And if the answer does involve opening a terminal and typing, the user is very likely to respond with fear and trembling and a request for a great deal of hand-holding.

Even people with experience programming and also experienced exiftool users can be exiftool dummies:
I became a linux-using windows refugee when Vista came out. Even back in my windows days, I always kinda preferred using the command line over working with a gui. Over the years I've used, written and/or modified programs in at least a dozen different languages, including fortran (showing my age), basic, c, command-line SPSS, a bunch of other more or less obscure YAPLs, and most recently, php. Alas, I don't know perl.

Despite having used exiftool more or less routinely for several years, I feel like an exiftool dummy. Whenever I want to do something new with exiftool, I always have to painstakingly work out how to write a new exiftool command.

At risk of exposing just how much of an exiftool dummy I really am, here are some example problems I've had:

1. Going from making a change to one image, to making changes recursively:
Phil gives examples of how to accomplish many different tasks. I think it would help if each task had at least two examples, one for making a change to only one image, the next for working recursively over a bunch of images and folders. I have a great deal of trouble trying to modify "single image" commands to work recursively.

2. dates and directories, "d":
I find myself confusing the syntax for dates and directories - I just automatically assume "d" means "dates" if I want to work with dates and means "directory" if I want to work with directories.

3. file names and extensions when working recursively:
The following command was relatively easy to figure out. It recursively opens "example.thumb.jpg", reads the contents of "TagsList", and writes the contents of "TagsList" to the corresponding raw file that has "example" as the file name and "cr2" as the file extension:
exiftool -v -tagsfromfile %d%f.thumb.jpg -TagsList -r -ext cr2 /media/b3/test

Going the other way to write all the metadata in the raw file back to the corresponding jpeg took a lot of trial and error:
exiftool -v -tagsfromfile %d%-.6f.cr2 -r -overwrite_original -ext jpg /media/b3/test

What I tried first was
exiftool -v -tagsfromfile %d%f.cr2 -TagsList -r -ext thumb.jpg /media/b3/test

And I am not sure why this doesn't work. A lot of mental thrashing went into figuring out first that I needed the "-.6" between the "%" and the "f".

4. Syntax and what to mentally call things like "-":
The following command copies a tag from one file to another:
exiftool -v -tagsfromfile %d%f.thumb.jpg -TagsList -r -ext cr2 /media/b3/test

And the following command removes certain tags from files:
exiftool -v -r -Copyright= -CopyrightNotice= -ext cr2 /media/b3/test/08

Notice the use of "-" in front of  the different metadata tags. One time it is used to copy a tag from one file to another, the next it is used to delete tags. Phil explains the syntax thusly:

"Tag values are written rather than being extracted if any tag name ends with a '=' symbol (or if the -tagsFromFile or -geotag options are used). The '=' may be prefixed by '+', '-' or '<' to add a value, remove a value or set a value from file. The following table outlines the different write syntaxes:"

What Phil says is succinct and accurate and eventually makes sense, with sufficient head-banging and trial and error (like reading a car repair manual - cryptic until after you figure out what you need to do).

My feeble brain keeps trying to make "-" mean "minus". Which is probably the source of one of my stumbling blocks. When using exiftool, what should I call "-"? "This tag"? I think I am on to something here. What should be the verbal "meaning" equivalent of things like "-", "+", "<", etc? Is it always the same? context-dependent?

5. Where do the modifiers go?
Another stumbling block that I keep running into: Where to put things like "-r", "-overwrite_original", "-ext" to get it to operate (if that is the right word) on the correct portion of the command, affect the right files.

Anyway, hopefully some of the above illustrates what this exiftool dummy might find helpful.

Elle Stone


Phil Harvey

Hi Elle,

Thank you very much for this informative post.  I understand most of your problems, but they have unfortunately renewed my fear about this "ExifTool for Dummies" document could easily turn into a book.  Perhaps I should change this to a "Quick Start" document.  I would eventually like to get more detailed than this, but baby steps first.  It is very useful to have the itemized list of problems you've had, and with enough posts like yours maybe it will get easier to write a comprehensive document.

As source material for that document, and for others who may be reading this, I will attempt to clarify some of the things that caused you problems:

Quote from: Elle Stone on January 20, 2011, 09:45:05 AM
1. Going from making a change to one image, to making changes recursively:

This is a common problem.  Basically you can put as many file names and/or directory names on the command line as you want.  I don't do this in my examples because they would quickly get too wide for the window, but I should work on documenting this better.

When you put a directory name on the command line, exiftool processes all files with recognized extensions in the directory (or all files that are writable by exiftool if writing information).  Adding the -r option causes exiftool to also recursively process files in any subdirectories of the specified directory.  The -ext option comes in very handy here to specify the types of files you want to process (unless you simply want all recognized files processed).

Quote2. dates and directories, "d":
I find myself confusing the syntax for dates and directories - I just automatically assume "d" means "dates" if I want to work with dates and means "directory" if I want to work with directories.

-d is always for dates.  The magic of exiftool lets you copy date/time values to any other tags, in which case the date/time formatting will apply to whatever tag you write. 

Another simply brilliant (IMHO) feature of Exiftool is that you can manipulate the file name as if it were metadata by writing the FileName tag.  So if you copy a date/time tag to the FileName, the -d option has the effect of specifying the format of the file name.  Really cool when you think about it.  I didn't design -d specifically to be used for file names, but the feature arose naturally from the design.

Quote3. file names and extensions when working recursively:
Going the other way to write all the metadata in the raw file back to the corresponding jpeg took a lot of trial and error:
exiftool -v -tagsfromfile %d%-.6f.cr2 -r -overwrite_original -ext jpg /media/b3/test

Yes.  This is very tricky, and well into some of the advanced features.  I'm glad you worked this out.

QuoteWhat I tried first was
exiftool -v -tagsfromfile %d%f.cr2 -TagsList -r -ext thumb.jpg /media/b3/test
And I am not sure why this doesn't work.

Basically, %f is the source file name, which ends with ".thumb" in your case.  You need to remove this for the -tagsfromfile argument because the CR2 file doesn't end with ".thumb".  Tricky I agree.  I made the syntax of %-.6f similar to the C printf() format strings so might this would be at least a bit easier to understand if you were a C programmer.

Quote4. Syntax and what to mentally call things like "-":

The only purpose of the dash (-) is so that exiftool will know the argument is not a file name.  It just says "this argument is an exiftool option", and nothing more.  Tag names on the command line are a specific type of option that you add to control the flow of information.  (Regarding this point, the "Under the Hood" of ExifTool page illustrates this information flow.)

QuoteMy feeble brain keeps trying to make "-" mean "minus". Which is probably the source of one of my stumbling blocks. When using exiftool, what should I call "-"? "This tag"? I think I am on to something here. What should be the verbal "meaning" equivalent of things like "-", "+", "<", etc? Is it always the same? context-dependent?

As I said, "-" at the start of a command line argument just specifies that this is an option.  The syntax of the options used to specify tags to extract or copy, and new values to write to tags is very complex, and I think this is the source of your confusion.  You can ignore the leading "-" from this point of view.  However, things like "-=" and "+-" inside an option are very significant, and are used to specify values to add or remove from a list.  Other combinations of ">","<","=","-" and "+" inside an argument are all simply conventions that I invented to be able to achieve the functionality that I wanted for ExifTool.  I borrowed ">" and "<" from the shell redirection syntax since you are effectively redirecting metadata from one tag to another with options like -DSTTAG<SRCTAG.  The deletion syntax -TAG= doesn't come from anywhere, but I thought this made sense since it implies that you want to assign the value of TAG to nothing.

Quote5. Where do the modifiers go?
Another stumbling block that I keep running into: Where to put things like "-r", "-overwrite_original", "-ext" to get it to operate (if that is the right word) on the correct portion of the command, affect the right files.

Yes.  I probably haven't explained this very well.  Essentially, all options apply to the entire command, so it doesn't matter where they go.  The only significant exceptions to this rule are the family of tag specification options -TAG... since these affect the order in which information is extracted, or the order in which new values are specified (which is significant only if you assign more than one value to a tag since it is the last value assigned that takes precedence).

Thanks again for your input.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Elle Stone

Hi, Phil,

Over the last few years, for one reason and another, I've read through a good deal of your documentation, and the more I use ExifTool, the more I learn from consulting the documentation.

But when I was first starting to use exiftool several years ago (did you have a forum back then?), the most useful "documentation" was other people's web sites, where they'd posted "how to" examples that I was able follow. Finding those helpful examples usually took a lot of trawling through websites.

Do you think it might be useful to new ExifTool users if you posted a collection of links to "how to" web pages? I bet a lot of the forum members have posted ExifTool "how to" examples to their own websites. I just put one up for the benefit of digiKam users. In case you are curious, here it is: http://ninedegreesbelow.com/2011/imaging/exiftool_digikam/exiftool_digikam.html

And no, my feelings won't be hurt if you do post links with examples and don't post mine!

Elle Stone