ExifTool for Dummies

[Phil Harvey]

Hi Elle,

Thanks for the suggestion and link, I hadn't seen that one yet.

Certainly providing links to other how-to pages would be useful.  Also, reading these pages is helpful to me.

- Phil

P.S. I started this forum about a year ago.  Before that (back to early 2005) we used the Image-ExifTool section of the CPAN forum (which wasn't very user-friendly).  I transferred all of the old CPAN posts to the Archives section of this forum when I created it.

[srichart4]

Phil,
I may be a perfect fit for the Dummy part.  I just downloaded Exiftool and can't even get in yet.  As to Perl, I know absolutely nothing.

I'm part of a big digitization project and we are going to generate a lot of images.  We want to add the meta data to each image, use your tool to extract it out for submission to the archives.  Up until a month or so ago, I didn't realize there was such a thing as meta data.  Now, I had seen some of it with other images, but actually knowing anything about what is stored and that there are specific fields, I hadn't a clue.

I will keep track of my problems and send them along. 

Sue (the Perfect Dummy >g)

[Phil Harvey]

Hi Sue,

I look forward to your input.

Just one point:  You don't need to know anything about Perl to use the exiftool application.  The only places you are exposed to any Perl at all with the application are through two of the more advanced features: The -if option syntax, and the configuration file.

Often the biggest hurdle in using the application is becoming familiar with the command line for your system, which really has nothing to do with exiftool or Perl at all.

- Phil

Edit: fixed typo

[StephenKiel]

Hi Phil,

First of all, thanks for putting together a great tool like exiftool, and then hanging in to explain how to get the most out of it.

I liked you comment about changing the help document from a "for dummies" to quick start.  It sets the expectation that it will be short and to the point. 

I think that part of the reason there is an initial steep learning curve is that the exiftool is like a swiss army knife with lots of different capabilities. Depending on what you want to do, it is hard for a new user to jump in to find out how to accomplish the task.  It is even harder to actually write an original command line for the task.

I think one thing that would help me is to take a command or two, dissect them and explain the mechanics of what is going on.  An example of a handy command line that I found and use, but could not explain is this:
exiftool -geotag test.gpx '-geotime<${DateTimeOriginal}-06:00' *.jpg

Beyond the obvious things like the track file name, the wildcarded filename, and the timezone offset this command line is not all that intuitive.  There is redirection, quoting, and options.  I think that pulling apart a couple of examples like this and explaining what is going on would be a big help.  You mentioned in earlier posts that you were concerned about having to explain shell quoting.  I do not think you need to or even should explain shell interpretation.  This information already exists.  What is difficult for a new user to understand is the interaction of the shell and the tool.  The quickstart document should get people to the point where they know where to start looking in order to take their next step.  If they need to gain more expertise in the shell or the actual tool they can pursue it on their own, the important step is to get the pointed in the right direction.

Stephen

[Phil Harvey]

Hi Stephen,

Thanks for the suggestion.  I like your idea of pulling apart some command line examples to help people understand the mechanics.

You're right about the Swiss army knife problem.  Last week I started working on an interactive command-line examples page to help people find commands to do what they want, but quickly realized that I was in way over my head due to the huge number of possible variations. [this is how far I got]

Also, a lot of the problem is just finding the information you need, and I spend a lot of time just acting as an index into the scattered documentation.  For example, the Geotagging with ExifTool page should explain your question about the geotagging command.  If not, please let me know exactly what you don't understand and I'll add it to the documentation.

Thanks again for your input!

- Phil

[StephenKiel]

Phil,

Thanks for the feedback and the offer to explain the geotag command further.  I did read the web page and I am still confused.  I do understand what the command is doing well enough to hack at it and make it do what I need, I just don't understand what it is doing well enough that I could write this command or one that is similar from scratch.  If going over this becomes too time consuming we can drop it. If on the other hand "capturing the confusion" helps to identify what type of things might be useful to explain in a "quick start" and it is helpful then perhaps we can work through this a little more.

It might be helpful to turn this around, and have me instead explain what I understand about this command, what I don't understand (and why) and then let you correct my understanding.  So here is my understanding and confusion.  Let me apologize in advance for it being long and kind of picky.

I read the page you sent "Geotagging with ExifTool" in order to further understand the command in my example:
exiftool -geotag test.gpx '-geotime<${DateTimeOriginal}-06:00' *.jpg

I then ran some experiments to understand it further.  The set of steps that I ran were:
exiftool -gps:all= example.jpg
exiftool example.jpg > example_tags_init
exiftool -geotag track.gpx example.jpg
exiftool example.jpg > example_tags_2nd
exiftool -geotag track.gpx '-geotime<${DateTimeOriginal}-06:00' example.jpg
exiftool example.jpg > example_tags_3rd
diff example_tags_init example_tags_2nd
diff example_tags_init example_tags_2nd > first_geotag
diff example_tags_2nd example_tags_3rd > second_geotag
grep 'Date/' example_tags_init > DateTimeOriginal

My conclusions are:

1) exiftool is the command, example.jpg is the target file, and -geotag is the operator (I will refrain from calling it a "tag" as I am confused about what a "tag" is – more on that in a bit).

2) the -geotag operator writes or overwrites several metadata variable name value pairs.  The variable names it writes are: GPS Version ID, GPS Latitude Ref, GPS Longitude Ref, GPS Altitude Ref, GPS Time Stamp, GPS Date Stamp, GPS Altitude, GPS Date/Time, GPS Latitude, GPS Longitude, GPS Position.

3) -geotime does not seem to operate on the metadata directly.  I could not get exiftool to use -geotime outside of being a modifier for -geotag (although I might not have figured out the right way to do it).  I am assuming that -geotime is only a modifier for the operator -geotag.

4) the -geotag operator without the -geotime modifier set the GPS Date/Time to 2010:09:11 15:11:28Z.  Date/Time Original was set to 2010:09:11 08:11:28.  The GPS Date seems to be affected by the timezone of the computer. (not clear from the documentation what this was going to be)

5) the -geotag operator with the -geotime modifier set all of the GPS variables to what seem to be correct values.

6) From my reading of the web page it seemed like the two commands should produce the same result:
exiftool -geotag track.gpx '-geotime<${DateTimeOriginal}-06:00' example.jpg
exiftool -geosync=-06:00:00 -geotag track.gpx example.jpg
The second command responded  -  Warning: No writable tags found

The things that are kind of confusing are:

1)  I am not sure what a tag is, the web page about geotaging refers to -geotag and -geotime as tags.  When I go to the tag index documentation page and check under a couple of the categories, it looks like that is referring to the metadata variable names.

  I don't see how the portion of the command '-geotime<${DateTimeOriginal}-06:00' gets turned into a time value.  Does exiftool fork itself and use this to compute time?  Is that why it is in single quotes – or what is the purpose of the single quotes?

9) ${DateTimeOriginal} looks kind of like a shell variable representation of the metadata variable Date/Time Original.  Is there a special list of metadata variables that are represented this way?  If all of them are available do you represent them by removing whitespace and special characters e.g.
Date/Time Original -> DateTimeOriginal
Create Date -> CreateDate
File Modification Date/Time -> FileModificationDateTime
Exif Image Width -> ExifImageWidth

10)  Is there documentation on operators like -geotag that indicates what additional modifiers (like track file name and time for GPS Time) they expect?

This is enough for now, and I apologize if it is too many questions.  It was my hope to capture what seems confusing, not to overwhelm with questions.

Thanks again,

Stephen

[Phil Harvey]

Hi Stephen,

I then ran some experiments to understand it further.  The set of steps that I ran were:

Wow.  You're already ahead of 99% of the exiftool users.  I'm impressed with your analytical approach.

1) exiftool is the command, example.jpg is the target file, and -geotag is the operator (I will refrain from calling it a “tag” as I am confused about what a “tag” is – more on that in a bit).

Yes.

2) the -geotag operator writes or overwrites several metadata variable name value pairs.  The variable names it writes are: GPS Version ID, GPS Latitude Ref, GPS Longitude Ref, GPS Altitude Ref, GPS Time Stamp, GPS Date Stamp, GPS Altitude, GPS Date/Time, GPS Latitude, GPS Longitude, GPS Position.

Yes:

"The ExifTool geotagging feature adds GPS tags to images based on data from a GPS track log file. The GPS track log file is loaded, and linear interpolation is used to determine the GPS position at the time of the image, then the following tags are written to the image:

GPSLatitude     GPSLongitude     GPSAltitude     GPSDateStamp
GPSLatitudeRef  GPSLongitudeRef  GPSAltitudeRef  GPSTimeStamp"

3) -geotime does not seem to operate on the metadata directly.

Correct.

4) the -geotag operator without the -geotime modifier set the GPS Date/Time to 2010:09:11 15:11:28Z.  Date/Time Original was set to 2010:09:11 08:11:28.  The GPS Date seems to be affected by the timezone of the computer. (not clear from the documentation what this was going to be)

Yes:

"If a value for Geotime is not given, it is taken from DateTimeOriginal for each image (as if "-Geotime<DateTimeOriginal" had been specified), but the value may be copied from any other date/time tag or set directly from a date/time string.

If the date/time tag does not include a timezone then one may be added (ie. "-Geotime<${CreateDate}-05:00"), otherwise the local system time is assumed."

5) the -geotag operator with the -geotime modifier set all of the GPS variables to what seem to be correct values.

Joy!

6) From my reading of the web page it seemed like the two commands should produce the same result:
exiftool -geotag track.gpx '-geotime<${DateTimeOriginal}-06:00' example.jpg
exiftool -geosync=-06:00:00 -geotag track.gpx example.jpg

The first command uses DateTimeOriginal with a timezone of -06:00.  The second command uses DateTimeOriginal with your local system timezone then shifts the time further by -06:00.  So the two commands are only equivalent if your local system timezone is UTC.

I agree that this is very confusing.  The -geosync tag is only necessary if the camera clock is not properly synchronized to GPS time (ie. it is a few minutes off, or drifts with time).

1)  I am not sure what a tag is, the web page about geotaging refers to -geotag and -geotime as tags.  When I go to the tag index documentation page and check under a couple of the categories, it looks like that is referring to the metadata variable names.

You already have more of a grasp than most people to be able to answer this question.  If you understand the concept of the Composite tags (tags derived from the values of other tags), then the Geotag tag is a similar concept, but the reverse.  There are many exiftool tags like this which do not represent fixed data in an image.  The Geosync tag is even further out there, and is used only as a modifier for Geotag as you discovered.  I could have coded these as command-line options instead of write-only tags, but this technique would result in a proliferation of command line options and corresponding API functions.  Implementing new features via special "tags" keeps the interface really clean (although I agree it is somewhat confusing).

  I don't see how the portion of the command '-geotime<${DateTimeOriginal}-06:00' gets turned into a time value.  Does exiftool fork itself and use this to compute time?  Is that why it is in single quotes – or what is the purpose of the single quotes?

The syntax '-DSTTAG<SRCTAG' is used to copy the value of one tag to another in the destination image.  An extension allows SRCTAG to be a string containing tag names prefixed by $ symbols.  So this extracts DateTimeOriginal from the image, appends -06:00, and writes this value to the Geotime tag.  This is where the magic happens:  Internally, writing Geotime triggers a set of complex calculations which take the input time value to interpolate in the GPS track specified by the Geotag value, and generate the new values to write for a number of GPS tags.

The single quotes are necessary to keep the shell from interpreting < as a redirection, and $ as a shell variable.  The shell strips the quotes from the command so when exiftool interprets the argument it does not see these.

9) ${DateTimeOriginal} looks kind of like a shell variable representation of the metadata variable Date/Time Original.  Is there a special list of metadata variables that are represented this way?  If all of them are available do you represent them by removing whitespace and special characters e.g.
Date/Time Original -> DateTimeOriginal
Create Date -> CreateDate
File Modification Date/Time -> FileModificationDateTime
Exif Image Width -> ExifImageWidth

Any tag name may be used.  But you are now falling prey to confusing the descriptions with tag names.  Use the -s option when extracting to see the tag names.  Otherwise the descriptions are printed by default (which may even be in another language if the -lang option is used).

10)  Is there documentation on operators like -geotag that indicates what additional modifiers (like track file name and time for GPS Time) they expect?

The geotagging tags are explained here.  I admit this description is very dry and concise, but it addresses most key features and refers the reader to the Geotagging documentation, which is a bit more descriptive.

------------------

Your questions are well beyond what I would consider for a quick start guide, although they do point out many of the very confusing aspects of exiftool which I'm sure many people wonder about.   Specifically, the implementation details of the geotagging feature are very tricky, and you already understand them at a level which is far beyond what I would expect for most users.

- Phil

[StephenKiel]

Phil,

Thanks for the kind words, patience, and help!  The more I dig into this, the more I appreciate the dilemma for quick starting folks.  I was hoping that pulling apart the command line they might decompose to kind of a hierarchy like operators that read | write metadata | rename files, and modifiers to alter the behavior of the operators.  Because the exif is kind of vague, and even things like time zone is vague (location and/or savings time) the tool needs a lot of flexibility.

Let me throw out another idea.

It occurs to me that there are a few fairly well defined things that I would want to do with exiftool: adjust local time if the clock camera is wrong, geotag using a shift between local and UTC, copy the information from one metadata variable to another (it seems like there are as many conventions to assign a comment to an image as there are tools to write them, digikam v picasa v microsoft organizer).  It also seems like there might be a lot of overlap between a large percentage of the users in the tasks they want to address.

Do you think that it might be possible / practical to associate a set of pseudo commands for the 20 or 30 most common exiftool tasks?  If the pseudo commands hide some of the syntax, order dependence, and  had a hierarchical structure it might be easy to document, and 90% of the questions might go away.  There will always be a few people that want to do something abstract, but I suspect that most users would be happy to run a command and get back to their pictures.  A command line user interface that hides some of the complexity might do that.  The command line is a desirable alternative to a GUI as it is easy to pipe the history to a README file in order to record what was done (and forgotten 6 months later).

Anyway, I will keep thinking about the quick start manual. Have a good weekend.

Stephen

[Phil Harvey]

Hi Stephen,

It occurs to me that there are a few fairly well defined things that I would want to do with exiftool: [...]
Do you think that it might be possible / practical to associate a set of pseudo commands for the 20 or 30 most common exiftool tasks?

I don't want to introduce another command syntax, however it would be possible to implement this by providing a set of batch files, and certainly adding a set of organized examples like this to the documentation is a good idea.

- Phil

[StephenKiel]

Phil,

I can see your point about not wanting to add any more variation to the current command syntax.  I did not really frame the suggestion too well. Your comment about batch might be along similar lines to what I was thinking,   Lt me outline my thoughts and give an example, if you don't think it is practical, that's OK, maybe a quick start document is the right way to go.

I was thinking of a shell that 1) performed command interpretation, 2) was prompting and self documenting, 3) allow modifier arguments in any order, 4) minimize punctuation in options.  I was thinking of something along these lines:

# exifshell

Code Select Expand

exifshell command interpreter for exiftool v 0.0.1
supports the following command arguments
  about    Display this message
  adjtime  Adjust local time when the picture was shot
  cpmdata  Copies metadata from one field to another
  export   Output selected metadata
  fname    Rename the files using Original Date / Time
  geotag   Geotag the images
  import   Import metadata from file
  ...

# exifshell geotag

Code Select Expand

Syntax for the exifshell with the geotag command argument is:
exifshell geotag -trackfile {trackfile} -timeshift {[-|+]hh:mm} file
  where {trackfile} is the gps track e.g. current.gpx
  and {hh:mm:ss} is the time difference between local and UCT time
Example:
exifshell geotag -trackfile current.gpx -timeshift -06:00 *.jpg

# exifshell geotag -tagfile my.gpx -timeshift -07:00 *.jpg

Code Select Expand

Geotagging selected files with my.gpx and -07:00:00 time shift
 
Which executes as:
exiftool -geotag my.gpx '-geotime<${DateTimeOriginal}-07:00' *.jpg

# exifshell geotag -timeshift -06:00 -trackfile my.gpx *.jpg

Should work the same as the previous command.

I think that name value options, prompting from the shell, and eliminating punctuation requirements will actually make the program pretty intuitive to use, and dramatically lower the amount of support that it needs.

[Phil Harvey]

Hi Stephen,

Interesting idea, I hadn't thought of something like this.  Sounds like a lot of work though, because I fear the capabilities of such an interface could never be complete and I would be forever adding extra features for people who wanted to do something slightly different.  This reminds me of my aborted effort in designing an interactive examples web page.

- Phil

[StephenKiel]

Phil,

If writing something like an exifshell interface did turn out to be a lot of work it would be kind of a failure, since the reason was to simplify things.  It might be something that could be contained though.  If stated goal was to handle the cases that 90% of the non-advanced users would use then there is a point where you just say no to new features.  If users want to do something that is more exotic there is always the option to just use exiftool directly (advanced users).  If someone wants to write something like GeoSetter or the exiftool GUI that uses exiftool, then the exifshell is not for them.  I suspect that given the option of using a well chosen default (like filename from date / timestamp) or coding their own, most people would choose the default. 

I was thinking that this would be easier than writing a quick start, but you have a better perspective from which to judge.

Stephen

[Phil Harvey]

Hi Stephen,

Thanks.  I'll let this idea simmer for a while and see if it grows on me.

- Phil

[Storhaug]

For me, the most helpful best practice for beginners would have been how to set the copyright, and how to license a photo under a Creative Commons license. I am now using these:

Setting copyright without CC license:

Code Select Expand

COPYRIGHT="Copyright, Firstname Lastname, \$createDate."; \
exiftool -P -overwrite_original_in_place -d %Y \
-Copyright\<"$COPYRIGHT" \
-XMP-dc:Rights\<"$COPYRIGHT" \
-XMP-xmpRights:Marked="True" \
*.jpg

Setting copyright and licensing for Attribution-ShareAlike 3.0 Unported (CC BY-SA 3.0):

Code Select Expand

COPYRIGHT="Copyright, Firstname Lastname, \$createDate. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA."; \
exiftool -P -overwrite_original_in_place -d %Y \
-Copyright\<"$COPYRIGHT" \
-XMP-dc:Rights\<"$COPYRIGHT" \
-XMP-xmpRights:Marked="True" \
-XMP-cc:License="http://creativecommons.org/licenses/by-sa/3.0/" \
*.jpg

The phrase All Rights Reserved is not required any more. You may read this thread in the Creative Commons forum:
http://forum.creativecommons.org/topic/148

I did quite some research about this and don't think that I am finished yet.

[Beholder3]

Hi,

coming back to the original question I can also support the view of a dummy user. 

My "feel" on the situation is:
- the documentation is complete but so extensive it's not too attractive for real inexperienced people who don't know where to start. It's more use to pros I think. The best thing to stop a user from using an application is always telling him "read the fucking manual" only once. You can bet he dumps the thing immediately if he's not desperate.
- I personally always look for examples which come close to my situation and then from the example look backwards to the abstract command help to try to figure out what happens.
- Any commands discussed here which do multiple things in one line I experience as counterproductive as they confuse me too much. They are good again for pros and specific optimal solutions but not for understanding. I think its like calculating things in Excel. If you are advanced user you try to achieve the perfect result in a single cell calculation where functions are nested to level 10. But the way I usually approach things first is I add a single calculation per column and do lots of columns first to be able to understand what I do. Then in the final tested version I put it all into one.

Result:
- I'd favor lots of generously explained examples which are structured in a helpful way so I am not confused by the mass of examples. The exewrcise wouldnt be helpful if it's just a huge flat list of examples. I underlined the three key words IMHO.

As a sidenote: Even minor details can be non-intuitive. For example for me as non English native speaker reading "DSTTAG<SRCTAG" at first was anything but intuitive because my eyes didnt really automatically turn the "DSTTAG" into "destination tag". I first justbread a wild bunch of letters.  Seems dumb probably but that's language dependend stuff mixed with abbreviations and missing cases.