ExifTool Forum

ExifTool => Developers => Topic started by: mrXif-geoTagger on November 02, 2022, 06:23:06 PM

Title: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 02, 2022, 06:23:06 PM
Hi all,
I'm writing an application that will enable me to name my scanned photo jpg files in a consistent way. New filenames consist of a date, location, event and individuals in the image and more. As new filenames are generated and files renamed, the data picked for the filename is populated in a listview control with lines per file and columns per data field. Multiple values for a field is comma separated (eg. names like: "John, Sarah, Kim"). In addition there's a "Description" text field for large blocks of texts. All strings can be written with Norwegian characters.

I have 10.000+ paper copies/positives to scan and don't want to start tagging in a way that turns out to be in violation of best practices. This is why I need your suggestions, ideas and thoughts.

More on the information to write:
Date: When was photo taken by camera (can be imprecise (will use #/space notation))
Location: GPS position, sublocation, city, state, country
Event: csv of words describing what's going on in photo (everyday life, birthday, Christmas, ...)
Individuals in the image: csv of "first lastname" of people in the image
Description/title: None/short/long description accompanying the photo
Source: Used for websites, newspapers and the like if needed

Questions:
1) What is the most effective way of writing the metadata to the jpegs (-stay_open, csv,..)?
2) What are the suggested fields to use (in ref to info I want to write)?

Me: Renaming jpg files
Using exiftool 12.49 on Windows 11 Home 64bit

Thank you!
Title: Re: Writing to many files and which fields are best practice
Post by: StarGeek on November 02, 2022, 08:04:17 PM
Quote from: mrXif-geoTagger on November 02, 2022, 06:23:06 PMI'm writing an application that will enable me to name my scanned photo jpg files in a consistent way.

My first piece of advice is don't try and re-invent the wheel.  There are plenty of Digital asset management (DAM) programs out there, ranging from free (DigiKam (https://www.digikam.org/)) to paid (LightRoom).  These programs will write the metadata to the files using the most common and useful tags.

QuoteMultiple values for a field is comma separated (eg. names like: "John, Sarah, Kim").

One thing to note is that while some metadata, such as keywords/tags, look like comma separated lists, but each piece of data is contained completely separate from the others. As an example, when you look at the raw XMP for the XMP:Subject tag (which is used to store keywords), you can see how each element is separate
  <dc:subject>
  <rdf:Bag>
    <rdf:li>first keyword</rdf:li>
    <rdf:li>Third Keyword</rdf:li>
    <rdf:li>&quot;Forth keyword&quot;</rdf:li>
    <rdf:li>Smith, John</rdf:li>
  </rdf:Bag>
  </dc:subject>
Programs that list the keywords as comma separated are just using that as an easy interface.  The last entry of "Smith, John" shows where a comma separated list would have problems.


QuoteIn addition there's a "Description" text field for large blocks of texts. All strings can be written with Norwegian characters.

This is getting into the specifics, but that is what the XMP:Description tag is for.


QuoteI have 10.000+ paper copies/positives to scan and don't want to start tagging in a way that turns out to be in violation of best practices. This is why I need your suggestions, ideas and thoughts.

Using a DAM as I mentioned will save data into the best locations.  For further research, you can read the IPTC Photo Metadata Standard (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata).  And while the Metadata Working Group is now defunct, it's Guidelines for Handling Image Metadata (https://web.archive.org/web/20180919181934/http://www.metadataworkinggroup.org/pdf/mwg_guidance.pdf) are also a common standard.  Most DAM programs follow both of these standards.

QuoteMore on the information to write:
Date: When was photo taken by camera (can be imprecise (will use #/space notation))
Location: GPS position, sublocation, city, state, country
Event: csv of words describing what's going on in photo (everyday life, birthday, Christmas, ...)
Individuals in the image: csv of "first lastname" of people in the image
Description/title: None/short/long description accompanying the photo
Source: Used for websites, newspapers and the like if needed

To break these down in case you don't use a DAM
Date:  Most of the time, this would be saved into the three main EXIF tags.  There is the EXIF:DateTimeOriginal, which is the time when the images was originally created.  The EXIF:CreateDate is the time when the image was turned into a computer file.  The EXIF standard calls this "DateTimeDigitized", but exiftool uses CreateData as a short cut and to help copying between tags.  Finally, there's EXIF:ModifyDate. This would be the time when the digitized image was last changed.

In digital cameras, the first two would be the same.  The image was both created and digitized in that moment.  They would differ if, for example, you scanned an old physical photo into the computer.  But when it comes down to it, that's getting far more exacting than needed.

There is a problem, though, when it comes to incomplete dates.  While the EXIF standard allows for it, very few programs allow for it.  The most common process is to give the minimum date/time known.  For example, if an images was taken today, but unknown time, it would be given a data of 2022:11:02 00:00:00.  If the day was unknown, it would be 2022:11:01 00:00:00.

The XMP standard for date/time allows for partial data, but most programs, if they read that data, would treat it as above.  You can find some details in FAQ #5 (https://exiftool.org/faq.html#Q5).

GPS Coordinates would be written into the EXIF GPS locations, GPSLatitude/GPSLongitude/GPSAltitude.  It is important to note that these are unsigned (positive) values.  The actual reference direction, N/S/E/W/Above/Below, are held in a separate Ref tag, GPSLatitudeRef/GPSLongitudeRef/GPSAltitudeRef.  It is important to make sure and fill out both.  Exiftool can use a wildcard to write both the coordinates and reference direction at the same time.  For example, this command would correctly write all four of the GPS coordinates (ignoring altitude for the moment).
exiftool -GPSLatitude*=40.6892 -GPSLongitude*=-74.0445 file.jpg

The other data would be held in the various location data tags.  Most common are CountryCode/Country/State/City/Location.  These are all XMP tags, but there are matching IPTC IIM tags that mostly have different names.  While they are widely used, the IPTC standard calls these Legacy tags in favor of th more complex Location Created and Location Shown in the Image tags (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#location-created).  These are Structured tags (see the Structured Information page (https://exiftool.org/struct.html)).  Personally, I just use the former legacy tags, as they are much simpler to deal with and still widely supported.

The person in the image would be written to the XMP:PersonInImage tag and possibly the keyword tag, XMP:Subject.  These are all list type tags.  See FAQ #17 (https://exiftool.org/faq.html#Q17).

Description/title: The long description would be written to XMP:Description.  Technically, the short description is supposed to be written to the XMP:Headline tag.  But for some reason, Adobe decided to work against the standard and use the XMP:Title tag, which is supposed to be a "short verbal and human readable name for the image, this may be the file name".  Compare Headline (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#headline) and Title (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#title) in the IPTC standards.  Basically, use either of them depending upon the software you use.

Source would most probably use the XMP:Source tag (see Source (https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#source)).  It may not be technically correct according to the standard, but it would easy to use and remember.

Quote1) What is the most effective way of writing the metadata to the jpegs (-stay_open, csv,..)?

Basically, whatever you are comfortable with.  Most of the time I'm just entering commands directly.  If you're calling from a program you've created, then using -@ (Argfile) option (https://exiftool.org/exiftool_pod.html#ARGFILE)/-stay_open option (https://exiftool.org/exiftool_pod.html#stay_open-FLAG)/-execute option (https://exiftool.org/exiftool_pod.html#execute-NUM) would be best if figure out how to work it.

QuoteUsing exiftool 12.49 on Windows 11 Home 64bit

On Windows, you will run into the FAQ #18 (https://exiftool.org/faq.html#Q18) for accented characters on the command line.  But using CSV or -@/-stay_open should avoid this problem I think.

Myself, I could never get the command line to deal with accented characters properly until I used this StackOverflow answer (https://stackoverflow.com/questions/57131654/using-utf-8-encoding-chcp-65001-in-command-prompt-windows-powershell-window/57134096#57134096), which may have font issues in some older programs.
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 03, 2022, 05:07:45 AM
Thank you again StarGeek, for your full answers. I'll read and digest this fully. I might also dispense some time on why I'm building my own DAM :- ) And it's not like I haven't researched this. I've scoured this site back and forth and gotten a lot of answers, but there are still things I'm wondering about. And you confirm that multiple values for a field could be a challenge. This speaks strongly in favor of the "stay_open" approach then probably, where I would loop over the values and add them one by one. Multiple values are the nature of both "persons in the image" and "event".

You're so fast :- ) and I had prepared a follow up post on this with my assumptions. Let me post it anyways and later I'll correct it according to your initial feedback. Here's what I wrote just after posting this yesterday:

Firstly I'll list the things I'm pretty sure is correct (cause many of you have helped me earlier on another similar project). I'm of course building on that as much as possible 🙂. Lastly are the things I'm less sure of. Correct me on everything if I'm mistaken. Usually am!

Am pretty sure about these:

Original date & time: When the camera took the photo
-MWG:DateTimeOriginal#="1968:08:    :  :  +01:00"
Will use # and space notation because for old photos you usually do not know the day, month or even year.
Questions: 1) Any suggestions on how to express that this is a picture taken in say summer of '69 (know the year, not the month but know the season).
2) Do I have to specify "MWG:"?
Based on earlier recommendations I'll stay away from the OS' file dates. Might also skip dates in GPS section. More on that later.

Event: What's going on in the photo? Everyday life, birthday, Christmas
Multiple values are separated by ", ".
-MWG:Keywords="{event1}, {event2}"

People in image: Who is in the photo? I'm working on writing a csv of "First Lastname" to the tag. Is there a place for this in other places than the one below?
-IPTC:PersonInImage="John McClain, Holly M. Gennero"

Description: Any additional information I would like to store with the image. Challenges here are length limits and can it handle Norwegian letters?
-MWG:Description="{long text with ÆØÅ}"

Less sure about these:

Location: Where's the photo taken?
For this the user has the option to do it simple and just name a place (this is what's put in the filename due to limited space). In the tag there are luckily designated places for all this. I assume if user goes with the easy approach and want to just name the place I can store "Sydney Opera House" like this:
-MWG:Location="Sydney Opera House"
If user opts for the more complex but precise approach I will store it like this:
-MWG:Location="Sydney Opera House"
-MWG:City="Sydney"
-MWG:State="NSW 2000"
-MWG:Country="Australia"
For the coordinates:
Correct me if I'm wrong but I believe I can write to EXIF and XMP at the same time by writing (Exiftool will sort out how to write it ref this(https://exiftool.org/forum/index.php?topic=10163.0)):
-GPSLongitude=151.2037806  -GPSLongitudeRef=151.2037806 -GPSLatitude=-33.8585187 -GPSLatitudeRef=-33.8585187

Need support on this one.
When tagging DateTimeOriginal and GPS coordinates, is it also important to set GPS time? I do not think so.

While I'm inserting tags, there are some others I can set at the same time:

Digitized date: I intend to use the file creation date for this. This will for the most part be correct (but can be modified in the listview control prior to writing).
-MWG:CreateDate="{date time}"

Photographer & Copyright: These strings are set in the settings section of my application, populated in the listview and can individually be altered per file.
-MWG:Creator="{name1}"
-MWG:Copyright="{Copyright © String 2022}"

Original filename: The information is there, so I'll also include this in the tag. Haven't found an MWG group tag for this. What's the best practice of embedding this?
-XMP:OriginalFilename="{current_filename.ext}"
-XMP:Title="{current_filename.ext}"
-IPTC:ObjectName="{current_filename.ext}"

Source: Every now and then you need to store news articles or similar and a source is appropriate. A source could be a web site or a newspaper. What would be the best practice of storing this information?
-IPTC:Source="{source}"  or  -IPTC:Creator="{source}"? Any place for this in EXIF and XMP? Is there an MWG for this?

Note: I will never write {curly brackets} to the tags, but here they represent something I will substitute with real content.
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 03, 2022, 10:40:12 AM
I acknowledge that I'm biting over a lot in this post, so let me move to a "concluded" list of what I see as agreed upon so far (find it at the bottom below).

What we're left with are these that need more discussion and insight:

Lists:
Event: MWG:Keywords="Party;Christmas"
I would assume that I could apply the strings above simply by specifying the separator (FAQ#17 (https://exiftool.org/faq.html#Q17)) like:
-sep ";" -MWG:Keywords="Party;Christmas"
I see what you write with duplicates. initially this will not be an issue for me as I'm writing to freshly scanned photos and I assume you are stating this as a principle.

People in image: IPTC:PersonInImage="McClain, John;Holly M. Gennero"
I see you suggest XMP:PersonInImage and I've found the same in IPTC. Are these two separate or the same? If separate, should I write both?

GPS coordinates:
According to you in "GPSLongitudeRef issue (https://exiftool.org/forum/index.php?topic=10163.msg52946#msg52946)" I could write coordinates (ignoring altitude) like:
-GPSLongitude=151.2037806  -GPSLongitudeRef=151.2037806 -GPSLatitude=-33.8585187 -GPSLatitudeRef=-33.8585187
the notation you suggest above is even briefer:
-GPSLatitude*=-33.8585187 -GPSLongitude*=151.2037806

Source: IPTC:Source, IPTC:Creator
Need to do more digging on this one.


Conclusion, best practice and best approach (feel free to challenge these as well):
I'll use MWG-tags (https://exiftool.org/TagNames/MWG.html) if possible as that infers "best practice" (I assume :- )
Digitized date: MWG:CreateDate
Original filename: XMP:OriginalFilename, XMP:Title, IPTC:ObjectName
Photographer/artist: MWG:Creator
Copyright: MWG:Copyright
Description: MWG:Description
Location: MWG:Location, MWG:City, MWG:State, MWG:Country (can't find country code in the MWG tag and deduct from that that country code is optional)

Original date & time: (variants with increasing precision)
-DateTimeOriginal#="1968:  :     :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1968"
-DateTimeOriginal#="1968:08:     :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1968:08"
-DateTimeOriginal#="1968:08:17   :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1968:08:17"
-DateTimeOriginal#="1968:08:17 12:34:45" -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1968:08:17 12:34:45"
Title: Re: Writing to many files and which fields are best practice
Post by: StarGeek on November 03, 2022, 11:29:42 AM
Quote from: mrXif-geoTagger on November 03, 2022, 05:07:45 AMOriginal date & time: When the camera took the photo
-MWG:DateTimeOriginal#="1968:08:    :  :  +01:00"
Will use # and space notation because for old photos you usually do not know the day, month or even year.
Questions: 1) Any suggestions on how to express that this is a picture taken in say summer of '69 (know the year, not the month but know the season).

This is mixed. Writing to MWG:DateTimeOriginal will write to multiple tags.  It will write the EXIF/XMP tags listed on the MWG tags page (https://exiftool.org/TagNames/MWG.html) and conditional write to IPTC if other IPTC tags exist.  This would be the proper format for the EXIF date/time tags,DateTimeOriginal/CreateDate/ModifyDate, but the correct way to write to the XMP tags would be simply "1968:08". This is covered in FAQ #5 linked above.

Checking this command, unfortunately, using "-MWG:DateTimeOriginal#="1968:08:    :  :  +01:00" will not work at all.  It looks like you will have to write to the EXIF:DateTimeOriginal and XMP:DateCreated separately. Additional problems with the time zone.  XMP:DateCreated will not accept a time zone on a partial date.  And while you can force EXIF:DateTimeOriginal to accept "1968:08:    :  :  +01:00" with the hashtag (shortcut for the -n (--printConv) option (https://exiftool.org/exiftool_pod.html#n---printConv)), I would not advise it.

I would say the best practice would be to write the date to EXIF with -DateTimeOriginal#="1968:08:    :  :  ", write the time zone with -OffsetTimeOriginal=+01:00, and the XMP with -XMP:DateCreated=1968:08".

One additional thing, for the incomplete EXIF date, it should be 5 spaces between the date and the time, e.g. "1968:08:(2spacesForDate)(1space)(2spacesForHour):(2spaces):(2spaces)".  It might just have been the forum or simply mis-type, but your post only had 4 spaces.

Any other DAM will probably read such a date as 1968:08:01 00:00:00 or possibly not at all.

Quote2) Do I have to specify "MWG:"?
Based on earlier recommendations I'll stay away from the OS' file dates. Might also skip dates in GPS section.

The MWG tags are a good starting place, as they will update multiple tags in the proper locations.  You can include the -use MWG option (https://exiftool.org/exiftool_pod.html#use-MODULE) and exiftool will automatically use the MWG tags when listed, i.e. setting just Description will set the tags associated with MWG:Description, but that still won't work on edge cases like the date/time above.

Programmably, it's probably a good idea to explicitly set the group name, whether it's MWG, EXIF, IPTC, or XMP.  In most cases, I usually suggest keeping tag names simple as possible.  For example, writing -DateTimeOriginal=2022:11:03 12:00:00 over writing -EXIF:DateTimeOriginal=2022:11:03 12:00:00 (group 0 name) or -ExifIFD:DateTimeOriginal=2022:11:03 12:00:00 (group 1 name) (see the GetAllGroups function (https://exiftool.org/ExifTool.html#GetAllGroups) for list of group names).  But this would be a case for setting the group explicitly. Using MWG is a good choice, as it does some other bookkeeping as well, but it's fine setting the group directly.

I'd also like to say it's better to use the Group 0 names than the more exacting Group 1 names.  This is especially important in the case of EXIF tags. It's better to write EXIF:DateTimeOriginal than ExifIFD:DateTimeOriginal because it allows exiftool to decide the best place to write it.  And the proper locations in EXIF aren't really as clear as the locations in XMP.  It's quite easy to read the data in a file and use that for EXIF but there are a lot of programs, and some cameras, that don't write EXIF data properly.  For example, some Sony cameras (https://exiftool.org/forum/index.php?topic=10693.0) incorrectly write the EXIF:ModifyDate to IFD1 instead of IFD0.  And it would be too easy to see that and assume IFD1 is the correct place to write the date.

QuoteEvent: What's going on in the photo? Everyday life, birthday, Christmas
Multiple values are separated by ", ".
-MWG:Keywords="{event1}, {event2}"

This is FAQ #17 (https://exiftool.org/faq.html#Q17).  I would suggest using the commands listed under "To prevent duplication".  In your example, you would use
-MWG:Keywords-="{event1}" -MWG:Keywords+="{event1}" -MWG:Keywords-="{event2}" -MWG:Keywords+="{event2}"
to prevent duplication.  Alternatively, you could read all previously keywords, remove duplicates, and the write them all again, overwriting the previous data.  As an command line example,  Keywords already has a value of "Event3", you would re-write that along with the new data.
C:\>exiftool -G1 -a -s -Keywords y:\!temp\Test4.jpg
[IPTC]          Keywords                        : Event3

C:\>exiftool -P -overwrite_original -Keywords=Event1 -Keywords=Event2 -Keywords=Event3 y:\!temp\Test4.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -Keywords y:\!temp\Test4.jpg
[IPTC]          Keywords                        : Event1, Event2, Event3

I have to cut this short at this point as I have an appointment. I'll come back to this and finish this post as well as your follow up post then.  This post may have spelling or other errors as I'm rushing out and don't have time to double check.
Title: Re: Writing to many files and which fields are best practice
Post by: StarGeek on November 03, 2022, 05:53:20 PM
Quote from: mrXif-geoTagger on November 03, 2022, 05:07:45 AMPeople in image: Who is in the photo? I'm working on writing a csv of "First Lastname" to the tag. Is there a place for this in other places than the one below?
-IPTC:PersonInImage="John McClain, Holly M. Gennero"

The tag is actually XMP:PersonInImage, not IPTC.  THis can be confusing because PersonInImage is part of the IPTC Extension schema.  The IPTC IIM standard is a much older standard dating back to the early '90s in regards to images (even older with regards to news print, which is the source).  According to the spec, it has limitations on how long the data in it can be, but these limitations are usually ignored by most modern programs.  The -m (-ignoreMinorErrors) option (https://exiftool.org/exiftool_pod.html#m--ignoreMinorErrors) must be included in any command to tell exiftool to ignore these character length limitations.

Since you mention using CSV file, this goes back to FAQ #17 (https://exiftool.org/faq.html#Q17), though FAQ #26 (https://exiftool.org/faq.html#Q26) is also relevant. In your example, you must also include the -sep option (https://exiftool.org/exiftool_pod.html#sep-STR--separator).  In this case it would be -sep ", ".  This is to tell exiftool that you want the keywords separated on the character string provided by the -sep option so you get a list of
John McClain
Holly M. Gennero

and not a single keyword of
"John McClain, Holly M. Gennero"

QuoteDescription: Any additional information I would like to store with the image. Challenges here are length limits and can it handle Norwegian letters?
-MWG:Description="{long text with ÆØÅ}"

Length wise, you'll want to use XMP tags, of which includes Description, as the IPTC IIM tags have aforementioned character length limits. Nearly all of the XMP tags don't have a limit on the length, though in the case of really long text, over 64KB, you might run into problems with some programs.  This is because the jpeg format has a limit of 64KB for any single block of data.  These is a way that allows XMP to be extended in to further blocks, but not all programs will correctly read the data.  You will most often see this on some forums where people are having problems because after a lot of edits, some of the LightRoom/Photoshop editing history tags (see XMP-xmpMM tags (https://exiftool.org/TagNames/XMP.html#xmpMM)) can end up with a huge amount of data, sometimes becoming a significant portion of the image file.

Sort of related is sometimes with Windows, when there's a large amount of metadata, the embedded thumbnail will have to cross this 64KB limit.  In those cases, Windows will not display the thumbnail as an icon.  Either some data needs to be removed or the embedded thumbnail can be removed, forcing Windows to read the actual image data to form the desktop icon.

With regards to the accented characters.  This will be something you'll have to try and work out.  Perl (which is what exiftool is written in) has some big problems dealing with accented characters on  the Windows command line (not on Mac/Linux shells).  The above mentioned FAQ #18 deals with this.  I think using CSV or -@ arg files will work around this, as exiftool will be reading from a file and not the command line.  I can't really check this out because I have my system set to UTF8 using this StackOverflow answer (https://stackoverflow.com/questions/57131654/using-utf-8-encoding-chcp-65001-in-command-prompt-windows-powershell-window/57134096#57134096) and it would require me to change, reboot, check, change back, reboot again to get back to normal, for me to check it out.

QuoteLocation: Where's the photo taken?
For this the user has the option to do it simple and just name a place (this is what's put in the filename due to limited space). In the tag there are luckily designated places for all this. I assume if user goes with the easy approach and want to just name the place I can store "Sydney Opera House" like this:
-MWG:Location="Sydney Opera House"
If user opts for the more complex but precise approach I will store it like this:
-MWG:Location="Sydney Opera House"
-MWG:City="Sydney"
-MWG:State="NSW 2000"
-MWG:Country="Australia"

This is go way to go.  Not only will the MWG tags write to the so called "legacy" tag locations, it will write to the newer LocationShown tag group, which is a structured tag and a complex subject.

QuoteFor the coordinates:
Correct me if I'm wrong but I believe I can write to EXIF and XMP at the same time by writing (Exiftool will sort out how to write it ref this(https://exiftool.org/forum/index.php?topic=10163.0)):
-GPSLongitude=151.2037806  -GPSLongitudeRef=151.2037806 -GPSLatitude=-33.8585187 -GPSLatitudeRef=-33.8585187

Not quite.  The problem in that link was that the person was trying to write the Reference tags in XMP, except they don't exist.  The reference direction is included in XMP (though someone needs to tell Apple this, because they screw this up constantly).

You listed command is correct in that it will write the EXIF coordinate tags and references correctly.  It's really much easier this way then including extra code to figure out if the reference direction is N or S/E or W.  The XMP coordinates would need to be written separately.
-GPSLongitude=151.2037806  -GPSLongitudeRef=151.2037806 -GPSLatitude=-33.8585187 -GPSLatitudeRef=-33.8585187 -XMP:GPSLongitude=151.2037806 -XMP:GPSLatitude=-33.8585187

That said, the XMP GPS tags really aren't that important outside of XMP sidecar files.  The GPS coordinates in the EXIF group are usually the tags read.

QuoteNeed support on this one.
When tagging DateTimeOriginal and GPS coordinates, is it also important to set GPS time? I do not think so.

This is something I haven't really thought about.  But there is a use case for it, though not necessarily for a single photographer.  If you're getting files from multiple images from separate locations, then, because GPS time is UTC, it would allow easier browsing by exact time.  But it is an edge case.  And additional case would be to if you wanted to create a GPS track from a bunch of images to upload to something like google maps.

If you have the correct time zone available, it is extremely easy to add.  All you have to do is set both the GPSDateStamp and GPS:GPSTimeStamp to the correct value and exiftool will automatically adjust the time.

Example
C:\>exiftool -P -overwrite_original -GPS*Stamp="2022:11:03 12:00:00-07:00" y:\!temp\Test4.jpg
    1 image files updated

C:\>exiftool -G1 -a -s -gps* y:\!temp\Test4.jpg
[GPS]           GPSVersionID                    : 2.3.0.0
[GPS]           GPSTimeStamp                    : 19:00:00
[GPS]           GPSDateStamp                    : 2022:11:03
[Composite]     GPSDateTime                     : 2022:11:03 19:00:00Z

While I'm inserting tags, there are some others I can set at the same time:

QuoteDigitized date:
<snip>
Photographer & Copyright:

Both sound fine.

QuoteOriginal filename: The information is there, so I'll also include this in the tag. Haven't found an MWG group tag for this. What's the best practice of embedding this?
-XMP:OriginalFilename="{current_filename.ext}"
-XMP:Title="{current_filename.ext}"
-IPTC:ObjectName="{current_filename.ext}"

There's no corresponding MWG tag for this, but otherwise fine except for what I noted above about how Adobe treats this data.

QuoteSource: Every now and then you need to store news articles or similar and a source is appropriate. A source could be a web site or a newspaper. What would be the best practice of storing this information?
-IPTC:Source="{source}"  or  -IPTC:Creator="{source}"? Any place for this in EXIF and XMP? Is there an MWG for this?

I linked above to the IPTC spec on SourceSource is named the same in both IPTC and XMP.

Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 04, 2022, 08:49:34 PM
Thanks a bunch StarGeek. I'm ready to dive in and here's my latest conclusion :-)

Conclusion, best practice and best approach (feel free to challenge these as well):
I'll use MWG-tags (https://exiftool.org/TagNames/MWG.html) if possible as that infers "best practice" (I assume). I'll test writing with -stay_open and not via CSV.

Digitized date: MWG:CreateDate
Original filename: XMP:OriginalFilename, XMP:Title, IPTC:ObjectName
Photographer/artist: MWG:Creator
Copyright: MWG:Copyright
Description: MWG:Description
Source: XMP:Source
Location: MWG:Location, MWG:City, MWG:State, MWG:Country (ignoring country code)

GPS coordinates:
-GPSLatitude*=34.0551881 -GPSLongitude*=-118.4132764
(omitting date/time and thus dodging the date issues with old dates and GPS :-)

Original date & time: (variants with increasing precision)
-DateTimeOriginal#="1988:  :    :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1988"
-DateTimeOriginal#="1988:11:    :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1988:11"
-DateTimeOriginal#="1988:11:18 :  :  " -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1988:11:18"
-DateTimeOriginal#="1988:11:18 23:31:10" -OffsetTimeOriginal=+01:00 -XMP:DateCreated=1988:11:18 23:31:10"

(padding with spaces where there are no digits)

Event: (possible multiple values, will ignore duplicate-issue)
-MWG:Keywords+="Christmas" -MWG:Keywords+="Party"

People in image: (possible multiple values, will ignore duplicate-issue)
-XMP:PersonInImage+="McClain, John" -XMP:PersonInImage+="Holly M. Gennero"

Wish me luck!
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 06, 2022, 07:42:51 PM
Hi all, I need your help!

Got the inspiration for the code below from this address: "Using stay_open and visual basic (https://exiftool.org/forum/index.php?topic=6725.msg33645#msg33645)"

There are no indications that the code worked and I've been toying with it for some time with no luck.

Can any of you see what is incorrect with the code?
The code simply writes the string "Make it work" to the MWG group tag "Description" of the image file x.jpg.

For the example to work, put the latest exiftool.exe and an image named x.jpg in the c:\tmp folder and run the code.

Public Sub testArgFile()
Dim fso As Object, wso As Object, w As Object, fileobj As Object
Dim ScriptObj As Object, ts As Object
Dim sCommand As String, sArgFile As String, sEXE As String, sImg as string

    'init
         sImg = "C:\tmp\x.jpg"
         sArgFile = "C:\tmp\EXIF.arg"
         sEXE = "c:\tmp\exiftool.exe"
         sCommand = """" & sEXE & """ -stay_open True @ """ & sArgFile & """"
   
    'Create empty arg file, start exiftool, wait for args...
         Set fso = CreateObject("Scripting.FileSystemObject")
         Set w = fso.CreateTextFile(sArgFile, True, False)
         Set fileobj = fso.getfile(sArgFile)
         w.Close
         Set wso = CreateObject("WScript.Shell")
         Set ScriptObj = wso.exec(sCommand)

     'write to arg file and tell exiftool to execute
        Set ts = fileobj.OpenAsTextStream(8, -1) '8=forappending, -1=unicode
        ts.writeline "-mwg:description=""Make it work"""
        ts.writeline sImg
        ts.writeline "-execute"
        ts.writeline vbLf
        ts.Close

    'Close exiftool
        Set ts = fileobj.OpenAsTextStream(8, -1)
        ts.writeline "-stay_open"
        ts.writeline "false"
        ts.writeline vbLf
        ts.Close
   
     'kill objects
        Set ts = Nothing:   Set ScriptObj = Nothing
        Set wso = Nothing:  Set fileobj = Nothing
        Set w = Nothing:    Set fso = Nothing
   
End Sub

When inspecting the arg file, it contains:
-xmp:description="Make it work"
C:\tmp\x.jpg
-execute
-stay_open
false




This code is not so useful for one image :-) but imaging writing 15-30 tags to 1000+ files...
Title: Re: Writing to many files and which fields are best practice
Post by: StarGeek on November 06, 2022, 08:59:47 PM
One note regarding incomplete date/times in EXIF.  There needs to be a space separating the date and time parts in addition to using two spaces when a value is missing.  So in the case of "1988:11", there would be 5 spaces total in between.  Example swapping hyphens for spaces
-DateTimeOriginal#="1988:11:-----:--:--"

Unfortunately, I can't help with the visual basic code.  And I've never actually looked into what's required of the -stay_open option.
Title: Re: Writing to many files and which fields are best practice
Post by: Phil Harvey on November 06, 2022, 09:09:47 PM
If you run exiftool with the argfile and it works, then you can narrow the problem to the visual basic interface.

- Phil
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 07, 2022, 03:38:03 AM
Quote from: Phil Harvey on November 06, 2022, 09:09:47 PMIf you run exiftool with the argfile and it works, then you can narrow the problem to the visual basic interface. - Phil

Good tip. I didn't think of that. Will try supplying the argfile on the command line when I get back to the lab. Are there any way of getting any feedback (for debug purposes) while using -stay_open? I will also inspect Task Manager to verify that exiftool.exe is running. It seems like it's not executing and I was wondering if I sent the -execute signal incorrectly. I've been toying with sending -execute & chr(10), chr(13), both and none, but no tag is written to the file.

The code also has a unnecessary closing and opening of the argfile, but that should just slow things down. Have tried with and without that and no difference. I've also asked this question in VBforums.com. They no little of exiftool though, but they might help me in the VB domain :-) Thanks, guys!
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 08, 2022, 06:50:01 AM
For anyone interested, my issue with instancing exiftool.exe and serving it commands (and successfully terminating :- ) has been solved by user wqweto over at the VBForums. Find working skeleton to use in post #4 in this thread over there: Issues with: Writing metadata to jpg using exiftool and -stay_open (https://www.vbforums.com/showthread.php?898246-Issues-with-Writing-metadata-to-jpg-using-exiftool-and-stay_open&p=5585294&viewfull=1#post5585294)
Title: Re: Writing to many files and which fields are best practice
Post by: Phil Harvey on November 08, 2022, 07:40:36 AM
Here is the code from that post (in case the other forum goes down):

Option Explicit

Private Sub Form_Load()
    Const JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE As Long = &H2000
    Const EXIFTOOL_EXE  As String = "D:\TEMP\exiftool-12.49\exiftool.exe"
    Dim oExec           As cExec
    Dim lNumCommands    As Long
    Dim lIdx            As Long
    Dim sOutput         As String
   
    Set oExec = New cExec
    oExec.Run EXIFTOOL_EXE, "-stay_open true -@ -", StartHidden:=True, LimitFlags:=JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE
    lNumCommands = 10
    For lIdx = 1 To lNumCommands
        oExec.WriteInput "-ver" & vbCrLf & "-execute" & lIdx & vbCrLf
    Next
    sOutput = pvWaitForOutput(oExec, "{ready" & lNumCommands & "}")
    Debug.Print sOutput
    oExec.WriteInput "-stay_open" & vbCrLf & "false" & vbCrLf
    Set oExec = Nothing
End Sub

Private Function pvWaitForOutput(oExec As cExec, sText As String) As String
    Dim sOutput         As String
   
    sOutput = oExec.ReadPendingOutput
    Do While InStr(sOutput, sText) = 0
        sOutput = sOutput & oExec.ReadOutput(1) & oExec.ReadPendingOutput
    Loop
    pvWaitForOutput = sOutput
End Function
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 23, 2022, 09:09:27 AM
Using the above method listed by Phil, I run this command on one file:
-DateTimeOriginal#=1901:01:01 00:00:00
-OffsetTimeOriginal=+01:00
-XMP:DateCreated="1901:01:01"
-MWG:Keywords+=input1event1
-MWG:Keywords+=input1event2
-MWG:Location=input2location
-MWG:City=input3city
-MWG:State=input4state
-MWG:Country=input5country
-GPSLatitude*=34.05494487166912
-GPSLongitude*=-118.41374080270626
-XPSubject=input6person1, input6person2
-XMP:PersonInImage+=input6person1
-XMP:PersonInImage+=input6person2
-XMP:Source=input7source
-MWG:Creator=input8artist
-MWG:Copyright=input9copyright
-MWG:Description=input10description
-comment=input10description
-XPComment=input10description
-XMP:OriginalFilename=test.jpg
-XMP:Title=test.jpg
-IPTC:ObjectName=test.jpg
-MWG:CreateDate="2022:11:23 13:31:54"
-overwrite_original
C:\tmp\test.jpg
-execute0


Anything listed here as "input#{field}" is my manual input and just a numbering and a reference to the {field} I'm addressing. After execution, the tag looks like this:
IPTC: Date Created: 1901:01:01
IPTC: Time Created: 00:00:00+01:00
ExifIFD: Date/Time Original: 1901:01:01 00:00:00
ExifIFD: Offset Time Original: +01:00
XMP-photoshop: Date Created: 1901:01:01
IPTC: Digital Creation Date: 2022:11:23
IPTC: Digital Creation Time: 13:31:54+01:00
ExifIFD: Create Date: 2022:11:23 13:31:54
XMP-xmp: Create Date: 2022:11:23 13:31:54
XMP-dc: Subject: input1event1, input1event2
IPTC: Keywords: input1event1, input1event2
XMP-iptcExt: Location Shown Sublocation: input2location
XMP-iptcCore: Location: input2location
IPTC: Sub-location: input2location
XMP-photoshop: City: input3city
XMP-iptcExt: Location Shown City: input3city
IPTC: City: input3city
XMP-photoshop: State: input4state
XMP-iptcExt: Location Shown Province State: input4state
IPTC: Province-State: input4state
XMP-photoshop: Country: input5country
XMP-iptcExt: Location Shown Country Name: input5country
IPTC: Country-Primary Location Name: input5country
XMP-iptcExt: Person In Image: input6person1, input6person2
IFD0: XP Subject: input6person1, input6person2
XMP-photoshop: Source: input7source
XMP-dc: Creator: input8artist
IPTC: By-line: input8artist
IFD0: Artist: input8artist
XMP-dc: Rights: input9copyright
IPTC: Copyright Notice: input9copyright
IFD0: Copyright: input9copyright
XMP-dc: Description: input10description
IPTC: Caption-Abstract: input10description
IFD0: Image Description: input10description
IFD0: XP Comment: input10description
File: Comment: input10description
GPS: GPS Longitude: 118.413741
GPS: GPS Latitude: 34.054945
GPS: GPS Latitude Ref: North
GPS: GPS Longitude Ref: West
XMP-getty: Original File Name: test.jpg
XMP-dc: Title: test.jpg
IPTC: Object Name: test.jpg

This is not the entire tag and I've grouped lines together... Any comments to this or is this "best practice"?  8)
Title: Re: Writing to many files and which fields are best practice
Post by: StarGeek on November 24, 2022, 06:48:43 PM
The only other comment I can make is one of my main rules: Get the data into the file (or sidecar). If you decide you did it wrong and want to put it someplace else, then exiftool can fix it for you, though you may have to come here to work out the details.
Title: Re: Writing to many files and which fields are best practice
Post by: mrXif-geoTagger on November 25, 2022, 08:00:15 AM
Quote from: StarGeek on November 24, 2022, 06:48:43 PM...Get the data into the file!

Couldn't agree more. This is exactly what I'm doing in my application!

The last thing i wonder about, and I wonder if I'm missing arguments for codepage or something, OR if it in fact is limitations in the tags: When I write the special Norwegian chars to the various fields, only these four were able to retain these correctly:
IFD0: Artist: in12Ærtistø
IFD0: Copyright: in13Cøpyright ÆØÅ(C)æøå
File: Comment: in15Descriptioøn_ÆØÅ(C)æøå
IFD0: Image Description: in15Descriptioøn_ÆØÅ(C)æøå

See a screenshot of my inputs/command/output here: https://ibb.co/1b3d7S4 (https://ibb.co/1b3d7S4)
(will be automatically deleted in 6 months)

This is the command (you can probably not see the special Norwegian characters):
-DateTimeOriginal#=1900:01:01 00:00:00
-OffsetTimeOriginal=+01:00
-XMP:DateCreated="1900:01:01"
-MWG:Keywords+=in1ØÆÅ
-MWG:Keywords+=in2æøå
-MWG:Location=in3locÆØÅ
-MWG:City=in4cityæøå
-MWG:State=in5stateæøå
-MWG:Country=in6countryÆØÅ
-GPSLatitude*=33
-GPSLongitude*=44
-XPSubject=in7Åse, in8Kjærsti, in9Bjørg, in10ÆØÅæøå
-XMP:PersonInImage+=in7Åse
-XMP:PersonInImage+=in8Kjærsti
-XMP:PersonInImage+=in9Bjørg
-XMP:PersonInImage+=in10ÆØÅæøå
-XMP:Source=in11Ævisøå
-MWG:Creator=in12Ærtistø
-MWG:Copyright=in13Cøpyright ÆØÅ(C)æøå
-MWG:Description=in15Descriptioøn_ÆØÅ(C)æøå
-comment=in15Descriptioøn_ÆØÅ(C)æøå
-XPComment=in15Descriptioøn_ÆØÅ(C)æøå
-XMP:OriginalFilename=x1æøå.jpg
-XMP:Title=x1æøå.jpg
-IPTC:ObjectName=x1æøå.jpg
-MWG:CreateDate="2022:11:07 15:45:25"
-overwrite_original
C:\tmp\test\x1æøå.jpg
-execute1


The (sorted & selected) output is:
IPTC: Time Created: 00:00:00+01:00
IPTC: Digital Creation Time: 15:45:25+01:00
IPTC: Date Created: 1900:01:01
XMP-photoshop: Date Created: 1900:01:01
ExifIFD: Date/Time Original: 1900:01:01 00:00:00
IPTC: Digital Creation Date: 2022:11:07
ExifIFD: Create Date: 2022:11:07 15:45:25
XMP-xmp: Create Date: 2022:11:07 15:45:25
GPS: GPS Latitude: 33.000000
GPS: GPS Longitude: 44.000000
GPS: GPS Longitude Ref: East
IFD0: Software: HDZphotoNamer v1.0.0
IPTC: Keywords: in1, in2
XMP-dc: Subject: in1???, in2???
XMP-photoshop: Source: in11?vis??
IPTC: By-line: in12
XMP-dc: Creator: in12?rtist?
IFD0: Artist: in12Ærtistø
IPTC: Copyright Notice: in13C
XMP-dc: Rights: in13C?pyright ???(C)???
IFD0: Copyright: in13Cøpyright ÆØÅ(C)æøå
IPTC: Caption-Abstract: in15Descriptio
XMP-dc: Description: in15Descriptio?n_???(C)???
File: Comment: in15Descriptioøn_ÆØÅ(C)æøå
IFD0: Image Description: in15Descriptioøn_ÆØÅ(C)æøå
IPTC: Sub-location: in3loc
XMP-iptcCore: Location: in3loc???
XMP-iptcExt: Location Shown Sublocation: in3loc???
IPTC: City: in4city
XMP-iptcExt: Location Shown City: in4city???
XMP-photoshop: City: in4city???
IPTC: Province-State: in5state
XMP-iptcExt: Location Shown Province State: in5state???
XMP-photoshop: State: in5state???
IPTC: Country-Primary Location Name: in6country
XMP-iptcExt: Location Shown Country Name: in6country???
XMP-photoshop: Country: in6country???
XMP-iptcExt: Person In Image: in7?se, in8Kj?rsti, in9Bj?rg, in10??????
GPS: GPS Latitude Ref: North
IPTC: Object Name: x1
XMP-dc: Title: x1???.jpg
XMP-getty: Original File Name: x1???.jpg
System: File Name: x1æøå.jpg

Anything I should do differently to preserve the special characters?

(just noticed that I can attach images here :- )
HDZphotoNamer.jpg
Title: Re: Writing to many files and which fields are best practice
Post by: Phil Harvey on November 25, 2022, 12:23:55 PM
If you are passing UTF-8 characters to ExifTool then you don't need to specify a -charset option, and the characters will be properly written to XMP.  If you are writing to IPTC, you should add -codedcharacterset=utf8 to the command, but only if the pre-existing IPTC doesn't already contain non-UTF8 characters.  See FAQ 10 (https://exiftool.org/faq.html#Q10) for all the details.

- Phil