One to rule them all - Big Photo Metadata Tag and Date cleanup

[Kugelblitz]

Hello,
Thank you for the exiftool. And the amazing support here! You guys are awesome!

I have about 200.000+ Images that I like to clean up the Tags / Keywords and the Date.
wonder if I can create a "if this to that" kind of exif command to cleanup all images.

Have uploaded a folder with sample images to my dropbox https://www.dropbox.com/sh/7o8n9zc2ajizn64/AAB_P-iS-65Y8cpuJ3GmnR3Aa?dl=0


- Fix encoding (red circels)
- get rid of to many keywords (blue crossed through)
- remove tags geo:lat= and geo:lon= (yellow highlight)
- equalise keyword separators (yellow highlighted "," and ";")
- change Filedate to Takendate (yellow highlight)

1st - MAKE THEM EQUAL

ENCODING
Some of the Metadata is encoded in different styles. Like "IPTC Encoding - Lation I (ANSI) (SP1252)"
I like to get them all to be converted to UTF-8

KEYWORD-SEPERATORS
I noticed the Tags (xmp-Subject / IPTC:Keywords) are seperated by differnet Seperators
sometimes comma "," is used sometimes semicolon ";" and sometimes even asterisk "*" 
I hope there is a defined standard for Separators that most programs understand and I love to set them to this "standard separator".
So Keywords shop up like "AUS,Australia,Sydney,AUS;Australia,Sydney"

I have tried to set it to a semicolon with this command but it did not always work.

Code Select Expand

-Sep ", " "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}"
I end up having all keywords separated with semicolons and then one block of the same keywords separated by comma too.
Wonder if that is the case cause it would create duplicate keywords?



2nd - REMOVE DUPLICATE KEYWORDS
I like to remove all duplicate keywords too.
Have found this command

Code Select Expand

-sep "##" "-keywords<${keywords;NoDups}"



3rd - GEOSETTER - LOCATION DATA
In the past I had checked to add the Location Data to the Keywords in Geosetter. Which was not a good idea.
For that I had already opend a Topic here "The "exiftool" Application / In need of help to fix some Errors I got processing all files on my Photo drive. https://exiftool.org/forum/index.php/topic,9288.msg47994.html#msg47994"
You helped me already with removing "geo:lat=******" und "geo:lon=******" Keywords added by geosetter with this command

Code Select Expand

-sep xxx "-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" "-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}"

That cleaned up the Tags quite a bit.
I love to take it a step further.

Geosetter has also added the Country Code, Country Name, Region/State, Location and City to the Tags/Keywords.
I think the County and City as Keywords is enough and I like to get rid of the keywords for the Country Code and Location
Because this data is also stored in the IPTC and XMP
iptc:Country-PrimaryLocationCode / Xmp:Country Code (AUS)
iptc:Country-PrimaryLocationName / Xmp:Country (Australia)
iptc:Province-State / xmp:State (New South Wales)
iptc:Sub-location / Xmp:Location (Newtown)
iptc:City / xmp:City (Sydney)

Is there any way to remove the Location Names from the Tags/keywords if the information is already in the right place in the Metadata. 

read the value of iptc:Country-PrimaryLocationCode and remove it from the keywords
read the value of iptc:Province-State and remove it from the keywords
read the value of iptc:Sub-location and remove it from the keywords
read the value of xmp:Country Code and remove it from the keywords
read the value of xmp:State and remove it from the keywords
read the value of xmp:Location and remove it from the keywords

EDIT ONLY IMAGES NO VIDEOS
I use this command to edit only the Image files and not the video Files.

Code Select Expand

-ext jpg -ext png -ext cr2 -ext dng -ext nef -ext nrw   --ext mov --ext avi --ext m4v --ext mp4

4th - FIX DATE
Picasa wrote the Filedate from the last edited date. I use this command to fix that.

Code Select Expand

"-DateTimeOriginal<CreateDate" "-DateTimeOriginal<CreateDate" "-FileCreateDate<CreateDate" "-FileModifyDate<CreateDate"

But some files like Scaned Images, Screenshots and so on do not have a Taken date.
Most of my Folders have this sceme "YYYY-MM-DD Location Event" 2018-10-24 Sydney beachday
I use this command to get the date from the folder and set the time to 12:00:00 o'clock so I know it is modified "by hand"

Code Select Expand

-if 'not $CreateDate' '-CreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? "$1 12:00:00" : undef}'

5th some ADDITIONAL COMMANDS THAT I USE

do not create duplicates by overwriting the original file.

Code Select Expand

-overwrite_original

FORCE WRITE

Code Select Expand

-forcewrite=exif -m

WORK IN THIS FOLDER AND ALL SUBFOLDERS

Code Select Expand

-r

WRITE STATISTIC AS A TXT FILE

Code Select Expand

1>processed_files_list-2018-10-23.txt

WRITE A ERROR LOG AS A TXT FILE

Code Select Expand

2>error_log_2018-10-23.txt


Thank you very much

[Phil Harvey]

ENCODING
Some of the Metadata is encoded in different styles. Like "IPTC Encoding - Lation I (ANSI) (SP1252)"
I like to get them all to be converted to UTF-8

This would likely have to be done by hand since it is difficult programmatically to tell what the encoding should be.

KEYWORD-SEPERATORS
I noticed the Tags (xmp-Subject / IPTC:Keywords) are seperated by differnet Seperators
sometimes comma "," is used sometimes semicolon ";" and sometimes even asterisk "*" 

This is a problem.  XMP:Subject and IPTC:Keywords should not contain ANY separators.  If they do, they were written incorrectly.  The should be stored as individual items.  See FAQ 17.

Geosetter has also added the Country Code, Country Name, Region/State, Location and City to the Tags/Keywords.
I think the County and City as Keywords is enough and I like to get rid of the keywords for the Country Code and Location
Because this data is also stored in the IPTC and XMP

Try this command:

read the value of iptc:Country-PrimaryLocationCode and remove it from the keywords
read the value of iptc:Province-State and remove it from the keywords
[...]

exiftool "-keywords-<iptc:Country-PrimaryLocationCode" "-keywords-<iptc:Province-State" ...

- Phil

[Kugelblitz]

Hello Phil,
thank you for your time and helpful reply.

Think the best way is to break it down and tackle one task at a time.

I have tried to search and remove the CountryCode, Province-State, and Sublocation with this command.

"-keywords-<iptc:Country-PrimaryLocationCode"
"-keywords-<iptc:Province-State"
"-keywords-<iptc:Sub-location"
"-keywords-<xmp:CountryCode"
"-keywords-<xmp:State"
"-keywords-<xmp:Location"
"-subject-<iptc:Country-PrimaryLocationCode"
"-subject-<iptc:Province-State"
"-subject-<iptc:Sub-location"
"-subject-<xmp:CountryCode"
"-subject-<xmp:State"
"-subject-<xmp:Location"

Code Select Expand

exiftool "-keywords-<iptc:Country-PrimaryLocationCode" "-keywords-<iptc:Province-State" "-keywords-<iptc:Sub-location" "-keywords-<xmp:CountryCode" "-keywords-<xmp:State" "-keywords-<xmp:Location" "-subject-<iptc:Country-PrimaryLocationCode" "-subject-<iptc:Province-State" "-subject-<iptc:Sub-location" "-subject-<xmp:CountryCode" "-subject-<xmp:State" "-subject-<xmp:Location"  -overwrite_original -r E:\Tests\2018-10-23\test 1>processed_files_list-2018-10-23.txt 2>error_log_2018-10-23.txt

But it did not remove anything from the Tags.
Not with the Files that have the Tags listed correctly (one by one) and not with the Files that have the Tags not split correctly.

What have I missed?

[Kugelblitz]

The Keyword separator sort of works.

Code Select Expand

exiftool -Sep ", " "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}" -overwrite_original -forcewrite=exif -m -r E:\Tests\2018-10-23\test 1>processed_files_list-2018-10-23.txt 2>error_log_2018-10-23.txt

BEFORE


AFTER


The Keywords are now correctly separated and listed underneath each other but one line with all the Keywords still exists (BEL, Belgien, Oostende, Turkijen, Vlaanderen).
How can I remove this "original" one before they have been separated ?

[Phil Harvey]

This would be less confusing if you used ExifTool to read the metadata.  I don't know what the other software is showing, which makes it hard to help.

But it did not remove anything from the Tags.
Not with the Files that have the Tags listed correctly (one by one) and not with the Files that have the Tags not split correctly.

What have I missed?

For this to work, the list items must be written separately and match exactly the value you are trying to remove.  To check if they are written separately, try extracting with exiftool -sep "XX" and see if they are separated by "XX".

How can I remove this "original" one before they have been separated ?

Use ExifTool to see where the "original" value is stored, and write that tag.

- Phil

[Kugelblitz]

Hello Phil,
thank you for your reply.

I had used geosetter to make the screenshots - Did not know how to extract the metadata with exiftool but I have looked it up.

exiftool shows this:

E:\Tests\2018-10-23\test>exiftool IMG_0800.JPG

Code Select Expand

ExifTool Version Number         : 10.96
File Name                       : IMG_0800.JPG
Directory                       : .
File Size                       : 2.1 MB
File Modification Date/Time     : 2016:07:13 13:20:40+02:00
File Access Date/Time           : 2018:10:25 09:07:44+02:00
File Creation Date/Time         : 2016:07:13 13:20:40+02:00
File Permissions                : rw-rw-rw-
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 6s
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 9.3.2
Modify Date                     : 2016:07:13 13:20:40
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/2137
F Number                        : 2.2
Exposure Program                : Program AE
ISO                             : 25
Time Zone Offset                : 2
Exif Version                    : 0221
Date/Time Original              : 2016:07:13 13:20:40
Create Date                     : 2016:07:13 13:20:40
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/2137
Aperture Value                  : 2.2
Metering Mode                   : Average
Flash                           : Auto, Did not fire
Focal Length                    : 4.2 mm
Subject Area                    : 839 1292 1009 1013
Sub Sec Time Original           : 364
Sub Sec Time Digitized          : 364
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 4032
Exif Image Height               : 3024
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Digital Zoom Ratio              : 1.340425532
Focal Length In 35mm Format     : 39 mm
Scene Capture Type              : Standard
Lens Info                       : 4.15mm f/2.2
Lens Make                       : Apple
Lens Model                      : iPhone 6s back camera 4.15mm f/2.2
GPS Version ID                  : 2.2.0.0
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 11:20:40
GPS Speed Ref                   : km/h
GPS Speed                       : 0
GPS Img Direction Ref           : True North
GPS Img Direction               : 317.3
GPS Map Datum                   : WGS-84
GPS Dest Bearing Ref            : True North
GPS Dest Bearing                : 317.3037975
GPS Date Stamp                  : 2016:07:13
GPS Horizontal Positioning Error: 5 m
Compression                     : JPEG (old-style)
Thumbnail Offset                : 1200
Thumbnail Length                : 9616
XMP Toolkit                     : Image::ExifTool 10.96
Country Code                    : BEL
Location                        : Turkijen
Subject                         : BEL, Belgien, Oostende, Turkijen, Vlaanderen
Date/Time Digitized             : 2016:07:13 13:20:40+02:00
Country                         : Belgien
State                           : Vlaanderen
Current IPTC Digest             : 8fbb56800a89dc3d627f9fc609659179
Application Record Version      : 4
Date Created                    : 2016:07:13
Time Created                    : 13:20:40+02:00
Keywords                        : BEL, Belgien, Oostende, Turkijen, Vlaanderen
City                            : Oostende
Sub-location                    : Turkijen
Province-State                  : Vlaanderen
Country-Primary Location Code   : BEL
Country-Primary Location Name   : Belgien
IPTC Digest                     : b443520a10119da99c2550175e6d0efb
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.2
Date/Time Created               : 2016:07:13 13:20:40+02:00
GPS Altitude                    : 11.9 m Above Sea Level
GPS Date/Time                   : 2016:07:13 11:20:40Z
GPS Latitude                    : 51 deg 14' 8.16" N
GPS Latitude Ref                : North
GPS Longitude                   : 2 deg 55' 1.25" E
GPS Longitude Ref               : East
GPS Position                    : 51 deg 14' 8.16" N, 2 deg 55' 1.25" E
Image Size                      : 4032x3024
Megapixels                      : 12.2
Scale Factor To 35 mm Equivalent: 9.4
Shutter Speed                   : 1/2137
Create Date                     : 2016:07:13 13:20:40.364
Date/Time Original              : 2016:07:13 13:20:40.364
Thumbnail Image                 : (Binary data 9616 bytes, use -b option to extract)
Circle Of Confusion             : 0.003 mm
Field Of View                   : 49.6 deg
Focal Length                    : 4.2 mm (35 mm equivalent: 39.0 mm)
Hyperfocal Distance             : 2.45 m
Light Value                     : 15.3

E:\Tests\2018-10-23\test>exiftool -sep "XX" E:\Tests\2018-10-23\test\IMG_0800.JPG

Code Select Expand

ExifTool Version Number         : 10.96
File Name                       : IMG_0800.JPG
Directory                       : E:/Tests/2018-10-23/test
File Size                       : 2.1 MB
File Modification Date/Time     : 2018:10:25 09:14:58+02:00
File Access Date/Time           : 2018:10:25 09:14:58+02:00
File Creation Date/Time         : 2016:07:13 13:20:40+02:00
File Permissions                : rw-rw-rw-
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 6s
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 9.3.2
Modify Date                     : 2016:07:13 13:20:40
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/2137
F Number                        : 2.2
Exposure Program                : Program AE
ISO                             : 25
Time Zone Offset                : 2
Exif Version                    : 0221
Date/Time Original              : 2016:07:13 13:20:40
Create Date                     : 2016:07:13 13:20:40
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/2137
Aperture Value                  : 2.2
Metering Mode                   : Average
Flash                           : Auto, Did not fire
Focal Length                    : 4.2 mm
Subject Area                    : 839 1292 1009 1013
Sub Sec Time Original           : 364
Sub Sec Time Digitized          : 364
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 4032
Exif Image Height               : 3024
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Digital Zoom Ratio              : 1.340425532
Focal Length In 35mm Format     : 39 mm
Scene Capture Type              : Standard
Lens Info                       : 4.15mm f/2.2
Lens Make                       : Apple
Lens Model                      : iPhone 6s back camera 4.15mm f/2.2
GPS Version ID                  : 2.2.0.0
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 11:20:40
GPS Speed Ref                   : km/h
GPS Speed                       : 0
GPS Img Direction Ref           : True North
GPS Img Direction               : 317.3
GPS Map Datum                   : WGS-84
GPS Dest Bearing Ref            : True North
GPS Dest Bearing                : 317.3037975
GPS Date Stamp                  : 2016:07:13
GPS Horizontal Positioning Error: 5 m
XP Keywords                     :
Compression                     : JPEG (old-style)
Thumbnail Offset                : 1212
Thumbnail Length                : 9616
XMP Toolkit                     : Image::ExifTool 10.96
Country Code                    : BEL
Location                        : Turkijen
Last Keyword XMP                :
Subject                         : BELXXBelgienXXOostendeXXTurkijenXXVlaanderen
Date/Time Digitized             : 2016:07:13 13:20:40+02:00
Country                         : Belgien
State                           : Vlaanderen
Current IPTC Digest             : 8fbb56800a89dc3d627f9fc609659179
Application Record Version      : 4
Date Created                    : 2016:07:13
Time Created                    : 13:20:40+02:00
Keywords                        : BEL, Belgien, Oostende, Turkijen, Vlaanderen
City                            : Oostende
Sub-location                    : Turkijen
Province-State                  : Vlaanderen
Country-Primary Location Code   : BEL
Country-Primary Location Name   : Belgien
IPTC Digest                     : b443520a10119da99c2550175e6d0efb
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.2
Date/Time Created               : 2016:07:13 13:20:40+02:00
GPS Altitude                    : 11.9 m Above Sea Level
GPS Date/Time                   : 2016:07:13 11:20:40Z
GPS Latitude                    : 51 deg 14' 8.16" N
GPS Latitude Ref                : North
GPS Longitude                   : 2 deg 55' 1.25" E
GPS Longitude Ref               : East
GPS Position                    : 51 deg 14' 8.16" N, 2 deg 55' 1.25" E
Image Size                      : 4032x3024
Megapixels                      : 12.2
Scale Factor To 35 mm Equivalent: 9.4
Shutter Speed                   : 1/2137
Create Date                     : 2016:07:13 13:20:40.364
Date/Time Original              : 2016:07:13 13:20:40.364
Thumbnail Image                 : (Binary data 9616 bytes, use -b option to extract)
Circle Of Confusion             : 0.003 mm
Field Of View                   : 49.6 deg
Focal Length                    : 4.2 mm (35 mm equivalent: 39.0 mm)
Hyperfocal Distance             : 2.45 m
Light Value                     : 15.3

Looks like it separates the items in the Subject line but not in the Keyword line.

[Phil Harvey]

OK.  So you need to add Keywords to the command where you were separating the keywords.  You were just doing XPKeywords and Subject.

- Phil

[Kugelblitz]

Hello Phil,
thank you for the Hint. As helpful as always.

I used the following command and now the Keywords are written separately.

Code Select Expand

exiftool -Sep ", " "-Keywords<${Keywords;s/\;/\,/g}" "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}" -overwrite_original E:\Tests\2018-10-23\test\IMG_0800.JPG

I have checked if they are written separately with the exiftool -sep "XX" command and they are separated by "XX".

Code Select Expand

E:\Tests\2018-10-23\test>exiftool -sep "XX" E:\Tests\2018-10-23\test\IMG_0800.JPG
ExifTool Version Number         : 10.96
File Name                       : IMG_0800.JPG
Directory                       : E:/Tests/2018-10-23/test
File Size                       : 2.1 MB
File Modification Date/Time     : 2018:10:25 14:17:48+02:00
File Access Date/Time           : 2018:10:25 14:17:48+02:00
File Creation Date/Time         : 2016:07:13 13:20:40+02:00
File Permissions                : rw-rw-rw-
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 6s
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 9.3.2
Modify Date                     : 2016:07:13 13:20:40
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/2137
F Number                        : 2.2
Exposure Program                : Program AE
ISO                             : 25
Time Zone Offset                : 2
Exif Version                    : 0221
Date/Time Original              : 2016:07:13 13:20:40
Create Date                     : 2016:07:13 13:20:40
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/2137
Aperture Value                  : 2.2
Metering Mode                   : Average
Flash                           : Auto, Did not fire
Focal Length                    : 4.2 mm
Subject Area                    : 839 1292 1009 1013
Sub Sec Time Original           : 364
Sub Sec Time Digitized          : 364
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 4032
Exif Image Height               : 3024
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Digital Zoom Ratio              : 1.340425532
Focal Length In 35mm Format     : 39 mm
Scene Capture Type              : Standard
Lens Info                       : 4.15mm f/2.2
Lens Make                       : Apple
Lens Model                      : iPhone 6s back camera 4.15mm f/2.2
GPS Version ID                  : 2.2.0.0
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 11:20:40
GPS Speed Ref                   : km/h
GPS Speed                       : 0
GPS Img Direction Ref           : True North
GPS Img Direction               : 317.3
GPS Map Datum                   : WGS-84
GPS Dest Bearing Ref            : True North
GPS Dest Bearing                : 317.3037975
GPS Date Stamp                  : 2016:07:13
GPS Horizontal Positioning Error: 5 m
Compression                     : JPEG (old-style)
Thumbnail Offset                : 1200
Thumbnail Length                : 9616
XMP Toolkit                     : Image::ExifTool 10.96
Country Code                    : BEL
Location                        : Turkijen
Subject                         : BELXXBelgienXXOostendeXXTurkijenXXVlaanderen
Date/Time Digitized             : 2016:07:13 13:20:40+02:00
Country                         : Belgien
State                           : Vlaanderen
Current IPTC Digest             : 072a5008353712a9b28715e3fff5fc36
Application Record Version      : 4
Date Created                    : 2016:07:13
Time Created                    : 13:20:40+02:00
Keywords                        : BELXXBelgienXXOostendeXXTurkijenXXVlaanderen
City                            : Oostende
Sub-location                    : Turkijen
Province-State                  : Vlaanderen
Country-Primary Location Code   : BEL
Country-Primary Location Name   : Belgien
IPTC Digest                     : b443520a10119da99c2550175e6d0efb
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.2
Date/Time Created               : 2016:07:13 13:20:40+02:00
GPS Altitude                    : 11.9 m Above Sea Level
GPS Date/Time                   : 2016:07:13 11:20:40Z
GPS Latitude                    : 51 deg 14' 8.16" N
GPS Latitude Ref                : North
GPS Longitude                   : 2 deg 55' 1.25" E
GPS Longitude Ref               : East
GPS Position                    : 51 deg 14' 8.16" N, 2 deg 55' 1.25" E
Image Size                      : 4032x3024
Megapixels                      : 12.2
Scale Factor To 35 mm Equivalent: 9.4
Shutter Speed                   : 1/2137
Create Date                     : 2016:07:13 13:20:40.364
Date/Time Original              : 2016:07:13 13:20:40.364
Thumbnail Image                 : (Binary data 9616 bytes, use -b option to extract)
Circle Of Confusion             : 0.003 mm
Field Of View                   : 49.6 deg
Focal Length                    : 4.2 mm (35 mm equivalent: 39.0 mm)
Hyperfocal Distance             : 2.45 m
Light Value                     : 15.3

so what is "the best" character to separate Keywords?
I personal do not like that comma "," semicolons ";" or tabs are used to split information in a database or cvs txt document - for the reason that these are very common punctuation marks that are easily used in the content as well. Unfortunately I do not know if there is a dedicated character or combination of charters to split Data in a file txt document. 


So now that the Keywords/Tags are correctly separated I have tried to remove them from "Subject" and "Keywords / XPKeywords" with the following command.

Code Select Expand

exiftool "-keywords-<iptc:Country-PrimaryLocationCode" "-keywords-<iptc:Province-State" "-keywords-<iptc:Sub-location" "-keywords-<xmp:CountryCode" "-keywords-<xmp:State" "-keywords-<xmp:Location" "-XPKeywords-<xmp:CountryCode" "-XPKeywords-<xmp:State" "-XPKeywords-<xmp:Location" "-XMP:Subject-<xmp:CountryCode" "-XMP:Subject-<xmp:State"  "-XMP:Subject-<xmp:Location" "-XMP:LastKeywordXMP-<xmp:CountryCode" "-XMP:LastKeywordXMP-<xmp:State" "-XMP:LastKeywordXMP-<xmp:Location" -overwrite_original E:\Tests\2018-10-23\test\IMG_0800.JPG

But that did not work. The Tags are still there.

Code Select Expand

E:\Tests\2018-10-23\test>exiftool E:\Tests\2018-10-23\test\IMG_0800.JPG
ExifTool Version Number         : 10.96
File Name                       : IMG_0800.JPG
Directory                       : E:/Tests/2018-10-23/test
File Size                       : 2.1 MB
File Modification Date/Time     : 2018:10:25 14:31:22+02:00
File Access Date/Time           : 2018:10:25 14:31:22+02:00
File Creation Date/Time         : 2016:07:13 13:20:40+02:00
File Permissions                : rw-rw-rw-
File Type                       : JPEG
File Type Extension             : jpg
MIME Type                       : image/jpeg
Exif Byte Order                 : Big-endian (Motorola, MM)
Make                            : Apple
Camera Model Name               : iPhone 6s
Orientation                     : Horizontal (normal)
X Resolution                    : 72
Y Resolution                    : 72
Resolution Unit                 : inches
Software                        : 9.3.2
Modify Date                     : 2016:07:13 13:20:40
Y Cb Cr Positioning             : Centered
Exposure Time                   : 1/2137
F Number                        : 2.2
Exposure Program                : Program AE
ISO                             : 25
Time Zone Offset                : 2
Exif Version                    : 0221
Date/Time Original              : 2016:07:13 13:20:40
Create Date                     : 2016:07:13 13:20:40
Components Configuration        : Y, Cb, Cr, -
Shutter Speed Value             : 1/2137
Aperture Value                  : 2.2
Metering Mode                   : Average
Flash                           : Auto, Did not fire
Focal Length                    : 4.2 mm
Subject Area                    : 839 1292 1009 1013
Sub Sec Time Original           : 364
Sub Sec Time Digitized          : 364
Flashpix Version                : 0100
Color Space                     : sRGB
Exif Image Width                : 4032
Exif Image Height               : 3024
Sensing Method                  : One-chip color area
Scene Type                      : Directly photographed
Exposure Mode                   : Auto
White Balance                   : Auto
Digital Zoom Ratio              : 1.340425532
Focal Length In 35mm Format     : 39 mm
Scene Capture Type              : Standard
Lens Info                       : 4.15mm f/2.2
Lens Make                       : Apple
Lens Model                      : iPhone 6s back camera 4.15mm f/2.2
GPS Version ID                  : 2.2.0.0
GPS Altitude Ref                : Above Sea Level
GPS Time Stamp                  : 11:20:40
GPS Speed Ref                   : km/h
GPS Speed                       : 0
GPS Img Direction Ref           : True North
GPS Img Direction               : 317.3
GPS Map Datum                   : WGS-84
GPS Dest Bearing Ref            : True North
GPS Dest Bearing                : 317.3037975
GPS Date Stamp                  : 2016:07:13
GPS Horizontal Positioning Error: 5 m
Compression                     : JPEG (old-style)
Thumbnail Offset                : 1200
Thumbnail Length                : 9616
XMP Toolkit                     : Image::ExifTool 10.96
Country Code                    : BEL
Location                        : Turkijen
Subject                         : BEL, Belgien, Oostende, Vlaanderen
Date/Time Digitized             : 2016:07:13 13:20:40+02:00
Country                         : Belgien
State                           : Vlaanderen
Current IPTC Digest             : eac8ac5b2471b03bf77f2f32cca4af0b
Application Record Version      : 4
Date Created                    : 2016:07:13
Time Created                    : 13:20:40+02:00
Keywords                        : BEL, Belgien, Oostende, Vlaanderen
City                            : Oostende
Sub-location                    : Turkijen
Province-State                  : Vlaanderen
Country-Primary Location Code   : BEL
Country-Primary Location Name   : Belgien
IPTC Digest                     : b443520a10119da99c2550175e6d0efb
Image Width                     : 4032
Image Height                    : 3024
Encoding Process                : Baseline DCT, Huffman coding
Bits Per Sample                 : 8
Color Components                : 3
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Aperture                        : 2.2
Date/Time Created               : 2016:07:13 13:20:40+02:00
GPS Altitude                    : 11.9 m Above Sea Level
GPS Date/Time                   : 2016:07:13 11:20:40Z
GPS Latitude                    : 51 deg 14' 8.16" N
GPS Latitude Ref                : North
GPS Longitude                   : 2 deg 55' 1.25" E
GPS Longitude Ref               : East
GPS Position                    : 51 deg 14' 8.16" N, 2 deg 55' 1.25" E
Image Size                      : 4032x3024
Megapixels                      : 12.2
Scale Factor To 35 mm Equivalent: 9.4
Shutter Speed                   : 1/2137
Create Date                     : 2016:07:13 13:20:40.364
Date/Time Original              : 2016:07:13 13:20:40.364
Thumbnail Image                 : (Binary data 9616 bytes, use -b option to extract)
Circle Of Confusion             : 0.003 mm
Field Of View                   : 49.6 deg
Focal Length                    : 4.2 mm (35 mm equivalent: 39.0 mm)
Hyperfocal Distance             : 2.45 m
Light Value                     : 15.3

[Phil Harvey]

Here is a test I ran here.  This should work for you too (although I might recommend updating to the most recent version of ExifTool):

Code Select Expand

> exiftool a.jpg -keywords="BEL, Belgien, Oostende, Vlaanderen" -sep ", "
    1 image files updated
> exiftool a.jpg -Country-PrimaryLocationCode=BEL
    1 image files updated
> exiftool a.jpg -keywords
Keywords                        : BEL, Belgien, Oostende, Vlaanderen
> exiftool a.jpg "-keywords-<iptc:Country-PrimaryLocationCode"
    1 image files updated
> exiftool a.jpg -keywords
Keywords                        : Belgien, Oostende, Vlaanderen

- Phil

[Kugelblitz]

Hello Phil,
thanks again!

Have updated to the latest version of exiftool from today.

These lines worked individually! Thank you.

Code Select Expand


exiftool IMG_0800.JPG "-keywords-<iptc:Country-PrimaryLocationCode"
exiftool IMG_0800.JPG "-keywords-<iptc:Province-State"
exiftool IMG_0800.JPG "-keywords-<iptc:Sub-location"
exiftool IMG_0800.JPG "-XMP:Subject-<xmp:CountryCode"
exiftool IMG_0800.JPG "-XMP:Subject-<xmp:State"
exiftool IMG_0800.JPG "-XMP:Subject-<xmp:Location"

How can I combine all Commands in one big task?
Or build a batch that runs all the following commands on all Image Files in a folder and all subfolders?

Code Select Expand

-ext jpg -ext png -ext cr2 -ext dng -ext nef -ext nrw   --ext mov --ext avi --ext m4v --ext mp4
-Sep ", " "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g
"-keywords-<iptc:Country-PrimaryLocationCode"
"-keywords-<iptc:Province-State"
"-keywords-<iptc:Sub-location"
"-XMP:Subject-<xmp:CountryCode"
"-XMP:Subject-<xmp:State"
"-XMP:Subject-<xmp:Location"
"-XMP:LastKeywordXMP-<xmp:CountryCode"
"-XMP:LastKeywordXMP-<xmp:State"
"-XMP:LastKeywordXMP-<xmp:Location"
"-keywords-<xmp:CountryCode"
"-keywords-<xmp:State"
"-keywords-<xmp:Location"
"-XPKeywords-<xmp:CountryCode"
"-XPKeywords-<xmp:State"
"-XPKeywords-<xmp:Location"
-sep "##" "-keywords<${keywords;NoDups}"
-sep xxx "-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" "-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" 
"-DateTimeOriginal<CreateDate" "-DateTimeOriginal<CreateDate" "-FileCreateDate<CreateDate" "-FileModifyDate<CreateDate"
-if 'not $CreateDate' '-CreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? "$1 12:00:00" : undef}'
-forcewrite=exif -m
-overwrite_original
-r
1>processed_files_list-2018-10-23.txt
2>error_log_2018-10-23.txt

[Phil Harvey]

These lines worked individually!

You should be able do do these all in one command.

How can I combine all Commands in one big task?

This would one way, but before doing something like this you need to get the constituent commands working properly:

exiftool -if "not $CreateDate" "-CreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? qq($1 12:00:00) : undef}" -execute -Sep ", " "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}" -execute -sep "##" "-keywords<${keywords;NoDups}" -execute "-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" "-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" "-DateTimeOriginal<CreateDate" "-DateTimeOriginal<CreateDate" "-FileCreateDate<CreateDate" "-FileModifyDate<CreateDate" -execute "-keywords-<iptc:Country-PrimaryLocationCode" "-keywords-<iptc:Province-State" "-keywords-<iptc:Sub-location" "-XMP:Subject-<xmp:CountryCode" "-XMP:Subject-<xmp:State" "-XMP:Subject-<xmp:Location" "-XMP:LastKeywordXMP-<xmp:CountryCode" "-XMP:LastKeywordXMP-<xmp:State" "-XMP:LastKeywordXMP-<xmp:Location" "-keywords-<xmp:CountryCode" "-keywords-<xmp:State" "-keywords-<xmp:Location" "-XPKeywords-<xmp:CountryCode" "-XPKeywords-<xmp:State" "-XPKeywords-<xmp:Location" -common_args -r -overwrite_original -m -P -ext jpg -ext png -ext cr2 -ext dng -ext nef -ext nrw DIR

For such a long command, it may be easier to put the arguments in an argfile and use the -@ option:

exiftool -@ a.args -common_args -r -overwrite_original -m -P -ext jpg -ext png -ext cr2 -ext dng -ext nef -ext nrw DIR

with this a.args file:

Code Select Expand

-if
not $CreateDate
-CreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? qq($1 12:00:00) : undef}
-execute
-Sep
,
-XPKeywords<${XPKeywords;s/\;/\,/g}
-XMP:Subject<${XMP:Subject;s/\;/\,/g}
-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}
-execute
-sep
##
-keywords<${keywords;NoDups}
-execute
-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}
-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}
-DateTimeOriginal<CreateDate
-DateTimeOriginal<CreateDate
-FileCreateDate<CreateDate
-FileModifyDate<CreateDate
-execute
-keywords-<iptc:Country-PrimaryLocationCode
-keywords-<iptc:Province-State
-keywords-<iptc:Sub-location
-XMP:Subject-<xmp:CountryCode
-XMP:Subject-<xmp:State
-XMP:Subject-<xmp:Location
-XMP:LastKeywordXMP-<xmp:CountryCode
-XMP:LastKeywordXMP-<xmp:State
-XMP:LastKeywordXMP-<xmp:Location
-keywords-<xmp:CountryCode
-keywords-<xmp:State
-keywords-<xmp:Location
-XPKeywords-<xmp:CountryCode
-XPKeywords-<xmp:State
-XPKeywords-<xmp:Location

- Phil

[Kugelblitz]

Thank God Phil..

I had created a slow batch "FOR /R %A IN (*.jpg *.png *.cr2 *.dng *.nef *.nrw ) DO (exiftool this "%A" & exiftool that "%A" & exiftool thatalso "%A") monster that worked but ran forever in my test directory.

Code Select Expand

exiftool -Sep ", " "-Keywords<${Keywords;s/\;/\,/g}" "-XPKeywords<${XPKeywords;s/\;/\,/g}" "-XMP:Subject<${XMP:Subject;s/\;/\,/g}" "-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}" -overwrite_original E:\Tests\2018-10-23\test\

FOR /R %A IN (*.jpg *.png *.cr2 *.dng *.nef *.nrw ) DO (exiftool.exe "%A" "-keywords-<iptc:Country-PrimaryLocationCode" -overwrite_original & exiftool.exe "%A" "-keywords-<iptc:Province-State" -overwrite_original & exiftool.exe "%A" "-keywords-<iptc:Sub-location" -overwrite_original & exiftool.exe "%A" "-XMP:Subject-<xmp:CountryCode" -overwrite_original & exiftool.exe "%A" "-XMP:Subject-<xmp:State" -overwrite_original & exiftool.exe "%A" "-XMP:Subject-<xmp:Location" -overwrite_original & exiftool.exe "%A" "-XMP:LastKeywordXMP-<xmp:CountryCode" -overwrite_original & exiftool.exe "%A" "-XMP:LastKeywordXMP-<xmp:State" -overwrite_original & exiftool.exe "%A" "-XMP:LastKeywordXMP-<xmp:Location" -overwrite_original & exiftool.exe "%A" "-keywords-<xmp:CountryCode" -overwrite_original & exiftool.exe "%A" "-keywords-<xmp:State" -overwrite_original & exiftool.exe "%A" "-keywords-<xmp:Location" -overwrite_original & exiftool.exe "%A" "-XPKeywords-<xmp:CountryCode"  & exiftool.exe "%A" "-XPKeywords-<xmp:State" -overwrite_original & exiftool.exe "%A" "-XPKeywords-<xmp:Location" -overwrite_original & exiftool "%A" -sep "##" "-keywords<${keywords;NoDups}" -overwrite_original & exiftool "%A" -sep xxx "-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}" "-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}"  -overwrite_original & exiftool "%A" "-DateTimeOriginal<CreateDate" "-DateTimeOriginal<CreateDate" "-FileCreateDate<CreateDate" "-FileModifyDate<CreateDate" -overwrite_original)

I tried your code but it did not work right at first. The line with the "geo:lat" messed up the separators so I have put it before they are sorted in the a.args File. 
Filling in the missing taken date did not work.

Code Select Expand

-if
not $CreateDate
-CreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? qq($1 12:00:00) : undef}
-execute
-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}
-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff)/}
-execute
-sep
##
-keywords<${keywords;NoDups}
-execute
-Sep
,
-Keywords<${Keywords;s/\;/\,/g}
-XPKeywords<${XPKeywords;s/\;/\,/g}
-XMP:Subject<${XMP:Subject;s/\;/\,/g}
-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}
-execute
-keywords-<iptc:Country-PrimaryLocationCode
-keywords-<iptc:Province-State
-keywords-<iptc:Sub-location
-XMP:Subject-<xmp:CountryCode
-XMP:Subject-<xmp:State
-XMP:Subject-<xmp:Location
-XMP:LastKeywordXMP-<xmp:CountryCode
-XMP:LastKeywordXMP-<xmp:State
-XMP:LastKeywordXMP-<xmp:Location
-keywords-<xmp:CountryCode
-keywords-<xmp:State
-keywords-<xmp:Location
-XPKeywords-<xmp:CountryCode
-XPKeywords-<xmp:State
-XPKeywords-<xmp:Location
-execute
-DateTimeOriginal<CreateDate
-DateTimeOriginal<CreateDate
-FileCreateDate<CreateDate
-FileModifyDate<CreateDate

Your code ran in a fraction of the time and is for sure the way better solution!
Thank You very very very very much Phil

[StarGeek]

I hate to point this out after all that work, but XPKeywords is a semicolon separated string, not a list type tag as is Keywords and Subject.  But it is also a pretty useless tag, as only Windows reads it (as far as I've been able to tell) and I'd suggest just removing it altogether.

[Kugelblitz]

Hello Phil,
Hello StarGeek,

thanks for your helpful replies.

Have tested the commands over and over again. For better "debugging" I have spitted the argfile into several files. "w-remove-tags-geolat-geolon.args", "w-separate-tags.args" and "time.args" work.
The args in "remove-geo-name-tags.args" work "individually" as single commands but not together in the args file.

To start it all I run the Command

Code Select Expand

exiftool -@ w-remove-tags-geolat-geolon.args -@ w-separate-tags.args -@ remove-geo-name-tags.args -common_args -r -overwrite_original -forcewrite=exif -m -P -ext jpg -ext png -ext cr2 -ext dng -ext nef -ext nrw "E:\Tests\2018-10-23\2018-10-23 test" 1>processed_files_list-2018-10-23.txt 2>error_log_2018-10-23.txt

With the content of these args Files.

w-remove-tags-geolat-geolon.args - works - removes the geolat and geolong and other unnecessary tags

Code Select Expand

-execute
-subject<${subject@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff|iPhone|iphone)/}
-Keywords<${Keywords@;$_=undef if /^(geo:lat=|geo:lon=|ffffffffffffffff|iPhone|iphone)/}

w-separate-tags.args- works - Splits the line of Tags into individually separated tags.

Code Select Expand

-execute
-Sep
,
-Keywords<${Keywords;s/\;/\,/g}
-XMP:Subject<${XMP:Subject;s/\;/\,/g}
-XMP:LastKeywordXMP<${XMP:LastKeywordXMP;s/\;/\,/g}

remove-geo-name-tags.args - does not work right - shall remove the Geoname "CountryCode, State, Location" from the Tags but they still exist after the process.
I checked the Subject Tags and Keywords are correctly separated.

Code Select Expand

-execute
-XMP:Subject-<xmp:CountryCode
-XMP:Subject-<xmp:State
-XMP:Subject-<xmp:Location
-keywords-<iptc:Country-PrimaryLocationCode
-keywords-<iptc:Province-State
-keywords-<iptc:Sub-location
-keywords-<xmp:CountryCode
-keywords-<xmp:State
-keywords-<xmp:Location
-XMP:LastKeywordXMP-<xmp:CountryCode
-XMP:LastKeywordXMP-<xmp:State
-XMP:LastKeywordXMP-<xmp:Location

It does not work all together in the args file but each line works if I use them separately as a single command like here.

Code Select Expand


exiftool.exe "E:\Tests\2018-10-23\2018-10-23 test\" "-keywords-<xmp:CountryCode" -overwrite_original
exiftool.exe "E:\Tests\2018-10-23\2018-10-23 test\" "-XMP:Subject-<xmp:State" -overwrite_original
exiftool.exe "E:\Tests\2018-10-23\2018-10-23 test\" "-XMP:Subject-<xmp:Location" -overwrite_original

and so on..
Would be great if you give me a hint how that can work in the args file too.

time.args - is not in the command yet cause it is work in progress - the lines work but I am not sure yet what the best order will be. There are so many possible "Date" fields.
Create Date, Date/Time Original, GPS Date/Time , Date/Time Digitized, Modify Date, File Modification Date/Time, File Creation Date/Time
I want to preserve the oldest (original) Time of the File or if not possible get the Date from the Foldername.

So it will be a "if this is empty then use that. And if that is empty too then use this" Solution.
Also need to check if a time equal 12:00:00 because then I have set it manual and put in the "correct estimated date" of the picture.

Code Select Expand

-if
not $gpstimestamp
-gpstimestamp<datetimeoriginal"
-execute
-if
not $FileModifyDate
-FileModifyDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? qq($1 12:00:00) : undef}
-execute
-if
not $FileCreateDate
-FileCreateDate<${directory;$_ = /(\d{4}-\d{2}-\d{2})/ ? qq($1 12:00:00) : undef}
-execute
-if
not $CreateDate
-CreateDate<FileModifyDate
-execute
-if
not $DateTimeOriginal
-DateTimeOriginal<FileModifyDate
-execute
-DateTimeOriginal<CreateDate
-DateTimeOriginal<CreateDate
-FileCreateDate<CreateDate
-FileModifyDate<CreateDate

This tool is so powerful!
Thank you guys.

[Phil Harvey]

Sorry, I don't have time to read this right now.

I was hoping that you would have all the tools you needed to venture out on your own by now.  But if you still need help I'll be back tomorrow and check in on how things are going.  If you could keep your posts a bit shorter it would help.

- Phil