Fixing Lightroom messed up files with Exiftool (Mac)

Started by pacman, January 29, 2016, 04:00:49 PM

Previous topic - Next topic

pacman

The Mac version of Adobe Lightroom (LR) has a bug (some might call it a feature) which changes and updates the Mac OSX file "Creation date" to the current date/time whenever a keyword is added or changed. Fortunately those files can be fixed by copying the EXIF date back to the Creation date (until a new keyword is written to the file and the procedure has to be repeated).

LR geeks might only rely on the EXIF data for such info but I would like the Creation date to be correct as well. Exiftool to the rescue! I have a couple of script ideas but I need help with the actual commands, so how do I do the following?:

1) Read the date/time from the file's EXIF date tag (if available), then write it to the Mac OSX file "Creation date". If the EXIF date tag isn't available, skip the process for that file and move to the next one in the folder.
I understand there are several EXIF date tags (Date time original, Date time digitized and date time) but I don't know which one would be correct to read from. In LR I see they're all the same date/time for each photo.


2) For scanned images (usually not already having any EXIF date tag):
read the date/time from the Mac OSX file "Creation date" then create and write a new EXIF date tag.
Assuming the Creation date hasn't been mangled by LR already (I plan to use this script as part of the scanning process) the files should then have an EXIF date which can be used to re-write the (mangled by LR) file "Creation date" now and then.
Again I don't know which date tag would be best suited for this.


3) As an extra safety feature, add the date/time to the actual filename. I assume the best method would be to read it from an EXIF tag (and not the file "Creation date") as that would be stable. It should then skip any file without an EXIF date tag (again, not sure which date tag should be used) and name files in the following format: YYYYMMDD-HHMMSS_OriginalFilename.extension


StarGeek

If I recall, Exiftool can't directly change the creation date (FileCreateDate) on the mac (I'm not a mac user).  If you check the note under FileCreateDate, it says

"Note that although ExifTool does not currently support FileCreateDate on other systems, the OS X file creation date may be accessed via the MDItemFSCreationDate tag when the MDItemTags API option is set, and may be written indirectly via the FileModifyDate tag by following these steps: 1. Rewrite the file to set the filesystem creation and modification times to the current time, 2. Set FileModifyDate to the desired creation time, then 3. Restore FileModifyDate to its original value)"

For scanned images, I think the appropriate tag would be CreateDate.  The Exif spec calls this tag DateTimeDigitized.  There's also an XMP:DateTimeDigitized tag but I would give priority to the exif tag.

Hopefully someone more mac aware can help out, since I'm unable to test out the specifics.  There are a few other threads in this forum about Macs and creation date, so you might try searching through those to see if they can help.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

pacman

Quote from: StarGeek on January 29, 2016, 04:22:37 PM
If I recall, Exiftool can't directly change the creation date (FileCreateDate) on the mac (I'm not a mac user).  If you check the note under FileCreateDate, it says

"Note that although ExifTool does not currently support FileCreateDate on other systems, the OS X file creation date may be accessed via the MDItemFSCreationDate tag when the MDItemTags API option is set, and may be written indirectly via the FileModifyDate tag by following these steps: 1. Rewrite the file to set the filesystem creation and modification times to the current time, 2. Set FileModifyDate to the desired creation time, then 3. Restore FileModifyDate to its original value)"

I'm not sure what that means -if there's a way around the limitation or if it just won't overwrite the file "Creation date". If it won't, at least I have a nice drag & drop tool which will: A Better Finder Attributes (using the "Copy EXIF timestamp to Creation date" preset action). Alas it's not scriptable which probably means I can't use it in conjuction with Exiftool so that I can have one script which does it all (IMHO simpler and more user friendly than having to remember several steps to do).


QuoteFor scanned images, I think the appropriate tag would be CreateDate.  The Exif spec calls this tag DateTimeDigitized.  There's also an XMP:DateTimeDigitized tag but I would give priority to the exif tag.

It's a bit confusing since all those three tags have the same time/date in my photos. But I tried out a demo of the Vuescan (available for Mac, Windows and Linux) scanning software (improved scanning software over what comes with most scanners) and it creates what is displayed as "Date Time Digitized" in Lightroom. So I suppose that's the EXIF tag to go for.


QuoteHopefully someone more mac aware can help out, since I'm unable to test out the specifics.  There are a few other threads in this forum about Macs and creation date, so you might try searching through those to see if they can help.

I've done some searching/browsing in the forum but most of it is a bit over my head.
Anyway, as Exiftool doesn't seem to be able to overwrite the OSX file "Creation date" I'll move on to what Exiftool can probably do. Can someone please tell me the commands for doing the following:

a) create a date/time EXIF tag if it isn't already there (typically scanned files) and copy the file Creation date over to it:
- CHECK if the file contains a date/time EXIF tag
- if YES, do nothing with the file (and move on to check the next file)
- if NO, READ the date/time info from the OSX file "Creation date" and WRITE a new EXIF date/time tag with that info

b) add the time/date to the file name:
- CHECK if the file contains a date/time EXIF tag
- if YES, READ the date/time from that tag, then WRITE it to the filename while keeping the existing filename using this format: YYYYMMDD-HHMMSS_originalfilename.extension
- if NO, do nothing with the file (and move on to check the next file)

c) Can Exiftool be used to find all files in a folder/sub-folders which DON'T contain any date/time EXIF tags and display them in the OSX Finder?  That way I can further fix those files with the above procedure (step a). I would usually try this using the OSX Smart search feature with a suitable search criteria, but alas I can't find any EXIF date/time tags for that.


pacman

OK, I've gotten the following command line from someone else which I assumed would read the OSX File Creation date, then create an EXIF date/time tag in the file and writing the same date as in the OSX file Creation date. But it appears to read the OSX file Modification date instead:

exiftool -r '-alldates<FileModifyDate' -keywords+=DateModified  -if '!$datetimeoriginal' . -overwrite_original


I'm guessing tag which needs changing is either FileModifyDate or DateModified (I really don't understand how the command sequence above works). Can someone point med in the right direction please?

StarGeek

Try FileCreateDate instead of FileModifyDate.

I think you mean ModifyDate instead of DateModified, as I can't find a tag with that name and trying to set it produces an error.

To add a tag to a keyword, you need to use +< instead of +=.  The latter directly sets a value instead of copying a tag.  In this case, you would end up with a Keyword of "DateModified"

Also, your command will set the keyword to the previous ModifyDate, not the value you are currently setting.

So try this command:
exiftool -r '-alldates<FileCreateDate' '-keywords+<FileCreateDate' -if '!$datetimeoriginal' . -overwrite_original
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

pacman

#5
Thanks, but unfortunately I got loads of error messages (apparently one for each file) like this when I enter the commands (shown in bold) in the OSX Terminal:

$ cd folder_path_of_images_to_be_fixed
$ exiftool -r '-alldates<FileCreateDate' '-keywords+<FileCreateDate' -if '!$datetimeoriginal' . -overwrite_original
Warning: No writable tags set from ./filename1.jpg
  (same error message repeated here for the rest of the 126 JPG files in the same folder)
    1 directories scanned
    0 image files updated
  126 image files unchanged
$

So I'm a little confused about how this works, and why there's "keyword" mentioned here. Does the whole thing work by creating a NEW file which is a COPY of the original one, but this time with an EXIF date/time tag, so essentially the existing keywords I've added in the past to the file also needs to be copied over? And not as I previously thought; the image file gets MODIFIED by adding the date/time tag to the EXIF header?

Update: I noticed there being a couple of apostrophe characters surrounding some commands not used in the previous example (shown in red below):

exiftool -r '-alldates<FileCreateDate' '-keywords+<FileCreateDate' -if '!$datetimeoriginal' . -overwrite_original

I removed the apostrophes but got another error message instead, so obviously that didn't work either:

-bash: FileCreateDate: No such file or directory


StarGeek

Quote from: pacman on February 01, 2016, 05:52:58 PM
Update: I noticed there being a couple of apostrophe characters surrounding some commands not used in the previous example (shown in red below):

exiftool -r '-alldates<FileCreateDate' '-keywords+<FileCreateDate' -if '!$datetimeoriginal' . -overwrite_original

Yes, they are necessary to prevent the command line from treating it as a file redirection command. 

As for the rest, I'm not sure what is the proper tag for you to use.  Looking through the forum, FileCreateDate support on the Mac seems questionable, hopefully someone with Mac experience can weigh in.

Try using this command to see if you can see the file tag that has the proper date that you want to copy.
exiftool -system:time:all -s
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

pacman

This is strange, because I don't get the file Creation date at all by issuing that command:

$ exiftool -system:time:all -s path/filename.jpg
FileModifyDate                  : 2015:06:15 23:38:16+02:00
FileAccessDate                  : 2016:02:02 06:56:16+01:00
FileInodeChangeDate        : 2016:02:02 06:55:54+01:00
$

... while doing a File-Get info in the OSX desktop Finder for the same file it says the file was created on March 29th 2015 (seen screenshot attachement). This is the date I want to transfer over to the newly created date/time EXIF tag.

StarGeek

I knew I had read something recent about Macs and creation date, but I only just found the thread.  Previously Phil didn't have a way that he could get Exiftool to read a Macs FileCreateDate but he recently figured out a way.

Make sure you're running at least ExifTool v. 10.09  and try using MDItemContentCreationDate instead of FileCreateDate and see if that does what you want.  Test it out first before running on a whole directory.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

pacman

I've got version 10.10 installed.
If I've understood the changes correctly, the following did seem to work according to a quick test:

$ exiftool -r '-alldates<MDItemContentCreationDate' '-keywords+<MDItemContentCreationDate' -if '!$datetimeoriginal' . -overwrite_original
Warning: [minor] Creating non-standard EXIF in PNG - ./non-exif_CCF_2.png
Warning: [minor] Creating non-standard EXIF in PNG - ./Screen Shot 2016-01-18 at 11.47.35.png
    1 directories scanned
    6 image files updated
$

I'm not sure what is meant by "creating non-standard EXIF in PNG" -aren't PNG files supposed to have EXIF data in them or is it the command line above? In any case they didn't get any EXIF date/time tags. The other test files are JPG and TIFF which all seemed to work.
I also noticed that the time in the modified file is exactly 1 hour behind the original file.

Someone in a different forum suggested the following usage of exiftool to create

$ exiftool -r '-alldates<FileModifyDate' -keywords+=DateModified  -if '!$datetimeoriginal' .
Warning: [minor] Creating non-standard EXIF in PNG - ./non-exif_CCF_2.png
Warning: [minor] Creating non-standard EXIF in PNG - ./Screen Shot 2016-01-18 at 11.47.35.png
    1 directories scanned
    6 image files updated
$ exiftool -delete_original -r .
    1 directories scanned
    6 image files found
    6 originals will be deleted!  Are you sure [y/n]? y
    6 original files deleted
$ exiftool -restore_original .
    1 directories scanned
    6 image files found
    0 original files found
$

I assume this is more or less the same thing, but here the date (compared to the original file's Creation date) is correct. Again the PNG files aren't getting any date/time EXIF tags though.

StarGeek

Quote from: pacman on February 02, 2016, 05:29:04 PM
I'm not sure what is meant by "creating non-standard EXIF in PNG" -aren't PNG files supposed to have EXIF data in them or is it the command line above? In any case they didn't get any EXIF date/time tags.

No, the PNG spec does not support EXIF data.  Exiftool (and Exiv2, I think) support a non-standard work around that allows adding EXIF data to PNGs.  Other programs are unlikely to be able to read EXIF data added by Exiftool to a PNG file.

QuoteI also noticed that the time in the modified file is exactly 1 hour behind the original file.
No idea what might be happening there.  You can try adding the -P option, but again, I'm too ignorant of the Mac file system.  You also might try -overwrite_original_in_place.

QuoteSomeone in a different forum suggested the following usage of exiftool to create
<snip commands>

You can try FileModifyDate and see if that works better for you.  As I said before, I can find no data on a DateModified tag, which is why I suggested ModifyDate.

The second command just deletes any backup files that are created when you run the command.  You can avoid the creation of the backup files with -overwrite_original or -overwrite_original_in_place.  The third command does nothing, as the backup files were deleted with the second command, note the "0 original files found" response.  Neither of the last two commands will do anything to change the dates of the files.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

pacman

#11
I've tried lots of things now but the date differs (between the OSX file and EXIF dates) no matter what.
Could it have something to do with time zone settings, on my Mac or in the camera?

Actually I noticed that with some files the OSX file "Creation date" differs with -1 hour in the EXIF while others show -2 hours in the EXIF. It occured to me that it could have to do with the timezone, but the majority of images I'm testing with are scanned images, and of course there's no clock to adjust. And in my cameras I believe the time is set without any relation to another (i.e. timezone shift), so that doesn't make sense either.
I also tried scanning a document, set the Mac clock to a different timezone 7 hours ahead, scanned a new image, then set the time back on the Mac. I expected the OSX Creation dates to be vastly different but in fact the hours were exactly the same! Apparently the actual time isn't imprinted in the file but a time relative to the Mac's clock. I'm confused....  ???
Perhaps exiftool is doing it correctly and it's really an OS thing. Still I seem to recall that I did manage to create an exif date with it which gave the file the correct date (i.e. same as the OSX file date) but some I had some other issues with it (didn't copy the correct file date or something...).

Time issues aside, I understand that in most of the examples the OSX file modification date has been used as the date source and not the OSX file creation date. You mentioned some issues with the latter on this particular computer platform/OS, but I think I really should use the creation date which would be the date I scanned the image. Usually both creation and modification dates are the same, but I have seen instances where they're not, so I used "A Better Finder Attributes" to change the Creation and Modification dates, then see which date exiftool had used after creating the EXIF date/time tag and transferring the file date over to it. This is the command I ended up with which took the OSX file Creation date into the newly created EXIF date/time tag:

$ exiftool -r '-alldates<MDItemContentCreationDate' -if '!$datetimeoriginal' . -overwrite_original_in_place

I did get an error concerning the PNG file I had in that same folder. Upon issuing the above command it says:

Warning: [minor] Ignored empty rdf:Seq list for exif:ISOSpeedRatings - ./testimage.png


That PNG was actually a regular JPG from a camera (meaning it had all the necessary EXIF date tags to begin with) opened in OSX's Preview app, then "Save as" selecting the PNG format. Upon reopening the file I can clearly see the EXIF date tags present, like with the JPG file. And Lightroom also recognized the date, so it appears EXIF tags can be saved in them. The question of course is how to do with with PNG files not already having the EXIF date tag to start with.

UPDATE: I just compared the EXIF and file dates of some regular camera JPG images and they were consistent with the OSX file Creation and Modification dates, so it appears it's not an OS thing as far as I can see.



Phil Harvey

#12
Sorry for the delay in responding.

Unfortunately the OSX MDItemFSCreationDate is in UTC.  For example:

> exiftool a.jpg -MDItemFSCreationDate
MD Item FS Creation Date        : 2016:02:05 16:34:27+0000


However, EXIF doesn't support a time zone.  I'm not sure exactly what to suggest here.  Perhaps ExifTool should be adjusting these MDItem date/time values to local time?

- Phil

Edit:  I think I have convinced myself that local times would be more useful, so ExifTool 10.11 will convert UTC MDItem date/time values to local time, like this:

> exiftool a.jpg -MDItemFSCreationDate
MD Item FS Creation Date        : 2016:02:05 11:34:27-05:00
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

pacman

Thanks for looking into this Phil.
Will that Exiftool update be available soon, and would converting the files with it enable the time to display correctly (i.e. same as the OSX file Creation date) within software such as Lightroom, in the same way as digital camera files have a consistency between the EXIF date tags and the OSX file Creation date?

Phil Harvey

My hope is that it will.

The new version is probably about a week away.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).