Fixing ALL date and time metadata for recovered files?

[tenthumbs]

I just came back from a trip (seems like all my date/time issues start with that phrase...) and had a minor disaster with some (fortunately not all) of my photos.
Now I need to extract the date/time info from exif data for each file, and write that to ALL the date/time fields for a set of files.
In on a Mac. 

If you don't care about How I Got Here, you can skip the section below in color and jump to the question itself (I include the origin story in case anyone cares or needs to know what happened)...

Here's what happened:
Upon my return home, I copied all images from 5 different devices (two phones, two compact P&S cameras, one DSLR) to my dedicated photo storage, an 8TB (external) hard drive. Lots of photos. These each went into an overall "trip" folder, containing 5 sub-folders, one for each source device.  All images appeared to copy over OK - after each copy, I did a quick review of a handful of images, checking each in an image preview app.  After all were copied to my dedicated photo drive, I put the memory cards back in each camera and formatted them (you can guess where this is going, right?).

I fired up Lightroom and started adding the trip photos to Lightroom.  I added all the photos from the phones and compact cameras with no problem. When I went to add the photos from my DSLR, I got an error, along the lines of "there are no photos there to add." I scoffed.  "Yes there are, about 800".  I looked again.  In the Mac Finder, sure enough, I see 800+ files in the folder for the DSLR, a 50/50 mix of JPGs and RW2 (Panasonic RAW) files.  I open up a couple of the JPGs in the Preview app, a couple of the RW2s in Photoshop, they all look fine.  Silly Lightroom, I think.  I go to add them again.  Same error.  Hmmm.  Then I stop and think, gee, those newly added images sitting on my 8 TB external HD have not been backed up yet - I have this drive set to automatically back up once a week, on Saturday nights. It was a Wednesday.  So, to be safe, I open up another external HD, and I copy over all the newly added files.  The copy fails and I get a read error ("could not complete the request because xxxxxx.RW2 could not be read").  Hmm.  Well, maybe I should't try copying all 2000 files at once.  I select just 10 of the DSLR files and copy them. Same error.  Hmmm.  I try copying just one file - the first JPG - and that copies.  I try copying the second file, which was the RW2 that it failed on. Error message again. OK, I figure I have one bad file, I skip that and try the next 10 (half JPGs, half RW2s).  Same error.  Hmmm.  OK, I figure it's time for a restart of the machine - the computer had been on for a couple days, so I exited out of all open apps, restarted. After restarting, I open up my primary 8 TB external HD, drill down into the new Trip folder and open the DSLR folder where I copied ~800 files.  That folder shows as empty.  WHAAA?  Sure enough, empty.  All the other subfolders for the other cameras and phones appear OK, all of those images are there, but the DSLR folder with the images I care most about (of course) - empty.  No backup.  Sigh.

I spend the last week trying to find a good File Recovery app, in hopes of snatching back these files.  I found one that worked (mostly) for restoring files from the formatted memory card (in case anyone is interested, I used an app called FireRecovery, which was not great but worked well enough I suppose).  Fast forward to today - now I have backed up everything newly added (in two separate places) so I'm feeling confident that no more loss is imminent.

I now have the recovered files, that's about about 800, half are JPGs, half are RW2 files (same images, two versions of each).  And yes, everything is now backed up (multiple locations).

Here are the issues I'd like to fix:

1. The Mac OS-level file attributes for Creation Date and Modification Date on ALL the files is wrong (those are now set to the date/time that I recovered them from the memory card, which makes sense, although its; not what I want).
2. Some (although not all) of the internal metadata for all these files is wrong (about half the values are set to the date/time that the files were recovered).  The "bad" date/time info seems consistent for all the JPGs (that is, what's wrong with all the JPGs seems consistent), and the "bad" date/time info for all the RW2s is consistent, too (but is different from what's wring with the JPGs).
3. Opening up both JPGs and RW2 files in Photoshop and using File > Info... I find that there's at least some metadata fields that still have the correct date/time data for all files.  This is good news.

What I want to do is extract the "correct" (original) date/time from one of these metadata fields, and then overwrite ALL the date/time data for EVERY metadata field, and the Mac OS-level file creation/modification dates, for each file, using that "correct" (original) date/time value pulled from that file's metadata.  Of course, that value will be different for each set of two images (one JPG and one RW2).

Looking over all the date/time metadata in Photoshop, I can see that all of the following values (in both the JPG and RW2 files) appears to be OK for every file - so I assume that I could use the date/time values from any of these fields as the "correct" value:

• File Info > Origin > Date Created (includes time stamp)
• File Info > IPTC > Date Created (includes time stamp)
• File Info > Audio Data > Date Created (includes time stamp)
• File Info > RAW data > xmp:CreateDate (includes time stamp)
• File Info > RAW data > photoshop:DateCreated (includes time stamp)

I'm assuming that I can run ExifTool and pull that "good" date/time value (from any one of the above items) for each file, and overwrite all other date/time values for that file, applying this command globally for all the files in a given folder.

Correct?  Can someone help get me started on that command with the correct syntax and parameters?  (I've used ExifTool only a couple times before, so I'm still fairly new with it).

I hope the above all makes sense.  A big thank you in advance for the help.

[Phil Harvey]

The first step is to use ExifTool to figure out what tags are right and what are wrong:

exiftool -a -s -G FILE

Then copy the correct dates into the wrong ones.  For example, if EXIF:DateTimeOriginal is correct, and all the others are wrong, you could do this:

exiftool "-exif:createdate<exif:datetimeoriginal" "-exif:modifydate<exif:datetimeoriginal" "-file:filemodifydate<exif:datetimeoriginal" "-file:filecreatedate<exif:datetimeoriginal" DIR

- Phil

[tenthumbs]

Thanks.

I have checked using ExifTool and verified that the values in the field/tag EXIF:DateTimeOriginal is indeed correct for each file (of course, the value is slightly different for each file, but they are all correct).  So far, so good.

I've opened up the Mac Terminal app, I've navigated to the location where I have some of the files copied (for test purposes, it's on an a separate external drive, to limit the scope of any damage I do if I get something horribly wrong).  I'm in the correct directory (which is named "metadatafix") on that volume (I've confirmed that I've navigated to the correct directory by listing the 10 files I put there).

I enter this (using single straight quotes since I'm on a Mac):

exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal'

The output is: No file specified

I has assumed it would perform the operation on all the files in the directory, but obviously it needs some additional argument or other trick.

Just to clarify: the goal would be to perform that operation for all files in the target (current) directory -- that is, for each file, it should pull the value from EXIF:DateTimeOriginal and overwrite that value to all the other date/time values for that file (and for each file this will be a slightly different time).

If I add the (current) directory name at the end like so:

exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal' metadatafix

Then the output is:

Warning: Error opening file - metadatafix
Error: File not found - metadatafix
    0 image files updated
    1 files weren't updated due to errors

I'm sure this is basic Terminal/Command Line 101, but I'm rusty on that.  The goal is to tell ExifTool to "do this to each file in the current directory".  What have I omitted?

Thanks again for the help.

[StarGeek]

exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal'

The output is: No file specified

I has assumed it would perform the operation on all the files in the directory, but obviously it needs some additional argument or other trick.

You're missing the directory to operate on.  To operate on the current directory, you can use a single dot ->.<-  So the command would be
exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal' .

Also, rather than selecting a font and a color, you can use the button above to make sure the command line examples are in a monospace font.

If I add the (current) directory name at the end like so:

exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal' metadatafix

If you're are currently in the directory metadatafix and you specify name without the full path on the command line, what you are actually saying is metadatafix/metadatafix i.e. a subdirectory of metadatafix with the name metadatafix.

[tenthumbs]

Ah, the missing dot. Thank you.

OK, I added that.  So I used this:
exiftool '-exif:createdate<exif:datetimeoriginal' '-exif:modifydate<exif:datetimeoriginal' '-file:filemodifydate<exif:datetimeoriginal' '-file:filecreatedate<exif:datetimeoriginal' .

That yielded a surprise:
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1199968.JPG
Warning: Error opening file - ./._P1199968.JPG
Error: File not found - ./._P1199968.JPG
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1199968.RW2
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200362.JPG
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1199969.JPG
Warning: Error opening file - ./._P1200362.JPG
Error: File not found - ./._P1200362.JPG
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1199969.RW2
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200362.RW2
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200180.JPG
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200365.JPG
Warning: Error opening file - ./._P1200365.JPG
Error: File not found - ./._P1200365.JPG
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200365.RW2
Warning: Error 256 running "setfile" to set FileCreateDate - ./P1200180.RW2
    1 directories scanned
   10 image files updated
    3 files weren't updated due to errors

The OS also popped a dialog informing me:
"The 'Setfile' command requires the command line developer tools.Would you like to install the tools now?  (Choose Install to continue. Choose Get Xcode to install Xcode and the command line developer tools from the app store).

I chose to Install.  OK, that's done...

OK, trying again now...

Good...that seemed to work better on my handful of test files.  Next, a larger bunch...OK.  Now to throw all 800+ files at it and see how that goes...

Looks like success.  Some edge issues (see below) but a success nonetheless.

Looking over the results -- I typically work in Photoshop -- I note that there are a few minor issues with some time stamps as viewed within Photoshop (e.g.: where the date was corrected but the time is still wrong) - however, these are (I think) Photoshop's internal listing of dates, not EXIF tags strictly speaking, and (I'd guess) probably stored in some proprietary fashion, not using standard EXIF tags - so it's not a shock to me that a few of these values don't quite line up.  In case you guys care, here's an example:

In Photoshop, File > File Info > Basic...the Creation Date is now correct but the time portion of that field is wrong.  Also, in the same path the Modification date (and time) are also wrong.  However, when viewing the file in the Mac OS (Finder), the creation date and time, and the modification date and time, are all corrected. 

This isn't a deal-breaker for me - and ExifTool is probably doing what it's supposed to do; I'm guessing that Photoshop keeps track of those values in some non-standard way (in fact, there appear to be multiple places Photoshop lists an overall file creation and modification date and time).  Not big deal for me.

The good news is that it appears ExifTool has "corrected" almost all the date/time values that I see (even when looking at them in obscure little corners of Photoshop), and best of all, it's "corrected" them for the values that matter to me - so the files appear to the OS with the correct creation/modification dates/times, and I'm assuming those "corrected" values will appear (and sort correctly) in the other tools I use (hopefully Lightroom, but we'll see about that).

So, happy ending (happy enough) for me.  A week ago, these photos appeared to be "lost". I managed to snatch them back, I only appear to have maybe lost 4 or 5 shots total, and the recovered files now have (mostly) correct date/time values.  That's good enough for me in this case.  I've lost a week of my life trying to put this stuff back together, now I can move on.

Thank you both for your help, and for creating/improving/supporting this very useful tool.  I'm sure I'll need to apply it again one of these days when some other minor disaster happens.

And now, I'm off to back up everything...

[Phil Harvey]

Interesting that MacOS popped up a dialog for you to install Xcode.  I didn't know that would happen.  Sorry you had to install such a big package (ie. GB's worth of Xcode) to get just the setfile functionality.

In Photoshop, File > File Info > Basic...the Creation Date is now correct but the time portion of that field is wrong.

Is this perhaps a time zone problem?  Or is the time completely wrong?

The good news is that it appears ExifTool has "corrected" almost all the date/time values that I see

Great.

Thank you both for your help, and for creating/improving/supporting this very useful tool.

Glad to help.

- Phil

[Phil Harvey]

I just checked, and you can download the MacOS command line tools as a separate package from here (you need to have an apple account): https://developer.apple.com/download/more/

The command line tools are about 200 MB.  You don't need to download the full 6 GB of XCode.

- Phil