AFK for a bit

[StarGeek]

So I'm definitely going to be AFK for a while.  One of my hard drives is acting flaky and is sometimes reading corrupted data, but only sometimes.  The same image that is messed up when loaded, loads perfectly fine a few minutes later.  To add to the mess, it took me some time to figure out which drive it was, because it is part of the drive pool.

So I'm removing the drive from the pool and I'm going to be spending quite some time extracting files from my backups and running DupeGuru to see which files might have been corrupted when they were copied off the drive.

It's a real pain because the drive doesn't show any SMART warnings and the last surface check that StableBit scanner ran about 2 weeks ago showed no problems.

Wish me luck.

[Phil Harvey]

Good luck StarGeek.  I've got my fingers crossed hoping that your recovery goes well.  (Gee, it sounds like you're in the hospital or something, but from that point of view it's good that it's only your hard drive.)

- Phil

[StarGeek]

So I'm still dealing with this.  There were 1.5 million files with possible corruption and I've got it down to under a million now.  About 40-50% of that first batch were files in my Steam game library, so that was just running a verify check on all the possibly corrupt games and letting Steam fix them.

The problem was really messy, as it was corrupting files only some of the time.  So even though I have incremental backups, the file could have been corrupted in one backup but fine a few days later, then corrupted again.  But I narrowed it down to having started between 1 and 2 weeks from my first post, so I could extract files from 2 weeks+ and they would be pretty much ok.

The main problem was that in that 2 week period, I had literally made at least 10k+ metadata edits, so I couldn't just extract from backup and continue.  I have to extract, compare to see if duplicated, and copy metadata if not the same.  And it would take forever if I only tried to extract the individual files from backup.  Much easier to extract full directories and then run DupeGuru to remove the duplicates.  Then run exiftool -TagsFromFile to copy over the data.

Still a huge amount of work to get back to where I started but I have the recovery workflow down now so I can start dealing with other things while a lot of the recovery operations run in the background.

[Phil Harvey]

It is a relief to know that you found a solution, and it sounds like your good backup strategy saved you here, but it also sounds like a lot of work sorting through what went bad.  Probably something for which your scripting talents will pay dividends.

- Phil

[obetz]

I have to extract, compare to see if duplicated, and copy metadata if not the same.

Can't resist to mention "Beyond Compare" from Scooter Software.

Oliver

[StarGeek]

Can't resist to mention "Beyond Compare" from Scooter Software.

Googled that, link was colored to show I had visited it (link for anyone else).  But it's paid while DupeGuru is free and covers what I need.

[obetz]

Can't resist to mention "Beyond Compare" from Scooter Software.

Googled that, link was colored to show I had visited it (link for anyone else).  But it's paid while DupeGuru is free and covers what I need.

Beyond Compare is no dupe finder *) but a directory and file comparison tool.

It has lots of special comparison options and also the possibility to add own comparisons. For example using ExifTool to extract metadata and present it side by side.

It's priceless.

Oliver

*) sadly. And it doesn't handle moves/renames as FreeFileSync does.

[StarGeek]

For example using ExifTool to extract metadata and present it side by side.

Now that is very interesting.