Journey in transitioning from Windows Live Photo Gallery to Digikam

[Frh]

Hello,

After a lot of research, and thanks to this forum and exiftool, I've been able to address a series of problems with my collection of family pictures.  I figured that documenting my findings might help others facing similar issues, and would be a way of giving back.  I wouldn't have been able to solve this without the content in this forum, and I have been immensely impressed by the responsiveness and knowledge demonstrated by its participants.  You have answered many of my questions without me having to ask: thank you!

So here goes my situation: I'm a casual photographer, with a collection of ~85k family pictures and archives of old scanned documents gathered from our extended family (so a variety of sources, and a variety of editing journeys for these pictures).  We used Windows Live Photo Gallery (WLPG) for tagging and commenting on pictures for years, until this month when I decided it was past time to look for an alternative.  I chose Digikam for its ability to store metadata in the files themselves, a must have for me after lots of work lost with Picasa's demise a few years back.  Thanks to the script shared on this forum https://exiftool.org/forum/index.php?topic=4361.0, I was able to get Digikam to recognize the face tags we had painstakingly entered over the years, in particular on scans of 50+ year old pictures of distant relatives.
Unfortunately, I noticed that editing some of the files with Digikam would cause the loss of a fair bit of the original metadata: original face tags, makernotes, etc.  As a newbie it took me a while to get to the bottom of it, but the issue boiled down to the corruption of metadata by WLPG documented on this site, corruption that exiftool is able to work around (e.g. by guessing the correct endianness for the Makernotes?), but that causes Digikam to choke. Digikam uses Exiv2 under the hood: when invoked via the CLI, Exiv2 would report issues related to the makernotes, even on files I thought I had cleaned up with the *first* command documented in the FAQ 20b.
exiv2 -pa -u file.jpg

Code Select Expand

Error: Directory Canon with 6144 entries considered invalid; not read.

What solved this issue for me was the discovery that the likely root cause for the exiv2 error message was the byte order of the makernote section.  In re-reading FAQ 20b, I realized the *second* command might address the issue, since it leverages the byte order of the makernotes.

The full command I ended up using (powershell, so ` is the escape character):

$dir = "E:\Photos"
exiftool -if "`$warning" -exif:all= -tagsfromfile `@ -exif:all -unsafe -icc_profile -thumbnailimage -codedcharacterset=utf8 -F -ext jpg $dir

To be clear, this information is already present in other places in this forum, but I figured my use case (transition Windows Live Photo Gallery -> Digikam) would be common enough that this might save others some time to have this together.
Once I ran this command on my entire collection, I reviewed a large sample of files before/after, and found that by restoring the correct byte order mark, the script not only allowed exiv2 to read makernotes -in turn making it possible for Digikam to preserve them when editing files-, but it also corrected the data displayed by exiftool, e.g. reporting valid entries for lens, camera temperature, etc. that previously were empty or non-sensical.

I'd welcome any suggestion for improvements for the command above, and if I had one suggestion it would be to highlight a little more clearly when exiftool detects likely byte-order mark issues in the metadata to make diagnosis easier.  Again, I've been impressed with Exiftool and this community: thank you!

François

[Phil Harvey]

Hi François,

Byte-order differences between Exif and maker notes are common, and may be caused by a lot of different software (not ExifTool though, unless you try really hard).  When creating Exif from scratch, ExifTool will use the byte order of the maker notes to avoid problems like this.

But specifically for Canon maker notes, ExifTool doesn't have to guess the byte order.  Canon stores a byte order record at the end of the makernote block, and uses this if it exists.  Exiv2 should do the same.  But even without this, Exiv2 should be able to guess the byte order because it knows that 6144 (hex 0x1800) is too high for the number of entries in the Canon directory, and it should realize that 0x0018 (the opposite byte order) is likely correct.

- Phil