Fixing files from Google Photos via Takeout

[ExifShmexif]

I received such helpful advice from searching and reading this forum, and the tool itself is amazing, so first of all thank you to all, especially Phil and stargeek. And in some meagre fashion, I wanted to return something by writing up my experience for anyone moving files out of Google Photos via Takeout. Many of the commands here are collected from other posts on this forum, and I'm sure I'll get a few things wrong, but hopefully it'll be helpful.

This will be most useful for anyone who:

• Has a lot of files to export from Photos
• Has images from a variety of sources
• Used the "original" quality storage option when saving images in Photos
• Wants to recreate and file those images once downloaded
• Uses Windows
• ExifTool is already in your $path

Getting images out of Google Photos can be done in two ways.

• Use takeout.google.com. This lets you download some or all of your photos via a zip file. You can export albums, or alternatively Takeout groups images by year into albums for you (these aren't real albums, they're only visible in takeout.) If you do this, each file will have an associated .json file, which contains the image's metadata (e.g. date it was taken, or which people are in the photo.)
• Use get.google.com/albumarchive. This lets you download all images in an album -- so if your collection is small enough to add everything to an album, it's a one click download of everything.

I'll assume you're doing the first of these. Assuming you selected images from all years in your export, you'll see a set of folders each corresponding to a year. Note: if your collection is larger than 50GB, and you have multiple zips to download and expand I strongly recommend you combine these into one. Takeout doesn't ensure that the media file and associated json are in the same zip, so combining them will fix it.

For any edited file, Takeout also exports both the original and edited files. If you prefer to not have doubles in your collection, you can delete the edits with:

del /s /q *-edited.jpg

You'll also notice that the file creation date for your media has been updated to today. That's probably correct as Takeout sees it, but for media that doesn't have EXIF data with the create date in -- like MP4 files -- your viewing software will probably just use the creation date for ordering, and these files will appear in the wrong place. So, from the Takeout root directory:

exiftool -d "%s" -tagsfromfile %d%f.%e.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -overwrite_original -ext mp4 -ext jpg -r .

I've also read that geolocation data may need to be added back to the EXIF data from the JSON file. I didn't find that to be the case for me -- location data seemed intact for the files. But just in case, again from the root:

exiftool -tagsfromfile %d%f.%e.json -description -title "-gpslatitude<GeoDataLatitude" "-gpslatituderef<GeoDataLatitude" "-gpslongitude<GeoDataLongitude" "-gpslongituderef<GeoDataLongitude" ... -ext mp4 -ext jpg -r .

Then, time to get rid of the .json files:

del /s /q *.json

The next part depends on how you want to organise the files. If you want to just dump everything into a directory and let your viewer sort it out -- maybe something like Synology moments -- you're probably done. I personally wanted to organise the files a little better and recreate a directory structure with my best guess of which camera took the image. This has the advantage of helping you identify which photos came from a friend, or any other source.

exiftool "-directory<%d/${model;}" -r .

You also may know that some actions strip exif data, e.g. uploading an image and sending it via some messaging apps or social networks. Additionally, as mentioned, mp4 files generally don't have exif. So if you want to sort further, I found it helpful to sort by image size:

exiftool "-directory<%d/${ImageSize;}" .

.... which helped me identify and group together images that were saved from various sources, like screenshots or movies. (Note that this will save media from the same source in different folders for portrait and landscape, but it's not a big deal to put those together.)

(One final TODO here, is to find a way to edit creation date for files that don't have an associated json file or any exif data, but present the date in the filename, something like 20130602_175206.jpg or 2012-10-16 09.39.07.jpg.)



I hope this has been helpful in some way. Thanks again!

[Phil Harvey]

Thanks for this write-up.  I'm sure it will be useful for someone else who wants to do this.

- Phil

[Nirmitlamed]

Wanted to say thank you. It was helpful a lot!

[Tyler]

In this command:
exiftool -d "%s" -tagsfromfile %d%f.%e.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -overwrite_original -ext mp4 -ext jpg -r .

You mention mp4 and jpg but if there are any other extensions, what happens to those?

[Phil Harvey]

Files with other extensions will be ignored.

- Phil

[ythiebault]

Thanks a lot for your post, it helped me a lot !

I had to adapt it in order to upload an album to a photo book creator service. In case it would help someone else in a common situation:

Code Select Expand

# suffix used for edited files. Depend on the lang.
EDITED_FILE_SUFFIX="-edited"

# Replace original files by their edited version
for edited_file in *$EDITED_FILE_SUFFIX* ; do
    original_file=$(echo "$edited_file" | sed s/$EDITED_FILE_SUFFIX//g)
    rm "$original_file"
    mv -v "$edited_file" "./${original_file}"
done

# Convert .HEIC to .jpeg (for iphone/mac users)
for file in *.HEIC; do
    sips -s format jpeg "$file" --out "${file%.HEIC}.jpeg"
    mv -v "$file.json" "${file%.HEIC}.jpeg.json"
    rm $file
done

# Update metadata with exiftool
exiftool -d "%s" -tagsfromfile %f.%e.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -overwrite_original -ext jpg -ext jpeg -ext mp4 -ext mov .


[Edi Nugroho]

I tried the code above, i.e.

Code Select Expand

exiftool -d "%s" -tagsfromfile %d%f.%e.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -overwrite_original -ext mp4 -ext jpg -r .
but an error message was displayed: "Warning: Error opening file -xxxxnameOfFile".

What went wrong?

[Phil Harvey]

It looks like maybe you used something that looks like a dash but isn't.  This can happen if you cut-and-paste a command.  Use the dash (minus sign) on the keyboard for the leading dashes.

Or... Is this a .bat file?  If so, all % characters need to be doubled in ExifTool arguments.

- Phil

[StarGeek]

I strongly suggest double-checking to see if you actually need to do this. Google didn't remove any data when the file was uploaded, so the only problem is if data was entered on the website or if no data was there to begin with.

Run this command to look at all the date/time data. If there is already proper time stamps such as an already existing DateTimeOriginal, then you don't want to run the above command
exiftool -time:all -G1 -a -s file.jpg

[revo2wheels]

Absolutely 100% a problem for most files from 500GB Download from Google photos takeout. I'm very grateful to the OP for this fix.

[alipio]

It is definitely a problem for me.

I had mixed results with Google Takeout: most of the time it came with all the data, but a 4.52GB worth of media is coming  up stripped no matter what I do. I'm lucky that I learned about NAS, Syncthing, Immich, etc. because currently I am only using Google Photos as a duplicate to share media.

Thanks for OP!

[StarGeek]

but a 4.52GB worth of media is coming  up stripped no matter what I do.

Are you 100% certain that it contained data when it was uploaded? If there was no data in the file when it was uploaded, then data isn't going to suddenly appear.

I just check a video I uploaded to Google photos in 2016. It is a binary exact match to the original on my drive.

File system data such and created date you first see when you check the properties on the desktop are always going to be lost because the file isn't on a local drive anymore.

[MrSimmo]

Hi all,

Just helping out a little, I found that when I ran the above it failed as Google seem to have changed the filenames in the Takeouts to include 'supplemental-metadata' to the JSON file name recently. With the original command I was getting the 'Error opening file [JSON]' error.


I also extended the command to skip files which already had the date/time taken in the exif data, and also to include other file types (as my photos are heif/heic and some videos seem to be mov instead of mp4). Also I added the large file support argument as exiftool skipped all my videos otherwise:

Here is my updated command:
exiftool -d "%s" -tagsfromfile %d%f.%e.supplemental-metadata.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -if 'not defined $DateTimeOriginal' -overwrite_original -ext mp4 -ext jpg -ext heic -ext m4v -ext mov -api largefilesupport=1  -r .


Note: I'm getting the warning "Warning: This tag is Windows/Mac only in File:FileCreateDate (ValueConvInv)" when I run the command on my Linux NAS, but it writes the exif data fine.


Also here is my updated command to fix the missing GPS location data:

exiftool -d "%s" -tagsfromfile %d%f.%e.supplemental-metadata.json "-gpslatitude<GeoDataLatitude" "-gpslatituderef<GeoDataLatitude" "-gpslongitude<GeoDataLongitude" "-gpslongituderef<GeoDataLongitude" -if 'not defined $GPSLatitude' -overwrite_original -ext mp4 -ext jpg -ext heic -ext m4v -ext mov -api largefilesupport=1  -r .


Hope its useful, cheers

[StarGeek]

As I previously said

Google didn't remove any data when the file was uploaded, so the only problem is if data was entered on the website or if no data was there to begin with.

So double-check your files to make sure you're not overwriting the original data, because when it comes to the date/time values, Google saves it as UTC, not the original time. Copying from the Google takeout files overwrites the correct date with an incorrect one.

[MrSimmo]

Hi all,

Just helping out a little, I found that when I ran the above it failed as Google seem to have changed the filenames in the Takeouts to include 'supplemental-metadata' to the JSON file name recently. With the original command I was getting the 'Error opening file [JSON]' error.


I also extended the command to skip files which already had the date/time taken in the exif data, and also to include other file types (as my photos are heif/heic and some videos seem to be mov instead of mp4). Also I added the large file support argument as exiftool skipped all my videos otherwise:

Here is my updated command:
exiftool -d "%s" -tagsfromfile %d%f.%e.supplemental-metadata.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" "-FileCreateDate<PhotoTakenTimeTimestamp" "-FileModifyDate<PhotoTakenTimeTimestamp" -if 'not defined $DateTimeOriginal' -overwrite_original -ext mp4 -ext jpg -ext heic -ext m4v -ext mov -api largefilesupport=1  -r .


Note: I'm getting the warning "Warning: This tag is Windows/Mac only in File:FileCreateDate (ValueConvInv)" when I run the command on my Linux NAS, but it writes the exif data fine.


Also here is my updated command to fix the missing GPS location data:

exiftool -d "%s" -tagsfromfile %d%f.%e.supplemental-metadata.json "-gpslatitude<GeoDataLatitude" "-gpslatituderef<GeoDataLatitude" "-gpslongitude<GeoDataLongitude" "-gpslongituderef<GeoDataLongitude" -if 'not defined $GPSLatitude' -overwrite_original -ext mp4 -ext jpg -ext heic -ext m4v -ext mov -api largefilesupport=1  -r .


Hope its useful, cheers

One more addition (sorry cant find a way to edit my original post) - I just found that Google Photos Takeout also has a random collection of CreateDate tags, where some exist (and are correct) and some are missing; but these don't correlate to the missing DateTimeOriginal tag.

So I run this AFTER the first command: (Note it uses the newly created DateTimeOriginal tag and ignores files that already have this tag in place - the reason why its a separate command is because the files that are missing may not be the same as the files where DateTimeOriginal is missing):

exiftool "-CreateDate<DateTimeOriginal" -if 'not defined $CreateDate' -overwrite_original -ext mp4 -ext jpg -ext heic -ext m4v -ext mov -api largefilesupport=1  -r .