How to merge image with his .json file (Google Photos takeout)

[Nick_0278]

Hello,

I wish to leave Google photos, and I have downloaded all my photos using Google Takeout. Now I have a boatload of folders containing both json files (that I think contains exif data) and images. The structure looks like this:

home/user/Takeout/Google Photos/2019-09-09/xy.jpg
home/user/Takeout/Google Photos/2019-09-09/xy.jpg.json.
or
home/user/Takeout/Google Photos/2020-01-09/zz.jpg
home/user/Takeout/Google Photos/2020-01-09/zz.jpg.json.

I want to join the json data with the correct images. I have been looking at Exiftool, but I founded only pieces of code like this

Code Select Expand

exiftool -tagsfromfile '%d/%F.json' '-ImageTag<JsonTag' FileOrDir
and don't know what and how exactly to replace the placeholders.

What I'm looking for is a way to join all the data with the correct images in bulk.
I'm trying to do this in linux command line. Is there a way/Whats the way to do this?

-Nick

[Phil Harvey]

Hi Nick,

Basically you run this command on the json file to see that information it contains:

exiftool -s /home/user/Takeout/Google Photos/2019-09-09/xy.jpg.json

that will give the list of possible Json tags for '-ImageTag<JsonTag'

Then you just need to figure out where you want to write each tag in EXIF or XMP.  Then create one command to copy all the tags you want.

I can be more specific if you would attach one of the .json files here.

- Phil

[Nick_0278]

Thank you for the fast answer!
That's the output I get

Code Select Expand

ExifToolVersion                 : 11.85
FileName                        : 20170723_165646.jpg.json
Directory                       : .
FileSize                        : 714 bytes
FileModifyDate                  : 2020:02:28 01:07:32+01:00
FileAccessDate                  : 2020:04:15 20:40:27+02:00
FileInodeChangeDate             : 2020:04:15 20:14:48+02:00
FilePermissions                 : rw-rw-r--
FileType                        : JSON
FileTypeExtension               : json
MIMEType                        : application/json
CreationTimeFormatted           : 04.03.2019, 08:35:55 UTC
CreationTimeTimestamp           : 1551688555
Description                     :
GeoDataAltitude                 : 0.0
GeoDataLatitude                 : 42.68277777777777
GeoDataLatitudeSpan             : 0.0
GeoDataLongitude                : 23.289722222222224
GeoDataLongitudeSpan            : 0.0
GeoDataExifAltitude             : 0.0
GeoDataExifLatitude             : 42.68277777777777
GeoDataExifLatitudeSpan         : 0.0
GeoDataExifLongitude            : 23.289722222222224
GeoDataExifLongitudeSpan        : 0.0
ImageViews                      : 0
ModificationTimeFormatted       : 28.02.2020, 09:07:32 UTC
ModificationTimeTimestamp       : 1582880852
PhotoTakenTimeFormatted         : 23.07.2017, 13:56:46 UTC
PhotoTakenTimeTimestamp         : 1500818206
Title                           : 20170723_165646.jpg


That's one of my .json files:

- Nick

[Phil Harvey]

Hi Nick,

OK, here is a command that will copy all of the useful information from json files like this to their corresponding jpg files in a directory:

exiftool -ext jpg -tagsfromfile %d%f.%e.json -description "-gpslatitude<geodatalatitude" "-gpslatituderef<geodatalatitude" "-gpslongitude<geodatalongitude" "-gpslongituderef<geodatalongitude" "-gpsaltitude<geodataaltitude" "-createdate<creationdatetimestamp" "-datetimeoriginal<phototakentimetimestamp" "-modifydate<modificationtimetimestamp" -d %s DIR

Here I am using the "%s" date/time formatting option to accept timestamps in seconds since the epoch, as done in this recent post.

- Phil

[Nick_0278]

Thanks, this is definitely the solution to my problem.
The only thing I'm not quite sure about is the last parameter of the command: 'DIR'

The thing is that I have a folder xy which contains many subfolders. These subfolders, (e.g. xy/xy2 | xy/xy2 | xy/xy3 ...)
contain the photos (e.g. xy/xy2/test.jpg). Now I want to use this command on all photos in the subfolders of xy/ but I don't know what I have to use for 'DIR'.

When I try:

Code Select Expand

exiftool -ext jpg -tagsfromfile %d%f.%e.json "-gpslatitude<geodatalatitude" "-gpslatituderef<geodatalatitude" "-gpslongitude<geodatalongitude" "-gpslongituderef<geodatalongitude" "-gpsaltitude<geodataaltitude" "-createdate<creationdatetimestamp" "-datetimeoriginal<phototakentimetimestamp" "-modifydate<modificationtimetimestamp" -d %s xy/

I get this 'error'

Code Select Expand

1 directories scanned
0 image files read

wich is logical, because in the directory itself aren't any photos.Do I need something like -r for recursively?

- Nick

[StarGeek]

One very important thing to take note of is that Google Photos does not remove the metadata from the files.  The files you downloaded have exactly the same data that was there when uploaded.

Unless you have made modifications to the data on the Google Photos website, you do not need to merge the json data into the files.  The data is already there.

If you take a look and compare the GeoData* tags with the GeoDataExif* tags, you'll see that they are the same.  the GeoDataExif* tags are what was in the file when it was uploaded and are still in the file.

[Phil Harvey]

If StarGeek is correct, all this is for nothing, but yes, the -r option processes subdirectories.

- Phil

[Nick_0278]

Thanks StarGeek, you're right. I don't understand why Google is giving you all the JSON files again. This was also the reason why I thought they were separate.

Nevertheless, I really thank you for your helpfulness.

So, that this thread can still be useful for someone, is that the result:
(If you've got also a Googlephotos export you do not need to do anything!)

To search only in the given directory for images and merge them with their json files:

exiftool -ext jpg -tagsfromfile %d%f.%e.json -description '-gpslatitude<geodatalatitude' '-gpslatituderef<geodatalatitude' '-gpslongitude<geodatalongitude' '-gpslongituderef<geodatalongitude' '-gpsaltitude<geodataaltitude' '-createdate<creationdatetimestamp' '-datetimeoriginal<phototakentimetimestamp' '-modifydate<modificationtimetimestamp' -d %s xy/

-Where xy/ must be replaced with your directory

To search also in Subfolder(s) of the given directory for images, and them merge them:

exiftool -r -ext jpg -tagsfromfile %d%f.%e.json -description '-gpslatitude<geodatalatitude' '-gpslatituderef<geodatalatitude' '-gpslongitude<geodatalongitude' '-gpslongituderef<geodatalongitude' '-gpsaltitude<geodataaltitude' '-createdate<creationdatetimestamp' '-datetimeoriginal<phototakentimetimestamp' '-modifydate<modificationtimetimestamp' -d %s xy/

- Where xy/ must be replaced with your directory
- the -r stands for recursively

EDIT:
should I somehow mark the thread as solved?

- Nicko

[StarGeek]

Thanks StarGeek, you're right. I don't understand why Google is giving you all the JSON files again.

Some people spend a lot of time editing what data they can on Google photos such as adding descriptions and setting locations for images that didn't have GPS coordinates.  That can be a lot of work that would otherwise be lost.

[h_correia]

Hello all,

I registered here just to give feedback that I had the exact same issue as Nick_0278 and, contrary to his and StarGeek's experience, when I downloaded all my photos from Google photos, for some reason, many came with the original CreateDate and as such, MacOS correctly identifies the date in which they were taken, but others have a CreateDate and ModifyDate from the day I downloaded them. Curiously, the original PhotoTakenTimeTimestamp was still there in the json file, but not in the jpg. Using the commands above, I was able to correctly set the jpgs' PhotoTakenTimeTimestamp. I'm including an example from an uncorrected jpg and from a corrected one, in case anyone is curious.

Uncorrected:

Code Select Expand

ExifToolVersion                 : 12.13
FileName                        : 1470856620843.jpg
Directory                       : /Volumes/Data/Photos/Takeout/Google Photos/Photos from 2016/Test
FileSize                        : 177 KiB
FileModifyDate                  : 2020:12:19 19:33:18-03:00
FileAccessDate                  : 2020:12:29 18:55:24-03:00
FileInodeChangeDate             : 2020:12:19 19:33:18-03:00
FilePermissions                 : rwxrwxrwx
FileType                        : JPEG
FileTypeExtension               : jpg
MIMEType                        : image/jpeg
JFIFVersion                     : 1.01
ExifByteOrder                   : Little-endian (Intel, II)
Software                        : Picasa
ExifVersion                     : 0220
ColorSpace                      : sRGB
ExifImageWidth                  : 1080
ExifImageHeight                 : 1731
InteropIndex                    : R98 - DCF basic file (sRGB)
InteropVersion                  : 0100
ImageUniqueID                   : 31d453461447e1d40000000000000000
Compression                     : JPEG (old-style)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ThumbnailOffset                 : 312
ThumbnailLength                 : 5431
XMPToolkit                      : XMP Core 5.5.0
ImageWidth                      : 1080
ImageHeight                     : 1731
EncodingProcess                 : Baseline DCT, Huffman coding
BitsPerSample                   : 8
ColorComponents                 : 3
YCbCrSubSampling                : YCbCr4:2:0 (2 2)
ImageSize                       : 1080x1731
Megapixels                      : 1.9
ThumbnailImage                  : (Binary data 5431 bytes, use -b option to extract)



Corrected:

Code Select Expand


ExifToolVersion                 : 12.13
FileName                        : 1470266269146.jpg
Directory                       : /Volumes/Data/Photos/Takeout/Google Photos/Photos from 2016/Test
FileSize                        : 146 KiB
FileModifyDate                  : 2020:12:29 18:50:14-03:00
FileAccessDate                  : 2020:12:29 19:20:28-03:00
FileInodeChangeDate             : 2020:12:29 18:50:14-03:00
FilePermissions                 : rwxrwxrwx
FileType                        : JPEG
FileTypeExtension               : jpg
MIMEType                        : image/jpeg
JFIFVersion                     : 1.01
ExifByteOrder                   : Little-endian (Intel, II)
Software                        : Picasa
ModifyDate                      : 2020:09:06 06:33:47
ExifVersion                     : 0220
DateTimeOriginal                : 2016:08:03 23:17:49
CreateDate                      : 2016:08:09 21:01:22
ColorSpace                      : sRGB
ExifImageWidth                  : 1080
ExifImageHeight                 : 1731
InteropIndex                    : R98 - DCF basic file (sRGB)
InteropVersion                  : 0100
ImageUniqueID                   : 57918d38f50752ac0000000000000000
GPSVersionID                    : 2.3.0.0
GPSLatitudeRef                  : North
GPSLongitudeRef                 : East
GPSAltitude                     : 0 m
Compression                     : JPEG (old-style)
XResolution                     : 72
YResolution                     : 72
ResolutionUnit                  : inches
ThumbnailOffset                 : 554
ThumbnailLength                 : 4211
XMPToolkit                      : Image::ExifTool 12.13
Description                     :
ImageWidth                      : 1080
ImageHeight                     : 1731
EncodingProcess                 : Baseline DCT, Huffman coding
BitsPerSample                   : 8
ColorComponents                 : 3
YCbCrSubSampling                : YCbCr4:2:0 (2 2)
ImageSize                       : 1080x1731
Megapixels                      : 1.9
ThumbnailImage                  : (Binary data 4211 bytes, use -b option to extract)
GPSLatitude                     : 0 deg 0' 0.00" N
GPSLongitude                    : 0 deg 0' 0.00" E
GPSPosition                     : 0 deg 0' 0.00" N, 0 deg 0' 0.00" E



[Aleburd]

I have to agree with h_correia.  While StarGeek's experience is certainly true of the geo-location and camera information in the files I've seen from Google, the Capture Date has been removed from the EXIF file leaving the computer to see date created (the date the file was zipped by Google) as the capture date.  Most of the value in any photo library is to have the photos sorted by date and with this information only available in the json file there's no choice but to attempt a merge.  The good news is ExifTool works perfectly for this use case.

[StarGeek]

Make sure you're not confusing the file system timestamps for the embedded time related tags. These timestamps you first see under Properties are file system timestamps, not embedded EXIF data.


These are of course going to be lost because the file is no longer on the drive. And when you re-acquire the file, these are going to be set to when you downloaded the file, because it's a new and completely different file.  The contents are the same, but to the OS, it's a new file.

[StarGeek]

Though thinking further, my tests didn't include a file that didn't have any embedded timestamps.  And I can't recall offhand if Google Photos saves that original file system timestamp and displayed that as the date online.

I can see how it might appear that Google is deleting data if it is saving the file system timestamp for sorting purposes.  I'll have to do some testing.

[Aleburd]

Yeah - it's dropping the date taken date from the files.  I looked through the files and did find some that do have the capture date.  The inconsistency is a little concerning.  The attached image with the Windows details tab marry up to this json file:

Code Select Expand

{
  "title": "DSCN0524.JPG",
  "description": "",
  "imageViews": "0",
  "creationTime": {
    "timestamp": "1312564956",
    "formatted": "Aug 5, 2011, 5:22:36 PM UTC"
  },
  "modificationTime": {
    "timestamp": "1608567500",
    "formatted": "Dec 21, 2020, 4:18:20 PM UTC"
  },
  "photoTakenTime": {
    "timestamp": "1312564956",
    "formatted": "Aug 5, 2011, 5:22:36 PM UTC"
  },
  "geoData": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 0.0,
    "longitude": 0.0,
    "altitude": 0.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  }
}

[RMcD]

https://legault.me/post/correctly-migrate-away-from-google-photos-to-icloud

>exiftool -r -d %s -tagsfromfile "%d/%F.json" "-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLatitudeRef<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" "-GPSLongitudeRef<GeoDataLongitude" "-Keywords<Tags" "-Subject<Tags" "-Caption-Abstract<Description" "-ImageDescription<Description" "-DateTimeOriginal<PhotoTakenTimeTimestamp" -ext "*" -overwrite_original -progress --ext json <DirToProcess>

Found this which seems to do what you want.

I just want to confirm because I have a flat folder with my entire 1.6TB Google Archive. It would be annoying to have to redownload it if this breaks it. 200k images/videos, and 200k .json files.

I want to add/replace the EXIF information on the images with that from the json's when the name of the thing matches. My HDD is almost full so I need it to replace the image and not create a new one.

To be fair looking at the list I haven't seen a single date taken wrong