Processing json files in Google Takeout downloads

[doninbc]

I think this is a common use case for Android phone users who want to get copies of their media from Google.

In order to download my media files I access Google Takeout which downloads all my files. Instead of putting the file metadata in the files themselves, Google includes a .json file for each media file that contains the metadata that they wish to share.The metadata includes a PhotoTakenTime value which can be viewed in a Timestamp or Formatted format. Both are string values.See typical .json file attached.

Over the years I've attempted to use that data to modify my files so they can be easily sorted in date order. I tried using the data to set up various tags in the files but it's hard to find a method that works for all media files, so I've settled on using the date info that includes date taken info in the filename so that when you sort by name you're also sorting by date. This has worked relatively well but I have one issue that I've been working on.

The date data comes in either a Unix time value formatted as a string or as a Date formatted by Google. Neither of these are great for use in a filename IMHO. What I would like to do is create my own custom formatting of the data value. So the problem is to convert either a Unix numeric time formatted as a string or a preformatted date string into my own format that works as a filename. I've tried -d, -p, Datefmt(), UnixDate(), RegEx, and other suggested options but I haven't quite gotten it to work as I would like. Any suggestions would be appreciated.

[Phil Harvey]

Google takeout should not modify the metadata in your files.  There should be no need to use the metadata from the JSON files.  Read here.

- Phil

[StarGeek]

I had forgotten about that long post. I need to add it to my notes.

But as said, Google does not modify your file. If the file is photo from the camera, then it will still have the time stamps embedded in the file and those are the ones you should copy from. Use this command to look at the embedded time stamps
exiftool -time:all -G1 -a -s file.jpg

There are two cases where you might want to use the JSON files. First is when you added/edited data on the Google website. In that case, the new data only exists in the JSON files.

The second case is when the file never had any embedded time stamps to begin with. For example, screenshots and Text Messages/WhatsApp/FaceBook/Insta/other social media sites. The first rarely (if every) embeds time stamps in the file and everything else strips away metadata.

In both of these cases, Google will use the FileModifyDate from the original file.

But there's an additional problem with the time stamps in the JSON file. The time stamp cannot be used as is. It is in UTC. And Google may not be consistent in how it converted it. It may have properly converted the local time to UTC, or it may have used the local time as if it was UTC, i.e. it may have taken the time of 12:00:00 Eastern time and used that as if it was noon UTC. I've seen both. So either way, the time stamp will need to be adjusted to the local time.

[doninbc]

Unfortunately, because Google exports my media in .zip files, any way I've been able to unzip the files alters the important date info. I've tried Windows and 7-zip to extract the files but the date info is always modified and the DateTaken tag only shows up on a small percentage of image files.

And besides, the main thing that I'm trying to do is to download my media and sort it in original date order.

So what I've been trying to do, with limited success, is to utilize the json date tags to rename all files with the date in the name so sorting by names is equivalent to sorting by date. It seems straight forward but I have run into a number of issues. Here is the json info I'm working with.

"photoTakenTime": {
    "timestamp": "1261785856",
    "formatted": "Dec 26, 2009, 12:04:16 AM UTC"
  },

I can rename the my files with the 'formatted' tag but the format is not great and it doesn't sort correctly. I can use the 'timestamp' and it works but as is (UnixDate) it is not very informative. What I would like to do, but have not quite been able to do is to, is to use the 'timestamp' tag, read as a numeric string, and then custom formatted into the appropriate name format (e.g. 2023_08_14_ ...). I've come close but just can't get it to work.

So what I would like help with is a combination of an exiftool command and config file that would at minimum do what I requested above. If it also processed the other tags in the json file like geoData and subject info that would be great, and I assume that you would not lose any of the embedded tags in the files. See Google Takeout Export definitions for details or see attached json file.

[Phil Harvey]

This thread discusses how to set the date/time tags from the unix timestamp in these json files.  (The key is to use -d %s)

- Phil