google takeout places parentheses wrongly in json file name?

[gflyer]

Hi all,
I am totally new. I am trying to migrate google photos to icloud.
when processing takeout files with exiftool I did notice a bunch of errors when opening json files.
looking closely it seems to me that if the original photo file name has parenthese, the associated json file name has the parentheses in the wrong place.
For example:
Photo name: DSC00041(2).JPG
Json file: DSC00041.JPG(2).json

If I change the name of the json file to DSC00041(2).JPG.json, then it works fine.

Did I get something wrong?
Is there a workaround to make this work?

Sorry if this is a silly question, but as I said I am really new to this.

Thank you.
Best,
Guido

[StarGeek]

First and foremost, Google did not remove any metadata from your files.  The metadata in the files is exactly the same as when you uploaded them.  The only time you actually need to do anything is if you made changes on the Google website.  If you didn't change anything on the Google site, you do not need to run exiftool.

Otherwise, the problem here is as you say, the parenthesis are in the wrong position.  Your best bet is to rename the json files with this command
exiftool -ext json -r -if '$Filename=~/(\.[^.]+)(\(\d+\)).json$$/i'  '-Filename<${Filename;s/(\.[^.]+)(\(\d+\)).json$/$2$1.json/}' /path/to/Takeout/

[gflyer]

Thank you for the kind answer.
The reason why I am using exiftool is because if I upload the takeout from google directly on apple photos, for some reason obscure to me, a portion of the pictures do not get the date as it was on google photos, but instead they get the date of the time I have generated the takeout.
The reason again is a mistery to me.
By running exiftool a portion of the picutres get the correct date/time.

Now I am going to try to fix the parentheses issues and see if I recover even more.

If you have any hint why the date is "forgotten" the moment I upload the takeout to apple photos, I would be extremely grateful.

Thank you again.
Best regards,
Guido

[StarGeek]

Apple Photos appears to be reading the file system timestamps, which are not embedded it the file but instead a property of the underlying OS.  They would have been lost when the file was uploaded to Google because, obviously, they were no longer on your hard drive.

If these were images taken from a camera, I would suggest something like
exiftool "-FileModifyDate<DateTimeOriginal" "-FileCreateDate<DateTimeOriginal" /path/to/files/
Or if you look at the file name and it's obviously a date/time name, you could use
exiftool "-FileModifyDate<FileName" "-FileCreateDate<FileName" /path/to/files/

But if you are copying the time into DateTimeOriginal from the Google takeout and Apple doesn't read it otherwise, then you are probably on the right path. 

[gflyer]

wow that is a lot of helpful information. Thank you. I will try the first command.

In the meantime I have tried to rename files to fix the parenthesis and then run the exiftool and I still have a bunch of photos that do not get the current date.
All these photos didn't have the json file associated with them and so exiftool faild to update them.
The file names are very long and meaningless.

For example:
Warning: Error opening file - ./image-ec23c323a2965fe6cf8f4330980ab3c6ae122f10a.jpg.json
Nothing changed in ./image-ec23c323a2965fe6cf8f4330980ab3c6ae122f10a.jpg

[gflyer]

When I run

exiftool "-FileModifyDate<DateTimeOriginal" "-FileCreateDate<DateTimeOriginal" /path/to/files/

I get a number of warnings like:
Warning: No writable tags set from ./image-aa6b71a11a5b6efe3a36e863a505dde620d853e4f.jpg

All the files affected are exactly the ones that do not get the proper date set when imported in Apple Photos.

[StarGeek]

That just means they didn't have a correct date embedded to begin with or had the metadata stripped, which happens to images from social media sites.

The example you give with the "Error opening file" means that either the json file doesn't exist or it's locked or corrupt.  Those files you'll probably have to process manually.

[gflyer]

I am making a lot of progress.
I was able to fix the date of MOV files by using

exiftool "-QuickTime:CreateDate<DateTimeOriginal"  example.MOV

So currently the only files that are getting the wrong date are the ones I have mentioned above with very long filenames and no json file.
If I run
exiftool image-aa6b71a11a5b6efe3a36e863a505dde620d853e4f.jpg  -time:all -G1 -a

I get the following
[System]        File Modification Date/Time     : 2021:09:27 11:54:09+02:00
[System]        File Access Date/Time           : 2021:09:28 23:14:51+02:00
[System]        File Inode Change Date/Time     : 2021:09:28 23:06:32+02:00


So I guess I might just have to give up on those or process them manually.

[StarGeek]

One thing to take into account with the MP4/Mov files is that the QuickTime:CreateDate tag is supposed to be in UTC, while the DateTimeOriginal in a video file is supposed to be local time.  If you copy DateTimeOriginal to the QuickTime:CreateDate, no adjustment for the time zone difference will happen.  You can add the -api QuickTimeUTC option to have exiftool adjust the time automatically.

[gflyer]

ah!!!
I know what happened with those files.
I guess takeout has a limited number of characters for the file names and so the json file name is trimmed before the json extension

image-fab40b5c1a287ca303710030125d340dea494cf3.json   
image-fab40b5c1a287ca303710030125d340dea494cf30.jpg

So what I have to do now is to trim the last character of all the json files and I should be able to process those.

exiftool -s image-fab40b5c1a287ca303710030125d340dea494cf3.json

ExifToolVersion                 : 12.31
FileName                        : image-fab40b5c1a287ca303710030125d340dea494cf3.json
Directory                       : .
FileSize                        : 949 bytes
FileModifyDate                  : 2021:09:27 11:54:05+02:00
FileAccessDate                  : 2021:09:28 23:06:21+02:00
FileInodeChangeDate             : 2021:09:28 23:06:32+02:00
FilePermissions                 : -rw-r--r--
FileType                        : JSON
FileTypeExtension               : json
MIMEType                        : application/json
CreationTimeFormatted           : Mar 31, 2016, 5:27:40 PM UTC
CreationTimeTimestamp           : 1459445260
Description                     :
GeoDataAltitude                 : 0.0
GeoDataLatitude                 : 0.0
GeoDataLatitudeSpan             : 0.0
GeoDataLongitude                : 0.0
GeoDataLongitudeSpan            : 0.0
GeoDataExifAltitude             : 0.0
GeoDataExifLatitude             : 0.0
GeoDataExifLatitudeSpan         : 0.0
GeoDataExifLongitude            : 0.0
GeoDataExifLongitudeSpan        : 0.0
ImageViews                      : 10
PhotoLastModifiedTimeFormatted  : Jun 5, 2021, 11:12:46 AM UTC
PhotoLastModifiedTimeTimestamp  : 1622891566
PhotoTakenTimeFormatted         : Jun 1, 2015, 5:17:19 PM UTC
PhotoTakenTimeTimestamp         : 1433179039
Title                           : image-fab40b5c1a287ca303710030125d340dea494cf308087dacd00b8650a446eda9-V.jpg

Now I have to figure how to trim those file names in bulk.

things are looking up!

[StarGeek]

Using exiftool
exiftool -ext jpg "-Filename=%30f.%e" /path/to/files/

Any jpeg with a name longer than 30 characters, not counting the (dot)extension, will be truncated to 30 characters.  Hopefully there won't be any filename collisions.

[gflyer]

I have been processing takeout files year by year and it looks good now. Not perfect but good enough for me.
The number of photos with wrong dates now is very small and I can manage manually.
This is really fantastic.

I have some video files with missing creation tag, but it is in the json file.
I guess I cannot fix them with exiftool, correct?

[Phil Harvey]

ExifTool can read json files, and will copy tags from json to image files.

- Phil

[StarGeek]

Your command would be along these lines
exiftool -ext mp4 -d %s -TagsFromFile %d%f.%e.json "-createdate<creationdatetimestamp" /path/to/files/

You can run exiftool directly on the json file (use the command in FAQ #3) to see what names exiftool gives for each of the json tags.

[gflyer]

Yes I understand that.
My question was related to video files because in the error message it says that exiftool cannot write mpg

ExifTool can read json files, and will copy tags from json to image files.

- Phil

[gflyer]

Hello. Thank you again for your help.
I have tried the command below but I get this error.
Am I doing something wrong?
the "PhotoTakenTimeTimestamp" tag is from the json file.

exiftool -ext MPG -d %s -TagsFromFile %d%f.%e.json "-createdate<PhotoTakenTimeTimestamp" MOV00789.MPG

Error: Writing of MPG files is not yet supported - MOV00789.MPG
0 image files updated
1 files weren't updated due to errors


exiftool -a -G1 -s MOV00789.MPG.json

[ExifTool]      ExifToolVersion                 : 12.31
[System]        FileName                        : MOV00789.MPG.json
[System]        Directory                       : .
[System]        FileSize                        : 1725 bytes
[System]        FileModifyDate                  : 2021:09:27 10:39:18+02:00
[System]        FileAccessDate                  : 2021:09:29 14:39:10+02:00
[System]        FileInodeChangeDate             : 2021:09:29 18:04:16+02:00
[System]        FilePermissions                 : -rw-r--r--
[File]          FileType                        : JSON
[File]          FileTypeExtension               : json
[File]          MIMEType                        : application/json
[JSON]          CreationTimeFormatted           : Mar 31, 2016, 6:02:13 PM UTC
[JSON]          CreationTimeTimestamp           : 1459447333
[JSON]          Description                     :
[JSON]          GeoDataAltitude                 : 0.0
[JSON]          GeoDataLatitude                 : 0.0
[JSON]          GeoDataLatitudeSpan             : 0.0
[JSON]          GeoDataLongitude                : 0.0
[JSON]          GeoDataLongitudeSpan            : 0.0
[JSON]          GeoDataExifAltitude             : 0.0
[JSON]          GeoDataExifLatitude             : 0.0
[JSON]          GeoDataExifLatitudeSpan         : 0.0
[JSON]          GeoDataExifLongitude            : 0.0
[JSON]          GeoDataExifLongitudeSpan        : 0.0
[JSON]          ImageViews                      : 37
[JSON]          PhotoLastModifiedTimeFormatted  : Sep 11, 2021, 3:33:26 AM UTC
[JSON]          PhotoLastModifiedTimeTimestamp  : 1631331206
[JSON]          PhotoTakenTimeFormatted         : Dec 31, 2012, 10:47:58 AM UTC
[JSON]          PhotoTakenTimeTimestamp         : 1356950878
[JSON]          Title                           : MOV00789.MPG

Your command would be along these lines
exiftool -ext mp4 -d %s -TagsFromFile %d%f.%e.json "-createdate<creationdatetimestamp" /path/to/files/

You can run exiftool directly on the json file (use the command in FAQ #3) to see what names exiftool gives for each of the json tags.

[gflyer]

I have used the command below and it seems to work. Apple photo now get the correct date

exiftool -ext MPG -d %s -TagsFromFile %d%f.%e.json "-FileModifyDate<PhotoTakenTimeTimestamp" MOV00789.MPG 

[StarGeek]

My question was related to video files because in the error message it says that exiftool cannot write mpg

See the Supported File Types table.  Mpeg files are read only.  You can't write any data to them with exiftool.

I believe most, if not all, Mpegs can be losslessly converted to mp4 with a program such as FFMPeg.  An example command would be
ffmpeg -i input.mpg -map_metadata 0 -c copy output.mp4

[gflyer]

I was able to write the FileModifyDate and when I import in Apple Photo date is correct now.


exiftool -a -G1 -s MOV00789.MPG

[ExifTool]      ExifToolVersion                 : 12.31
[System]        FileName                        : MOV00789.MPG
[System]        Directory                       : .
[System]        FileSize                        : 303 MiB
[System]        FileModifyDate                  : 2012:12:31 10:47:58+01:00
[System]        FileAccessDate                  : 2012:12:31 10:47:58+01:00
[System]        FileInodeChangeDate             : 2021:09:29 22:11:58+02:00
[System]        FilePermissions                 : -rw-r--r--
[File]          FileType                        : MPEG
[File]          FileTypeExtension               : mpg
[File]          MIMEType                        : video/mpeg
[MPEG]          MPEGAudioVersion                : 1
[MPEG]          AudioLayer                      : 2
[MPEG]          AudioBitrate                    : 64 kbps
[MPEG]          SampleRate                      : 32000
[MPEG]          ChannelMode                     : Single Channel
[MPEG]          ModeExtension                   : Bands 4-31
[MPEG]          CopyrightFlag                   : True
[MPEG]          OriginalMedia                   : True
[MPEG]          Emphasis                        : None
[MPEG]          ImageWidth                      : 640
[MPEG]          ImageHeight                     : 480
[MPEG]          AspectRatio                     : 1:1
[MPEG]          FrameRate                       : 30 fps
[MPEG]          VideoBitrate                    : Variable
[Composite]     ImageSize                       : 640x480
[Composite]     Megapixels                      : 0.307

My question was related to video files because in the error message it says that exiftool cannot write mpg

See the Supported File Types table.  Mpeg files are read only.  You can't write any data to them with exiftool.

I believe most, if not all, Mpegs can be losslessly converted to mp4 with a program such as FFMPeg.  An example command would be
ffmpeg -i input.mpg -map_metadata 0 -c copy output.mp4