ExifTool Forum

ExifTool => Newbies => Topic started by: andreasmb on April 16, 2020, 09:40:13 PM

Title: Write exif data from Flickr JSON files
Post by: andreasmb on April 16, 2020, 09:40:13 PM
Hi there!

My question is very similar to the user kkaiser in this thread - https://exiftool.org/forum/index.php?topic=10780.0 (https://exiftool.org/forum/index.php?topic=10780.0), but unlike that thread, I want to write some of the metadata in the flickr JSON files to the image's Exif data.

Long story short – I'm trying to leave Flickr. And there's a lot of metadata that I've added to Flickr over the years that I'd very much like to keep – tags, album names, titles, and descriptions. This metadata is unfortunately in separate JSON files. I'd like to some of the fields in the JSON file and write it to the appropriate Exif tags on the image, so that I can integrate my Flickr photos into the rest of my collection in Lightroom.

After spending my day googling for various solutions, Exiftool seems to be the most expedient and powerful solution for this. I watched a couple of videos, read some documentation, but it seems like what I want to do is not beginners stuff. So I'm hoping for advice about which commands to look into and how many steps it should take. (Also – I don't want to mess up the 15k images that I have.) I've done some programming in the past, but apparently not enough to find my way to the right commands.

The first challenge is mapping the JSON file to the corresponding image (or MP4/MOV video). This seems like the most difficult step, since the file names are not the same. But they share a unique ID. The JSON file has this file naming convention:
photo_89015302.json (The number is the image ID).

The image file naming convention is:
mySpecialTitle_89015302_o.jpg ("mySpecialTitle" could be anything, including strings of random numbers)

Second challenge is the metadata mapping. I'd like to transfer the JSON data to the Exif data like this:
"Tags" -> Keywords
"Albums" -> Keywords (I don’t think there’s a clean way to make Flickr albums into Lightroom ’Sets’, so I’ll keep them as keywords and re-make the sets in Lightroom later)
"Name" -> Object Name
"Description" -> Caption-Abstract


Then, lastly, I'd like to rename the files, so that "date taken" is added a prefix with this format - YYYY-MM-DD. (This is something I can probably figure out myself, but I'd still appreciate any pointers.)

The time that this would save me is immeasurable, so I would be in your debt if I could make any progress on this.
Title: Re: Write exif data from Flickr JSON files
Post by: StarGeek on April 16, 2020, 10:19:49 PM
But they share a unique ID. The JSON file has this file naming convention:
photo_89015302.json (The number is the image ID).

The image file naming convention is:
mySpecialTitle_89015302_o.jpg ("mySpecialTitle" could be anything, including strings of random numbers)

Does the jpeg file always have two characters after the ID number?  And are the json files in the same directory as the image files?  If so, you would be able to map it in a command like this:
exiftool -TagsFromFile %dphoto_%-8.2f.json "-IMAGETAG<JSONTAG" /path/to/files

The %-8.2f part take the base name (mySpecialTitle_89015302_o), strip away the two trailing characters and take the last 8 characters from the end (89015302).  This is prefixed by the directory %d, the string photo_, and have the .json extension added.

The "-IMAGETAG<JSONTAG" is where you would remap the JSONTAG name into the tag in the image file.

Quote
Second challenge is the metadata mapping. I'd like to transfer the JSON data to the Exif data like this:
"Tags" -> Keywords
"Albums" -> Keywords (I don’t think there’s a clean way to make Flickr albums into Lightroom ’Sets’, so I’ll keep them as keywords and re-make the sets in Lightroom later)
"Name" -> Object Name
"Description" -> Caption-Abstract

Run this on one of the json files
exiftool -s file.json

Exiftool will output the names of the tags in the json file.  You would decide which tag you want to copy and where you want to copy it.  You seem to have the basic mapping planned out, you just need to verify the names of the json tags.

I don't have any Flickr json files on hand at the moment, but I'll try find time to get some tomorrow so I can advise further.
Title: Re: Write exif data from Flickr JSON files
Post by: andreasmb on April 16, 2020, 11:10:52 PM
Wow, StarGeek, thank you for the reply! I'll start experimenting with the commands you suggested. In the meantime, here's some answers to your questions –

Does the jpeg file always have two characters after the ID number? 
Yes, there's always a "_o" at the end of every media file name.

And are the json files in the same directory as the image files? 

No, but I could move them all into the same folder. It would be a bit of manual work, but it's not unsurmountable.

The images are batched into separate folders, like this:

...and the json files for all the media files are also batched into a smaller set of separate folders, but there's only two:


Here's an example of what one of those JSON files look like –

https://www.dropbox.com/s/xrfczynami7b2hb/2020-04%20flickr%20JSON%20example%20-%20photo_57298148.json?dl=0 (https://www.dropbox.com/s/xrfczynami7b2hb/2020-04%20flickr%20JSON%20example%20-%20photo_57298148.json?dl=0)

Hopefully that's helpful. A tricky thing with the with JSON files is that I don't know how to reference nested fields. When running exiftool on the JSON file, I get a comma-separated list of values called "Albums Title" and "Tags Tag", which is good, but I think exiftool doesn't allow for spaces in the tag titles, so I'm not sure how I can refer to them. Tags.Tag? TagsTag?
Title: Re: Write exif data from Flickr JSON files
Post by: StarGeek on April 16, 2020, 11:18:26 PM
...and the json files for all the media files are also batched into a smaller set of separate folders, but there's only two:

  • data-download 72157713880626132_375a3c67e881_part1
  • data-download 72157713880626132_375a3c67e881_part2

If you can drop them all into the same directory without any filename collisions, then you could replace the %d in the above command with the path to that directory, e.g.
exiftool -TagsFromFile /path/to/jsonfiles/photo_%-8.2f.json "-IMAGETAG<JSONTAG" /path/to/imagefiles/
Title: Re: Write exif data from Flickr JSON files
Post by: andreasmb on April 16, 2020, 11:40:50 PM
That seems like it should do the trick!

However, I'm getting an error when trying to read from the JSON file.

Code: [Select]
Warning: Error opening file - /json/photo_57298148.json
Here's a picture of file structure and the test command:
https://www.dropbox.com/s/t322vo23j18hipu/2020-04%20exiftool%20test%20output.png?dl=0[/s]

Edit: I had an extraneous forward slash in front of my folder path, which caused the issue.
Title: Re: Write exif data from Flickr JSON files
Post by: andreasmb on April 17, 2020, 02:26:55 AM
Everything seems to be working well! Thanks a million @StarGeek!

One minor hiccup seems to be that copying values from two places ("TagsTag" and "AlbumsTitle") and writing to the same field ("keywords") doesn't work. One overwrites the other, so I'm left with the values from either "TagsTag" or "AlbumsTitle". Ideally, there's the equivalent of a concat() function that lets me join or "add to" an existing exif field.

Is it possible to "add to" an exif tag? 
Title: Re: Write exif data from Flickr JSON files
Post by: Phil Harvey on April 17, 2020, 07:52:47 AM
One minor hiccup seems to be that copying values from two places ("TagsTag" and "AlbumsTitle") and writing to the same field ("keywords") doesn't work.

Use -addTagsFromFile instead of -tagsFromFile to do this.  (For more information, read Note 5 of the -tagsFromFile section in the exiftool application documentation.)

- Phil
Title: Re: Write exif data from Flickr JSON files
Post by: andreasmb on April 17, 2020, 07:38:18 PM
Brilliant... just brilliant. I'm definitely donating to the upkeep of this excellent piece of software!