JSON import fails for CanonVRD tags

[andre-7d]

I found a few topics about reading/writing Canon DPP3 and DPP4 recipes, but none seems to go the path I'm trying to explore, which is to map specific tag IDs and values between these two tag blocks.

Let me describe what I'm reading/writing and then ask the question.

I obtain all DPP3 tags from the CanonVRD group. In the simplest case, I can get a list of tags with tag IDs with this command:

Code Select Expand

exiftool -a -H -s -G5
    -CanonVRD:TIFF-Trailer-CanonVRD-EditData-VRD1:all @_MG_2280.CR2
In more practical terms, once I know the tag IDs, I just get them with direct tag references, like this:

Code Select Expand

exiftool -a -H -s
    -CanonVRD:ID-0x020e:RGBCurvePoints
    -CanonVRD:ID-0x0038:RawBrightnessAdj @_MG_2280.CR2
In order to apply these values as DPP4 recipes, I get a list of all DPP4 tag IDs with this command (with a couple more TIFF trailer groups).

Code Select Expand

exiftool -a -H -s -G5
    -CanonVRD:TIFF-Trailer-CanonVRD-Edit4Data-DR4:all
    -CanonVRD:TIFF-Trailer-CanonVRD-Edit4Data-DR4-CropInfo:all @_MG_2280.CR2
I then apply values from DPP3 to the DPP4 tags:

Code Select Expand

exiftool -a -H -s -G5
    -CanonVRD:ID-0x20001:RawBrightnessAdj=0.33
    -CanonVRD:ID-0x0007:RGBCurvePoints="(0,0) (74,61) (173,183) (255,255)"
    @_MG_2280.CR2
Some tags need to be remapped, such as CropLeft/CropTop in DPP3 need to be set as CropX/CropY in DPP4.

This brings me to the question. I figured JSON import would be the most straightforward way to remap all the values, rather than specify them all on the command line, so I tried to import this JSON file:

Code Select Expand

[{
  "SourceFile": "@_MG_2280.CR2",
  "RGBCurvePoints": {
    "id": "0x0007",
    "val": "(0,0) (74,61) (173,183) (255,255)"
  },
  "RawBrightnessAdj": {
    "id": "0x20001",
    "val": 1.33
  }
}]
, and exiftool gives me this message:

Code Select Expand

exiftool -v -json=dr4.json @_MG_2280.CR2
Reading JSON file dr4.json
======== @_MG_2280.CR2
Setting new values from JSON database
Can't write a structure to CanonVRD:RGBCurvePoints
Can't write a structure to CanonVRD:RawBrightnessAdj
Nothing changed in @_MG_2280.CR2
    0 image files updated
    1 image files unchanged
If I use explicit TIFF group, similar to how this command generates:

Code Select Expand

exiftool -a -H -G:5 -json
    -CanonVRD:RGBCurvePoints
    -CanonVRD:RawBrightnessAdj @_MG_2280.CR2
, which looks like this:

Code Select Expand

[{
  "SourceFile": "@_MG_2280.CR2",
  "TIFF-Trailer-CanonVRD-Edit4Data-DR4-ToneCurve:RGBCurvePoints": {
    "id": "0x0007",
    "val": "(0,0) (74,61) (173,183) (255,255)"
  },
  "TIFF-Trailer-CanonVRD-Edit4Data-DR4:RawBrightnessAdj": {
    "id": "0x20001",
    "val": 1.33
  }
}]
, it doesn't work either and errors out with this:

Code Select Expand

exiftool -v -json=dr4.json @_MG_2280.CR2
Reading JSON file dr4.json
======== @_MG_2280.CR2
Setting new values from JSON database
Sorry, TIFF-Trailer-CanonVRD-Edit4Data-DR4-ToneCurve:RGBCurvePoints doesn't exist or isn't writable
Sorry, TIFF-Trailer-CanonVRD-Edit4Data-DR4:RawBrightnessAdj doesn't exist or isn't writable
Nothing changed in @_MG_2280.CR2
    0 image files updated
    1 image files unchanged
Any thoughts on how I can make these JSON imports work?

DPP4 tags seem to come from CanonVRD::CanonDR4 and I thought I'm missing this group reference somewhere, but it appears that exiftool flattens CanonDR4 into the CanonVRD group, so specifying TIFF trailer seems to be the intended way, but maybe I'm misinterpreting this.

Any help would be greatly appreciated.

[StarGeek]

This will probably require an answer from Phil, who is away until next week.

[andre-7d]

After some experimentation, here's JSON syntax that does work.

Code Select Expand

[{
  "SourceFile": "@_MG_2280.CR2",
  "CanonVRD:ID-0x0007:RGBCurvePoints": "(0,0) (74,61) (173,183) (255,255)",
  "CanonVRD:ID-0x20001:RawBrightnessAdj": -1.33
}]

It appears that exiftool tries to write the value verbatim, without recognizing an expanded JSON variant and looking for var.

If anyone can comment on whether there could be a tag ID conflict here between VRD and DR4 tags, I would appreciate a comment.

In other words, intuitively, I would expect (conceptually) CanonVRD:CanonDR4:RawBrightnessAdj, so the tag is in its own group. Is using CanonVRD:ID-* nomenclature safe in terms of tag ID collisions?

[andre-7d]

@StarGeek

Sorry, didn't notice your reply. Thanks for the heads up. Happy to wait for Phil to comment.

[Phil Harvey]

Right.  -D and -H JSON outputs are not compatible with the import file format.  I'll add this to the documentation.

I can't comment on the tag ID collision possibility without studying this in more detail, but I don't have the time to do this right now.

- Phil

[andre-7d]

Thanks for responding, Phil.

One way to avoid referencing tag IDs (conceptually) is to explicitly specify their group, so tag names are not ambiguous, but CanonDR4 group seems to be collapsed into its parent - the CanonVRD group, so CanonVRD:RawBrightnessAdj refers to whichever happens to be first in the list, and exiftool can report either, depending on that order.

Is there a reason DPP3 and DPP4 tags are not in their separate groups, so CanonVRD:RawBrightnessAdj refers only to the DPP3 value and CanonVRD:CanonDR4:RawBrightnessAdj refers to the DPP4 value?

[andre-7d]

Also, wanted to clarify/correct this post (cannot edit it anymore).

https://exiftool.org/forum/index.php?topic=14288.msg76959#msg76959

This sentence from that post should read:

It appears that exiftool tries to write the value verbatim, without recognizing the expanded JSON variant and using values in val.

The JSON in the example in that post itself was manually constructed with the group name and tag IDs.

The only way I'm thinking converting DPP3 recipes to DPP4 would work is via a JSON transform step. In other words,

1. Run this command to get all recipe tags:

Code Select Expand

exiftool -a -json -H -G:5 -CanonVRD:* @_MG_2280.CR2

2. Run a JSON transform to extract TIFF-Trailer-CanonVRD-EditData-VRD1 tags and use their IDs and values to construct a matching value using the DPP4 tag ID value. In other words, find each of these pairs:

Code Select Expand

[{
  "SourceFile": "@_MG_2280.CR2",
  "TIFF-Trailer-CanonVRD-EditData-VRD1:RawBrightnessAdj": {
    "id": "0x0038",
    "val": 1.33
  },
  "TIFF-Trailer-CanonVRD-Edit4Data-DR4:RawBrightnessAdj": {
    "id": "0x20001",
    "val": 0
  }
}]

, and construct the JSON exiftool will accept to change all listed values via the tag ID syntax, like this.

Code Select Expand

[{
  "SourceFile": "@_MG_2280.CR2",
  "CanonVRD:ID-0x0007:RGBCurvePoints": "(0,0) (74,61) (173,183) (255,255)",
  "CanonVRD:ID-0x20001:RawBrightnessAdj": 1.33,
...
}]

3. Apply the resulting JSON file:

Code Select Expand

exiftool -json=vrd-to-dr4.json @_MG_2280.CR2


[Phil Harvey]

OK.  In ExifTool 12.53 I will change the family 1 group (ie. "location") of the DR4 tags to CanonDR4.  Older VRD tags will stay as CanonVRD.  This will allow you address all DR4 tags with

exiftool -canondr4:all FILE

or all older VRD tags with

exiftool -1canonvrd:all FILE

using the optional leading family number on the group name.

- Phil

[andre-7d]

This is great. It is much appreciated. Thank you, Phil.

[DanielOquinn]

Thanks for helping me out as well.