DNG files missing recorded corrections

[Wendtjanice]

The ExifTool has been working great. Recently thought we used it and now it is not bringing over important information.

We use the ExifTool to make XMP files from DNG files. It has always provided all the information of the adjustments completed to the DNG file. But recently it stopped working. Now it is not bringing over the White Balance correction. It put it to a default instead of the recorded value.

I have tested the DNG file with several other programs to make sure that the White Balance is within the DNG and it shows in Lightroom 4, 5 and 6 as well as in Bridge. But when I use the tool to strip out the information the White Balance is no longer correct.

It was working fine last month but now it is no longer working. Do you know what might be happening?

Thanks, Janice

[Phil Harvey]

Hi Janice,

Can you tell me what exiftool command you are using and what information is no longer coming over?  Also, any ideas what could have changed in the last month to stop this from working?  Did you change the command?  Update ExifTool?  Or change something else?  If it was an ExifTool update, what versions were/are you using?

- Phil

[Wendtjanice]

Dear Phil,

What is not coming over is the White balance settings.
We used your tool in Windows. It works the same with all the different computers we have tested. Windows, 7, 8 and 10.
Also we have tried the latest update as well as previous versions.
Photoshop has updated a couple of times since we last had the tool working.
We tested DNG files that we created from raw files as well and the White balance will not come over on any of the new DNG files as well.
All versions of Lightroom can read the White balance settings. Versions of Lightroom that can read it is Lightroom 4, Lightroom 5 and Lightroom 6.
I save the data to the DNG with the corrections and then test in Bridge with the original files to see if the White Balance comes over with it but it does not. Also the text does not show the adjustment.
This image should show a white balance setting of custom  Temperature 5550 Tint +9. The data in the XMP does not reflect this adjustment.
I am providing the DNG that we adjusted, saved and then used the ExifTool.
Thank you, Janice Wendt

[Hayo Baan]

The DNG file does contain the colour temperature as well as the tint:

Code Select Expand

$ exiftool -G0:1 -ColorTemperature -Tint 07022016_DerekSara_Lidia_WED-2769.dng
[XMP:XMP-crs]   Color Temperature               : 5550
[XMP:XMP-crs]   Tint                            : +9


If you don't get this, I'd like to know what the exact command is that you used to extract the info. Also what version of exiftool are you using?

[Wendtjanice]

We are using the 10.24. on windows.

I know I see it when I read it in the DNG file but it is not coming over to the XMP file.

Thank you,
Janice Wendt

[Hayo Baan]

How do you create the XMP file?

[Wendtjanice]

We drag the set of files over into the exif tool and it creates the xmp files for us.
It is creating all the other data but the White balance is not coming over.
We are using Windows 7.

[Wendtjanice]

This is the syntax that we are using to write the xmp
exiftool -k -o %d%f.xmp

Is this creating the issue that is not writing the temp and tint?

[Phil Harvey]

Only tags with the same names in XMP will be copied over using this technique.  If there are other tags you want copied over you will either need to redirect them to another tag or create a suitably-named user-defined tag.

Check the XMP tags documentation for a complete list of available XMP tags.

- Phil

[Wendtjanice]

White balance is a normal tag that would always be contained. It can be read in the metadata and always a part of XMPs that Adobe creates.
I guess I do not understand why it is not being saved in the xmp.

[dsinai]

Hi Phil - I work with Janice and I am trying to figure out the correct syntax to create XMPs that pull everything that is stored inside the DNG.

This is what I named the file:  exiftool(-k -a -u -g1 -o %d%f.xmp)

I found this in your examples:
exiftool -a -u -g1 a.jpg
Print all meta information in an image, including duplicate and unknown tags, sorted by group (for family 1)

Even with this name, the tool is not extracting the correct white balance information in the XMP.

Can your reply with how I should name the tool in order to get every piece of meta data that is contained in a DNG file?
Thanks
David

[Hayo Baan]

As Phil wrote, with the -o option only the known tags in the XMP name space are exported. The ColorTemperature tag in the XMP-crs group is not in this list. It probably should be there though (@Phil?).

However, instead of using -o %d%f.xmp why don't you simply extract the xmp block from the file using -b -XMP -o %d%f.xmp ? This will extract all xmp (exactly!) as it is stored in the file, note though that this will not create xmp tags from e.g., exif data that wasn't already in xmp (something your original command might have done in some cases).

[Phil Harvey]

OK, I see the problem with XMP-crs:ColorTemperature.  It is flagged as a tag to Avoid when writing (indicated by the "/" after the format in the tag documentation).  There are a number of XMP-crs tags that ExifTool avoids writing due to name conflicts with XMP-exif tags.  However, there is no XMP-exif:ColorTemperature (or any other XMP:ColorTemperature for that matter), so it shouldn't be flagged as "Avoid".  I will change this in the next release.

But we need to figure out what you are trying to do.

A) If you want to copy the EXIF tags from the DNG into XMP, then your command is correct (but you can drop the -a -u -g1 because they don't do anything).

B) If you want to copy XMP tags from the DNG to XMP, then you should preserve the original groups by adding -xmp:all.  If you do this, then ExifTool will write the XMP-crs:ColorTemperature.  (Either that, or copy the xmp as a block, as Hayo suggested.  Then even tags unknown to ExifTool will be copied.)

A+B) Or maybe you want to do both (which may be a good idea), in which case you should use this command:

exiftool -o %d%f.xmp -all -xmp:all -ext dng DIR

(I have added -ext dng so only DNG files will be processed.)

Unfortunately you can't put -xmp:all in the brackets of the exiftool.exe name because the colon is not allowed in a Windows file name.  Instead, you could either create a shortcut that adds these arguments or use a .bat file, whichever you prefer.

- Phil

[dsinai]

Phil and Hayo - We appreciate your quick responses.

Thanks
David

[dsinai]

Please look out for a donation from david@sinaifamily.com.  We love your tool!

[Phil Harvey]

Received.  Thanks!!

- Phil

[Wendtjanice]

Thank you for working with us. Unfortunately 
exiftool -o %d%f.xmp -all -xmp:all -ext dng DIR
Does not work the white balance is still not writing.

-b -XMP -o %d%f.xmp  This is not working because the image rating and a lot of other data is missing so it is not all writing to the xmp files.


When this is used -xmp:all  the data can all be read but a XMP file is not created. The information stays in the terminal window.


Thank you, Janice

[Phil Harvey]

Hi Janice,

Sorry, you are correct.  My mistake -- I should read the documentation.  The implied destionation group of "XMP" wasn't specific enough.  This is the command to use:

exiftool -o %d%f.xmp -all "-xmp:all>all:all" -ext dng DIR

This will copy the XMP to the same namespace in the destination file.

- Phil

[Wendtjanice]

Dear Phil,

Now I get the lens information but the white balance disappeared again.

It seems that I cannot get both the lens information and the white balance, with the temperature and tint to come through.

Janice Wendt

[Wendtjanice]

Dear Phil,

This is the data the tool can read from the DNG.
How do I get all of this to come over into the XMP?

When I get on the other fails but all of the data is there.


ExifTool Version Number         : 10.24
File Name                       : 07022016_DerekSara_Lidia_WED-2839.dng
Directory                       : /Users/janicewendt/Desktop/Joel test/Mactest
File Size                       : 2.1 MB
File Modification Date/Time     : 2016:07:27 14:07:53-07:00
File Access Date/Time           : 2016:07:28 14:46:55-07:00
File Inode Change Date/Time     : 2016:07:27 14:07:53-07:00
File Permissions                : rw-r--r--
File Type                       : DNG
File Type Extension             : dng
MIME Type                       : image/x-adobe-dng
Exif Byte Order                 : Little-endian (Intel, II)
Make                            : Canon
Camera Model Name               : Canon EOS 6D
Strip Offsets                   : 130198
Orientation                     : Rotate 270 CW
Rows Per Strip                  : 171
Strip Byte Counts               : 9661
Software                        : Adobe Photoshop Lightroom 4.4 (Windows)
Modify Date                     : 2016:07:07 12:01:18
Artist                          : Lidia Sharapova
Image Width                     : 3000
Image Height                    : 2000
Bits Per Sample                 : 8 8 8
Compression                     : Lossy JPEG
Photometric Interpretation      : Linear Raw
Samples Per Pixel               : 3
Planar Configuration            : Chunky
Tile Width                      : 512
Tile Length                     : 512
Tile Offsets                    : (Binary data 182 bytes, use -b option to extract)
Tile Byte Counts                : (Binary data 148 bytes, use -b option to extract)
Black Level Repeat Dim          : 1 1
Black Level                     : 0 0 0
White Level                     : 255 255 255
Default Scale                   : 1 1
Default Crop Origin             : 0 0
Default Crop Size               : 3000 2000
Anti Alias Strength             : 1
Best Quality Scale              : 1
Opcode List 2                   : (Binary data 256 bytes, use -b option to extract)
Subfile Type                    : Reduced-resolution image
Preview Image Start             : 139860
Preview Image Length            : 48412
Y Cb Cr Coefficients            : 0.299 0.587 0.114
Y Cb Cr Sub Sampling            : YCbCr4:2:0 (2 2)
Y Cb Cr Positioning             : Co-sited
Reference Black White           : 0 255 128 255 128 255
XMP Toolkit                     : Adobe XMP Core 5.6-c011 79.156380, 2014/05/21-23:38:37
Creator Tool                    : Adobe Photoshop Lightroom 4.4 (Windows)
Metadata Date                   : 2016:07:27 14:07:53-07:00
Rating                          : 1
Format                          : image/dng
Lens                            : 35mm
Approximate Focus Distance      : 2.22
Flash Compensation              : 0
Firmware                        : 1.1.6
Date Created                    : 2016:07:02 11:02:10.00
Document ID                     : xmp.did:9113c810-5869-4147-8f02-e2c8b088186c
Original Document ID            : 0C8B71834D0C6918E9ABE304C2A231CB
Instance ID                     : xmp.iid:04ebded1-eef3-4461-a73a-e2a42cbb65de
Version                         : 9.6
Process Version                 : 6.7
White Balance                   : Custom
Auto White Version              : 134348800
Color Temperature               : 4650
Tint                            : +5
Saturation                      : +5
Sharpness                       : 55
Luminance Smoothing             : 10
Color Noise Reduction           : 25
Vignette Amount                 : 0
Shadow Tint                     : 0
Red Hue                         : 0
Red Saturation                  : 0
Green Hue                       : 0
Green Saturation                : 0
Blue Hue                        : 0
Blue Saturation                 : 0
Vibrance                        : +5
Hue Adjustment Red              : 0
Hue Adjustment Orange           : 0
Hue Adjustment Yellow           : 0
Hue Adjustment Green            : 0
Hue Adjustment Aqua             : 0
Hue Adjustment Blue             : 0
Hue Adjustment Purple           : 0
Hue Adjustment Magenta          : 0
Saturation Adjustment Red       : 0
Saturation Adjustment Orange    : 0
Saturation Adjustment Yellow    : 0
Saturation Adjustment Green     : 0
Saturation Adjustment Aqua      : -3
Saturation Adjustment Blue      : -33
Saturation Adjustment Purple    : 0
Saturation Adjustment Magenta   : 0
Luminance Adjustment Red        : 0
Luminance Adjustment Orange     : 0
Luminance Adjustment Yellow     : 0
Luminance Adjustment Green      : 0
Luminance Adjustment Aqua       : 0
Luminance Adjustment Blue       : 0
Luminance Adjustment Purple     : 0
Luminance Adjustment Magenta    : 0
Split Toning Shadow Hue         : 0
Split Toning Shadow Saturation  : 0
Split Toning Highlight Hue      : 0
Split Toning Highlight Saturation: 0
Split Toning Balance            : 0
Parametric Shadows              : 0
Parametric Darks                : 0
Parametric Lights               : 0
Parametric Highlights           : 0
Parametric Shadow Split         : 25
Parametric Midtone Split        : 50
Parametric Highlight Split      : 75
Sharpen Radius                  : +0.8
Sharpen Detail                  : 15
Sharpen Edge Masking            : 40
Post Crop Vignette Amount       : 0
Grain Amount                    : 0
Luminance Noise Reduction Detail: 50
Color Noise Reduction Detail    : 50
Luminance Noise Reduction Contrast: 0
Color Noise Reduction Smoothness: 50
Lens Profile Enable             : 1
Lens Manual Distortion Amount   : 0
Perspective Vertical            : 0
Perspective Horizontal          : 0
Perspective Rotate              : 0.0
Perspective Scale               : 100
Perspective Aspect              : 0
Perspective Upright             : 0
Perspective X                   : 0.00
Perspective Y                   : 0.00
Auto Lateral CA                 : 1
Exposure 2012                   : +0.20
Contrast 2012                   : 0
Highlights 2012                 : -55
Shadows 2012                    : +36
Whites 2012                     : +31
Blacks 2012                     : -5
Clarity 2012                    : +10
Defringe Purple Amount          : 0
Defringe Purple Hue Lo          : 30
Defringe Purple Hue Hi          : 70
Defringe Green Amount           : 0
Defringe Green Hue Lo           : 40
Defringe Green Hue Hi           : 60
Dehaze                          : 0
Tone Map Strength               : 0
Convert To Grayscale            : False
Tone Curve Name                 : Medium Contrast
Tone Curve Name 2012            : Linear
Camera Profile                  : Adobe Standard
Camera Profile Digest           : 98BA1AFA1155D0472068BB57D3655975
Lens Profile Setup              : Custom
Lens Profile Name               : Adobe (Nikon AF-S NIKKOR 24-120mm f/4G ED VR) v2
Lens Profile Filename           : NIKON D3X (Nikon AF-S NIKKOR 24-120mm f4G ED VR) v2 - RAW.lcp
Lens Profile Digest             : 5D975A315226B9BFF48E008D11F2D4DC
Lens Profile Distortion Scale   : 100
Lens Profile Chromatic Aberration Scale: 100
Lens Profile Vignetting Scale   : 100
Upright Version                 : 151388160
Upright Center Mode             : 0
Upright Center Norm X           : 0.5
Upright Center Norm Y           : 0.5
Upright Focal Mode              : 0
Upright Focal Length 35mm       : 35
Upright Preview                 : False
Upright Transform Count         : 6
Upright Four Segments Count     : 0
Has Settings                    : True
Crop Top                        : 0.082346
Crop Left                       : 0.014774
Crop Bottom                     : 1
Crop Right                      : 0.884412
Crop Angle                      : 1.43539
Crop Constrain To Warp          : 0
Has Crop                        : True
Already Applied                 : False
Raw File Name                   : 07022016_DerekSara_Lidia_WED-2839.dng
Creator                         : Lidia Sharapova
Rights                          : (c)Lidia Sharapova
History Action                  : derived, saved, saved
History Parameters              : converted from image/x-canon-cr2 to image/dng, saved to new location
History Instance ID             : xmp.iid:9113c810-5869-4147-8f02-e2c8b088186c, xmp.iid:04ebded1-eef3-4461-a73a-e2a42cbb65de
History When                    : 2016:07:07 12:01:18-05:00, 2016:07:27 14:07:53-07:00
History Software Agent          : Adobe Photoshop Lightroom 4.4 (Windows), Adobe Photoshop Lightroom 6.6 (Macintosh)
History Changed                 : /, /metadata
Derived From Document ID        : 0C8B71834D0C6918E9ABE304C2A231CB
Derived From Original Document ID: 0C8B71834D0C6918E9ABE304C2A231CB
Tone Curve                      : 0, 0, 32, 22, 64, 56, 128, 128, 192, 196, 255, 255
Tone Curve Red                  : 0, 0, 255, 255
Tone Curve Green                : 0, 0, 255, 255
Tone Curve Blue                 : 0, 0, 255, 255
Tone Curve PV2012               : 0, 0, 255, 255
Tone Curve PV2012 Red           : 0, 0, 255, 255
Tone Curve PV2012 Green         : 0, 0, 255, 255
Tone Curve PV2012 Blue          : 0, 0, 255, 255
Copyright                       : (c)Lidia Sharapova
Exposure Time                   : 1/160
F Number                        : 2.5
Exposure Program                : Manual
ISO                             : 1600
Sensitivity Type                : Recommended Exposure Index
Recommended Exposure Index      : 1600
Exif Version                    : 0230
Date/Time Original              : 2016:07:02 11:02:10
Create Date                     : 2016:07:02 11:02:10
Shutter Speed Value             : 1/160
Aperture Value                  : 2.5
Exposure Compensation           : 0
Max Aperture Value              : 1.4
Metering Mode                   : Multi-segment
Flash                           : Off, Did not fire
Focal Length                    : 35.0 mm
Focal Plane X Resolution        : 152
Focal Plane Y Resolution        : 152
Focal Plane Resolution Unit     : mm
Custom Rendered                 : Normal
Exposure Mode                   : Manual
Scene Capture Type              : Standard
Serial Number                   : 323051001273
Lens Info                       : 35mm f/?
Lens Model                      : 35mm
Lens Serial Number              : 0000000000
Image Number                    : 0
DNG Version                     : 1.4.0.0
DNG Backward Version            : 1.4.0.0
Unique Camera Model             : Canon EOS 6D
Color Matrix 1                  : 0.7546 -0.1435 -0.0929 -0.3846 1.1488 0.2692 -0.0332 0.1209 0.637
Color Matrix 2                  : 0.7034 -0.0804 -0.1014 -0.442 1.2564 0.2058 -0.0851 0.1994 0.5758
Camera Calibration 1            : 0.9844 0 0 0 1 0 0 0 0.9569
Camera Calibration 2            : 0.9844 0 0 0 1 0 0 0 0.9569
Analog Balance                  : 1 1 1
As Shot Neutral                 : 0.58783 1 0.490187
Baseline Exposure               : 0.25
Baseline Noise                  : 0.8
Baseline Sharpness              : 1.25
Linear Response Limit           : 1
Camera Serial Number            : 323051001273
DNG Lens Info                   : 35mm f/?
Shadow Scale                    : 1
Calibration Illuminant 1        : Standard Light A
Calibration Illuminant 2        : D65
Raw Data Unique ID              : 00000000D10421E142314F6FD5E8995E
Original Raw File Name          : 07022016_DerekSara_Lidia_WED-2839.CR2
Camera Calibration Sig          : com.adobe
Profile Calibration Sig         : com.adobe
Profile Name                    : Adobe Standard
Profile Hue Sat Map Dims        : 90 30 1
Profile Hue Sat Map Data 1      : (Binary data 137566 bytes, use -b option to extract)
Profile Hue Sat Map Data 2      : (Binary data 138373 bytes, use -b option to extract)
Profile Embed Policy            : Allow Copying
Profile Copyright               : Copyright 2012 Adobe Systems, Inc.
Forward Matrix 1                : 0.7763 0.0065 0.1815 0.2364 0.8351 -0.0715 -0.0059 -0.4228 1.2538
Forward Matrix 2                : 0.7464 0.1044 0.1135 0.2648 0.9173 -0.182 0.0113 -0.2154 1.0292
Preview Application Name        : Adobe Photoshop Lightroom
Preview Application Version     : 4.4
Preview Settings Digest         : 92127ca164f19e53bcea1ee0340d1f58
Preview Color Space             : sRGB
Preview Date Time               : 2016:07:07 12:01:18-05:00
Profile Look Table Dims         : 36 8 16
Profile Look Table Data         : (Binary data 89143 bytes, use -b option to extract)
Noise Profile                   : 0.000212738231479362 3.95745008879821e-07 0.000212034790569924 3.95863756131935e-07 0.000208419928282597 4.00886066383125e-07
Original Default Final Size     : 5472 3648
Original Best Quality Size      : 5472 3648
Original Default Crop Size      : 5472 3648
New Raw Image Digest            : 2416c7b0a36406ebe3ccaff78b0b4996
Aperture                        : 2.5
Image Size                      : 3000x2000
Megapixels                      : 6.0
Preview Image                   : (Binary data 48412 bytes, use -b option to extract)
Scale Factor To 35 mm Equivalent: 1.8
Shutter Speed                   : 1/160
Circle Of Confusion             : 0.016 mm
Field Of View                   : 31.5 deg
Focal Length                    : 35.0 mm (35 mm equivalent: 63.8 mm)
Hyperfocal Distance             : 29.75 m
Lens ID                         : Sigma 35mm f/1.4 DG HSM
Light Value                     : 6.0
MacBook-Pro:~ janicewendt$

[Phil Harvey]

It is more complicated that you think.  You should read FAQ number 3

The listing you gave in your last post doesn't help at all because you probably didn't extract duplicate tags and I can't tell what groups they are in.  For the DNG you posted:

Code Select Expand

> exiftool ~/Desktop/07022016_DerekSara_Lidia_WED-2769.dng -a -G1 -whitebalance -colortemperature
[XMP-crs]       White Balance                   : Custom
[ExifIFD]       White Balance                   : Auto
[XMP-crs]       Color Temperature               : 5550

So there are two white balance tags (with different values).

But if you use the last command I gave, this is what I get:

Code Select Expand

> exiftool -o %d%f.xmp -all "-xmp:all>all:all" -ext dng ~/Desktop/
    1 directories scanned
    1 image files created
> exiftool ~/Desktop/07022016_DerekSara_Lidia_WED-2769.xmp -a -G1 -whitebalance -colortemperature
[XMP-crs]       White Balance                   : Custom
[XMP-exif]      White Balance                   : Auto
[XMP-crs]       Color Temperature               : 5550

Both WhiteBalance tags and the ColorTemperature tag were copied just fine to the XMP.  So I don't understand the problem you are having.

- Phil