I wrote a Perl script to whittle down the ~230 lines of EXIF data on a Nikon D800E .NEF file and generate a summary report. I also extract the JpgFromRaw for weeding but Nikon only provides 3 of the 65 interesting NEF tags in the JpgFromRaw.
My goal is to have the same EXIF data in the convenient JPG extract as in the original NEF.
I wrote a script to transplant the missing items, adding 9 custom tags to the .ExifTool_config for tags which were "not writable" or had other errors in SetNewValue().
The results are only 34% successful with exiftool, latest version 10.46:
- 65 tags with SetNewValue($tag, $val) -> TRUE
- 1 tag mangled (FlashMode:: JPG tag value !== NEF value)
- 22 tags copied VERBATIM
- 42 tags successfully set to new value but NOT subsequently found
1 tag mangled:
JPG tag 'FlashMode' -> 'On' != NEF_tag 'Fired, TTL Mode'!
22 tags copied VERBATIM:
Aperture ColorSpace CreateDate ExposureCompensation ExposureMode ExposureTime Flash FlashCompensation FocalLength ISO LensID LightValue MeteringMode Model Orientation SequenceNumber SerialNumber ShutterCount ShutterSpeed SubSecCreateDate UserComment WB_GRBGLevels
42 tags successfully set, but missing in JPG:
AFAperture AFAreaMode AFPointsUsed CommanderChannel CommanderGroupAManualOutput CommanderGroupAMode CommanderGroupA_TTL-AAComp CommanderInternalFlash CommanderInternalManualOutput CommanderInternalTTLComp ContrastDetectAF ContrastDetectAFInFocus DOF ExposureBracketValue ExposureDifference ExternalFlashCompensation ExternalFlashExposureComp ExternalFlashFlags FOV FlashCommanderMode FlashControlBuilt-in FlashControlMode FlashExposureComp4 FlashGNDistance FlashGroupACompensation FlashGroupAControlMode FlashSetting FlashShutterSpeed FlashSource FlashSyncSpeed FlashType FocusDistance FocusMode FocusPosition HyperfocalDistance MinFocalLength PhaseDetectAF PrimaryAFPoint RepeatingFlashOutputExternal ShootingMode VibrationReduction WhiteBalance
UserDefined tags:
ShutterCount, Aperture, LightValue, DOF, FOV, HyperfocalDistance, LensID, ShutterSpeed, WB_GRBGLevels
Of the 9 attempted UserDefines, 6 work, 3 fail (%Image::ExifTool::UserDefined)
OK: ShutterCount, Aperture, LightValue, LensID, ShutterSpeed, WB_GRBGLevels
BAD: DOF, FOV, HyperfocalDistance,
------------------------------------------------------
/home/brianp/bin/.ExifTool_config excerpt:
%Image::ExifTool::UserDefined = (
# All EXIF tags are added to the Main table, and WriteGroup is used to
# specify where the tag is written (default is ExifIFD if not specified):
'Image::ExifTool::Exif::Main' => {
# Example 1. EXIF:NewEXIFTag
0xd000 => {
Name => 'ShutterCount',
Writable => 'int32u',
WriteGroup => 'IFD0',
},
0xd002 => {
Name => 'Aperture',
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd003 => {
Name => 'LightValue',
Writable => 'rational64s',
WriteGroup => 'IFD0',
},
0xd004 => {
Name => 'DOF', << FAILS >>
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd005 => {
Name => 'FOV', << FAILS >>
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd006 => {
Name => 'HyperfocalDistance', << FAILS >>
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd007 => {
Name => 'LensID',
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd008 => {
Name => 'ShutterSpeed',
Writable => 'string',
WriteGroup => 'IFD0',
},
0xd009 => {
Name => 'WB_GRBGLevels',
Writable => 'string',
WriteGroup => 'IFD0',
},
},
------------------------------------
Full source of transplant function:
%ET_OPT=(FastScan=>1, FixBase=>1, MakerNotes=>1, Duplicates=>0, Binary=>1,
IgnoreMinorErrors=>1, List=>1, Sort=>'Input');
sub nef_exif_to_jfn() {
my(@etag, %t2v, $tag, $val, $nfn, $jfn, $ftag, $et, $jt, %ii, $verbose, $ii);
my($success, $estr, $good, $bad, $no_exif_backup, $debug, %good, %ji, @good);
my($ntag);
use Image::ExifTool qw(:Public);
$nfn='za-2017.0308-263694.nef'; # Nikon NEF
$jfn='za-2017.0308-263694.jfn.jpg'; # exiftool -b -JpgFromRaw
$debug=1; $verbose=1; $good=0; $bad=0; $no_exif_backup=0; %good=();
@etag=qw(AFAperture AFAreaMode AFPointsUsed Aperture ApproximateFocusDistance Caption Caption-Abstract Category ColorSpace ColorTemperature CommanderChannel CommanderGroupA_TTL-AAComp CommanderGroupAManualOutput CommanderGroupAMode CommanderInternalFlash CommanderInternalManualOutput CommanderInternalTTLComp ContrastDetectAF ContrastDetectAFInFocus CreateDate DOF ExposureBracketValue ExposureCompensation ExposureDifference ExposureMode ExposureTime ExternalFlashCompensation ExternalFlashExposureComp ExternalFlashFlags Flash FlashCommanderMode FlashCompensation FlashControlBuilt-in FlashControlMode FlashExposureComp4 FlashFocalLength FlashGNDistance FlashGroupACompensation FlashGroupAControlMode FlashMode FlashSetting FlashShutterSpeed FlashSource FlashSyncSpeed FlashType FocalLength FocusDistance FocusMode FocusPosition FOV HyperfocalDistance ImageAltText ImageNumber IntellectualGenre ISO Keywords LensID LightValue MeteringMode MinFocalLength Model Orientation PhaseDetectAF PrimaryAFPoint RepeatingFlashCountExternal RepeatingFlashOutputExternal Scene SequenceNumber SerialNumber ShootingMode ShutterCount ShutterSpeed SubjectCode SubSecCreateDate SupplementalCategories Tint Title UserComment VibrationReduction WB_GRBGLevels WhiteBalance);
$ftag=clone(\@etag); # Clones array, returns ref. ImageInfo mangles @
$et = new Image::ExifTool;
$jt = new Image::ExifTool;
%ii = %{$et->ImageInfo($nfn, \%ET_OPT, $ftag)}; $ii=0;
printf("M: $ii) Found: %s\n\n Searched for [%d]:\n%s\n\n",
&dumphash(\%ii, 'II hash', 0, 1), scalar @etag, join(" ", @etag))
if $verbose;
$ii=-1;
foreach $tag (sort keys %ii) {
$ii++;
$val = $ii{$tag};
($success, $estr) = $jt->SetNewValue($tag, $val);
if($success) {
print("GOOD tag $tag -> '$val'\n");
$good{$tag} = $val; next; }
print("$ii) ERROR on tag $tag -> $estr\n");
$tag = "EXIF:$tag"; # Prepend 'EXIF:' to raw tag
($success, $estr) = $jt->SetNewValue($tag, $val);
if($success) {
print("$ii) FIXED tag $tag\n"); $good{$tag} = $val; next; }
print("$ii) Failed again on tag $tag\n");
$bad++; next;
}
$good = scalar keys %good;
$ii++; # Convert from 0 based index to 1 based COUNT
print("NETJ: Tried $ii new values; $good good, $bad bad in jpg $jfn\n\n");
>>NETJ: Tried 65 new values; 65 good, 0 bad in jpg za-2017.0308-263694.jfn.jpg
if($ii == $good) {
&write_exif($jt, $jfn, $no_exif_backup, $debug); }
else { printf("Good $good != tried $ii, NOT writing JPG\n\n"); }
print("\n=============================\nChecking:\n");
undef $et; undef $jt; @good=();
$ftag=clone(\@etag); # Clones array, returns ref. ImageInfo mangles @
$jt = new Image::ExifTool;
%ji = %{$jt->ImageInfo($jfn, \%ET_OPT, $ftag)}; $ii=0;
printf("M: $ii) Found: %s\n\n Searched for [%d]:\n%s\n\n",
&dumphash(\%ji, 'II hash', 0, 1), scalar @etag, join(" ", @etag))
if $verbose;
$ii=-1; $good=0; $bad=0;
foreach $tag (sort keys %ji) {
$val = $ji{$tag};
if(defined(($ntag=$ii{$tag})) && ($ntag eq $val)) {
$good++;
push @good, $tag;
delete $ii{$tag}; # Tag -> val duplicated on II & JI
next; }
printf("ERROR %d! JPG tag '$tag' -> '$val' != NEF_tag '$ntag' !\n",
$bad++);
}
printf("M: $good good, $bad bad;\n%d tags copied VERBATIM: %s\n\n%s\n%s\n",
$good, join(" ", @good), &dumphash(\%ii, 'NEF Orphan tags', 0, 1),
join(" ", sort keys %ii));
}
=====================================
NEF EXIF extract Report (with XX indicating ERROR):
XX AFArea => Dynamic, 9 points, A7
Aperture => 8.0
XX DOF_FOV => 0.18m (1.69-1.87m), 23.7°
ExposureCompensation => -0.67
XX ExposureDifference => -1.40 -> Error=-0.7 EV Dark
XX ExposureTime => 1/250 sec, ExpMode=Manual, VibRed=On
FileName => za-2017.0308-263694.nef, 73 MB << Expected difference
XX Flash => Fired Rear, iTTL-BL +0.3EV, Commander On (-2.0EV)
XX FlashShutterSync => 1/8 - 1/250 sec
FocalLength => 82.0 mm
XX FocusDistance => 1.78 m
XX FocusMode => AF-C
XX HyperfocalDistance => 34.00 m
ISO => 640
Lens => AF-S Nikkor 28-300mm f/3.5-5.6G ED VR
LightValue => 11.3
MeteringMode => Multi-segment
Model => Nikon D800E 3000955
ShutterCount => 263694
SubSecCreateDate => 2017-03-08 14:13:04.90
UserComment => BPB_3943.nef
WB_GRBGLevels => 256 526 338 256
XX WhiteBalance => Auto1
-----------------------------------------------
EXIF values Nikon copies to JpgFromRaw
CreateDate => 2017-03-08 15:09:21
FileName => za-2017.0308-263835.jfn.o.jpg, 1497 kB
ShutterCount => 263835
----------------------------------------
Result of NEF EXIF transplanted into the JPG
Aperture => 8.0
ExposureCompensation => -0.67
ExposureTime => 1/250 sec, ExpMode=Manual
FileName => za-2017.0308-263694.jfn.jpg, 2.2 MB
Flash => On, Return not detected
FlashCompensation => -2
FlashMode => On
FocalLength => 82.0 mm
ISO => 640
Lens => AF-S Nikkor 28-300mm f/3.5-5.6G ED VR
LightValue => 11.3
MeteringMode => Multi-segment
Model => Nikon D800E 3000955
ShutterCount => 263694
ShutterSpeed => 1/250
SubSecCreateDate => 2017-03-08 14:13:04.90
UserComment => BPB_3943.nef
WB_GRBGLevels => 256 526 338 256
==================================================
Net result from log file:
ERROR 0! JPG tag 'FlashMode' -> 'On' != NEF_tag 'Fired, TTL Mode' !
M: 22 good, 1 bad;
22 tags copied VERBATIM: Aperture ColorSpace CreateDate ExposureCompensation ExposureMode ExposureTime Flash FlashCompensation FocalLength ISO LensID LightValue MeteringMode Model Orientation SequenceNumber SerialNumber ShutterCount ShutterSpeed SubSecCreateDate UserComment WB_GRBGLevels
{43}, NEF Orphan tags
=================================================
And, the command line version confirms the Perl script results exactly:
brianp@raptor:~/tmp/ltza$ /usr/local/bin/exiftool -ver -*AFAperture -*AFAreaMode -*AFPointsUsed -*CommanderChannel -*CommanderGroupAManualOutput -*CommanderGroupAMode -*CommanderGroupA_TTL-AAComp -*CommanderInternalFlash -*CommanderInternalManualOutput -*CommanderInternalTTLComp -*ContrastDetectAF -*ContrastDetectAFInFocus -*DOF -*ExposureBracketValue -*ExposureDifference -*ExternalFlashCompensation -*ExternalFlashExposureComp -*ExternalFlashFlags -*FOV -*FlashCommanderMode -*FlashControlBuilt-in -*FlashControlMode -*FlashExposureComp4 -*FlashGNDistance -*FlashGroupACompensation -*FlashGroupAControlMode -*FlashMode -*FlashSetting -*FlashShutterSpeed -*FlashSource -*FlashSyncSpeed -*FlashType -*FocusDistance -*FocusMode -*FocusPosition -*HyperfocalDistance -*MinFocalLength -*PhaseDetectAF -*PrimaryAFPoint -*RepeatingFlashOutputExternal -*ShootingMode -*VibrationReduction -*WhiteBalance za-2017.0308-263694.jfn.jpg
10.46
Flash Mode : On
Latest ExifTool version=10.46, 1 tag (FlashMode) mangled, everything else MIA.
brianp@raptor:~/tmp/ltza$ /usr/local/bin/exiftool -*FlashMode* za-2017.0308-263694.jfn.o.jpg za-2017.0308-263694.jfn.jpg za-2017.0308-263694.nef
======== za-2017.0308-263694.jfn.o.jpg << extracted from NEF
======== za-2017.0308-263694.jfn.jpg << After transplant
Flash Mode : On
======== za-2017.0308-263694.nef << Raw_file
Flash Mode : Fired, TTL Mode
FlashMode scrambled
TLDR; I suspect most of the problems you are having are due to an attempt to create individual makernote tags, which can't be done. Am I correct? If so, did you consider copying the makernotes as a block?: $et->SetNewValuesFromFile($nefFile, "MakerNotes",...);
- Phil
So it actually looks like you want ALL information that is in the NEF to be (also) available in the JPG you extracted?
Why don't you simply copy everything over then? Have a look at the copying examples (http://www.exiftool.org/exiftool_pod.html#COPYING-EXAMPLES) on how this could be achieved.
Note: The embedded JPG you extract does not contain ANY metadata at all! It's just image data so any metadata you find is the metadata that is mandatory to make it a valid jpg, nothing else...