Hi Phil,
The Panasonic Raw data extraction is broken in the new version. The following flags were working well in 8.95, but produce no output in the new version:
exiftool -PanasonicRaw_0x0119 -xmlformat *.RW2
Do we know what the problem is?
Thanks
Andrew
I see that you've changed the name of this tag to DistortionInfo, which is OK but I had expected the old one to keep working as well. I need a tagname which returns the whole block of data as a single unit, as the old version did, not the individual integer values. Please can you restore this ASAP?
I also notice that you've added extraction of the values according to Raphael Rigo's algorithm. You need to know that this algorithm hasn't been tested and is incorrect for most MFT lenses.
I'm running a little project to get a better handle on these rules. See www.andrewj.com/mft (http://www.andrewj.com/mft) for details. In the meantime it's probably not a good idea to hard-code Raphael's interpretation of the values.
Andrew
Hi Andrew,
Thanks for the information. You can override this tag with a user-defined tag to return the data block if you want.
- Phil
Hi Phil,
I'm not sure I know how to do that, and I don't want anything complicated as I'm building the command to handle this in code. It would still be better if there was a tag which just gets the block.
If I do have any choice in the matter, the ideal for me would be that ExifTool has a tag which outputs this (and possible the similar CA correction block) as the hex representation (a single string of 32 characters). I wouldn't then have to xml-encode and decode the values.
Can you do this?
Thanks
Andrew
This config file will do what you want:
%Image::ExifTool::UserDefined = (
'Image::ExifTool::PanasonicRaw::Main' => {
0x0119 => {
Name => 'DistortionInfo',
PrintConv => 'unpack("H*",$val)',
},
},
);
1; # end
Then just use "-config CONFIG_FILE_NAME" as the first option in your command.
- Phil
Hi Phil,
That's not a valid approach, as I'm deploying my exiftool command to a large number of other users, and I have no control over their config files.
Please restore the ability to get this information in a single tag, ideally as a hex string, or as an ascii string as previously. Either is acceptable, but I really do need this as a single tag.
Thanks
Andrew
Hi Andrew,
Quote from: andrewj on August 31, 2012, 09:23:47 AM
That's not a valid approach, as I'm deploying my exiftool command to a large number of other users, and I have no control over their config files.
The
-config option applies the specified config file which you would ship as part of your application, overriding any config file that the user may have -- this isn't acceptable?
- Phil
No, I'm building a command line in code, within a plug-in to the Bibble/ASP RAW processor, and I have no access to the user's exiftool config. Whatever I do has to work with a bog-standard exiftool install.
Please, pretty please, can we just restore something like the previous behaviour. This worked beautifully, and I didn't have to jump through any hoops.
Thanks
Andrew
Hi Andrew,
I'm not sure you understand. This does work with a bog-standard exiftool install. You don't need access to the User's config file.
Or are you saying that it is not possible for you to include this custom config file in your distribution? If so, this is a problem.
Permanently restoring the previous behaviour is not an option. The whole idea of ExifTool is to decode embedded metadata and present it in a meaningful way to the user. Unknown tags only stay as binary blobs until their meaning is known.
- Phil
Dear Phil,
There is no way for me to include a custom config file with my tool. Please accept that. I need to use features directly available from the command line with no other settings or files.
The interpretation of the values is not fixed, at least not at the moment. Raphael Rigo's work was a start, but I have positively established that his algorithm is not correct for the majority of lenses. Therefore the labels you are currently putting on some of the values are incorrect.
What we do know is that this block contains geometric correction data, and that it's a block of 16, 16-bit values. Therefore I can't see that it compromises exiftool if you support a way to get the individual values, plus a way to get the values as a block.
I am hoping that the project I am running will find a better algorithm to turn this data into the correction values used to drive RAW processors. When we get that, you will be very welcome to add that interpretation of the values, but I suspect that many people will still need access to the underlying array of values.
Does this make sense?
Andrew
Hi Andrew,
I understand your situation now. Unfortunate, but not unworkable.
You can get what you need by parsing the output of the -v3 option, independent of how ExifTool decodes this information.
- Phil
Dear Phil,
Sorry, that's not what I need at all. I need to store a small number of values per image in an XML file per image, which are read at a later date when the image is processed. I am interested in and already use several other values as well as the Panasonic correction data. Each value needs to sit neatly in a single XML element. I have an existing programming model which works well for all the other data, and worked well for the Panasonic data as a single base-64 string.
Please can we make it simple, rather than complicated?
I'd like to suggest a compromise solution, as otherwise I have to abandon what was going to be a promising project, or tell the other participants that they can only use exiftool 8.95 or lower. I don't want to do either of those things.
How About:
DistortionInfo00 .. DistortionInfo15 return the individual values
DistortionInfoAll returns the 32 byte block as a hex-encoded string
Then later, when we understand the algorithm, we can add DistortionValueA etc.
Can you do this, or something similar?
Thanks
Andrew
Hi Andrew,
Quote from: andrewj on August 31, 2012, 02:03:14 PM
Sorry, that's not what I need at all. I need to store a small number of values per image in an XML file per image, which are read at a later date when the image is processed. I am interested in and already use several other values as well as the Panasonic correction data. Each value needs to sit neatly in a single XML element. I have an existing programming model which works well for all the other data, and worked well for the Panasonic data as a single base-64 string.
I understand. You had something that works and you are angry that I have broken it.
QuotePlease can we make it simple, rather than complicated?
Unfortunately, simple for you is very complicated for me. I would have to introduce a new model to allow the same tag to be accessed by different names (other than the existing user-defined tag mechanism), and a new interface to present this new functionality to the user. All because your unique situation doesn't allow you to use a config file. It seems like a lot of work for me, and an unnecessary complication for ExifTool, since the user-defined tag option works for everyone else. :(
QuoteI'd like to suggest a compromise solution, as otherwise I have to abandon what was going to be a promising project, or tell the other participants that they can only use exiftool 8.95 or lower. I don't want to do either of those things.
Yes. That would be unfortunate. All because you can't include a config file with your distribution. :(
- Phil
Quote from: Phil Harvey on August 31, 2012, 08:09:54 PM
Unfortunately, simple for you is very complicated for me. I would have to introduce a new model to allow the same tag to be accessed by different names (other than the existing user-defined tag mechanism), and a new interface to present this new functionality to the user.
If this is a genuine restriction, then the best thing you can do with this tag is present the "raw" data. Based on my research so far there is not going to be a simple "one size fits all" algorithm where value 5 is always correction parameter a, for example. What you have done at the moment is not correct, and Raphael who hosts syscall.eu acknowledges that. I suspect users of this data will need to choose between different algorithms to find those which work for their lenses.
Please understand that I am very restricted in what I can do here. Whatever I do needs to work in a single command line to extract both this data and the other values I need, and I have no control over the exterior environment. I really need a "tag" which does this, which we had before.
If you can't support direct output of this data any longer, then I have two suggestions:
1. Can I build the user defined tag in the same command line which uses it, in a way which will work across platforms?
2. Can you build this in as a "standard" user defined tag?
Thanks
Andrew
Quote from: andrewj on September 01, 2012, 12:33:00 AM
If you can't support direct output of this data any longer, then I have two suggestions:
Andrew, isn't there a third option: using the last ExifTool version that worked for you instead of the latest release?
Uwe
That will work for me, but I'm trying to run a project with several participants, and then I want to build the lens correction logic into a plugin available to all users of Bibble or ASP. I can't place that restriction on that whole community.
Thanks
Andrew
Hi Andrew,
Quote from: andrewj on September 01, 2012, 12:33:00 AM
1. Can I build the user defined tag in the same command line which uses it, in a way which will work across platforms?
No. There is no mechanism to do this.
Quote2. Can you build this in as a "standard" user defined tag?
There are no "standard" user-defined tags (other than the ones in the sample config file).
- Phil
Andrew,
What about hard-coding the config file that Phil was talking about, and writing it to a temporary file when your plug-in is started? Then you could simply run the config file through ExifTool and access the data you need. Wouldn't that work for you?
Uwe
@Uwe - Thanks. I'd come up with a similar scheme myself, and I think it will work, so that will solve my immediate problem.
@Phil - I am worried that exiftool now only has a known, invalid interpretation of this data. I still think that it would make most sense to make the integer array available directly as well. I will keep you posted as we find out more about the data, but please review whether the current approach is the right one.
Thanks all,
Andrew
Thanks Andrew,
I'm glad you figured out a way to work with the user-defined tag.
The decoding of this information will certainly evolve as more is learned.
About your web page: I can't find any mention of the actual formula that you are using. Raphael gives his formula (although he fails to explain his variables), but his A and C are reversed compared to yours.
- Phil
Hi Phil,
There's a second page under the link "known algorithms" which lists all the algorithms I've identified so far:
http://www.andrewj.com/mft/algorithms.asp (http://www.andrewj.com/mft/algorithms.asp)
Hopefully this gives you a feel for the potential complexity.
Andrew
Hi Andrew,
Thanks, but I saw this page already. This page just lists the way you calculate the constants for some formula, but I don't see the formula. Raphael used this formula:
Ru = scale*(Rd + a*Rd^3 + b*Rd^5 + c*Rd^7)
but didn't explain the meaning of Ru and Rd (although I think I can guess).
It seems your formula is different. Perhaps this:
Ru = scale*(Rd + a*Rd^7 + b*Rd^5 + c*Rd^3)
- Phil
Dear Phil,
The formula doesn't change, it's just the standard formula for lens geometric correction (see, for example, WikiPedia), and is embedded in the RAW processing software. "All" we have to do is work out the parameters a, b and c. These should derive from the MFT data, but it's not a 1:1 correspondence.
Andrew
Hi Andrew,
Maybe I'm stupid, but I just don't get it. Wikipedia describes Brown's distortion model (http://en.wikipedia.org/wiki/Lens_Distortion#Software_correction), which is a completely different formula, not involving a, b or c at all.
It could be that your a, b and c correspond to K1, K2, K3 in the Brown's equation (and the tangential distortion coefficients are all zero?), but this is different from Raphael's equation (http://syscall.eu/#pana) which used powers of 3, 5 and 7 and not 2, 4 and 6 as in Brown's model.
- Phil
Edit: Also, you still haven't explained why your "a" and "c" are reversed from Raphael's.
Hi Phil,
Maybe you have hit on one of the sources of confusion. I have a RAW processor into which I can put three values labelled a, b and c. I assumed that these are the same as in Raphael's model, but I had established that a and c seem to be reversed between the two. You have spotted that this may not be the case. I will do some more research and confirm whether or not the formula used by Bibble is the same as Raphael's or a different variant.
I haven't seen anything which explains how the formulae which use even powers relate to the formulae which use odd powers. Assuming it's not just a simple factorisation then this could also explain why I am getting different results, and in some cases the variation doesn't seem to be linearly related to the data in the RAW.
I think fundamentally a, b and c are equivalent (but via some translation I don't yet understand) to K1, K2, K3 etc. I suspect tangential distortions can be ignored as no lens in the MFT class has these by design. However some of my own experiments suggest that there may be cases where the ideal correction is different in two axes, which would be supported by the Browns equations and the number of data points in the RAW data, but as far as I am aware most image processing programs only allow for three parameters and assume radial symmetry.
I'll investigate further and keep you posted. If you have any moments of inspiration please let me know,
Andrew