Problem with IPTC tags having length greater than 32767 B

[arv]

Hello,

I noticed a problem when editing IPTC tags having length greater than 32767 B.

While the length of data <= 32767 (0x7FFF) structure looks like:
1c02 b7 XXXX data_of_tag... - where XXXX is correct length of data

but when the length >= 32768 (0x8000) then structure of IPTC tag changes:
1c02 b7 8004 XXXX XXXX data_of_tag... - where XXXX XXXX is length of data and 0x8004 is any fixed value

Is this correct or intended ?


I have encountered few problems with that. I use this script to reproduce the problem:

Code Select Expand


# Perl v5.16.3, ExifTool v9.76 and v10.00
my $exifTool = new Image::ExifTool;
$exifTool->Options(IgnoreMinorErrors => 1); # I'm aware of oversizing

%Image::ExifTool::UserDefined = (
'Image::ExifTool::IPTC::ApplicationRecord' => {
183 => {
Name => 'User_Defined_183',
Format => 'string[0,16]', # even for string[0,35000]
},
},
);

my $str = 'x' x 33024; # hex: 81 00

my $info = $exifTool->ImageInfo('test.jpg');
my $tagName = $Image::ExifTool::IPTC::ApplicationRecord{183}{'Name'};
$exifTool->SetNewValue($tagName => $str, Group => 'IPTC', AddValue => 0, Replace => 0);
# or delete it:
#$exifTool->SetNewValue($tagName);



Problems:

1. If the file does not contain any segment FFED then tag 2:183 is not added and is not created segment FFED.

2. If the file contains at least one IPTC tag then tag 2:183 is added as follows:

Code Select Expand


0000000: ffd8 ffe0 0010 4a46 4946 0001 0100 0001  ......JFIF......
0000010: 0001 0000 ffed 8136 5068 6f74 6f73 686f  .......6Photosho
0000020: 7020 332e 3000 3842 494d 0404 0000 0000  p 3.0.8BIM......
0000030: 8119 1c02 5000 0474 6573 741c 0200 0002  ....P..test.....
0000040: 0004 1c02 b780 0400 0081 0078 7878 7878  ...........xxxxx
0000050: 7878 7878 7878 7878 7878 7878 7878 7878  xxxxxxxxxxxxxxxx
...
0008130: 7878 7878 7878 7878 7878 7878 7878 7878  xxxxxxxxxxxxxxxx
0008140: 7878 7878 7878 7878 7878 7800 ffdb 0043  xxxxxxxxxxx....C


1c02 b7 8004 0000 8100 data_of_tag... <-- different structure for this tag


3. If the tag 2:183 is followed by some other tags and I delete it then the all remaining tags are removed to the end of the segment:

before delete (tag 2:183 is from a different source, not from ExifTool, it has different structure: 1c02 b7 XXXX data_of_tag...),
we have six IPTC tags: 0x00, 0x74, 0x6E, 0xB7, 0x78, 0x50:

Code Select Expand


...
0001660: 49aa 7fff d900 ffed d060 5068 6f74 6f73  I........`Photos
0001670: 686f 7020 332e 3000 3842 494d 0404 0000  hop 3.0.8BIM....
0001680: 0000 b65c 1c02 0000 0200 021c 0274 0000  ...\.........t..
0001690: 1c02 6e00 0120 1c02 b7b6 3978 7878 7878  ..n.. ....9xxxxx
00016a0: 7878 7878 7878 7878 7878 7878 7878 7878  xxxxxxxxxxxxxxxx
...
000ccc0: 7878 7878 7878 7878 7878 7878 7878 7878  xxxxxxxxxxxxxxxx
000ccd0: 7878 7878 1c02 7800 0120 1c02 5000 0120  xxxx..x.. ..P..
000cce0: 3842 494d 0425 0000 0000 0010 2bc9 e595  8BIM.%......+...
000ccf0: 6ae7 5bf2 6fe3 aa07 99cc ff99 3842 494d  j.[.o.......8BIM
000cd00: 043a 0000 0000 00e5 0000 0010 0000 0001  .:..............
...


after delete we have only the first three: 0x00, 0x74, 0x6E:

Code Select Expand


...
0001660: 49aa 7fff d900 ffed 1a16 5068 6f74 6f73  I.........Photos
0001670: 686f 7020 332e 3000 3842 494d 0404 0000  hop 3.0.8BIM....
0001680: 0000 0012 1c02 0000 0200 021c 0274 0000  .............t..
0001690: 1c02 6e00 0120 3842 494d 0425 0000 0000  ..n.. 8BIM.%....
00016a0: 0010 2bc9 e595 6ae7 5bf2 6fe3 aa07 99cc  ..+...j.[.o.....
00016b0: ff99 3842 494d 043a 0000 0000 00e5 0000  ..8BIM.:........
00016c0: 0010 0000 0001 0000 0000 000b 7072 696e  ............prin
...


Perhaps this is because ExifTool uses a different data structure for storing the length of the big data block.

I hope that is clearly described.

Regards.

[Phil Harvey]

While the length of data <= 32767 (0x7FFF) structure looks like:
1c02 b7 XXXX data_of_tag... - where XXXX is correct length of data

but when the length >= 32768 (0x8000) then structure of IPTC tag changes:
1c02 b7 8004 XXXX XXXX data_of_tag... - where XXXX XXXX is length of data and 0x8004 is any fixed value

Is this correct or intended ?

This is an extended IPTC record.  Check the IPTC IIM spec for details.

1. If the file does not contain any segment FFED then tag 2:183 is not added and is not created segment FFED.

I tested this and your script adds 2:183 just fine when there is no metadata in the file to begin with.

I also tried deleting the tag from a file containing other IPTC records and didn't see a problem.

What version of ExifTool are you using?

Could you post or email me a sample file that shows the problem?

- Phil

[arv]

While the length of data <= 32767 (0x7FFF) structure looks like:
1c02 b7 XXXX data_of_tag... - where XXXX is correct length of data

but when the length >= 32768 (0x8000) then structure of IPTC tag changes:
1c02 b7 8004 XXXX XXXX data_of_tag... - where XXXX XXXX is length of data and 0x8004 is any fixed value

Is this correct or intended ?

This is an extended IPTC record.  Check the IPTC IIM spec for details.

Indeed, I'm very sorry for not accurate diagnosis.

1. If the file does not contain any segment FFED then tag 2:183 is not added and is not created segment FFED.

I tested this and your script adds 2:183 just fine when there is no metadata in the file to begin with.

And again indeed and again I'm sorry. 
What is happening with me...

I also tried deleting the tag from a file containing other IPTC records and didn't see a problem.

What version of ExifTool are you using?

Could you post or email me a sample file that shows the problem?

I send you a sample file, but I suppose this is because using the incorrect record structure for storing the length.
I tested on ExifTool v9.76 and Perl v5.16.3

Thanks and sorry for the unnecessary confusion.