-AllDates+= changes filesize

[Archive]

[Originally posted by ecto on 2007-03-29 17:02:14-07]

Hello.

First of all thanks for a -great- program. Keep it up!

As I have lots of images where the DateTime is one hour wrong due to me forgetting to change the camera's time when daylight savings time kicked in, I'm using "exiftool -P -overwrite_original -AllDates+=1 DIR". This reduces the filesize of my images by roughly 10-30 KB (originals are ~1 MB). This occurs even if I don't use -overwrite_original.

This made me a bit scared at first, but the images seem intact and the Exif is there. So I'm curious - why is the filesize reduced? What is changed/stripped? From other posts I've gathered that you, Phil, is very aware of data integrity so I take it for granted that it's not the image data itself that is changed. I've searched the FAQ and the forums but didn't find any answer to this.

[Archive]

[Originally posted by exiftool on 2007-03-29 17:18:11-07]

The EXIF standard allows for arbitrary padding between all
pieces of information, and many camera models pad the meta
information with unused blocks which are usually filled with binary
zeros.  I suppose the reason for this is to simplify the algorithms that
the camera uses to allow variable-sized information to be placed at
fixed offsets in the output file.

When ExifTool rewrites an image, it does not copy the unreferenced
padding blocks if they exist.  For some images, this can reduce the
image size many kilobytes.

I hope this helps, but I can be more specific if I know exactly what
type of image you are editing:  JPEG images?  From what camera
model?

- Phil

[Archive]

[Originally posted by ecto on 2007-03-29 18:13:03-07]

Aha. Yes, your answer helped - thanks for the quick response!

Since the padding is arbitrary I guess that Exif software searches for tagnames and not for specific offsets, so there shouldn't be any compatibility problems reading Exif data even if there's lots of zeros or none, correct? I tried two programs, ACDSee 3.1 (yeah - it's old but still the fastest image viewer imo) and Exifer 2.1.5, and the Exif data is there.

The files are JPEG files from a Minolta DiMAGE Xt, firmware version 1.00. I also tested it on several images each (JPEGs) from these camera models:

* Pentax Optio S5i - no filesize change

* Pentax K100D - filesize reduced similar to the DiMAGE Xt

* Sony DSC-T7 - filesize increased 4-5 bytes

* Sony DSC-T7 (same as above) but with underwater house - filesize reduced 1-2 bytes

I uploaded examples for reference:

Minolta DiMAGE Xt Original

Minolta DiMAGE Xt reduced

Sony DSC-T7 original

Sony DSC-T7 increased

Sony DSC-T7 with underwater house original

Sony DSC-T7 with underwater house reduced

Hope this helps if you want to investigate it further. Interesting that the filesize increased on images from the Sony DSC-T7.. but reduced with the underwater house.

-Danny

[Archive]

[Originally posted by ecto on 2007-03-29 18:25:34-07]

Aha. Yes, your answer helped - thanks for the quick response!

Since the padding is arbitrary I guess that Exif software searches for tagnames and not for specific offsets, so there shouldn't be any compatibility problems reading Exif data even if there's lots of zeros or none, correct? I tried two programs, ACDSee 3.1 (yeah - it's old but still the fastest image viewer imo) and Exifer 2.1.5, and the Exif data is there.

The files are JPEG files from a Minolta DiMAGE Xt, firmware version 1.00. I also tested it on several images each (JPEGs) from these camera models:

* Pentax Optio S5i - no filesize change

* Pentax K100D - filesize reduced similar to the DiMAGE Xt

* Sony DSC-T7 - filesize increased 4-5 bytes

* Sony DSC-T7 (same as above) but with underwater house - filesize reduced 1-2 bytes

I uploaded examples for reference:

Minolta DiMAGE Xt Original

Minolta DiMAGE Xt reduced

Sony DSC-T7 original

Sony DSC-T7 increased

Sony DSC-T7 with underwater house original

Sony DSC-T7 with underwater house reduced

Hope this helps if you want to investigate it further. Interesting that the filesize increased on images from the Sony DSC-T7.. but reduced with the underwater house.

-Danny

[Archive]

[Originally posted by exiftool on 2007-03-29 18:31:55-07]

Yes, the DiMAGE Xt JPEG images contain 3 large blocks (1-5 KB each).
Two of them are filled with hex 0xff's, and the third is random data
or data from previous preview images as far as I can tell.

Software that properly reads the TIFF/EXIF structure will have no problems
with the reformatted information.  Software that does have problems
is brain-damaged and should be fixed (I have seen things like this
happen for early releases of some software packages, but eventually
they do the right thing and this gets fixed).

If you are interested, you can use the -htmldump option
to see exactly where these unused blocks are, and what they contain.

I plan to add the following FAQ entry in the next release to help people
with similar questions in the future (note that the Windows PhotoInfo
tool also addresses a similar question in their FAQ):

Why is my image smaller after I use ExifTool to write information?"

There are different reasons this can happen, but the bottom-line answer
is that when ExifTool writes an image, the meta information may be
restructured in such a way that it takes less space than in the original file.

For instance, the EXIF/TIFF standard allows for blocks of
unreferenced data to exist in an image. Some digital cameras
write JPEG or TIFF-based RAW files which contain large blocks of
unused data, usually filled with binary zeros. The reason for
this could be to simplify camera algorithms by allowing
variable-sized information to be written at fixed offsets in the
output image. When ExifTool rewrites an image it does not copy
these unused blocks. This can result in a significant reduction
in file size for some images.

Also, the size of an XMP record may easily shrink or grow when
it is rewritten, even if no meta information is changed. This is
partly due to the fact that the XMP specification recommends a
few KB of padding at the end of the record (ExifTool adds 2424
bytes by default), and partly due to the flexibility of the XMP
format which allows the information to be written in various
styles, some of which are more compact than others.

- Phil

[Archive]

[Originally posted by ecto on 2007-03-29 19:36:04-07]

Yes, that FAQ entry seems like a good idea. I guess there are more people than I out there who wonders where the bytes go.

I used the -htmldump options and as you say, there are unused blocks. Interesting, you learn something new every day.

Thanks for your extremely quick responses. Really great support to a really great program. I now feel safe that the image data is still intact.

-Danny

[Archive]

[Originally posted by ecto on 2007-03-29 21:32:50-07]

Yes, that FAQ entry seems like a good idea. I guess there are more people than I out there who wonders where the bytes go.

I used the -htmldump options and as you say, there are unused blocks. Interesting, you learn something new every day.

Thanks for your extremely quick responses. Really great support to a really great program. I now feel safe that the image data is still intact.

-Danny

[Archive]

[Originally posted by ecto on 2007-03-29 22:50:25-07]

Yes, that FAQ entry seems like a good idea. I guess there are more people than I out there who wonders where the bytes go.

I used the -htmldump options and as you say, there are unused blocks. Interesting, you learn something new every day.

Thanks for your extremely quick responses. Really great support to a really great program. I now feel safe that the image data is still intact.

-Danny

[Archive]

[Originally posted by exiftool on 2007-03-29 23:33:10-07]

You're welcome,

You're welcome,

You're welcome,



- Phil

[Archive]

[Originally posted by ecto on 2007-03-30 10:21:53-07]

Sorry about the double-triple posts. I actually only pressed Submit once - but then every time i reloaded the start page to see if you had answered (I even closed the tab and re-entered the URL in a new one, so no POST data should've been there) it seems it got posted again (Opera 9.1, WinXP). Weird.

Anyway, I'll stop bloating the forum now.