ExifTool Dies on One Specific Photo

[elopsentorml]

I've split this off of my Google Takeout thread because it seems specific to one photo.  I ran a "merge" command to copy some information from my Google Takeout .json folders to their associated photo folders:

Code Select Expand

exiftool -r -wm cg -tagsfromfile "%d/%F.json" "-Caption-Abstract<Description" "-ImageDescription<Description" -Description "-DateTimeOriginal<PhotoTakenTimeTimestamp" -d %s "-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLatitudeRef<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" "-GPSLongitudeRef<GeoDataLongitude" -overwrite_original "D:\GooglePhotos"

After a very short time (seconds), that process failed without any kind of message.  It just stopped.  There was no summary at the end of things.  Just a new command prompt.  I ran a validate command against everything (similar to this):

Code Select Expand

exiftool -validate -warning -error -a test.jpg

and piped it into a text file.  That showed just warnings.  No errors.

Then I ran my initial "merge" command but with the -verbose option.  I took the last file where it stopped and looked at it:

Code Select Expand

[ExifTool]      ExifToolVersion                 : 11.77
[File]          FileName                        : forestport 001.jpg
[File]          Directory                       : D:/1939-01-01
[File]          FileSize                        : 62 kB
[File]          FileModifyDate                  : 2019:11:30 08:01:51-08:00
[File]          FileAccessDate                  : 2019:11:30 08:01:51-08:00
[File]          FileCreateDate                  : 2019:11:29 20:42:45-08:00
[File]          FilePermissions                 : rw-rw-rw-
[File]          FileType                        : JPEG
[File]          FileTypeExtension               : jpg
[File]          MIMEType                        : image/jpeg
[File]          ExifByteOrder                   : Little-endian (Intel, II)
[File]          CurrentIPTCDigest               : 3b6de139c6e8a694079289c325b3e831
[File]          ImageWidth                      : 640
[File]          ImageHeight                     : 520
[File]          EncodingProcess                 : Baseline DCT, Huffman coding
[File]          BitsPerSample                   : 8
[File]          ColorComponents                 : 3
[File]          YCbCrSubSampling                : YCbCr4:2:0 (2 2)
[JFIF]          JFIFVersion                     : 1.02
[JFIF]          ResolutionUnit                  : inches
[JFIF]          XResolution                     : 200
[JFIF]          YResolution                     : 200
[EXIF]          ImageDescription                : redacted Clan in Forestport
[EXIF]          XResolution                     : 72
[EXIF]          YResolution                     : 72
[EXIF]          ResolutionUnit                  : inches
[EXIF]          Software                        : Adobe Photoshop Elements 4.0
[EXIF]          ModifyDate                      : 2005:10:17 09:32:08
[EXIF]          YCbCrPositioning                : Centered
[IPTC]          ApplicationRecordVersion        : 2
[IPTC]          Keywords                        : redacted, Portraits
[IPTC]          Caption-Abstract                : redacted Clan in Forestport
[XMP]           XMPToolkit                      : Image::ExifTool 11.77
[XMP]           Description                     : redacted Clan in Forestport
[XMP]           Subject                         : redacted, Portraits
[XMP]           NativeDigest                    : 36864,40960,40961,37121,37122,40962,40963,37510,40964,36867,36868,33434,33437,34850,34852,34855,34856,37377,37378,37379,37380,37381,37382,37383,37384,37385,37386,37396,41483,41484,41486,41487,41488,41492,41493,41495,41728,41729,41730,41985,41986,41987,41988,41989,41990,41991,41992,41993,41994,41995,41996,42016,0,2,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,20,22,23,24,25,26,27,28,30;D41D8CD98F00B204E9800998ECF8427E
[XMP]           NativeDigest                    : 256,257,258,259,262,274,277,284,530,531,282,283,296,301,318,319,529,532,306,270,271,272,305,315,33432;7187616F350C7C8B8750E27A55214AC4
[XMP]           CreatorTool                     : Adobe Photoshop Elements 4.0
[XMP]           ModifyDate                      : 2005:10:17 09:32:08-05:00
[Composite]     ImageSize                       : 640x520
[Composite]     Megapixels                      : 0.333

Code Select Expand

[ExifTool]      ExifToolVersion                 : 11.77
[File]          FileName                        : forestport 001.jpg.json
[File]          Directory                       : D:/1939-01-01
[File]          FileSize                        : 771 bytes
[File]          FileModifyDate                  : 2019:04:05 14:19:20-07:00
[File]          FileAccessDate                  : 2019:11:29 20:42:45-08:00
[File]          FileCreateDate                  : 2019:11:29 20:42:45-08:00
[File]          FilePermissions                 : rw-rw-rw-
[File]          FileType                        : JSON
[File]          FileTypeExtension               : json
[File]          MIMEType                        : application/json
[JSON]          CreationTimeFormatted           : Jan 3, 2018, 12:33:42 AM UTC
[JSON]          CreationTimeTimestamp           : 1514939622
[JSON]          Description                     : Sometime around 1939 in Forestport, NY: . redacted Clan in Forestport (I just guessed at 1939 for a date)
[JSON]          GeoDataAltitude                 : 0.0
[JSON]          GeoDataLatitude                 : 0.0
[JSON]          GeoDataLatitudeSpan             : 0.0
[JSON]          GeoDataLongitude                : 0.0
[JSON]          GeoDataLongitudeSpan            : 0.0
[JSON]          GeoDataExifAltitude             : 0.0
[JSON]          GeoDataExifLatitude             : 0.0
[JSON]          GeoDataExifLatitudeSpan         : 0.0
[JSON]          GeoDataExifLongitude            : 0.0
[JSON]          GeoDataExifLongitudeSpan        : 0.0
[JSON]          ImageViews                      : 0
[JSON]          ModificationTimeFormatted       : Apr 5, 2019, 8:19:21 PM UTC
[JSON]          ModificationTimeTimestamp       : 1554495561
[JSON]          PhotoTakenTimeFormatted         : Jan 1, 1939, 5:00:00 PM UTC
[JSON]          PhotoTakenTimeTimestamp         : -978246000
[JSON]          Title                           : forestport 001.jpg

The one thing that looks off to me is that the PhotoTakenTimeTimestamp in the .json file is negative and that's what I'm using to merge a Created Date into the .jpg file.  But, I don't know if that's bad or not.

I ran that "validate" command against the files and it came back "OK":

Code Select Expand

D:\1939-01-01>exiftool -validate -warning -error -a "d:\1939-01-01"
======== d:/1939-01-01/forestport 001.jpg
Validate                        : OK
======== d:/1939-01-01/forestport 001.jpg.json
Validate                        : OK
    1 directories scanned
    2 image files read

Then, I ran that initial "merge" command against them and it once again just stopped:

Code Select Expand

D:\1939-01-01>exiftool -verbose -wm cg -tagsfromfile "%d/%F.json" "-Caption-Abstract<Description" "-ImageDescription<Description" -Description "-DateTimeOriginal<PhotoTakenTimeTimestamp" -d %s "-GPSAltitude<GeoDataAltitude" "-GPSLatitude<GeoDataLatitude" "-GPSLatitudeRef<GeoDataLatitude" "-GPSLongitude<GeoDataLongitude" "-GPSLongitudeRef<GeoDataLongitude" -overwrite_original "d:\1939-01-01"
======== d:/1939-01-01/forestport 001.jpg
Setting new values from d:/1939-01-01/forestport 001.jpg.json

D:\1939-01-01>

I have no idea why ExifTool is failing on this, but I thought I'd pass it on.

[elopsentorml]

And, just in case another sample helps, I ran into another one:

Code Select Expand

[ExifTool]      ExifToolVersion                 : 11.77
[File]          FileName                        : redacted at Birth.jpg.json
[File]          Directory                       : D:/GooglePhotos/1959-02-03
[File]          FileSize                        : 711 bytes
[File]          FileModifyDate                  : 2019:04:05 14:19:14-07:00
[File]          FileAccessDate                  : 2019:11:29 20:42:45-08:00
[File]          FileCreateDate                  : 2019:11:29 20:42:45-08:00
[File]          FilePermissions                 : rw-rw-rw-
[File]          FileType                        : JSON
[File]          FileTypeExtension               : json
[File]          MIMEType                        : application/json
[JSON]          CreationTimeFormatted           : Dec 30, 2017, 3:59:34 PM UTC
[JSON]          CreationTimeTimestamp           : 1514649574
[JSON]          Description                     : redacted
[JSON]          GeoDataAltitude                 : 0.0
[JSON]          GeoDataLatitude                 : 0.0
[JSON]          GeoDataLatitudeSpan             : 0.0
[JSON]          GeoDataLongitude                : 0.0
[JSON]          GeoDataLongitudeSpan            : 0.0
[JSON]          GeoDataExifAltitude             : 0.0
[JSON]          GeoDataExifLatitude             : 0.0
[JSON]          GeoDataExifLatitudeSpan         : 0.0
[JSON]          GeoDataExifLongitude            : 0.0
[JSON]          GeoDataExifLongitudeSpan        : 0.0
[JSON]          ImageViews                      : 0
[JSON]          ModificationTimeFormatted       : Apr 5, 2019, 8:19:15 PM UTC
[JSON]          ModificationTimeTimestamp       : 1554495555
[JSON]          PhotoTakenTimeFormatted         : Feb 3, 1959, 5:00:00 PM UTC
[JSON]          PhotoTakenTimeTimestamp         : -344242800
[JSON]          Title                           : redacted at Birth.jpg

Code Select Expand

[ExifTool]      ExifToolVersion                 : 11.77
[File]          FileName                        : redacted at Birth.jpg.json
[File]          Directory                       : D:/GooglePhotos/1959-02-03
[File]          FileSize                        : 711 bytes
[File]          FileModifyDate                  : 2019:04:05 14:19:14-07:00
[File]          FileAccessDate                  : 2019:11:29 20:42:45-08:00
[File]          FileCreateDate                  : 2019:11:29 20:42:45-08:00
[File]          FilePermissions                 : rw-rw-rw-
[File]          FileType                        : JSON
[File]          FileTypeExtension               : json
[File]          MIMEType                        : application/json
[JSON]          CreationTimeFormatted           : Dec 30, 2017, 3:59:34 PM UTC
[JSON]          CreationTimeTimestamp           : 1514649574
[JSON]          Description                     : redacted
[JSON]          GeoDataAltitude                 : 0.0
[JSON]          GeoDataLatitude                 : 0.0
[JSON]          GeoDataLatitudeSpan             : 0.0
[JSON]          GeoDataLongitude                : 0.0
[JSON]          GeoDataLongitudeSpan            : 0.0
[JSON]          GeoDataExifAltitude             : 0.0
[JSON]          GeoDataExifLatitude             : 0.0
[JSON]          GeoDataExifLatitudeSpan         : 0.0
[JSON]          GeoDataExifLongitude            : 0.0
[JSON]          GeoDataExifLongitudeSpan        : 0.0
[JSON]          ImageViews                      : 0
[JSON]          ModificationTimeFormatted       : Apr 5, 2019, 8:19:15 PM UTC
[JSON]          ModificationTimeTimestamp       : 1554495555
[JSON]          PhotoTakenTimeFormatted         : Feb 3, 1959, 5:00:00 PM UTC
[JSON]          PhotoTakenTimeTimestamp         : -344242800
[JSON]          Title                           : redacted at Birth.jpg

Again, the PhotoTakenTimeTimestamp is negative.  I'm assuming the validate command and the

Code Select Expand

exiftool -r -all= -tagsfromfile @ -all:all -unsafe -icc_profile -overwrite_original -F "D:\GooglePhotos"

command I did didn't catch it (if it's actually a problem) because its not a standard tag.

[StarGeek]

That date is definately a problem since the time stamp in the JSON PhotoTakenTimeTimestamp is in Epoch time which starts at 00:00:00 UTC on 1 January 1970.  I didn't even realize that it could be counted prior to that date until I read the wiki page.

I'm guessing that the perl time routines that exiftool uses can't deal with a negative Epoch time stamp.  For time stamps before 1970, try using -d "%b %d, %Y, %I:%M:%S %p" and the JSON PhotoTakenTimeFormatted instead.

[elopsentorml]

Do I need to report this somewhere as a bug?  According to that Wikipedia link:

The Unix epoch is the time 00:00:00 UTC on 1 January 1970.[2] There is a problem with this definition, in that UTC did not exist in its current form until 1972; this issue is discussed below. For brevity, the remainder of this section uses ISO 8601 date and time format, in which the Unix epoch is 1970-01-01T00:00:00Z.

The Unix time number is zero at the Unix epoch, and increases by exactly 86400 per day since the epoch. Thus 2004-09-16T00:00:00Z, 12677 days after the epoch, is represented by the Unix time number 12677 × 86400 = 1095292800. This can be extended backwards from the epoch too, using negative numbers; thus 1957-10-04T00:00:00Z, 4472 days before the epoch, is represented by the Unix time number −4472 × 86400 = −386380800.

So, all photos taken/modified before 1970 with the unformatted date/time code will cause this problem.

I was leery of using the formatted field because I'm not sure if all the software and services I've used over the last several decades have formatted those numbers the same way.  Also, I can't find the FAQ talking about date/time codes (I know I've seen it before).  Can you point me in the right direction?

EDIT:  Hopefully the following is good for those format codes:

https://linux.die.net/man/3/strftime

[elopsentorml]

I'm running the command with the formatted string now.  Just about everything popped up a warning about "garbage at end of string in strptime:  UTC in ExifIFD:DateTimeOriginal (PrintConvInv)."  It looks like the date didn't get copied over.  Perhaps because of the "UTC" after the actual formatted string in the .json file?

[StarGeek]

Just about everything popped up a warning about "garbage at end of string in strptime:  UTC in ExifIFD:DateTimeOriginal (PrintConvInv)."

Oops, sorry, I didn't copy/paste the complete line from my notes. Use
-d "%b %d, %Y, %I:%M:%S %p UTC"

[elopsentorml]

That got it.  Thanks.

[Phil Harvey]

I don't have time right now to read this in detail, but if you haven't already provided the files necessary to reproduce this problem, could you please email them to me along with the command you used (philharvey66 at gmail.com).

Thanks.

It sounds like I may have to put in some protection against the standard time routines crashing on odd input values.

- Phil

[StarGeek]

Here's a quicky json file I made from a previous Google Takeout of mine.  I just changed the date to match one elopsentorml's examples.  Any image file can be used.

Code Select Expand

{
  "title": "test1.Jpg",
  "description": "XMP-tiff:ImageDescription",
  "imageViews": "0",
  "creationTime": {
    "timestamp": "1548714348",
    "formatted": "Jan 28, 2019, 10:25:48 PM UTC"
  },
  "modificationTime": {
    "timestamp": "1554915781",
    "formatted": "Apr 10, 2019, 5:03:01 PM UTC"
  },
  "geoData": {
    "latitude": 5.346782999999999,
    "longitude": -3.985286,
    "altitude": 2222.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "geoDataExif": {
    "latitude": 5.346782999999999,
    "longitude": -3.985286,
    "altitude": 2222.0,
    "latitudeSpan": 0.0,
    "longitudeSpan": 0.0
  },
  "photoTakenTime": {
    "timestamp": "-978246000",
    "formatted": "Jan 1, 1939, 5:00:00 PM UTC"
  }
}

The command to duplicate would be as follows.  I can confirm that on my end exiftool just exits without any output
exiftool -TagsFromFile test1.Jpg.json "-DateTimeOriginal<PhotoTakenTimeTimestamp" -d %s test1.Jpg

[Phil Harvey]

Thanks StarGeek,

It seems that this problem may be specific to the Windows libraries because it works on the Mac and sets DateTimeOriginal to "1939:01:01 12:00:00".

I'll try this on Windows when I get a chance, but it looks like this will be a Windows-only patch.

- Phil

[Phil Harvey]

Thanks for this report.

OK.  I can reproduce this on Windows.  Ouch.  The Perl interpreter dies when calling the strptime function with this message:



This is particularly egregious because I was already protecting the strptime call inside an eval statement (which would normally guard against any sort of error in the function from affecting ExifTool).  I will put in a specific patch to protect against negative epoch times in Windows, and ExifTool 11.78 should give you this warning instead of crashing:

        Can't convert negative epoch time in ExifIFD:DateTimeOriginal (PrintConvInv)

I really hope there aren't other argument combinations that cause strptime to crash as well.

- Phil