exiftoogui v. 4.16.0.0, exiftool v. 8.60, win xp sp2.
I was trying to add an exif copyright entry to all (jpg) files in a directory with 2170 jpg files (also contained 2 non-image files which did not show up in exiftoogui listing).
I started by selecting all files in the exiftoolgui file listing, then setting 2 exif fields from default values. After running for some time (minutes), I noted that exiftoolgui appeared to be hung, unwilling to refresh its window. Examination with sysinternals process explorer showed 2 exiftool processes running, but accumulating no cpu, and none of the exiftool* processes had any image files open, only the directory containing them was open. In addition, none of the processes was accumulating i/o reads or writes or changing the amount of memory being used. Further examination showed that 257 files had been processed (a number suspiciously close to 256).
Further attempts to run exiftoolgui on only the remaining 1913 files failed immediately with the pop-up "no message from exiftool", even after killing all exiftool* processes and starting from scratch. However, changing a single file continued to work flawlessly.
I suspect the problem is some issue with the number of arguments being passed to exiftool (since unlike Linux, windows has a small limit on the number of arguments possible in a command line).
One other suspicious thing I noticed was that windows had a high load of deferred procedure calls at the time. So, it's conceivable that all that was happening was that Windows was being overwhelmed with all the calls coming from exiftool or exiftoolgui. If that's the case, then exiftool might consider throttling the rate at which it issues calls, e.g. wating for completion of batches of some number before issuing more. I'm not sure, however, whether Windows, much lesss the C std lib, gives you the chance to find that out. Alternately, exiftoolgui might consider issuing such calls to exiftool in limited size batches, which might be easier to implement than at the C system call level.
It's getting late here. I'll read your post (carefully) tomorrow.
Bogdan
I can also report that selecting up to about 500 files at a time also seems to work just fine. I am beginning to wonder if the original problem may have occurred because some file may have been locked by some other process, though I was not trying to write to any of them with any other software. At some point I will again try doing it to a large number and see if the problem recurs. It did, however, recur multiple times in trying to modify approximately 1900 files.
I spoke too soon. After initially starting out ok, the attempt with about 500 files is now hung. This time, there is no large deferred procedure call overhead going on, so that was previously probably caused by some other programs.
The command line that Process Explorer reports for what was given to exiftool is below (I have replaced my name with x's since I don't want my name on the public internet, but the number of x's is the same as the number of original characters).
The modifications got made through file IMG_1350-3.JPG. As best I can tell, that's 229 image files that were modified, out of 525 in the command line.
exiftool -overwrite_original -P -v0 -IFD0:Artist="xxxxxxxxxxxxxxxx" -IFD0:Copyright="© by xxxxxxxxxxxxxxxx" "IMG_1174.JPG" "IMG_1175.JPG" "IMG_1176.JPG" "IMG_1177.JPG" "IMG_1178.JPG" "IMG_1179.JPG" "IMG_1180.JPG" "IMG_1181.JPG" "IMG_1182.JPG" "IMG_1183.JPG" "IMG_1184.JPG" "IMG_1185.JPG" "IMG_1186.JPG" "IMG_1187.JPG" "IMG_1188.JPG" "IMG_1189.JPG" "IMG_1190.JPG" "IMG_1191.JPG" "IMG_1192.JPG" "IMG_1193.JPG" "IMG_1194.JPG" "IMG_1195.JPG" "IMG_1196.JPG" "IMG_1197.JPG" "IMG_1198.JPG" "IMG_1199.JPG" "IMG_1200.JPG" "IMG_1201.JPG" "IMG_1202.JPG" "IMG_1203.JPG" "IMG_1204.JPG" "IMG_1205.JPG" "IMG_1206.JPG" "IMG_1207.JPG" "IMG_1208.JPG" "IMG_1209.JPG" "IMG_1210.JPG" "IMG_1211.JPG" "IMG_1212.JPG" "IMG_1213.JPG" "IMG_1214.JPG" "IMG_1215.JPG" "IMG_1216.JPG" "IMG_1217.JPG" "IMG_1218.JPG" "IMG_1219.JPG" "IMG_1220.JPG" "IMG_1221.JPG" "IMG_1222.JPG" "IMG_1223.JPG" "IMG_1224.JPG" "IMG_1225.JPG" "IMG_1226.JPG" "IMG_1227.JPG" "IMG_1228.JPG" "IMG_1229.JPG" "IMG_1229-1.JPG" "IMG_1230.JPG" "IMG_1231.JPG" "IMG_1232.JPG" "IMG_1233.JPG" "IMG_1234.JPG" "IMG_1235.JPG" "IMG_1236.JPG" "IMG_1237.JPG" "IMG_1238.JPG" "IMG_1238-1.JPG" "IMG_1239.JPG" "IMG_1240.JPG" "IMG_1241.JPG" "IMG_1241-1.JPG" "IMG_1242.JPG" "IMG_1242-1.JPG" "IMG_1243.JPG" "IMG_1244.JPG" "IMG_1245.JPG" "IMG_1246.JPG" "IMG_1247.JPG" "IMG_1248.JPG" "IMG_1249.JPG" "IMG_1250.JPG" "IMG_1251.JPG" "IMG_1252.JPG" "IMG_1253.JPG" "IMG_1254.JPG" "IMG_1255.JPG" "IMG_1255-1.JPG" "IMG_1256.JPG" "IMG_1256-1.JPG" "IMG_1257.JPG" "IMG_1257-1.JPG" "IMG_1258.JPG" "IMG_1258-1.JPG" "IMG_1258-2.JPG" "IMG_1259.JPG" "IMG_1259-1.JPG" "IMG_1260.JPG" "IMG_1260-1.JPG" "IMG_1261.JPG" "IMG_1261-1.JPG" "IMG_1262.JPG" "IMG_1263.JPG" "IMG_1264.JPG" "IMG_1265.JPG" "IMG_1266.JPG" "IMG_1267.JPG" "IMG_1268.JPG" "IMG_1269.JPG" "IMG_1270.JPG" "IMG_1271.JPG" "IMG_1272.JPG" "IMG_1273.JPG" "IMG_1274.JPG" "IMG_1275.JPG" "IMG_1275-1.JPG" "IMG_1275-2.JPG" "IMG_1276.JPG" "IMG_1277.JPG" "IMG_1278.JPG" "IMG_1279.JPG" "IMG_1280.JPG" "IMG_1281.JPG" "IMG_1282.JPG" "IMG_1283.JPG" "IMG_1284.JPG" "IMG_1285.JPG" "IMG_1286.JPG" "IMG_1287.JPG" "IMG_1288.JPG" "IMG_1289.JPG" "IMG_1290.JPG" "IMG_1291.JPG" "IMG_1292.JPG" "IMG_1293.JPG" "IMG_1294.JPG" "IMG_1295.JPG" "IMG_1296.JPG" "IMG_1297.JPG" "IMG_1298.JPG" "IMG_1299.JPG" "IMG_1299-1.JPG" "IMG_1299-2.JPG" "IMG_1300.JPG" "IMG_1301.JPG" "IMG_1301-1.JPG" "IMG_1301-2.JPG" "IMG_1301-3.JPG" "IMG_1302.JPG" "IMG_1303.JPG" "IMG_1304.JPG" "IMG_1305.JPG" "IMG_1306.JPG" "IMG_1307.JPG" "IMG_1308.JPG" "IMG_1309.JPG" "IMG_1310.JPG" "IMG_1311.JPG" "IMG_1311-1.JPG" "IMG_1312.JPG" "IMG_1313.JPG" "IMG_1314.JPG" "IMG_1315.JPG" "IMG_1315-1.JPG" "IMG_1316.JPG" "IMG_1316-1.JPG" "IMG_1317.JPG" "IMG_1317-1.JPG" "IMG_1318.JPG" "IMG_1318-1.JPG" "IMG_1319.JPG" "IMG_1319-1.JPG" "IMG_1320.JPG" "IMG_1321.JPG" "IMG_1322.JPG" "IMG_1322-1.JPG" "IMG_1323.JPG" "IMG_1323-1.JPG" "IMG_1324.JPG" "IMG_1325.JPG" "IMG_1326.JPG" "IMG_1326-1.JPG" "IMG_1327.JPG" "IMG_1327-1.JPG" "IMG_1328.JPG" "IMG_1328-1.JPG" "IMG_1329.JPG" "IMG_1329-1.JPG" "IMG_1329-2.JPG" "IMG_1330.JPG" "IMG_1330-1.JPG" "IMG_1331.JPG" "IMG_1331-1.JPG" "IMG_1332.JPG" "IMG_1332-1.JPG" "IMG_1333.JPG" "IMG_1333-1.JPG" "IMG_1334.JPG" "IMG_1334-1.JPG" "IMG_1335.JPG" "IMG_1335-1.JPG" "IMG_1336.JPG" "IMG_1337.JPG" "IMG_1337-1.JPG" "IMG_1338.JPG" "IMG_1338-1.JPG" "IMG_1339.JPG" "IMG_1340.JPG" "IMG_1341.JPG" "IMG_1342.JPG" "IMG_1343.JPG" "IMG_1343-1.JPG" "IMG_1344.JPG" "IMG_1344-1.JPG" "IMG_1344-2.JPG" "IMG_1344-3.JPG" "IMG_1345.JPG" "IMG_1345-1.JPG" "IMG_1346.JPG" "IMG_1346-1.JPG" "IMG_1347.JPG" "IMG_1348.JPG" "IMG_1348-1.JPG" "IMG_1348-2.JPG" "IMG_1348-3.JPG" "IMG_1349.JPG" "IMG_1350.JPG" "IMG_1350-1.JPG" "IMG_1350-2.JPG" "IMG_1350-3.JPG" "IMG_1350-4.JPG" "IMG_1350-5.JPG" "IMG_1351.JPG" "IMG_1352.JPG" "IMG_1353.JPG" "IMG_1354.JPG" "IMG_1355.JPG" "IMG_1356.JPG" "IMG_1356-1.JPG" "IMG_1357.JPG" "IMG_1358.JPG" "IMG_1358-1.JPG" "IMG_1359.JPG" "IMG_1360.JPG" "IMG_1361.JPG" "IMG_1362.JPG" "IMG_1363.JPG" "IMG_1364.JPG" "IMG_1364-1.JPG" "IMG_1365.JPG" "IMG_1365-1.JPG" "IMG_1366.JPG" "IMG_1367.JPG" "IMG_1368.JPG" "IMG_1369.JPG" "IMG_1370.JPG" "IMG_1371.JPG" "IMG_1371-1.JPG" "IMG_1372.JPG" "IMG_1373.JPG" "IMG_1374.JPG" "IMG_1374-1.JPG" "IMG_1375.JPG" "IMG_1375-1.JPG" "IMG_1376.JPG" "IMG_1376-1.JPG" "IMG_1377.JPG" "IMG_1377-1.JPG" "IMG_1378.JPG" "IMG_1378-1.JPG" "IMG_1379.JPG" "IMG_1379-1.JPG" "IMG_1380.JPG" "IMG_1380-1.JPG" "IMG_1381.JPG" "IMG_1382.JPG" "IMG_1382-1.JPG" "IMG_1383.JPG" "IMG_1384.JPG" "IMG_1384-1.JPG" "IMG_1385.JPG" "IMG_1385-1.JPG" "IMG_1386.JPG" "IMG_1386-1.JPG" "IMG_1387.JPG" "IMG_1387-1.JPG" "IMG_1388.JPG" "IMG_1388-1.JPG" "IMG_1389.JPG" "IMG_1390.JPG" "IMG_1391.JPG" "IMG_1391-1.JPG" "IMG_1392.JPG" "IMG_1392-1.JPG" "IMG_1393.JPG" "IMG_1394.JPG" "IMG_1395.JPG" "IMG_1396.JPG" "IMG_1397.JPG" "IMG_1397-1.JPG" "IMG_1398.JPG" "IMG_1399.JPG" "IMG_1400.JPG" "IMG_1401.JPG" "IMG_1402.JPG" "IMG_1402-1.JPG" "IMG_1403.JPG" "IMG_1404.JPG" "IMG_1405.JPG" "IMG_1406.JPG" "IMG_1407.JPG" "IMG_1407-1.JPG" "IMG_1408.JPG" "IMG_1409.JPG" "IMG_1410.JPG" "IMG_1410-1.JPG" "IMG_1411.JPG" "IMG_1412.JPG" "IMG_1413.JPG" "IMG_1414.JPG" "IMG_1415.JPG" "IMG_1415-1.JPG" "IMG_1416.JPG" "IMG_1416-1.JPG" "IMG_1417.JPG" "IMG_1417-1.JPG" "IMG_1418.JPG" "IMG_1419.JPG" "IMG_1420.JPG" "IMG_1420-1.JPG" "IMG_1421.JPG" "IMG_1422.JPG" "IMG_1422-1.JPG" "IMG_1423.JPG" "IMG_1423-1.JPG" "IMG_1424.JPG" "IMG_1424-1.JPG" "IMG_1425.JPG" "IMG_1425-1.JPG" "IMG_1426.JPG" "IMG_1426-1.JPG" "IMG_1427.JPG" "IMG_1428.JPG" "IMG_1428-1.JPG" "IMG_1429.JPG" "IMG_1430.JPG" "IMG_1431.JPG" "IMG_1431-1.JPG" "IMG_1432.JPG" "IMG_1433.JPG" "IMG_1434.JPG" "IMG_1434-1.JPG" "IMG_1435.JPG" "IMG_1435-1.JPG" "IMG_1436.JPG" "IMG_1436-1.JPG" "IMG_1436-2.JPG" "IMG_1436-3.JPG" "IMG_1437.JPG" "IMG_1437-1.JPG" "IMG_1438.JPG" "IMG_1439.JPG" "IMG_1440.JPG" "IMG_1441.JPG" "IMG_1442.JPG" "IMG_1442-1.JPG" "IMG_1443.JPG" "IMG_1443-1.JPG" "IMG_1444.JPG" "IMG_1444-1.JPG" "IMG_1445.JPG" "IMG_1446.JPG" "IMG_1447.JPG" "IMG_1448.JPG" "IMG_1449.JPG" "IMG_1449-1.JPG" "IMG_1450.JPG" "IMG_1450-1.JPG" "IMG_1451.JPG" "IMG_1452.JPG" "IMG_1452-1.JPG" "IMG_1453.JPG" "IMG_1453-1.JPG" "IMG_1454.JPG" "IMG_1455.JPG" "IMG_1455-1.JPG" "IMG_1456.JPG" "IMG_1456-1.JPG" "IMG_1457.JPG" "IMG_1457-1.JPG" "IMG_1458.JPG" "IMG_1458-1.JPG" "IMG_1459.JPG" "IMG_1459-1.JPG" "IMG_1460.JPG" "IMG_1460-1.JPG" "IMG_1461.JPG" "IMG_1461-1.JPG" "IMG_1462.JPG" "IMG_1463.JPG" "IMG_1463-1.JPG" "IMG_1464.JPG" "IMG_1464-1.JPG" "IMG_1465.JPG" "IMG_1465-1.JPG" "IMG_1466.JPG" "IMG_1466-1.JPG" "IMG_1467.JPG" "IMG_1467-1.JPG" "IMG_1468.JPG" "IMG_1468-1.JPG" "IMG_1469.JPG" "IMG_1470.JPG" "IMG_1471.JPG" "IMG_1472.JPG" "IMG_1473.JPG" "IMG_1474.JPG" "IMG_1475.JPG" "IMG_1476.JPG" "IMG_1477.JPG" "IMG_1478.JPG" "IMG_1479.JPG" "IMG_1480.JPG" "IMG_1481.JPG" "IMG_1481-1.JPG" "IMG_1482.JPG" "IMG_1483.JPG" "IMG_1484.JPG" "IMG_1485.JPG" "IMG_1486.JPG" "IMG_1487.JPG" "IMG_1488.JPG" "IMG_1489.JPG" "IMG_1490.JPG" "IMG_1491.JPG" "IMG_1492.JPG" "IMG_1493.JPG" "IMG_1494.JPG" "IMG_1495.JPG" "IMG_1496.JPG" "IMG_1496-1.JPG" "IMG_1497.JPG" "IMG_1498.JPG" "IMG_1499.JPG" "IMG_1500.JPG" "IMG_1501.JPG" "IMG_1502.JPG" "IMG_1503.JPG" "IMG_1504.JPG" "IMG_1505.JPG" "IMG_1506.JPG" "IMG_1507.JPG" "IMG_1508.JPG" "IMG_1509.JPG" "IMG_1510.JPG" "IMG_1511.JPG" "IMG_1512.JPG" "IMG_1513.JPG" "IMG_1514.JPG" "IMG_1515.JPG" "IMG_1516.JPG" "IMG_1517.JPG" "IMG_1518.JPG" "IMG_1519.JPG" "IMG_1520.JPG" "IMG_1521.JPG" "IMG_1522.JPG" "IMG_1523.JPG" "IMG_1523-1.JPG" "IMG_1523-2.JPG" "IMG_1523-3.JPG" "IMG_1524.JPG" "IMG_1525.JPG" "IMG_1526.JPG" "IMG_1527.JPG" "IMG_1527-1.JPG" "IMG_1527-2.JPG" "IMG_1528.JPG" "IMG_1529.JPG" "IMG_1530.JPG" "IMG_1531.JPG" "IMG_1532.JPG" "IMG_1533.JPG" "IMG_1534.JPG" "IMG_1535.JPG" "IMG_1536.JPG" "IMG_1537.JPG" "IMG_1538.JPG" "IMG_1539.JPG" "IMG_1540.JPG" "IMG_1541.JPG" "IMG_1542.JPG" "IMG_1543.JPG" "IMG_1544.JPG" "IMG_1545.JPG" "IMG_1546.JPG" "IMG_1547.JPG" "IMG_1548.JPG" "IMG_1548-1.JPG" "IMG_1548-2.JPG" "IMG_1549.JPG" "IMG_1550.JPG" "IMG_1551.JPG" "IMG_1552.JPG" "IMG_1553.JPG" "IMG_1554.JPG" "IMG_1555.JPG" "IMG_1556.JPG" "IMG_1556-1.JPG" "IMG_1556-2.JPG" "IMG_1556-3.JPG" "IMG_1557.JPG" "IMG_1558.JPG" "IMG_1559.JPG" "IMG_1560.JPG" "IMG_1561.JPG" "IMG_1562.JPG" "IMG_1563.JPG" "IMG_1564.JPG" "IMG_1565.JPG" "IMG_1566.JPG" "IMG_1567.JPG" "IMG_1568.JPG" "IMG_1569.JPG" "IMG_1570.JPG" "IMG_1571.JPG"
If a file is locked, then exiftool should return an error. This shouldn't cause a hang.
- Phil
Quote from: Phil Harvey on August 04, 2011, 08:50:48 PM
If a file is locked, then exiftool should return an error. This shouldn't cause a hang.
- Phil
That theory was a longshot.
I'm inclined to believe it has to do with either too many files, or too many args in the command line. However, I don't offhand remember what limits windows xp places on those. I suppose it could also be a problem with exiftool parsing the command line, e.g. because it got truncated by windows and say e.g. quotes in the truncated command line are now no longer balanced as a result. There are many ways to lose.
I've just thrown a quick view here to check for news in this regard. Right now I have no time (must go to work) to explain some things, which might help you (and me) to narrow the couse of this issue.
I'm thankfull for any further details about your findings, but please be patient with my answer.
Bogdan
Ok... I'll be as short as possible on this (you'll understand why).
I've said, since latest GUI version, there's no limit on number of selected files. But as it seems, I should add "in most cases". The thing is, when implementing this feature, I forgot doing that for Exif, Iptc and Xmp edit [ ^ ] buttons!! I still can't believe this happened :-[
That is, these three buttons (and hopefully nothing else) still behaves in old fashion: if total length of all filenames (incl ExifTool parameters) exceeds ~30kb, then "unpredicted" things can happen.
I will fix this in next few days.
However, if all files are selected in filelist pane, then above limitation doesn't exist (because in this case, GUI passes *.* wildcard expression toward ExifTool).
Bogdan
Thanks for the info, and the quick diagnosis.
However, I believe that the first time I had this problem, I had in fact selected all files in the file list. That was the instance where it modified 257 of those files before hanging. But, it's possible that I only thought I had selected all the files, and had somehow missed one, so I will have to check this again. It probably makes sense for me to wait to check that until after you fix the other case. Also, note that in that case, although I had selected all the files in the file list (I believe), there were another 2 files in that directory that were not image files and therefore were not shown in the file list.
Yes, I suggest you wait until I fix this known bug first -don't waste energy :)
If all files are selected, then GUI simply sends (for example):
exiftool -Exif:Artist="MyName" -Exif:Copyright="C by Myname" *.*
-or in case file extension filter is used:
exiftool -Exif:Artist="MyName" -Exif:Copyright="C by Myname" *.jpg
In this case I see no way, command line could exceed ~32k characters (Windows) limit.
Yes, I can imagine one can skip first or last file when selecting all files (to be sure, Ctrl+A should be used). And from GUI's perspective: only (in GUI) visible files matter when GUI "counts" them. If some of selected files are not image files, then, after process is finished, ExifTool will report (via GUI) about that -that is, this shouldn't cause any "unexpected" error.
To save ExifTool's time for reading (not needed) files, file extension filter should be used if possible -especially if working on large amount of files.
Talking about... do you really regulary keep 2000+ image files in single folder? Just wondering. :)
Bogdan
Bogdan,
I'm also having the problem in Win7 while using the "Quick" pane to enter Artist, Copyright and the IPTC object names, location, and keywords. My folder has 3000 images (CR2 RAW+JPG)
It hung after updating 252 files.
Chris
Hi Chris,
No, I've checked Edit in Quick mode (again) and there shouldn't be a problem related to amount of files. That is, it must be something else causing this. But without further info, I can't say what that would be or where to look at.
If it hung after 252 files, try to select and process only about 300 files to narrow down where this actually happens.. by shoosing less files, there's a bigger chance to findout what causes this.
Bogdan
Bogdan,
Thank you for your patience and assistance!
After my prior error, I tried again with 300 or 200 files but it kept getting hung (even after killing the residual exiftool.exe processes in Task Manager). I figured it was because I was selecting less than all the files.
This morning I tried 100 files in the same directory and it completed with the following pop-up message:
Warning: [minor] Adusted MakerNotes base by 4010 - IMG_4450.JPG
Warning: [minor] Adusted MakerNotes base by 4010 - IMG_4451.JPG
Warning: [minor] Adusted MakerNotes base by 4010 - IMG_4452.JPG
Warning: [minor] Adusted MakerNotes base by 4010 - IMG_4453.JPG
Warning: [minor] Adusted MakerNotes base by 4010 - IMG_4454.JPG
100 image files updated
...etc...
Note: The images I selected went from IMG_4450 through IMG_4499 (both CR2 and JPG)
I've also attached a screen print of the error message. If you need a screen print of the full screen, let me know how to e-mail it to you.
Thanks!
Chris
Bogdan,
Further investigation using just one file and process of elimination reveals that the error occurs when updating anything in the EXIF portion, and only for the first time on that file. For example, if I take an untouched file and modify any one row in the EXIF portion of the Quick Tab for a JPG file, I will get the error message. However, if I make subsequent changes to any of the other EXIF tags, I do not receive any error.
I do not receive any error messages if I make a change ONLY to the IPTC section. And only on JPG files. CR2 RAW files give me no problem.
On a hunch I also tried, for one JPG file the "Exif: LensInfo from MakerNotes..." tool and received the exact same error message. Again, only on the JPG, not the CR2.
In all cases above, the information was updated despite the error messages
Chris
Hi Cris,
Such errors ("Warning: [minor]...") occur in cases when ExifTool discovers metadata inside file is not conform with specifications -in most cases, this can happen after metadata was modified with some "other" tool.
However, in such case (minor error) ExifTool is clever enought and "adjust" metadata correctly -that's why this error appears only first time you modify them with ExifTool.
When working on hundreds/thousands of corrupted files, a list with errors can be huge. To keep message reasonable small, GUI shows only smaller amount of them -which should be enough, to alert you, that there is/was something wrong.
If such messages ("minor" errors) disturb you, you can use menu Options>Ignore existing minor errors and disable displaying them.
Your finding proves one thing: you have used some other tool previously, for modifying metadata in JPG files -and that tool corrupted your files.. and now ExifTool corrected existing mistake -isn't ExifTool great? :)
And how come CR2 files aren't affected? I believe, it's because the tool you've used before, probably wasn't capable to modify raw files...
Why there's no problem when modifying IPTC section only? I don't know.. Having a file, Phil could findout the reason for sure (after all, he made ExifTool). But once you run such file thru ExifTool, it gets "corrected" -thus, the cause disappears.
And a notice:
If you plan to work on large amount of files (in single folder) by using [ ^ ] buttons, then wait for next GUI version -it should happen tomorrow.
Bogdan
Bogdan,
ExifTool (and ExifToolGUI!) are indeed great. However, I have not used any other tool, other than Windows Explorer and Windows Photo Viewer, to view the files.
I did use TeraCopy to copy the files from my laptop to my external drives and again to my desktop. I don't think that would have done it but I will do some further research.
Chris
I believe you. But fact is, ExifTool gives warnings for some files.
Windows Photo Viewer... as far I can remember, this viewer isn't capable to use Exif:Orientation tag value to display (portrait oriented) images properly... hence, to see them vertically oriented, user must rotate them manually. If I'm right on this, then by doing this, jpg image file is being "corrected"... just thinking "loud". If I'm wrong, then there was something else.
Years ago, when I discovered, that metadata is being altered without my knowledge (by software I wouldn't believe), I started searching for the tool... and found ExifTool. Being impressed by it's capability (and support!), I decided to make "my private small GUI" for it -I mean, who on earth can remember these tag names and options :)
I don't say other good software doesn't exist... one just need to check consequences befure trusting fully. For example: Even I know Corel PSP is poor player in metadata area, I'm still using it for editing my photos.. but not for managing metadata.
Bogdan
I believe I used ctrl-A, but the past has a habit of becoming vague over time.
I used to keep smaller folders of images, but it became a nuisance for locating or browsing images. However a few thousand is the max in win xp because otherwise the filesystem becomes too slow.
Quote from: BogdanH on August 05, 2011, 04:08:05 PM
Yes, I suggest you wait until I fix this known bug first -don't waste energy :)
If all files are selected, then GUI simply sends (for example):
exiftool -Exif:Artist="MyName" -Exif:Copyright="C by Myname" *.*
-or in case file extension filter is used:
exiftool -Exif:Artist="MyName" -Exif:Copyright="C by Myname" *.jpg
In this case I see no way, command line could exceed ~32k characters (Windows) limit.
Yes, I can imagine one can skip first or last file when selecting all files (to be sure, Ctrl+A should be used). And from GUI's perspective: only (in GUI) visible files matter when GUI "counts" them. If some of selected files are not image files, then, after process is finished, ExifTool will report (via GUI) about that -that is, this shouldn't cause any "unexpected" error.
To save ExifTool's time for reading (not needed) files, file extension filter should be used if possible -especially if working on large amount of files.
Talking about... do you really regulary keep 2000+ image files in single folder? Just wondering. :)
Bogdan
I just counted the number of chars in the failing command line I posted earlier, and it is no more than 8236 bytes (the fastest way for me to count was to create a file of it and look at the size). Obviously, that is nowhere near 32k, so maybe there is something else going on as well, possibly with exiftool. However, again I'll wait til you get the known bug fixed before I try running more tests.
Quote from: BogdanH on August 05, 2011, 12:52:08 PM
Ok... I'll be as short as possible on this (you'll understand why).
I've said, since latest GUI version, there's no limit on number of selected files. But as it seems, I should add "in most cases". The thing is, when implementing this feature, I forgot doing that for Exif, Iptc and Xmp edit [ ^ ] buttons!! I still can't believe this happened :-[
That is, these three buttons (and hopefully nothing else) still behaves in old fashion: if total length of all filenames (incl ExifTool parameters) exceeds ~30kb, then "unpredicted" things can happen.
I will fix this in next few days.
However, if all files are selected in filelist pane, then above limitation doesn't exist (because in this case, GUI passes *.* wildcard expression toward ExifTool).
Bogdan
Quote from: BogdanH on August 06, 2011, 08:37:29 AM
Why there's no problem when modifying IPTC section only? I don't know..
The answer is simple: ExifTool only modifies the EXIF if you are writing EXIF information, for 2 reasons: 1) It's faster, and 2) I don't like the idea of ExifTool changing something unless you tell it to. If you write only IPTC tags, you shouldn't expect the EXIF to change.
Having said this, for most raw images the IPTC is actually stored inside the TIFF/EXIF structure. So for these images, editing the EXIF it is unavoidable. But for JPEG images, the EXIF and IPTC are in separate segments.
- Phil
Thanks for this clarification, though it doesn't currently affect me. I strongly agree with the sentiment about what exiftool should and shouldn't modify. However, for a neophyte like me, the story about raw images is new info and counterintuitive. If you don't already mention it somewhere in the FAQ, it might prove useful for people like me if you added it.
Quote from: Phil Harvey on August 06, 2011, 02:03:42 PM
Quote from: BogdanH on August 06, 2011, 08:37:29 AM
Why there's no problem when modifying IPTC section only? I don't know..
The answer is simple: ExifTool only modifies the EXIF if you are writing EXIF information, for 2 reasons: 1) It's faster, and 2) I don't like the idea of ExifTool changing something unless you tell it to. If you write only IPTC tags, you shouldn't expect the EXIF to change.
Having said this, for most raw images the IPTC is actually stored inside the TIFF/EXIF structure. So for these images, editing the EXIF it is unavoidable. But for JPEG images, the EXIF and IPTC are in separate segments.
- Phil
Quote from: pb on August 06, 2011, 02:14:51 PM
However, for a neophyte like me, the story about raw images is new info and counterintuitive. If you don't already mention it somewhere in the FAQ, it might prove useful for people like me if you added it.
It is true that I haven't documented this. The reason is that explaining it properly requires an explanation of metadata structure of the various image formats, which would also be useful, but is something that I have been trying to avoid. Also, this is an implementation-specific detail that I would prefer not to document in case I want to change the details of the implementation in a future ExifTool version.
- Phil
I have now tested multiple file hang using vesion 4.17 of exiftoolgui (and v 8.60 of exiftool). The problem is exactly the same as before, and "it" hangs at exactly the same point on the same directory (well, a copy of the original directory) -- after processing 257 files. This time I verified that the command line to exiftool indeed contains "*.*". The second spawned exiftool is hung -- it stops any further i/o activity and uses no more cpu.
I don't know how exiftoolgui is communicating with exiftool -- conceivably the hang could be due to some pipe problem -- so I ran the exiftool command directly from the command prompt. Running it that way does not hang, and finishes modifying the entire directory, without any problem.
My conclusion is that the problem is probably coming from whatever method is being used to communicate between exiftoolgui and exiftool. My guess is that you have some kind of pipe, and due to some error in handling the pipe a deadly embrace kind of situation is happening -- exiftool is blocked because the pipe is full (or waiting for it to be emptied), while exiftoolgui is blocked thinking that it needs more data in the pipe before emptying it, or maybe blocked in some intermediate operation before working on emptying the pipe again, for example some auto-refresh task (or blocked in some weird way, e.g. caused by an exiftool error message about fixing something in an image file). As usual, there are many ways to lose; I'm just guessing at some.
I should also mention one small irregularity, namely that using the copyright symbol from the command line that exiftoolgui sends to exiftool (as reported by Process Explorer) in the command I used to run exiftool directly from a shell results in a wrong symbol ending up in the exif field. I'm pretty confident that this is an independent charset problem, but just in case it's not, I'm mentioning it.
Now, that you've mentioned that: GUI uses standard MS CreateProcess routine for calling ExifTool and pipes, of course. I can imagine, that there (pipe in/out) can be a reason for this issue.
I've ran GUI with about 1500 mixed JPG and RAW files (several GBytes) and everything is fine -so, it's hard for me to findout where the problerm is. Anyway, I will take a closer look to my piping methods and if needed I'll make some "special" GUI version where you could change some related parameters, which should help locating what's needed to be changed.
Bogdan
PS: I'm in hurry.. just in short: to enter C-right symbol in GUI, you press (and hold) Alt key and type 0169 on numerical keyborad (it's a Windows thing).
One more thing that I should have mentioned in my last post was that I notice that a hang has occurred when the gui fails to redraw its window after having been occluded by something else. This might only be a symptom of the fact that it is hung, but it might also mean that something in the user interface hangs first, and that propagates to failing to read from the exiftool pipe. This might or might not be related to the fact that exiftool issues a fairly large number of warnings about minor problems in the exif data.
Quote from: BogdanH on August 10, 2011, 01:22:04 AM
Bogdan
PS: I'm in hurry.. just in short: to enter C-right symbol in GUI, you press (and hold) Alt key and type 0169 on numerical keyborad (it's a Windows thing).
Although that's useful info, that isn't what I was wondering about. Here's what was weird: sysinternals Process Explorer shows the command line that exiftoolgui gave to the shell to run exiftool. What it showed was the circle-c copyright symbol. I used the windows "copy to clipboard" (ctrl-c) and then pasted that command line into an emacs shell buffer. Emacs also then showed the symbol correctly. I then ran the command line, which ran just fine. Then when I used either exiftool or exiftoolgui to show what ended up in the copyright field, it showed a different pair of characters (now a pair, perhaps reflecting a conversion to unicode or other 16 bit coding.) What I am trying to figure out is exactly what all character conversions were taking place to cause this, and what character I should have handed to exiftool that would be exactly the character that exiftoolgui actually gave it in the command line it used. (BTW I am running a US English Windows installation, with everything set to US English.)
BTW, based on Phil Harvey's answer to a similar question on the exiftool forum (https://exiftool.org/forum/index.php/topic,3506.0.html), it's probably not such a great idea to use the copyright symbol in exif data, only in xmp data, since as I understand his response, strictly speaking it is not allowed. I think I will end up just saying "(c) Copyright" instead of using the symbol, since it is much more likely to survive obscure character set conversions.
PH Edit: Added link to thread in the exiftool board
@pb
First, thanks again for taking time by giving additional info. As said, I can hardly test my code, bacause I don't get this "hangs". Here is a link to "beta" version of GUI:
Link removed
-to start somewhere, I have significant increased pipe buffer.
Now, if you would be so kind to try it out... if buffer is/was the reason, then I expect that either: there's no "hang" anymore or much more than 257 of your files will be processed. If there will be no changes, I'll need to look elsewhere.
You've mentioned you get many "minor" errors.. have you tried to "check" Options>Ignore existing minor errors menu? Does it make any difference in number of processed files before "hang"?
Thank you,
Bogdan
PS: as soon you confirm downloading, BetaGUI will be deleted.
I haven't had time to try the beta gui yet, but I do have more information to report. Sorry I did not get a chance to tell you last night (my time zone - Pacific) so you would have seen it early your time.
I realized that I could check whether the problem was coming from the many warning messages that exiftool generates on my files, because as you mentioned to coz earlier in this thread, it fixes the problems on the first pass, and subsequent runs will not find those problems. So, I ran exiftoolgui on the same directory after I had run exiftool already once. The result was that exiftoolgui completed the entire directory of over 2000 files without hanging. This makes me believe that the problem is related to the many warning messages. Coz's complaint and resolution is consistent with this theory. I can send you the full output of exiftool if you want to analyze why it could cause a problem.
Doing this did uncover some other less serious problems, though:
1. After all exiftool runs had finished, exiftoolgui used 100% of cpu time on one cpu (I have a dual cpu machine), for more than one minute (maybe several minutes -- I did not time it) before it was able to refresh the ui and show a message box telling me about warnings/errors.
2. Then, after I clicked 'ok' on the message box, exiftoolgui again took a very long time at 100% cpu before it finally refreshed the ui again and became available for input. I did have autorefresh turned on, and was displaying only filesystem information (so presumably exiftoolgui did not have to read the contents of all files). Doing a manual refresh by clicking the 'refresh' button only takes a few seconds (at most 10 sec) on the same directory, so apparently a lot of unnecessary crunching is going on.
3. One of the error messages from exiftool was for a non-image file in the directory. Exiftool interprets *.* as meaning all files, while exiftoolgui only displays image files, and only certain image files. In my case, this was not a problem, but in the case of image files that are not supported by exiftoolgui's file pane, but which are supported by exiftool, this can result in exiftool processing files that the user was not aware would be processed.
Ok, I downloaded the beta and confirmed that I can run it. (Have not yet had time to test it on the problem directory yet.)
Quote from: BogdanH on August 11, 2011, 12:58:22 PM
@pb
First, thanks again for taking time by giving additional info. As said, I can hardly test my code, bacause I don't get this "hangs". Here is a link to "beta" version of GUI:
Beta1GUI (https://exiftool.org/gui/exiftoolgui417b1.zip)
-to start somewhere, I have significant increased pipe buffer.
Now, if you would be so kind to try it out... if buffer is/was the reason, then I expect that either: there's no "hang" anymore or much more than 257 of your files will be processed. If there will be no changes, I'll need to look elsewhere.
You've mentioned you get many "minor" errors.. have you tried to "check" Options>Ignore existing minor errors menu? Does it make any difference in number of processed files before "hang"?
Thank you,
Bogdan
PS: as soon you confirm downloading, BetaGUI will be deleted.
I'll wait for your comment on "beta"... thanks.
Bogdan
Quote from: BogdanH on August 11, 2011, 03:42:57 PM
I'll wait for your comment on "beta"... thanks.
Bogdan
Ok, I ran it with the beta. Same exact hang, at the same exact spot.
Here is the output from exiftool when I run it from the command line (actually from a windows cmd shell inside emacs). (I obfuscated my name, to keep it off the internet, but still same number of chars.) Note that exiftoolgui fails at file IMG_01771.jpg, i.e. IMG_01771.jpg is the last file where it modifies exif.
I've also attached a directory listing (emacs dired) of everything in the directory that I am running on.
Note that I have windows explorer set to use alphabetic (i.e. "stupid") (default in win2k and earlier) sort order, not "smart" (default in win xp) sort order where it sorts numbers by value rather than alphabetically. Emacs always uses "stupid" sort order, I believe.
Another curious thing I should mention, though probably not related to the problem, is that exiftool generates an error for the file "Picasa.ini" but not for the file ".picasa.ini". (These are droppings from Picasa, one from an older version, one from a newer version.) I guess exiftool does not attempt to process files that start with ".".
Also, incidentally, all the makernotes errors that exiftool discovers are generated by files that were written by some older version of Picasa, but files written by the most recent version do not generate the errors (i.e., exiftool "warning"s).
One other piece of information. If I kill the exiftool processes that were spawned by exiftoolgui (or more exactly, the exiftool process that gui spawned, and the exiftool process that exiftool spawned), then exiftoolgui unblocks and shows its message box with a (partial) list of exiftool warnings. Clicking "ok" then restores a fully functioning exiftoolgui. (But of course without modifying all the files that were requested.)
I can replicate the issue, so finding the cause should be easier now...
Added (about 12h later):
I'm out of ideas right now... for some reason, at some point, I simply can't catch error messages anymore, if too many files have errors -and increasing the buffer doesn't change a thing.
From my perspective, it looks like after writing "some" amount of error messages, ExifTool would suddenly change the way of writting them, and GUI can't cope with that. Yeah, I know it sounds stupid...
Reality is, in console, everything seems normal... so, I think, I need a break.
Bogdan
This is actually good news, since I'm sure you will now be able to find the problem. I predict it is due to some counterintuitive behavior of Windows when dealing with pipes.
Quote from: BogdanH on August 12, 2011, 02:40:11 AM
I can replicate the issue, so finding the cause should be easier now...
Added (about 12h later):
I'm out of ideas right now... for some reason, at some point, I simply can't catch error messages anymore, if too many files have errors -and increasing the buffer doesn't change a thing.
From my perspective, it looks like after writing "some" amount of error messages, ExifTool would suddenly change the way of writting them, and GUI can't cope with that. Yeah, I know it sounds stupid...
Reality is, in console, everything seems normal... so, I think, I need a break.
Bogdan
I've found the cause of "hanging" and fixed it
It wasn't a bug actually.. if at all, I would blame MS API documentation. Anyway, because of it's nature, it's not possible to prevent "hangs" 100% -that is, I hope nobody keeps 10000 files (with corrupted metadata) in single folder :)
Phil, thank you for providing these thousands of jpg examples (contaning "minor" errors) -without having them, I wouldn't be able to find the solution.
Bogdan
PS: Fixed GUI (v4.18) will be available for download within next 24h hours.
Great! Also, I'm glad I correctly predicted it would be some MS weirdness.
Quote from: BogdanH on August 13, 2011, 07:32:33 AM
I've found the cause of "hanging" and fixed it
It wasn't a bug actually.. if at all, I would blame MS API documentation. Anyway, because of it's nature, it's not possible to prevent "hangs" 100% -that is, I hope nobody keeps 10000 files (with corrupted metadata) in single folder :)
Phil, thank you for providing these thousands of jpg examples (contaning "minor" errors) -without having them, I wouldn't be able to find the solution.
Bogdan
PS: Fixed GUI (v4.18) will be available for download within next 24h hours.
To be honest, it's not that MS stuff wouldn't work as documented. It's just, (in this case) the thing I needed to know, isn't described precise enough.
Thanks again for all the info you gave me.
Bogdan
PS: new GUI v4.18 is ready for download.
Bogdan: I'm impressed you were able to fix this. Bugs like this can be very difficult to track down.
- Phil
Thank you for compliment, Phil.
To tell the truth, I was very, very close to give up on this (+advice like "keep modest amount of files in single directory" :) ).
Bogdan
So what was the problem, Bogdan? Can you elaborate or provide a link that describes it in more detail? Remember that there are fellow programmers out there who also use EXIFTOOL in their applications... :)
Thanks!
Bodgan,
Thanks for your persistance & patience. Great fix.
Chris
@ coz:
Thank you.
@MOL:
There might be more ways/techniques to "talk" with ExifTool, but one can't avoid using at least two API functions: CreateProcess and CreatePipe.
In this case, after checking almost everything else involved in this (my own procedures/variables in first place), I came to my CreatePipe usage. I've re-read some tutorials on the web, plus MS quide:
http://msdn.microsoft.com/en-us/library/aa365152%28v=vs.85%29.aspx
-here, nSize parameter is described as follows:
The size of the buffer for the pipe, in bytes. The size is only a suggestion; the system uses the value to calculate an appropriate buffering mechanism. If this parameter is zero, the system uses the default buffer size.
The answer that I'm still looking for, is:
-how much buffer exactly do I get, if value is zero?
-how much buffer exactly do I get from buffering "mechanism", if value is non-zero?
That is, I don't like writting code by "try/error" until it works in particular case only. Of course, to be "sure" one could choose buffer size of 1GB, but... does "program being memory hog" sounds familiar? :)
Bogdan
Thanks a lot, much appreciated!