News:

2023-03-15 Major improvements to the new Geolocation feature

Main Menu

Proposal for a robust and simple Windows version

Started by obetz, May 16, 2019, 07:23:36 AM

Previous topic - Next topic

obetz


mceachen

I've been able to run all the unit tests that do reading and writing for exiftool-vendored against both your 11.69 build, and, using your unparx.cmd, a new 11.71 build: https://ci.appveyor.com/project/mceachen/exiftool-vendored/builds/28279077.

I also ran an extensive read test (reading some 4.5k example images) to reproduce the Tags typings (see https://exiftool-vendored.js.org/interfaces/tags.html, but caution, it's a huge webpage!), and the differences were expected (due to timestamp differences and other expected system noise).

It looks stable to me! The added security of not running perl scripts out of a temporary directory is also quite appealing.

I had some issues with trying to reproduce your 11.69 zipfile by using your build instructions, and sent you an email directly, but the diff and steps I took are here, if anyone else cares to look: https://github.com/mceachen/exiftool-vendored.exe/commit/5294df9fd23b8149b8dc2d3f96ccb85f09232466. Given how frequently Phil pushes out versions and bugfixes, it'd be nice if other people could reliably reproduce the installer (or, best-case scenario: have Phil switch to this installer!)

Thanks again for your work here, Oliver!

obetz

Quote from: mceachen on October 22, 2019, 11:05:48 PM
I had some issues with trying to reproduce your 11.69 zipfile by using your build instructions, and sent you an email directly, but the diff and steps I took are here, if anyone else cares to look: https://github.com/mceachen/exiftool-vendored.exe/commit/5294df9fd23b8149b8dc2d3f96ccb85f09232466

Either use unparx on Phil's exiftool.exe or use my Strawberry Perl based package, don't mix them.

unparx.cmd from https://oliverbetz.de/pages/Artikel/Portable-Perl-Applications will unpack Phil's exiftool.exe to use it's contents without the pp magic. Phil's exiftool.exe is based on a different Perl distribution than "my" package.

The ExifTool package available from https://oliverbetz.de/pages/Artikel/ExifTool-for-Windows is based on Strawberry Perl.

The "several hundred different files" are simply the different Perl distributions.

It makes no sense to replace all the Perl related files manually just to get a new ExifTool version.

You need to follow the matching "build instructions". If you want to upgrade the Strawberry Perl based package, follow the instructions at the end of my related page: "Sources How to build the installer yourself".

Again: Do not use files unpacked from Phil's exiftool.exe (using unparx) together with my Strawberry Perl based package.

After all: Even though I can't keep up with Phil's pace, the versions I release should be sufficient for the needs of most users.

Oliver

Eriksson


mceachen

Quote from: obetz on October 23, 2019, 07:10:21 AM
Quote from: mceachen on October 22, 2019, 11:05:48 PM
I had some issues with trying to reproduce your 11.69 zipfile by using your build instructions, and sent you an email directly, but the diff and steps I took are here, if anyone else cares to look: https://github.com/mceachen/exiftool-vendored.exe/commit/5294df9fd23b8149b8dc2d3f96ccb85f09232466

Either use unparx on Phil's exiftool.exe or use my Strawberry Perl based package, don't mix them.

Ah! Sorry, I misread your instructions!

I converted your instructions into a short shell script: https://github.com/mceachen/exiftool-vendored.exe/blob/master/update.sh.

The build from that script passes all exiftool-vendored tests: https://ci.appveyor.com/project/mceachen/exiftool-vendored/builds/28336491.

This is great: it's wonderful to have ExifTool on Windows launch in fractions of a second, rather than tens of seconds.

Thanks again, Oliver!

obetz

thanks for providing this shell script.

Regarding the speed: Your system must be somewhat different from most other user's setup.

Some time ago I did a simple bechmark test: https://exiftool.org/forum/index.php/topic,10230.msg53763.html#msg53763

On my PC, Phil's exiftool.exe started in less than 250ms. My "par-less" exiftool starts in 150ms. This is faster, but I can't see how the original exiftool.exe might take "tens of seconds".

Also on https://exiftool-vendored.js.org/ you write "On Windows, for every invocation, exiftool installs a distribution of Perl and extracts the ~1000 files that make up ExifTool, and then runs the perl script". This should not be the case, the extracted files stay in the temp directory and are re-used at the next time.

Even a fresh install should not take "tens of seconds". Which virus scanner do you use?

mceachen

Quote from: obetz on October 24, 2019, 01:52:32 AM

Even a fresh install should not take "tens of seconds". Which virus scanner do you use?


I'm running tests on the latest Windows 10 (version 1903) on an older XPS 15 (i7 6700) on an SSD using Windows Defender.

I've just "refreshed" windows a month or two ago, but it does have a full development environment on it, and something there may be mucking with I/O performance, but I have other windows boxes that are just vanilla, nothing-else-installed test machines, and the prior version of exiftool would sometimes take 2 to sometimes >10 seconds (!!) to return the result of `exiftool -ver`.

Windows Defender spikes the CPU while I'm waiting for the version number, fwiw.

In any event, I was able to delete that comment, as I'm pushing out your new approach today: https://github.com/photostructure/exiftool-vendored.js/commit/6a3ade83d9ec3374af09c64d1845ede3d104c52c#commitcomment-35661276

obetz

Quote from: mceachen on October 25, 2019, 12:57:02 PM
I have other windows boxes that are just vanilla, nothing-else-installed test machines, and the prior version of exiftool would sometimes take 2 to sometimes >10 seconds (!!) to return the result of `exiftool -ver`.

that's strange, I never observed such a long delay. And I'm using ExifTool for many years now.

mceachen

Due to other reasons, I just had to do a "Reset this PC" on this machine, so it's Windows 10 with nothing other than what comes with a "clean" install, plus git for windows and vs2015:

Current official release, first run: `time exiftool -j image.jpg`

real    0m11.715s
user    0m0.000s
sys     0m0.015s

Immediate subsequent runs are much faster, though:

real    0m0.435s
user    0m0.015s
sys     0m0.031s


However your release, first time and every time: `time exiftool -j image.jpg`

real    0m0.249s
user    0m0.000s
sys     0m0.015s


mikelee33

Oliver:
I just today found and tried out your portable ExifTool. Super great idea, and it works flawlessly! I never did care for the self extracting (to the %temp% directory) workings, but understood that Phil knew this worked fine for most all users. As a software application developer I prefer your concept, as it installs cleanly in the program directory, and there is no clean up in the temp folder when an application is uninstalled.
Thank you for your hard work.
Michael Lee
https://gpstamper.com
Mike

obetz

thanks for your feedback.

I suggest to keep the original file times from my archive instead of the time when you extracted it.

Maybe the "zone identifier" issue? https://devblogs.microsoft.com/oldnewthing/20110504-00/?p=10743

Oliver

Malus

Hi Oli!

Ich wollte einfach mal Danke sagen, dass du dir Mühe machst, den Installer auf dem neusten Stand zu halten.  🙏
Ohne deinen Installer müsste ich ein Loch in die Sicherheit meiner Systeme bohren.

Malus

obetz

Hallo Malus,

danke für das Lob und die Erinnerung an 12.29 ;-)

Es freut mich, einen Beitrag leisten zu können. Das Erstellen ist halbwegs automatisiert. Ein paar ausführlichere Selbsttests (zusätzlich zu denen von Phil) für die Windows-Spezialitäten mache ich noch "zu Fuß", aber nicht bei jeder Version.

Thanks for the kudos,

I'm glad to be able to contribute. The build is rather automated. The more detailed self-tests (in addition to Phil's) for the Windows specialties I still do manually, but not for each version.

Oliver

Phil Harvey

This version of ExifTool reduced memory usage from >12GB to about 3.5GB for one user who was having out-of-memory problems.  See the github issue here.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

obetz

Quote from: Phil Harvey on December 30, 2021, 07:55:28 PM
This version of ExifTool reduced memory usage from >12GB to about 3.5GB for one user who was having out-of-memory problems

Interesting, I have no explanation... Maybe Strawberry Perl or the Perl version is the reason (we don't even know whether the 32 or 64 bit version was used).

Oliver