Exiftool "hijacking" 100% of my local network

Started by musicadi, November 20, 2023, 04:24:43 PM

Previous topic - Next topic

musicadi

Hi,

I am experiencing a weird issue when using exiftool to copy tags of large video files (aprox. 3 Gb), which are on my NAS.

What I'm doing is converting mov files to mp4 and then using exiftool to copy the tags from the mov file to the mp4 file.

I use this command: exiftool -tagsfromfile %d%f.mov -ext mp4

After the command is issued, exiftool is working, but my whole local network lags like crazy until the copying is finished.

I suspect the exiftool command is somehow overwhelming the router or the NAS.

Any thoughts?

Thank you!

My system:

exiftool 13.69
macOS 13.6.1
NAS: QNAP TS253A (connected to it through SMB)

StarGeek

Exiftool does have to read the entire file and output the edited version.  When you are editing a file over the network, it will be reading and writing at the same time.

You might look into installing exiftool on the NAS.  From what I understand from previous posts on these forums, a QNAP NAS can run linux programs.  All that would be required is installing Perl and then the exiftool code.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Neal Krawetz

As StarGeek said: You're looking at 2x-4x file transfers.

Every file must go from the NAS to the location computer (transfer #1).
Then exiftool may need to use a temp file. If you're sitting on a mount point, then this could be another 2x file transfers (write and then read).
Then exiftool needs to write the final output (last transfer).

(Without temp files, that's 2x the file size being send across the network. With temp files in a mounted directory, that's 4x.)

So yeah, that's a lot of bandwidth if you're processing lots of files.

Depending on the type of NAS, you might be able to remotely login, install exiftool, and run it locally on the NAS. (Synology and most other NAS's permit administrative logins.)  This will allow you to run exiftool without the network overhead.

Phil Harvey

Neal is correct, but I think it is 3 transfers not 4.

1. Read the source MOV
2. Read the source MP4
3. Write the destination temporary MP4
 - Rename the temporary file (no need to transfer over network)

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

D'oh, I totally missed that it was reading a MOV file as well as the MP4.  Yeah, that's a lot of network traffic.

Installing exiftool on the NAS would probably be the best move.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

musicadi

Hi everyone and thank you for your answers!

So basically it does several read/write operations simultaneously? Is this why it's using up the whole bandwidth? Is there an option to cap the bandwidth it uses?

Installing exiftool on the NAS is definitely a good idea. Also, having the destination file on my local machine would speed things up then, right? In this case exiftool would only need to read the source file from the NAS and all other operation would be done on the local machine. The only question is what the right syntax would be to achieve this?

Anyway, I only get this when working on a large video file (even if it's just one file). No problems when doing operations on small files (even with a large number of files).

Phil Harvey

Quote from: musicadi on November 21, 2023, 08:37:40 AMSo basically it does several read/write operations simultaneously?

No.  Sequentially.

QuoteIs there an option to cap the bandwidth it uses?

No.

QuoteIn this case exiftool would only need to read the source file from the NAS and all other operation would be done on the local machine.

Source files.

Add -o DESTINATION_DIR/ to the command to write to a different output directory.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

Quote from: musicadi on November 21, 2023, 08:37:40 AMIs there an option to cap the bandwidth it uses?

To expand a bit more, exiftool doesn't know anything about where the data is coming from. It has no network capabilities on its own. From its point of view, it's just reading/writing a file off of a disk (or from STDIN or a pipe).
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

musicadi

Quote from: Phil Harvey on November 21, 2023, 08:58:46 AMAdd -o DESTINATION_DIR/ to the command to write to a different output directory.

- Phil

Thank you! Does tag copying work if the files have the same filename, different extensions, but they are in different directories?

Phil Harvey

In your command, %d represents the directory of the target image.  If the source is in a different directory, change %d to that directory.

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

StarGeek

Or you can edit the %d with the options listed under the Advanced features section -w (-TextOut) option.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

musicadi

Quote from: Phil Harvey on November 21, 2023, 02:08:01 PMIn your command, %d represents the directory of the target image.  If the source is in a different directory, change %d to that directory.

- Phil

Perfect, thank you! It's definitely WAY faster this way and the network is not overwhelmed either.