Appending metadata to a database file as images are added to a folder?

[Charles Barbour]

We have installed a couple systems that allow faculty and students to easily create mp4 recordings. When the recording is done, they can copy the file to a thumbdrive or network share, upload it to Dropbox, delete it, etc.

I'd like to gather some metrics about how much video is being created on these systems and Exiftool seems like it can do it but I'm running into my ineptitude. (I may know who is using the system and when but if they delete them, I don't know how many videos they made.)

All the files are automatically saved to a specific folder. I'd like to watch that folder and as video files are created, I'd like to extract the metadata from only the newly created file before it is moved or deleted. (I don't want to have exiftool scan through months of video files every time a new one gets created.) Since the user might move the file immediately, that kind of precludes running a scheduled task. A scheduled task would also prevent power management from working correctly.

Since both machines are Macs, I thought an automator watch folder might be the easiest way to do this.

Basically here's the desired functionality.
A new mp4 file is added to the folder
Automator passes the filename to exiftool
I run exiftool as a shell script with the name as an argument.

Where I'm stuck is how to get exiftool to scan only the new file and get it to append that info to an existing file. I was thinking that the -csv option would work but it appears that you can't append files using that option.

Since all the files will have the same tags, -T would work (either by specifying the specific tags I want or just letting it dump them all).

What I can't seem to get my head wrapped around is how to make the command append a text file as each new video file is added and to specify where that file is stored. Ideally I'd like to store a file called video_metadata.txt in the Documents folder.

Using this, I can create and append the file in the same directory as the videos but that's not great.

Code Select Expand

for f in "$@"
do
exiftool -filename -createdate -duration -T -w+ %d/scriptout.txt "$f"
done


I can't seem to figure out how to append one file in another folder. It keeps creating individual files.

I'd also welcome any other thoughts on how to improve this process or make it more robust.

Thanks!

[Alan Clifford]

The key thing here seems to be that you don't have control of the files being deleted.

How about creating a second directory with hard links to the files?  Even if the user deletes the original file, it will continue to exist.  You will have complete control of the files in the second directory and can delete them after you have finished with them.

[Charles Barbour]

I thought about that but there were some potential issues:

• To respect their privacy, I want a user to have faith that if they delete their file, I don't have a copy.
• Since they're videos, the files are pretty large. As such, I'd prefer to not duplicate them, even in the short term, if I can come up with a viable alternative.
• Copying a file that might be a GB or more can take some time. Time when the user might be wanting to do something else with the file like copy or delete it. Also one of the systems is somewhat automated and it would simultaneously be copying the file to a thumbdrive.
• I have to imagine that extracting the metadata would be faster than trying to do a copy or is that not always true?

[Hayo Baan]

Hi Charles,

Alan actually suggest hard linking the files, not copying them. You create a hard link using the ln command. With a hard link, the file is at two places, but underneath it's the same file, creating a hard link is an almost instantaneous action. The only restriction is that the folders in which the hard linked files are, are on the same volume.

Hope this helps,
Hayo

[Charles Barbour]

Thank you, that is helpful. I still would prefer to not do that for reasons of privacy though.

[Phil Harvey]

Hi Charles,

Using this, I can create and append the file in the same directory as the videos but that's not great.

Code Select Expand

for f in "$@"
do
exiftool -filename -createdate -duration -T -w+ %d/scriptout.txt "$f"
done


I can't seem to figure out how to append one file in another folder. It keeps creating individual files.

Just specify the output folder name instead of %d.

Or alternatively, the usual method is to pipe the output when a single output file is desired:

Code Select Expand

exiftool -filename -createdate -duration -T >> OUTDIR/scriptout.txt "$f"

... but I'm not sure if this will work in whatever type of script you are using.

Also, you should be able to avoid looping by passing all the filenames to exiftool in one command, although you would have to think about how the quoting is handled.

- Phil

[Charles Barbour]

Thanks! I wound up using the following and it works great.

Thank you so much for this tool. Very helpful!

Code Select Expand

for f in "$@"
do
exiftool -filename -createdate -duration -filesize -avgbitrate -imagesize -T >> /$HOME/Documents/videometadata.txt "$f"
done

[Phil Harvey]

Very interesting.  I didn't know you could put arguments after the redirection.  The normal thing to do would be to put "$f" before >> on the command line.  But what you have done seems to work.

- Phil

[Charles Barbour]

One small follow up, question. When running exiftool, it appears that the timestamps of the videos are off by 4 hours.

It's currently almost noon here and a video I just recorded shows a time of 11:46 in Finder but 15:46 in the exiftool output.

I assume the create date of the file is based on UTC time. Is there any way to get exiftool to report the local time without changing the file itself? It would be nice if the output didn't need to be modified before use.

It looks like the -d option might do it but I'm unsure what to change.

Thanks!

[Phil Harvey]

Try -api QuickTimeUTC

- Phil