ExifTool Forum

ExifTool => Newbies => Topic started by: polayer2015 on August 04, 2017, 10:17:26 AM

Title: Ubuntu Automation
Post by: polayer2015 on August 04, 2017, 10:17:26 AM
I would like to automate my photo processing on Ubuntu 16.04.  My photos are upload via photosync on IOS to a qnap network folder.  Folder has been permanently mapped and is accessible by exiftool.  From there I would like to automate the renaming of each photo into a folder structure of Year/Month/Date.    I can do this separately via CLI, but would like to automate the process.  So far I have:

This renames the files based on the parameters I want to use.
exiftool -d %Y%m%d_%H%M%%-c.%%e "-filename<CreateDate" DIR

This places my photos into a folder structure of Year/Month/Date located at /home/myusraccount/2017. 
exiftool -d %Y/%B/%d "-directory<datetimeoriginal" *.jpg

I would like to merge the first two steps and move the files into my master photo collection located at /media/Multimedia/My Pictures.  This folder also uses the  Year/Month/Date structure to store photos.  I would like for this to take place automatically once new photos are uploaded from photosync via my iphone or new images are added from my DSLR.  Thanks in advance for any help with my setup.





Title: Re: Ubuntu Automation
Post by: Phil Harvey on August 04, 2017, 10:40:30 AM
I'm not clear.  You want to merge these 2 commands?  If you can use CreateDate for both, this would work:

exiftool -d %Y/%B/%d/%Y%m%d_%H%M%%-c.%%e "-filename<CreateDate" DIR

To do this automatically you could add this as a cron job, but whatever copies the images into the source directory must first copy to a temporary file then rename so that exiftool doesn't pick up partially-written files.

- Phil
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 04, 2017, 10:59:07 AM
Thanks for the quick reply.  This worked fantastic!  How can I set the destination folder that the files are moved to.  With this cli, the files are moved to /home/myusraccount/2017.  I would like the final destination to be /media/Multimedia/My Media/My Pictures.  This would require files to be copied into preexisting folders with the directory structure of year/month/date.  Thanks in advance.
Title: Re: Ubuntu Automation
Post by: Phil Harvey on August 04, 2017, 11:11:30 AM
Since we specified a relative directory, the destination directory will be relative to the current working directory, so you could either first "cd" to "/media/Multimedia/My Media/My Pictures", or alternatively specify an absolute directory like this:

exiftool -d "/media/Multimedia/My Media/My Pictures/%Y/%B/%d/%Y%m%d_%H%M%%-c.%%e" "-filename<CreateDate" DIR

- Phil
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 04, 2017, 01:09:12 PM
Thanks!  Once again this worked perfectly.  Just reviewed my work flow and had a few additional questions.  My wife and I will use the IOS app photosync to upload photos/videos to a network folder (/media/Multimedia/My Media/My Pictures/Need to sort).  I wanted see if exiftool could scan this folder recursively (/media/Multimedia/My Media/My Pictures/Need to sort/My Phone and /media/Multimedia/My Media/My Pictures/Need to sort/Wifes Phone).  The next question is how to handle photos and videos.  Photos are stored /media/Multimedia/My Media/My Pictures and videos are stored in /media/Multimedia/My Media/My Pictures/My Videos in a Year/Month/Day structure.  Sorry for complicating the process, but this would make my digital life so much easier if I could automate these steps.
Title: Re: Ubuntu Automation
Post by: Phil Harvey on August 04, 2017, 03:24:10 PM
Again, I'm not clear on what you are asking, but just add the -r option to process the source directory recursively.

- Phil
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 05, 2017, 11:56:43 AM
Thanks for the help so far!

Ran this:
exiftool -r -d "/media/Multimedia/My Media/My Pictures/%Y/%B/%d/%Y%m%d%%-c.%%e" "-filename<CreateDate" "/media/Multimedia/My Media/My Pictures/Need to Sort"

Got this:
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0551.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0441.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0440.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0439.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0437.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0356.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0309.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0308.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0306.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0297.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0273.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0255.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0252.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0196.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0194.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0159.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0160.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0156.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0155.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0146.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0133.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0562.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0476.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0438.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0484.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0550.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0372.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0370.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0364.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0363.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0342.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0331.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0311.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0314.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0310.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0307.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0287.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0254.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0256.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0239.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0222.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0215.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0211.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0192.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0189.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0182.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0166.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0165.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0157.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0148.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0145.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0315.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0369.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0573.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0362.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0359.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0324.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0242.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0290.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0253.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0361.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0143.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0371.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0487.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0225.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0483.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0330.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0303.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0296.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0514.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0320.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0300.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0367.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0243.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0323.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0542.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0301.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0227.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0150.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0144.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0135.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0100.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0007.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0006.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0564.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0137.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0283.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0183.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0184.JPG
    5 directories scanned
   37 image files updated
   89 image files unchanged




Seems like I have 89 images that are in limbo. Files are no longer in directory and were not moved to final destination.  Not sure how to address the writable tag issue.

To clarify my second request, I would like to separate .JPG's  and .MOV's .  Photosync (IOS App) uploads my photos and videos into a permanently mounted network folder located at /media/Multimedia/My Media/My Pictures/Need to sort/My Phone and /media/Multimedia/My Media/My Pictures/Need to sort/Wifes Phone.  From there I would like exiftool to recursively scan the folders, rename, and move to specific folders. Photos are stored in /media/Multimedia/My Media/My Pictures and videos are stored at /media/Multimedia/My Media/My Pictures/My Videos.  Both locations have year/month/date directory structures.  The goal is to run a scheduled exiftool command that can distinguish between .JPG/.MOV and place the files in the appropriate directory.  Hopefully this clarifies my request.
Title: Re: Ubuntu Automation
Post by: Hayo Baan on August 05, 2017, 01:20:01 PM
Regarding the non-writable tag issue, check to see if the images indeed have a CreateDate. They probably don't so you'll need to use another tag instead.

To select only JPG files add -ext JPG, similarly for MOV. That should easily solve the split of images and videos. To split between your phone and your wife's is a bit more of a challenge. But is doable too as long as you have good way to make the distinction. Is there perhaps a unique tag of part of the filename?
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 05, 2017, 05:01:31 PM
Quote from: Hayo Baan on August 05, 2017, 01:20:01 PM
Regarding the non-writable tag issue, check to see if the images indeed have a CreateDate. They probably don't so you'll need to use another tag instead.

Running this
exiftool -time:all -s '/media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0551.JPG'

Gave me this
FileModifyDate                  : 2017:08:05 06:12:20-05:00
FileAccessDate                  : 2017:08:05 07:33:03-05:00
FileInodeChangeDate             : 2017:08:05 06:12:20-05:00


I ran a few of the files and they all have similar information.  These files were created in mid June, July, and August.  Not sure why information was not recorded on wife's iphone.  Would referencing information from this thread (https://exiftool.org/forum/index.php?topic=5912.0%5B") fix the 89 files that have no writable tags?  Specifically, adding something like this to the command in case "-filename<CreateDate" creates the No writable tags set from issue?

exiftool -timecreated<datetimeoriginal -datecreated<datetimeoriginal file

Thanks for any advice or help!


Title: Re: Ubuntu Automation
Post by: Hayo Baan on August 06, 2017, 03:24:09 AM
It looks like all/most metadata of those files was stripped. This could happen if the photos were edited. In that case your use of datetime original won't help either. You either have to use e.g. FileModifyDate or use some other means to determine the correct date.

Using a non existing tag is what causes the exiftool error (and yes, the error message is kinda misleading in this case since it's actually a tag that is being read that is missing)

Edit: correction
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 06, 2017, 08:58:12 AM
Quote from: Hayo Baan on August 06, 2017, 03:24:09 AM
It looks like all/most metadata of those files was stripped. This could happen if the photos were edited. In that case your use of datetime original won't help either. You either have to use e.g. FileModifyDate or use some other means to determine the correct date.

Using a non existing tag is what causes the exiftool tag (and yes, the error message is kinda misleading in this case since it's actually a tag that is being read that is missing)

I think I found the culprit.  It looks like a combination of Photosync (IOS app) and the social media apps my wife is using.  Photosync and the social media apps appear to be modifying/deleting/or not creating exif data.  I will need to test further and possibly contact the developer to see why exif data is being stripped during modification/transfer.   

One final question.  I would like to create a directory structure of year/month/date for my files.  I'm currently using %Y/%B/%d, which works perfect.  Is there an argument that can create a directory structure like this:
2017
01 - January
(Then folders by date as needed 01...31)
02 - February
03 - March
...12 - December

Title: Re: Ubuntu Automation
Post by: Hayo Baan on August 06, 2017, 11:48:40 AM
With some perl trickery the directory structure you're after can be achieved:

-fileName'<${datetimeoriginal;
my @months=qw(January February March April May June July August September October November December);
my $months=join("|", @months);
my %monthNr; for (my $i=0; $i<@months; ++$i) { $monthNr{$months[$i]} = $i+1; }
s/($months)/sprintf("%02d",$monthNr{$1}) . " - $1"/e;}' -d "%Y/%B/%d"


Note: If you use -testName instead of -fileName you can see the resulting filename without actually moving the file. Very helpful :)
Title: Re: Ubuntu Automation
Post by: Phil Harvey on August 06, 2017, 03:30:42 PM
I don't have much time to think about what Hayo was trying to do, so I may be off base here, but I thought that creating the directory structure you want should be simple:

exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

See here (https://exiftool.org/TagNames/QuickTime.html) for more information.

- Phil
Title: Re: Ubuntu Automation
Post by: Hayo Baan on August 07, 2017, 01:46:05 AM
Quote from: Phil Harvey on August 06, 2017, 03:30:42 PM
exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

Doh ::) I couldn't find the code for the month number in the manual (how I overlooked that in the man page, I don't know...) so I made up this piece of perl trickery. But %m should do the trick too ;D
Title: Re: Ubuntu Automation
Post by: polayer2015 on August 07, 2017, 08:16:36 AM
Quote from: Phil Harvey on August 06, 2017, 03:30:42 PM
I don't have much time to think about what Hayo was trying to do, so I may be off base here, but I thought that creating the directory structure you want should be simple:

exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

See here (https://exiftool.org/TagNames/QuickTime.html) for more information.

- Phil

Thank you Phil and Hayo!  Your support has been amazing and I'm finally able to organize my digital life!  In case anyone wants to know, here is the code I tested.

exiftool -r -d "/home/jim/Desktop/Test%Y/%m - %B/%d/%Y%m%d%%-c.%%e" "-filename<CreateDate" "/media/Multimedia/My Media/My Pictures/Need to Sort"

This renames my JPG's/Mov's and places them into folders by year/month/date.  Numbers are added to the months to create logical ordering inside the year folder.  Thanks once again!   :)