Ubuntu Automation

[polayer2015]

I would like to automate my photo processing on Ubuntu 16.04.  My photos are upload via photosync on IOS to a qnap network folder.  Folder has been permanently mapped and is accessible by exiftool.  From there I would like to automate the renaming of each photo into a folder structure of Year/Month/Date.    I can do this separately via CLI, but would like to automate the process.  So far I have:

This renames the files based on the parameters I want to use.

Code Select Expand

exiftool -d %Y%m%d_%H%M%%-c.%%e "-filename<CreateDate" DIR

This places my photos into a folder structure of Year/Month/Date located at /home/myusraccount/2017. 

Code Select Expand

exiftool -d %Y/%B/%d "-directory<datetimeoriginal" *.jpg

I would like to merge the first two steps and move the files into my master photo collection located at /media/Multimedia/My Pictures.  This folder also uses the  Year/Month/Date structure to store photos.  I would like for this to take place automatically once new photos are uploaded from photosync via my iphone or new images are added from my DSLR.  Thanks in advance for any help with my setup.

[Phil Harvey]

I'm not clear.  You want to merge these 2 commands?  If you can use CreateDate for both, this would work:

Code Select Expand

exiftool -d %Y/%B/%d/%Y%m%d_%H%M%%-c.%%e "-filename<CreateDate" DIR

To do this automatically you could add this as a cron job, but whatever copies the images into the source directory must first copy to a temporary file then rename so that exiftool doesn't pick up partially-written files.

- Phil

[polayer2015]

Thanks for the quick reply.  This worked fantastic!  How can I set the destination folder that the files are moved to.  With this cli, the files are moved to /home/myusraccount/2017.  I would like the final destination to be /media/Multimedia/My Media/My Pictures.  This would require files to be copied into preexisting folders with the directory structure of year/month/date.  Thanks in advance.

[Phil Harvey]

Since we specified a relative directory, the destination directory will be relative to the current working directory, so you could either first "cd" to "/media/Multimedia/My Media/My Pictures", or alternatively specify an absolute directory like this:

Code Select Expand

exiftool -d "/media/Multimedia/My Media/My Pictures/%Y/%B/%d/%Y%m%d_%H%M%%-c.%%e" "-filename<CreateDate" DIR

- Phil

[polayer2015]

Thanks!  Once again this worked perfectly.  Just reviewed my work flow and had a few additional questions.  My wife and I will use the IOS app photosync to upload photos/videos to a network folder (/media/Multimedia/My Media/My Pictures/Need to sort).  I wanted see if exiftool could scan this folder recursively (/media/Multimedia/My Media/My Pictures/Need to sort/My Phone and /media/Multimedia/My Media/My Pictures/Need to sort/Wifes Phone).  The next question is how to handle photos and videos.  Photos are stored /media/Multimedia/My Media/My Pictures and videos are stored in /media/Multimedia/My Media/My Pictures/My Videos in a Year/Month/Day structure.  Sorry for complicating the process, but this would make my digital life so much easier if I could automate these steps.

[Phil Harvey]

Again, I'm not clear on what you are asking, but just add the -r option to process the source directory recursively.

- Phil

[polayer2015]

Thanks for the help so far!

Ran this:

Code Select Expand

exiftool -r -d "/media/Multimedia/My Media/My Pictures/%Y/%B/%d/%Y%m%d%%-c.%%e" "-filename<CreateDate" "/media/Multimedia/My Media/My Pictures/Need to Sort"

Got this:

Code Select Expand

Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0551.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0441.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0440.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0439.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0437.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0356.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0309.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0308.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0306.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0297.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0273.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0255.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0252.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0196.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0194.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0159.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0160.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0156.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0155.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0146.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0133.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0562.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0476.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0438.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0484.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0550.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0372.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0370.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0364.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0363.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0342.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0331.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0311.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0314.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0310.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0307.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0287.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0254.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0256.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0239.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0222.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0215.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0211.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0192.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0189.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0182.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0166.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0165.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0157.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0148.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0145.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0315.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0369.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0573.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0362.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0359.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0324.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0242.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0290.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0253.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0361.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0143.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0371.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0487.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0225.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0483.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0330.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0303.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0296.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0514.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0320.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0300.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0367.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0243.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0323.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0542.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0301.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0227.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0150.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0144.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0135.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0100.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0007.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0006.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0564.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0137.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0283.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0183.JPG
Warning: No writable tags set from /media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0184.JPG
    5 directories scanned
   37 image files updated
   89 image files unchanged



Seems like I have 89 images that are in limbo. Files are no longer in directory and were not moved to final destination.  Not sure how to address the writable tag issue.

To clarify my second request, I would like to separate .JPG's  and .MOV's .  Photosync (IOS App) uploads my photos and videos into a permanently mounted network folder located at /media/Multimedia/My Media/My Pictures/Need to sort/My Phone and /media/Multimedia/My Media/My Pictures/Need to sort/Wifes Phone.  From there I would like exiftool to recursively scan the folders, rename, and move to specific folders. Photos are stored in /media/Multimedia/My Media/My Pictures and videos are stored at /media/Multimedia/My Media/My Pictures/My Videos.  Both locations have year/month/date directory structures.  The goal is to run a scheduled exiftool command that can distinguish between .JPG/.MOV and place the files in the appropriate directory.  Hopefully this clarifies my request.

[Hayo Baan]

Regarding the non-writable tag issue, check to see if the images indeed have a CreateDate. They probably don't so you'll need to use another tag instead.

To select only JPG files add -ext JPG, similarly for MOV. That should easily solve the split of images and videos. To split between your phone and your wife's is a bit more of a challenge. But is doable too as long as you have good way to make the distinction. Is there perhaps a unique tag of part of the filename?

[polayer2015]

Regarding the non-writable tag issue, check to see if the images indeed have a CreateDate. They probably don't so you'll need to use another tag instead.

Running this

Code Select Expand

exiftool -time:all -s '/media/Multimedia/My Media/My Pictures/Need to Sort/iPhone/Camera Roll/IMG_0551.JPG'

Gave me this

Code Select Expand

FileModifyDate                  : 2017:08:05 06:12:20-05:00
FileAccessDate                  : 2017:08:05 07:33:03-05:00
FileInodeChangeDate             : 2017:08:05 06:12:20-05:00

I ran a few of the files and they all have similar information.  These files were created in mid June, July, and August.  Not sure why information was not recorded on wife's iphone.  Would referencing information from this thread fix the 89 files that have no writable tags?  Specifically, adding something like this to the command in case "-filename<CreateDate" creates the No writable tags set from issue?

Code Select Expand

exiftool -timecreated<datetimeoriginal -datecreated<datetimeoriginal file

Thanks for any advice or help!

[Hayo Baan]

It looks like all/most metadata of those files was stripped. This could happen if the photos were edited. In that case your use of datetime original won't help either. You either have to use e.g. FileModifyDate or use some other means to determine the correct date.

Using a non existing tag is what causes the exiftool error (and yes, the error message is kinda misleading in this case since it's actually a tag that is being read that is missing)

Edit: correction

[polayer2015]

It looks like all/most metadata of those files was stripped. This could happen if the photos were edited. In that case your use of datetime original won't help either. You either have to use e.g. FileModifyDate or use some other means to determine the correct date.

Using a non existing tag is what causes the exiftool tag (and yes, the error message is kinda misleading in this case since it's actually a tag that is being read that is missing)

I think I found the culprit.  It looks like a combination of Photosync (IOS app) and the social media apps my wife is using.  Photosync and the social media apps appear to be modifying/deleting/or not creating exif data.  I will need to test further and possibly contact the developer to see why exif data is being stripped during modification/transfer.   

One final question.  I would like to create a directory structure of year/month/date for my files.  I'm currently using %Y/%B/%d, which works perfect.  Is there an argument that can create a directory structure like this:

Code Select Expand

2017
01 - January
(Then folders by date as needed 01...31)
02 - February
03 - March
...12 - December

[Hayo Baan]

With some perl trickery the directory structure you're after can be achieved:

Code Select Expand

-fileName'<${datetimeoriginal;
my @months=qw(January February March April May June July August September October November December);
my $months=join("|", @months);
my %monthNr; for (my $i=0; $i<@months; ++$i) { $monthNr{$months[$i]} = $i+1; }
s/($months)/sprintf("%02d",$monthNr{$1}) . " - $1"/e;}' -d "%Y/%B/%d"

Note: If you use -testName instead of -fileName you can see the resulting filename without actually moving the file. Very helpful

[Phil Harvey]

I don't have much time to think about what Hayo was trying to do, so I may be off base here, but I thought that creating the directory structure you want should be simple:

exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

See here for more information.

- Phil

[Hayo Baan]

exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

Doh I couldn't find the code for the month number in the manual (how I overlooked that in the man page, I don't know...) so I made up this piece of perl trickery. But %m should do the trick too

[polayer2015]

I don't have much time to think about what Hayo was trying to do, so I may be off base here, but I thought that creating the directory structure you want should be simple:

exiftool "-directory<datetimeoriginal" -d "%Y/%m - %B/%d" DIR

See here for more information.

- Phil

Thank you Phil and Hayo!  Your support has been amazing and I'm finally able to organize my digital life!  In case anyone wants to know, here is the code I tested.

Code Select Expand

exiftool -r -d "/home/jim/Desktop/Test%Y/%m - %B/%d/%Y%m%d%%-c.%%e" "-filename<CreateDate" "/media/Multimedia/My Media/My Pictures/Need to Sort"

This renames my JPG's/Mov's and places them into folders by year/month/date.  Numbers are added to the months to create logical ordering inside the year folder.  Thanks once again!