I've been trying for a long time to create a universal RAW-file naming algorithm.
All offers found on the Internet from various photographers I consider defective.
Their naming systems are based on subjective preferences and do not take diversity of shooting processes.
Naming system must be based on hard constant parameters in metadata of particular file.
Basic requirements for file names:
1. Ability to fully restore file name if it was damaged.
Name will be associated with many processes and it should never change.
This is possible if all the data needed for naming will always be with the file. This data is metadata created by camera at shooting.
2. File name must be unique in the world.
Unfortunately, camera manufacturers don't think about it.
I checked the RAW-files of several professional popular cameras and did not find standalone data for the unique name.
As a result, I understood that there is not the same way for all cameras.
But the general principle of name creating exists.
The file name can be made unique using combination of:
date;
time;
camera model;
internal serial number of camera;
name of the RAW-file at birth.
Unfortunately, I have found that manufacturers do not write file name at birth (such as IMG_236) to metadata.
This is important for file differences and chronology within one second.
Also, not all cameras store a serial number or internal serial number.
The file name becomes long, but this is acceptable if you do not make a deep hierarchy of folders with long names.
Ideally, it would be good if when you receive a file from the camera you immediately write a new unique name into its metadata forever and restore it when the file name is damaged.
Now the result in theory looks like this:
2019/
20191021/
20191021_145325_CanonEOS5DMarkIV_ZA2562417_IMG_236.cr2
or without camera name
20191021_145325_ZA2562417_IMG_236.cr2
or with version
20191021_145325_ZA2562417_IMG_236_v02.jpg
You also need to remove spaces or other characters when name creating:
...Canon EOS 5D MarkIV... -> ...CanonEOS5DMarkIV...
I am interested in your thoughts on this matter.
Since the very beginning of my digital photography I have adapted a simple, but efficient naming convention, which I only needed to extend once (to cover case b below). Like you I wanted: file names to always be unique and derived from metadata. I also liked to be able to have the files sort based on time of shooting and I wanted to quickly see when the image was taken as well.
I ended up with the date in sortable order (YYYYMMDD) and time which, because you can shoot multiple images in a second, includes the subseconds (HHMMSSss), date and time separated by an underscore. This is quite unique, but not always:
a) Sometimes subseconds is not precise enough (e.g. the Nikon D4 only records 1/10 seconds, resulting every now an then with the same subseconds in a series of shots)
b) When shooting with multiple cameras/people it can happen that two shots have the exact same time, up to the subsecond)
I solved this as follows:
a) I add 1/100 to the subsecond in case there are two (or more, though I've never seen that happen) files of the same camera with exactly the same subsecond.
b) I add the abbreviated model name to the filename (e.g. _D500 for a Nikon D500, _5Dii for a Canon 5D mark II, i4s for an iPhone 4s, etc.).
True, this might not always be unique if I would ever use two versions of the same camera at the same time (but that could be solved by adding another distinguishing mark, derived e.g. from the serial number).
Notes:
- Times recorded this way are always local time. This has its own problems (travel, lost hour during daylight savings changeover), but it is the "clearest" and most informative for the user. And anyway local time is the only thing recorded by all cameras (if you're lucky there's a timezone, but not always).
- In above case of a) the subsecond change gets reflected in the metadata.
If you think very seriously about the naming of photo files (or any else) it turns out to be a deep philosophical question.
In reality, everything is unique itself and marking itself by itself.
But in practice, we need some kind of conditional naming system, which can not be without the area of agreements.
Even your and my systems (by the way, they are similar) are not universal on a global scale.
For example, we use the date and time only for our own convenience.
Can you give a complete example of your organization of files and folders?
Quote from: andreikorzhyts on July 15, 2019, 12:20:59 PM
If you think very seriously about the naming of photo files (or any else) it turns out to be a deep philosophical question.
In reality, everything is unique itself and marking itself by itself.
But in practice, we need some kind of conditional naming system, which can not be without the area of agreements.
Even your and my systems (by the way, they are similar) are not universal on a global scale.
Sure, not globally universal in that there could still be multiple photographs world-wide that have identical names. Going to UTC times would help at sorting them world-wide, but we'd still need something to make all – globally – unique, full serial number could be a potential tag to use (but not all cameras have that stored in a readable fashion). But hey, I don't care about "globally" unique, I care about "my" unique.
Quote from: andreikorzhyts on July 15, 2019, 12:20:59 PM
For example, we use the date and time only for our own convenience.
Yes, but what else should the naming convention support? I don't do this for someone else, only for myself. If need arises, (a selection of) my images can always be renamed to suit some other naming convention, e.g. for a client.
Quote from: andreikorzhyts on July 15, 2019, 12:20:59 PM
Can you give a complete example of your organization of files and folders?
All by images are set up like this
Personal pictures go in a folder by year, and then in a folder by "event" which is always named
YYYYMMDD description[, period in English], where period in English is optional (e.g. "generic" photos don't get that, they also get MMYY as 0000, e.g. most years have a folder called "YYYY0000 Misc"). Sometimes (e.g. on larger trips with visits to multiple locations), the "event" folder is again split into subfolders. These could be named freely e.g. "Day 03-05 Location", etc.).
My commissioned work goes into a folder which is subdivided into a folder for each client which itself is then again subdivided into "assignment", named similarly to "events" above.
Cheers,
Hayo
Apparently there have been no sha256 collisions yet so that could be a candidate for uniqueness
shasum -a 256 ahc_5510.nef
3b21a7f325090f1abf7044356e1de3e198802004d8d831ca25211e7726121c01 ahc_5510.nef
Thanks, Hayo.
YYYY0000 — I find this in a practical way for various non-standard cases, for example, with damaged dates and times.
Folder 00000000 will be in the root of the folder tree :).
But you use spaces in folder names. You have no difficulty with the commands in ExifTool?
Spaces are not recommended from the technical side.
I think it is correct to use only letters, numbers and underscores.
ExifTool has no problems with spaces in folder names, but these names need to be quoted or the spaces need to be escaped on the command line.
- Phil
QuoteApparently there have been no sha256 collisions yet so that could be a candidate for uniqueness
Yes, that's first thing about I was thinking a long time ago.
But SHA considers the full file with its metadata.
If you change the metadata, you must update the name.
If you do a SHA of the content only, it will be a conflict in the case of the overexposed file where all pixels 255.255.255 (I haven't checked, but it is logical).
But the overexposed files are not needed, so this conflict can be ignored.
Ok, next...
SHA as the name is not suitable for perception. We need the agreements I mentioned earlier. The best agreement is chronology.
In result (SHA from the content only):
20190523_124551_0d1776c13dc533df0fcb302de3faf56f722637a3_v2.cr2If we compare it with this,
20191021_145325_CanonEOS5DMarkIV_ZA2562417_IMG_236_v02.cr2we have lost perception and have not acquired shortness.
Still need to take into account the time zone.
20191021_145325_p0430_CanonEOS5DMarkIV_ZA2562417_IMG_236_v02.cr2
But not all cameras write it and it will have to add itself.
If you photographing on the orbit, it will be hell for you, several time zones in one session. :)
Quote from: Phil Harvey on July 16, 2019, 10:26:32 AM
ExifTool has no problems with spaces in folder names, but these names need to be quoted or the spaces need to be escaped on the command line.
Probably, I am too insure, but it's not just the ExifTool. You never know where your archive will be once, what kind of operating system it will be, what file system, how old it will be. etc. Therefore, I try not to use any symbols that were not in the dawn of the computer era. Only
a-Z0-9_.
Quote from: andreikorzhyts on July 16, 2019, 11:18:52 AM
Quote from: Phil Harvey on July 16, 2019, 10:26:32 AM
ExifTool has no problems with spaces in folder names, but these names need to be quoted or the spaces need to be escaped on the command line.
Probably, I am too insure, but it's not just the ExifTool. You never know where your archive will be once, what kind of operating system it will be, what file system, how old it will be. etc. Therefore, I try not to use any symbols that were not in the dawn of the computer era. Only a-Z0-9_.
Well, I just happen to like folder names that read well, hence the spaces (my image files themselves have no spaces).
Apart from DOS (though even there it was possible, I think), all operating systems I have ever used (including OS/2!) never had big problems with spaces in names, and all command-lines of those OSs have autocompletion which automatically takes care of (escaping) the spaces as well. So it's really quite seemless to work with spaces in files.
The only place I found spaces in filenames a no-go is when developing software with "make"; there spaces really trip you up. There are likely solutions for that too, but I have not bothered to look into it and simply keep my development in files/directories without spaces :D
QuoteWell, I just happen to like folder names that read well, hence the spaces...
To me on the contrary, underscores help to read better. Often the empty space between the graphemes of letters is difficult to distinguish from the space, the underscore monotonously indicates the division of words. If poor vision and small text ...
Quote from: andreikorzhyts on July 16, 2019, 10:40:16 AM
QuoteApparently there have been no sha256 collisions yet so that could be a candidate for uniqueness
If you do a SHA of the content only, it will be a conflict in the case of the overexposed file where all pixels 255.255.255 (I haven't checked, but it is logical).
But the overexposed files are not needed, so this conflict can be ignored.
I was only being half serious with the suggestion. However, regarding your point about the problem if all pixels are the same as in another photograph, then both photographs are actually the same photograph even if they were taken at different times and places.
Yes, I already wrote that this is a deep philosophical question.
This issue can be solved only by adopting restrictive agreements, for example, within the planet Earth (for date and time).
If the photos will be in another part of the Universe, you need to expand existing system or create an additional system.
################
I am currently coming to this base of naming system for photos (Windows)
This is short statement. There are a lot of nuances which are considered by this system and I don't write them, but I am ready to provide arguments for any remarks.
################
Reasons
It is necessary to create a relatively unique works name of the minimum length to avoid collisions in the archive with the maximum variety of shooting conditions.
Agregements
0. All works have correct and undamaged data.
1. Rules are not strict and are only basis, the system can be changed at your own risk if necessary.
2. Must be a staging folder for collect of works before their organization.
_\
3. Folder and file structure must be at the root of storage. This is to minimize the length of full name (Windows assigns maximum 260 characters for full name).
4. Use only
a-Z0-9_
symbols (only latin letters).
5. Names and hierarchy of folders are based on chronology of works.
STORAGE \ YYYY \ YYYYMMDD \ WORK
6. Names of works consist of their constant parameters sufficient for relative global uniqueness.
20190718 — date.
032000 — time.
f0300 — time zone (you may need to add it yourself). f — forward, b — backward.
CanonEOS5DMarkIV —model (this can be neglected, because the probability of matching serial numbers from different manufacturers is negligible).
ZA2561817 — internal serial number or serial number of camera (choose the shortest).
IMG_0024 — the first name of shot (you may need to add it yourself, but some cameras write something like that in metadata, example IMG_0024.cr2 and FileNumber:100-0024)
v02 — version of postprocessing (for non-raw files only).
Examples
Full compliance with the base of naming system
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_CanonEOS5DMarkIV_ZA2561817_IMG_0024.cr2
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_CanonEOS5DMarkIV_ZA2561817_IMG_0024.psd
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_CanonEOS5DMarkIV_ZA2561817_IMG_0024_v02.psd
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_CanonEOS5DMarkIV_ZA2561817_IMG_0024.jpg
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_CanonEOS5DMarkIV_ZA2561817_IMG_0024_v02.jpg
Individual options
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_ZA2561817_IMG_0024.cr2
D:\ 2019 \ 20190718 \ 20190718_032000_f0300_ZA2561817_IMG_0024_v02.jpg
D:\ 2019 \ 20190718_Minsk \ 20190718_032000_f0300_CanonEOS5DMarkIVZA2561817IMG0024.cr2
D:\ 2019 \ 20190718_Photography_workshop \ 20190718032000_f0300ZA2561817IMG0024.cr2
D:\ 2019 \ 20190718 \ 20190718_032000f0300_IMG0024.cr2
One of the most vulnerable variant
D:\ 2019 \ 20190718 \ 20190718_IMG0024.cr2
Interesting! Some remarks.
You will definitely have to add subseconds to the time to make that part more unique.
The serial number is good to use, but unless you think you will have shots from multiple versions of the same camera model, an abbreviated form of the model name would suffice and is much shorter.
I don't see why you need the original image file name in the name. It adds 8 characters which really don't tell you much (unless it is to fix the time problem I mentioned, but you can use the subseconds for that).
QuoteI don't see why you need the original image file name in the name. It adds 8 characters which really don't tell you much (unless it is to fix the time problem I mentioned, but you can use the subseconds for that).
I always keep the original out-of-camera files in case I (or my software) make a complete mess of the time adjustment and geotagging. Keeping the original image file name as part of the new name is then the most reliable route back to the original file if I ever need to redo my processing.
QuoteYou will definitely have to add subseconds to the time to make that part more unique.
I was thought about it. But the system should only consist of data that we can identify. Not all cameras write seconds and if the camera does not record seconds, we will not be able to restore them. For example, not all cameras write time zone, but we ourselves know what time zone it is and can add he.
QuoteThe serial number is good to use, but unless you think you will have shots from multiple versions of the same camera model, an abbreviated form of the model name would suffice and is much shorter.
The system takes into account the worst situation when shooting is conducted on several cameras synchronously and serially, and part of these cameras of the same model. If your situation isn't that complicated. you may not use the serial number (
1. Rules are not strict and are only basis, the system can be changed at your own risk if necessary.).
The system is initially made as wide as possible (but not excessive), so you do not have to finish it, but only simplify for yourself, if necessary.
QuoteI don't see why you need the original image file name in the name. It adds 8 characters which really don't tell you much (unless it is to fix the time problem I mentioned, but you can use the subseconds for that).
I already wrote, why cannot be use subseconds.
Remains there is only one parameter that can support the uniqueness of the file name — this is its first name, about uniquenes of which is taken care of by the camera itself.
In addition, you can save the file name in metadata, but uses only part of it.
IMG_0541 -> 0541
QuoteI always keep the original out-of-camera files in case I (or my software) make a complete mess of the time adjustment and geotagging. Keeping the original image file name as part of the new name is then the most reliable route back to the original file if I ever need to redo my processing.
Thanks, good argument. Force majeure circumstances currently have not been canceled.
Stumbled upon such a site. Maybe someone will come in handy.
dpbestflow.org (http://dpbestflow.org)
Thank you Andrei for raising this interesting topic.
Thank you also for the link to dpbestflow
The addons below are based on my experience with Nikon cameras.
* If your camera writes it, you can use the ShutterCount field as a complement, or even an alternative to subsectimeoriginal field. In practice I no longer include subsectimeoriginal.
* Since the beginning of my workflow I took precaution of, like tim_rylance, keeping the original file name as part of the new file name. More than once I used the "rename" command with a regexp, or a sed-based bash command line to rename files back. But it makes scripts a little more complicated if you want the forward rename operation to be idempotent (which means "be a no-op if works was already done"). exiftool syntax %-8f came in handy in that case.
* I prefer when the file name can be completely regenerated from the file content. But the original file (or even part of it) is not in metadata written by Nikon cameras.
I'm working on a workflow refinement where original file names are recorded associated with a hash of the content (for example with md5sum or sha*sum commands) in additional files. This way, the original file name is still preserved, but externally, thus does not consume any character in current name.
All in all, I now use this command:
exiftool -v '-FileName<${datetimeoriginal}_${shuttercount}.${filetypeextension}' -d %Y-%m-%d_%H.%M.%S somefiles
Which produces files like 2019-05-18_21.45.11_43664.nef
I would add camera serial number when needed.
Extras
When several cameras shoot the same event, it is still the norm that not all have clocks properly set. When making a unified stream of photos, the usual disorder happens.
My workflow refinement will include a program (I started it, it works perfectly but only covers the basic functionalities) that can automatically rename a bunch of files with proper timestamps based on hints. A good and relatively easy to obtain hint is to have all participants take a photograph of a common object that shows time, for example your watch. If not done the same day (for example, participants contact you later), you can ask them to *not* change their camera clock but take a photograph of their browser showing https://time.is/ and send it via e-mail.
Regards.
Thanks Stephane.
QuoteBut the original file (or even part of it) is not in metadata written by Nikon cameras.
You mean the file name?
Quote from: Andrei Korzhyts on August 02, 2019, 04:29:34 PM
Thanks Stephane.
QuoteBut the original file (or even part of it) is not in metadata written by Nikon cameras.
You mean the file name?
Yes, the file name. When original file name looks like DSC_1234.JPG or NEF/MOV. There's nothing like 1234 in the metadata. The shuttercount is there, but not correlated (mine has 5 digits). That said, the shuttercount in itself is better than the four-digit number if only because it is unique per device.
Here is a more complete command line that I may use:
exiftool -v '-FileName<${datetimeoriginal}_${Model;s/[^a-zA-Z0-9]/-/}_${SerialNumber}_${shuttercount}.${filetypeextension}' -d %Y-%m-%d_%H.%M.%S *.nef *.jpg
Again, the ";" syntax in exiftool comes in handy to adjust the camera model name and flatten it to a valid filename.
I have been thinking about the problem of naming photo files for a long time and a lot. My modest business depends on it. My desire for order forces me to create this theory.
Now it is called «ТЕОРИЯ И ПРАКТИКА ИМЕНОВАНИЯ RAW-ФАЙЛОВ ФОТОГРАФИЙ» (THEORY_AND_PRACTICE_OF_NAMING_RAW_FILES_OF_PHOTOS).
Here are my main conclusion I came to:
It is impossible to create one for all action plan for naming files.
I do not create a ready-made solution, but a universal algorithm that can be adapted to create a solution for your needs.
This is due to the boundless inconsistency between the camera manufacturers and the forced agreements that reality dictates to us.
I do not consider the following candidates for parts of the name and I giving reasons:
1. Subseconds.
They do not add informational content to the file name;
they do not consolidate uniqueness, only increase it;
they can be defined differently by the manufacturer (different fractions);
they are rarely written to the metadata.
2. Hash from content of file.
He do not add informational content to the file name and consumes space in the file name.
Option with photographing clocks interesting, clever and good, but only in controlled conditions and only with an accuracy of one second.
However, this is good practical advice for many cases.
Hi,
I have just joined the forum, so thats my excuse for posting to an older topic. I have just spent this Christmas holiday writing a utility application to rename and copy camera raw files. I'm not as concerned about creating a file name that is unique in the universe but I want them to be unique to me and most likely unique in the world.
I have adopted the following structure : SQL date _ Shoot time _ Shoot Name _ KeyWord _ Original Camera Name . Extension
For example a raw file named p1234567.rw2 is renamed 20191217_164520_Didcot_Joe_McNally_p1234567.rw2
This works for me because I only use a single camera on a shoot. If I were using more than one camera then I would think about adding an additional discriminator. This could be as simple as "Cam1" or "Cam2". No spaces or other harmful characters allowed.
I believe that "uniqueness" is all a question of odds versus complexity. My solution works for me and produces filenames that are unlikely to be duplicated by other photographers even if they were on the same shoot. If I were shooting from orbit I would add the mission name e.g. _Apollo_11_ or STS45, not that I see it being a problem for many people.
just my two pence
best wishes
Skids
My two cents: If you are just looking for something unique, then you could use the ExifTool NewGUID tag (https://exiftool.org/TagNames/Extra.html). But this tag is based on the current date/time, and not the date/time of the image.
- Phil
I'm going for a long time to this and now this is possible for using.
Порядок именования фотографий (ПИФ)
Naming order of photos (PIF)
pif.afoteris.com (http://pif.afoteris.com)