I have joined the forum to try and find a way to store the depth underwater pictures are taken in their exif files, extracting the data from a dive profile which contains the date, time and offset from that time and the depth at that point in time, as a CSV file.
I have tried searching the site but had no joy. Can anyone point me in the right direction ?
Clarification, do the files already contain the depth info or are you looking to add that info from another source?
If the information is already in the images and you want to extract it to a CSV file, then run the command in FAQ #3 (https://exiftool.org/faq.html#Q3) on one of the files. Look at the output and pick the tag names that you want to extract. Then your command would be something like
exiftool -CSV -TAG1 -TAG2 -TAG3 /path/to/files/ >path/to/Output.csv
SubSecDateTimeOriginal would probably be a good choice for the time stamp as that tag contains the date, time, and time zone if it exists in the file.
See the -csv option (https://exiftool.org/exiftool_pod.html#csv-CSVFILE) and FAQ #12 (https://exiftool.org/faq.html#Q12).
To be clear, the information is in the dive profile which is in the form of a CSV file containing the date and time the dive started, the offset from that time and the depth at that point in time. I want to extract the depth from that and add it to the exif of a picture taken at that time, possibly in the altitude field. Similar to programs that geotag photographs from a GPX track, but in this case the vertical possition.
I really can't see a way to do this with exiftool. I can only read CSV files in a certain format and can't perform any operations like adjusting the start time by the offset. I'd say it would require some scripting to accomplish.
I do this by extracting the depth from a Suunto dive computer log and matching it against the date set by the camera. (There are problems with the Suunto logs which are probably irrelevant to you).
There is an exif tag now for depth, waterdepth.
I haven't been diving since 2019 so have basically forgotten how my scripts do it but I'll get back to you.
My scripts read sml or xml files from suunto so not directly usuable for csv. And most of it seems to be related to the time inaccuracy on the suunto device and coping with timezones.
If you post a copy of one of your csv depth files, I'll see if I can come up with something without all the suunto mess.
How do you deal with time zones? Are both devices using the same local time for instance?
Thank you for your interest.
I'm actually using a Suunto Vyper but the software I use to download it to will output the profile without all the Suunto mess. Time zones are not a problem as all my diving is UK and camera and computer are kept in step. The main problem, as I see it, is that the profile lists the depth in relation to the time of the start of the dive, while exif uses the absolute time, though it is probably simple to add the start time and offset together within the script, compare it with the cameras time and then write the depth at that point to the exif data. Simple that is, if you can easily write scripts!
I don't see how to attach files to posts but the first part of csv file should show you what is in it:-
"dive number","date","time","sample time","sample depth","sample temperature","sample pressure"
"785","2023-06-28","10:28:00","0:00","0.0",,
"785","2023-06-28","10:28:00","0:20","7.01",,
"785","2023-06-28","10:28:00","0:40","8.839",,
"785","2023-06-28","10:28:00","1:00","9.754",,
"785","2023-06-28","10:28:00","1:20","10.668",,
"785","2023-06-28","10:28:00","1:40","10.973",,
and so on
Quote from: richardy on August 28, 2023, 06:02:50 AMI don't see how to attach files to posts
You can't using the "Quick Reply" at the bottom of a thread but if you hit the "Reply" or "Quote" button or if you hit "Preview" before posting, just below the text box will be "Attachments and other options"
Quote from: richardy on August 28, 2023, 06:02:50 AMThank you for your interest.
I'm actually using a Suunto Vyper but the software I use to download it to will output the profile without all the Suunto mess...
I've taken a copy of your csv data. You don't need to send a file.
But I think you will have the suunto mess.
Mess 1. Usually, when you set the date and time on a device such as a camera, although the seconds are not displayed, when you press "set", the seconds are set to zero. This does not happen on the vyper (at least, not on mine) so there can be up to 30 seconds error. Or is it 59 seconds? I've forgotten the details.
Mess 2. Look at the start time stamp in your csv. The seconds are zero. I found it is always zero in every file I've looked at. So that introduces another up to 30 seconds error. Or maybe 59 seconds.
These two messes don't sound too bad except that in a dive from surface to 30 metres, you can get an error of a couple of metres when matched against the camera time.
I actually go down to the bottom (or whatever), take a photo and set a mark in the suunto log and then work out the time error on each dive using the marker photos. It's a bit of a fiddly process.
But I'll have a go at doing something simple first with your csv, just to match the existing times together. I've used perl to parse a csv before.
Quote from: Alan Clifford on August 28, 2023, 03:49:56 PMI actually go down to the bottom (or whatever), take a photo and set a mark in the suunto log and then work out the time error on each dive using the marker photos. It's a bit of a fiddly process.
Looking at the adjustment files I create, I've had up to a 48 second discrepancy between the mark time on the Vyper and the photo time on devices that are supposed to have been set to the same time.
After looking through my logs, there is actually another time error. My vyper records a depth every 10 seconds. But the mark I always create appears to be on a 10 second as well, whilst you would expect it to be somewhere in between the 10 second depth records.
Sorry for the delay, we have had family staying! Thanks so much for all your comments. I don't think they will actually pose a great problem to me. The main purpose I will put the information to, is for splitting the pictures into habitats on our survey forms, for example, going down through kelp you reach a line when is disappears as the light levels become too low to support it, this results in a different habitat recoded separately. The demarcation is not precise so some error is acceptable. Also, we move slowly as there is a lot of detail to record, so time errors of a minute or so will probably not make a great difference to where we are. If you are at all interested in what we are doing have a look at https://youtu.be/NGzX-eZR-pM
I had a look at this this week and have written a script in perl if you are interested to try it on some test photos. It doesn't update the photos unless you specifically ask it to and will just display what it would do. I've got a Mac. I can try it on Linux but don't have access to Microsoft.
I gave it two photos and two .csv files and asked it give a more verbose output.
0 ../depthfromcsv/w300_1446.jpg 1688037385
0:5 Before time = 1687948180, depth = 10.973
1:4 Before time = 1688037380, depth = 10.668
1:5 After time = 1688037400, depth = 10.973
0 Depth found for 1688037385, depth = 10.744 in ../depthfromcsv/w300_1446.jpg
Not updating photo ../depthfromcsv/w300_1446.jpg with depth 10.744
--------------------------------------
1 ../depthfromcsv/w300_1453.jpg 1687948090
0:0 Before time = 1687948080, depth = 0.0
0:1 After time = 1687948100, depth = 7.01
1 Depth found for 1687948090, depth = 3.505 in ../depthfromcsv/w300_1453.jpg
Not updating photo ../depthfromcsv/w300_1453.jpg with depth 3.505
--------------------------------------
I'm afraid this is where I have to admit to one of my many weaknesses. It is a very long while since I used DOS and apart from a brief play with FORTH I have used Windows with only rare use of the Command Line. I even use ExifTool via the excellent ExifToolGUI v5.16, so I'm a little stumped when faced with scripts. The output from your script certainly seems to be what I'm looking for. I imagine I will need to download and install something like 'Perl Express' to run the script. However, I am very happy to experiment :-) HOWEVER there seems to be a problem I had not realised before. The positions of the images (lat & long) would appear to be held in my Lightroom catalogue or an XMP sidecar file, not directly into the exif data, I assume it is done this way to avoid corruption of the RAW file when changing it. It would seem sensible to follow the same process, but I can see things getting too complex to run batches, and I think a simple script to use individually when there is a specific need would be the way to go, and this would appear to be what you have already done. So if you could let me have a copy of the script and a rough idea of how I should run it, I will have a play and let you know how I get on. Thank you so much for spending your time pointing me in the right direction :-)
The script at the moment uses exif:datetimeoriginal from the photo and the csv file for depth. It creates an exif tag, exif:waterdepth in the photo.
You can't put an exif tag in an xmp file. Looking at the documentation for tags at https://exiftool.org/TagNames/XMP.html, I can see that there is an xmp tag that could be suitable XMP-exifEX:waterdepth. But I'd need to know how the name of the photo file and the xmp side car file are related and whether they are in the same directory or where. And any other software you are using might not read that tag.
So I'd guess the way to proceed is to try it one step at a time. Use a copy of your photo and related xmp file.
exiftool filename.xmp XMP-exifEX:waterdepth=20
where filename is the name of the xmp file related to the photo and look at the result in you viewing software.
I wouldn't have a clue what to do with data held in proprietary lightroom files.
Edit:
exiftool filename.xmp xmp:waterdepth=20
is sufficient
Thanks. I will have a play when I find a spare minute. I seem to have less spare time in retirement than I did when I had a job!!
I apologise for the delay, but I have finally tried this out. I'm afraid your syntax did not work for me with three types of file (ORF JPG ans XMP) available in the same folder:-
C:\Program Files (x86)\Metadata++>exiftool PB071875.xmp xmp:waterdepth=20
======== PB071875.xmp
ExifTool Version Number : 12.09
File Name : PB071875.xmp
Directory : .
File Size : 520 bytes
File Modification Date/Time : 2022:11:21 10:32:20+00:00
File Access Date/Time : 2023:10:01 14:12:06+01:00
File Creation Date/Time : 2022:11:21 10:32:18+00:00
File Permissions : rw-rw-rw-
File Type : XMP
File Type Extension : xmp
MIME Type : application/rdf+xml
XMP Toolkit : XMP Core 5.6.0
Sidecar For Extension : JPG
Metadata Date : 2022:11:21 10:32:18Z
Label : Yellow
Error: File not found - xmp:waterdepth=20
1 image files read
1 files could not be read
However I found this did work:-
C:\Program Files (x86)\Metadata++>exiftool -waterdepth=20 PB071875.xmp
1 image files updated
C:\Program Files (x86)\Metadata++>
If I specified xmp it added a value of 20 to waterdepth in the xmp file.
If I specified jpg it added a value of 20 to waterdepth within the jpg file
If I specified ORF (my RAW file) it added a value of 20 to waterdepth within the ORF file
In all cases a backup was created, so potential corruption is avoided. I carried out all commands in the same folder to avoid getting paths mixed up, but the basic principle works and I can easily choose where to write the value to see which works best in future workflows. The next step seems to be to work out how to connect your perl script to this to provide the value to be written to whichever file is chosen.
Thanks again for your time :-)
Sorry, I left out the - before xmp: :(
There are another decisions to be made.
In jpg files, you can use exif tags and there is an exif:waterdepth. Which you choose to use really depends on if you are using other software to view it and does that software see the value.
For raw files, do you want to update the raw file or an xmp sidecar file. If the latter, is the sidecar in the same directory as the raw file or in a subdirectory. Does the sidecar file have a naming scheme or is it just same_name_as_raw_file.xmp.
A script can be told to make these decisions based on the type of file but it needs to know the rules.
As is so often the problem, at the outset of a project like this the extent of its far reaching implications are not fully understood! As progress is made the extent of these becomes more obvious and decisions need to be made regarding priorities. The project can be split into three sections. Calculation of the depth, storage of the calculated depth and finally retrieval of the data. You seem to have solved the calculation successfully and the inevitable errors due to the frequency of data collection are acceptable.
The storage can be done, but has several decisions that need making. The first is where to put it, which partly depends on the final 'how to access it' problem, in that that has an effect on where is practical to store them. It is the 'how' that is more of a problem. Most of my pictures have details imported into the Lightroom catalogue when they are added, however a few, edited elsewhere before importing have XMP files. It would be more efficient to add the depth value to the RAW file itself, but this introduces the risk of corruption as the general principle of RAW files is that they are not changed. ExifTool gets round this by creating backups but, unless these are removed once the new file is shown to be sound, storage space doubles.
Retrieval of the data depends on what is being used to look for it. Lightroom would be simplest as that is where they already are, it also has options to read EXIF data, but not all of it. Probably the most obvious that can be read by Lightroom is altitude (waterdepth does not appear to be available). A simple positive value would be fine if I just referred to it in my head as depth rather than hight.
It seems the process of writing the data is the biggest problem as automated error checking seems very hard https://photo.stackexchange.com/questions/46919/is-there-a-tool-to-check-the-file-integrity-of-a-series-of-images so it seems the use of XMP sidecar files would be the way to move forward, but some means of creating one if it is not there would be needed. I suspect that afterwards the information might even be able to be incorporated into the catalogue and the XMP removed again, there is a 'read metadata from file' command, but that is a step too far at this stage, and in any case, they are only small files compared to a duplicate image!
I am off to Scotland for a week of diving next week so I will leave exploring these questions further until after I return, as I currently need to prepare.
What a week!! Part of the group got stuck for 2 nights by floods and landslips on the way, and then Covid hit. Only 2 of us dived every day (I was one of them). Then a road accident (not involving them) caused delays getting home. However we are all home and gear sorted. The next 3 days will be taken up visiting my son but I will then be free to spend more time on this project.
I'm afraid this project seems to have gone very cold. However, I think I have learnt enough to at least be able to apply the principle to individual pictures, which is probably all it will really be needed for, though the initial idea of running batches through would be even better. However to do anything myself I need to have the Perl script in #11 if that would be at all possible, as my newly gained knowledge does not stretch to writing those myself yet.
Sorry I seemed to have missed this. I haven't been looking at the forum for some time.