Main Menu

Windows metadata

Started by Phil Harvey, July 07, 2015, 07:07:50 AM

Previous topic - Next topic

Martin Z

#30
Oh god, it's even worse than I thought, lol! 🙈

In the process of writing a response back to you, I did 1-2 quick tests in EXIFtool, but rather than giving me info that I hoped would help me clarify my reply it ended up opening a whole 'nother can of worms! ;D  (stay tuned for another post from me within the next hour)

I will carry on writing my proper response to you, but I just had to pause and write this interim comment as it was just too crazy/funny/depressing not to, lol.

Martin Z

#31
Quote from: StarGeek on April 23, 2023, 01:17:34 PMThis is "it depends" and varies by your workflow and what programs you are using.  You don't want to update Description and not change Caption-Abstract if one of your programs is going to give priority to Caption-Abstract, as I believe this is what the MWG guide suggests.

Lol, so am not sure if it was deliberate or not but Description is probably actually a great example to dive into for a second as it shows how mixed-up and complicated metadata is (at least for me) and why I am finding it trickier than I expected to try and get my head round this...

Firstly, just to be clear* when you say Description, as you referenced Caption-Abstract (and thanks to your table above, I could cross-reference!) - I'm assuming that you're referring to... [what I, being more Windows/File property-centric, would generally refer to as]... Title? (please say yes! :)  )... and hoping to score max points - as this is an amalgamation of different tags into 1, this is a composite tag?**

* If the above sounds like I am nit-picking or being sarcastic, please note I this is not my intention at all and I am genuinely trying to understand the nuance and interaction amongst different properties better, and so just trying to disambiguate between the different properties otherwise I end-up confused and with a headache! [PS - Hope you like disambiguate, had to google that one! :)]

** If it's not a composite tag then just ignore that reference for now - we can pick composite tags later (if at all) but I don't want to lead things off at a different tangent.



Quote from: StarGeek on April 23, 2023, 01:17:34 PMIf you want to not deal with IPTC data, then you should probably remove it rather than just ignoring it.

So, to give you some context about my main interactions/editing of metadata as a whole (and highlight my current gaps/limits in knowledge)... So, it's not so much that I don't want to deal with IPTC data, it's more that I just don't get exposed to the data down to that level and so I genuinely could not tell you whether 99% of my images have IPTC tags or 0%.

PART 1: As a test...
  • I created a blank MP4 file
  • Via Windows/properties I set the Title to Title set from Windows property dialog
  • In many cases this is the max depth I'm aware of in terms of which properties are being stored, where, i.e. I'll set other properties and may import or manipulate those properties in Excel, AtomicParsley or some other tool, but I only know that data point as being the Title, I don't know if that's in a Microsoft tag, an XMP one, IPTC, all 3 or another tag entirely.

So, going full-circle - the reason for asking my questions initially and trying to understand whether "I should we be manually creating IPTC tags" was in essence trying to understand whether I should...
  • Carry on as I am, i.e. reading/writing tags at a "Windows"-level (and not worry whether the info is being stored in IPTC or XMP as I'm trusting EXIFtool (or A.N.other tool to manage that lower level storage for me)
  • Start setting IPTC tags explicitly so that for at least the scripts I write / metadata edits I make, I know for certain that they are updating/creating IPTC tags (in addition to others like XMP, etc)

PART 2: Following your comment, I thought I'd try to dig into this a bit deeper and clarify / educate myself (although this is what led to my 'worse than I thought' comment above :) )...
  • I used EXIFtool to do CSV export (using the default, i.e. "common" options)
  • The only column in the CSV containing Title set from Windows property dialog was Title
  • I know from another discussion we had (thanks!) that EXIFtool can include the tag groups with -G, so I did second CSV export
  • That CSV clarified however that the data is in...
  • NOT Microsoft, XMP, IPTC... but QuickTime:Title
  • Which isn't even on the list / one of the ones I was thinking of! ;D

Lol, like I say my hope was this would give me a better grasp on how IPTC might come into play (or at least show whether Windows was ignoring this in favour or say XMP for example), unfortunately in actuality this has probably muddied the water more than ever, lol.

StarGeek

Quote from: Martin Z on April 23, 2023, 04:44:54 PMFirstly, just to be clear* when you say Description, as you referenced Caption-Abstract (and thanks to your table above, I could cross-reference!) - I'm assuming that you're referring to... [what I, being more Windows/File property-centric, would generally refer to as]... Title? (please say yes! :)

No, I mean Description, as in the the Dublin Core XMP-dc:Description.  When I use the monspace colored font set with [tt][/tt] (see text just below the edit box), then I am listing the exact name of a tag.  When I use it otherwise, I will either give the context and probably capitalize it and put quotes on it, e.g. IPTC standard "Description", Windows property "Title", or just as a generic term where it will probably be lower cased.

The IPTC Description has two corresponding tags, IPTC:Caption-Abstract and XMP:Description.  The MWG standard also adds EXIF:ImageDescription.  Windows says I don't give a damn about standards and uses unrelated tags.

Quoteas this is an amalgamation of different tags into 1, this is a composite tag?**

Composite tags don't exist outside of exiftool.  Exiftool creates them on the fly based upon other tags in the file.  I will also use the monospaced font for Composite tags.

Quote[PS - Hope you like disambiguate, had to google that one! :)]

I used to heavily edit the wiki for an MMO, so I used it a lot.

Quoteit's more that I just don't get exposed to the data down to that level and so I genuinely could not tell you whether 99% of my images have IPTC tags or 0%.

This is why, unless you're really interested in the minutia of metadata, I always suggest using a Digital Asset Management (DAM) program of some sort (Adobe Bridge or Lightroom, DigiKam).  They provide a layer between the files and you to properly reconcile the data.  Most people don't really need to know where the data is located.

Side note, this unfortunately leads to people thinking that the file system timestamps are important, when it is the actual embedded data that is important.  This is what lead people to think that Google was deleting data from their images when they did a Google Takeout when the only issue was the loss of some file system data, which will always be lost when you upload a file to a website.  The unfortunate result of this is that people are going to lengths to copy the data back into the files, which is actually overwriting the original correct data with less accurate data.

QuoteI can carry on as I am in effect, i.e. reading/writing tags at a "Windows"-level (and not worry whether the info is being stored in IPTC

This is perfectly acceptable.  Windows will reconcile the data for you, albeit in it's own non-standard way.

QuoteXMP as I'm trusting EXIFtool (or A.N.other tool to manage that lower level storage for me)
Start setting IPTC tags explicitly so that for at least the scripts I write / metadata edits I make, I know for certain that they are updating/creating IPTC tags (in addition to others like XMP, etc)

As I said, unless you're actually interested in the low level details, use a good program that will take care of the details for you.

QuoteThat CSV clarified however that the data is in...[/color]
NOT Microsoft, XMP, IPTC... but QuickTime:Title
Which isn't even on the list / one of the ones I was thinking of! ;D

You jumped from image metadata, which is simply a universe of madness, into video data, which is a multiverse of madness.  QuickTime/video data has 3 subgroups and often the same tag name can appear in all 3 subgroups and in some cases, even multiple times in the same subgroup.  For example, Description can appear in the Keys group, the ItemList group (3 separate times!!!), and the UserData group.  Additionally, videos can hold all the regular XMP data, so there can be XMP-dc:Description.  To add to the fun, Adobe will only read the XMP data, but most other programs will read the Quicktime data.  And which tag those read can vary from program to program.

Obligatory xkcd
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Phil Harvey

Quote from: StarGeek on April 23, 2023, 06:39:02 PMYou jumped from image metadata, which is simply a universe of madness, into video data, which is a multiverse of madness.

I like it.  Sounds like a quote from some Marvel movie. ;)

But, oh so true!

- Phil
...where DIR is the name of a directory/folder containing the images.  On Mac/Linux/PowerShell, use single quotes (') instead of double quotes (") around arguments containing a dollar sign ($).

Martin Z

Quote from: StarGeek on April 23, 2023, 06:39:02 PMI mean Description, as in the the Dublin Core XMP-dc:Description.  When I use the monspace colored font set with (see text just below the edit box), then I am listing the exact name of a tag.

OK cool... Should I always assume that it's a XMP tag?
(Just thinking where there are multiple tags with the same name like Microsoft:Description and QuickTime:Description)



Quote from: StarGeek on April 23, 2023, 06:39:02 PMComposite tags don't exist outside of exiftool.  Exiftool creates them on the fly based upon other tags in the file.

Cool, thanks for the link and think that now clears it up in my head now... (FYI): I had seen in a few apps that Composite tags were listed separately to EXIF and seemed to imply it combined data from different groups - but reading the page again now (and with your extra info/context), I think I see my confusion... (To check): Composite tags DO combine data from different groups, incl. fields outside of EXIF, BUT the composite tags themselves exist within EXIF, so if you only loaded say XMP, Microsoft, Quicktime groups you would get no composites, but if you added EXIF as well, Composite tags would then be accessible, right?



Quote from: StarGeek on April 23, 2023, 06:39:02 PMI always suggest using a Digital Asset Management

Hmm, that's interesting - as in, I've worked with DAMs in a work setting but from what I've seen they've been pretty heavy duty (designed for things like product catalogues, or official company-wide media libraries, where there is a change-control approval process, etc)... I've never really thought of using one in a personal capacity.

That said, I'm not sure if it qualifies as a DAM - but my end goal is probably to manage the data in a SharePoint list (I work in SharePoint professionally, so am pretty familiar with it).



Quote from: StarGeek on April 23, 2023, 06:39:02 PMSide note, this unfortunately leads to people thinking that the file system timestamps are important, when it is the actual embedded data that is important.


Urgh, I would say don't even get me started, but you've done it now so it's too late! :-D

On the topic of timestamps (and metadata):
• One of my gripes is that most apps will update the 'modified' timestamp when you change the metadata. Yes for some instances this may be valid/useful, but it's really annoying when I say I delete some junk text from a title or add a keyword to an image and then Windows changes the modified date to today... NO, the image that was taken in 2019, hasn't been MODIFIED today(!).. urgh
• To make matters worse, this problem can be metadata-destructive, i.e. say the image didn't have the Date Taken date stored, and you were relying on the Date Modified being in 2019 to find the image. Trying to remove/clean-up the metadata in this case deleted/corrupted the data you were previously relying on
• Now, all my files are backed up with OneDrive...
- Which is great on one hand as in the case of the above, I can actually go and see a full time-stamped change history (and even restore the previous version if needed) and this has let me retain/recover metadata more than once!
- However it also creates some interesting scenarios (especially in the case of rolling back old files) as this can mean Windows shows a Modified date that is earlier than the Created Date... So apparently the file was modified before it even existed!? :-D
• Side note: It also frustrates me that PNGs and ICOs have virtually no metadata at all (I know PNGs are more complicated as they can store EXIF I believe, however support is limited at best so lets skip that for now), but anyway, I would like to put Metadata in EVERYTHING but sadly sometimes it just isn't an option.

[Side note: I was REALLY optimistic about "File Meta", which is a Windows app that lets you extend the File Properties for different types and so you CAN store keywords, title, description, etc for an ICO file... However the metadata is emulated (stored in a separate DB/file) so again, as I use OneDrive quite a lot to access files remotely and/or peace of mind in case my HDD died - I really would've preferred if this somehow hacked the metadata into the file's data itself somehow, but c'est la vie]



Quote from: StarGeek on April 23, 2023, 06:39:02 PMYou jumped from image metadata, which is simply a universe of madness, into video data, which is a multiverse of madness.

Lol, fair enough - I did realise after I posted that choosing an MP4 example probably didn't help but I didn't clock that at the time and in my defence it was just the sense of dismayed, thinking I will take the time and try and get to the bottom of this, I'll do some edits and then trace them through using EXIFtool which should give me a much clearer understanding of whether the data is being stored in EXIF, IPTC or XMP... So, eagerly awaiting to find out which of the THREE it was, having the result turn out to be (from my naïve perspective) a FOURTH option was just hilarious (and soul-destroying!) :-D

TLDR - Yes, metadata is absolute chaos! :-)

wywh

Quote from: Martin Z on April 26, 2023, 02:54:09 AMOne of my gripes is that most apps will update the 'modified' timestamp when you change the metadata. Yes for some instances this may be valid/useful, but it's really annoying when I say I delete some junk text from a title or add a keyword to an image and then Windows changes the modified date to today

A very common gripe in Photos.app forum is that moving images around often changes the file dates and people get mad when the original date is "lost". I wish Apple would provide a setting to preserve image and movie file dates, if possible.

I usually use the exiftool -P switch (Preserve file modification date/time) to prevent that and GraphicConverter does this by default.

BTW recently I wondered why a very tiny metadata change did not get backupped. The backup app uses file size and modification date to determine if a file should be updated on the destination. The app's author sounded a little irritated when I mentioned that exiftool -P switch, and said that is a flaw in the exiftool utility, but they call it a "feature" ;-)

The backup app has an option to check also md5 checksum but that makes the backup process much slower. I tested this and noticed that the md5 checksum is changed to another value with such exiftool's +1 second change in ExifIFD:DateTimeOriginal. I was somewhat surprised to find that if the md5 checksum is reverted back to the initial value if I the change ExifIFD:DateTimeOriginal back -1 second with exiftool.

I prefer to let exiftool or GraphicConverter to adjust image and movie file creation & modification dates the same as their internal metadata dates so it is easier to spot errors in dates. In macOS 13 those tools can readily set way older file dates than 1970 but the old dates (1900, for example) are usually reverted to something like 2036 after a reboot so the backup app sometimes re-clones such images although they are not really modified.

But anyway, that backup issue is so rare that I do not mind and use the default backup settings.

- Matti

StarGeek

Quote from: Martin Z on April 26, 2023, 02:54:09 AMOK cool... Should I always assume that it's a XMP tag?
(Just thinking where there are multiple tags with the same name like Microsoft:Description and QuickTime:Description)

It depends upon the context and the file type.  For the most part, yes, it would be the XMP:Description.  But for videos, it would also include the Quicktime:Description, specifically, the ItemList:Descripton.  Basically, whatever exiftool will write by default to that filetype.  IMO, Phil has done a fantastic job of picking the right group to write to for the various file types.

The Microsoft:Description tag isn't writable by exiftool (gah, 3 different versions of Microsoft:Description!) but basically I also won't be talking about anything that is marked with the "Avoid" flag.  Those would have to be specifically named anyway.


Quote(To check): Composite tags DO combine data from different groups, incl. fields outside of EXIF, BUT the composite tags themselves exist within EXIF, so if you only loaded say XMP, Microsoft, Quicktime groups you would get no composites, but if you added EXIF as well, Composite tags would then be accessible, right?

Composite tags are a group unto themselves.  And you would have to check the page to see what is combined.  In some cases, they will pull from specific sources.  For example Composite:DateTimeOriginal in an audio file such as MP3, will pull from four different specific ID3 group tags.  For an image, it will pull from three tags, DateTimeCreated, DateCreated, and TimeCreated.  Note that these do not have a group indicated.  Additionally, DateTimeCreated is itself a composite tag, created by combining IPTC:DateCreated, IPTC:TimeCreated.  So it can be created from specifically the IPTC:DateCreated and IPTC:TimeCreated, aka DateTimeCreated, or any other DateCreated, and TimeCreated tags, which could include XMP-photoshop:DateCreated, Microsoft:DateCreated, Red:DateCreated/TimeCreated, RIFF:DateCreated, etc, etc. (At least I think that's what will happen, I don't have any samples of these other tags and filetypes).

QuoteHmm, that's interesting - as in, I've worked with DAMs in a work setting but from what I've seen they've been pretty heavy duty (designed for things like product catalogues, or official company-wide media libraries, where there is a change-control approval process, etc)... I've never really thought of using one in a personal capacity.

They can also be called image management programs and there are plenty that are specifically designed for personal/photography business use.  The most well know is Lightroom.  IMO, the best FOSS option is DigiKam, though Darktable might be better if you want a DAM/editor in the same vein as LightRoom.  Picasa was another example.  I use IMatch, as it uses exiftool on the backend, has extremely powerful metadata options, and can index a huge amount of filetype beyond just images and videos.

QuoteOn the topic of timestamps (and metadata):
• One of my gripes is that most apps will update the 'modified' timestamp when you change the metadata. Yes for some instances this may be valid/useful, but it's really annoying when I say I delete some junk text from a title or add a keyword to an image and then Windows changes the modified date to today... NO, the image that was taken in 2019, hasn't been MODIFIED today(!).. urgh

But that isn't the purpose of the file system modify time... at least that's what my understanding is.  Going way back, it would be how to tell if a file needed to backed up.  The time stamp isn't there to indicate if the image has been modified, that's what the EXIF:ModifyDate is for.  It's there to incidate when the file itself has been modified.  Which it has been if you edit the metadata, indicating it needs to be updated.  Now a days, the often is an Archive attribute, at least on Windows, so depending upon the software, it might not be as important.

Quote• To make matters worse, this problem can be metadata-destructive, i.e. say the image didn't have the Date Taken date stored and you were relying on the Date Modified being in 2019 to find the image. Trying to remove/clean-up the metadata in this case deleted/corrupted the data you were previously relying on

This can vary by file source.  Photos and videos should have the embedded time stamp set.  Screenshots, from what I've seen, usually have the time stamp as part of the name.  Other social media sources will have any such data stripped.

I understand that figuring how to embed the time from the file system is beyond what most people are willing to invest the time in learning how to do, but if it's important, then it should be learned.  At the very least, if the file is properly backed up, you can always restore from backup to get the time right.

Quote• Now, all my files are backed up with OneDrive...

This is something that you might want to be careful about.  At some point, I read that Microsoft does not consider OneDrive to be a backup for images, that it will edit the metadata on files stored there.  Of course, I can't find the source any more, but it might be worth checking out.  Specifically, OneDrive would at the very least add XMP:LastKeywordIPTC and XMP:LastKeywordXMP, two Microsoft specific tags, to files uploaded that contained a IPTC:Keywords or XMP:Subject.  I can't follow up on this as I'm not willing to install OneDrive on my computer.

Quote• Side note: It also frustrates me that PNGs and ICOs have virtually no metadata at all (I know PNGs are more complicated as they can store EXIF I believe, however support is limited at best so lets skip that for now), but anyway, I would like to put Metadata in EVERYTHING but sadly sometimes it just isn't an option.

PNG files can support XMP and EXIF (EXIF used to be non-standard and IPTC is non-standard), but you are correct in that software support is limited, though it is growing.  I believe Google Photos has since added support for EXIF in PNGs.

Myself, my use of PNG files is limited, mostly for quick screenshots that I upload and don't keep a local copy.  Anything that I need to keep lossless is going to be saved as a tiff, as that has decades of metadata support. 

Quote[Side note: I was REALLY optimistic about "File Meta", which is a Windows app that lets you extend the File Properties for different types and so you CAN store keywords, title, description, etc for an ICO file... However the metadata is emulated (stored in a separate DB/file)

Hmmm... I faintly recall looking at it and I thought that it saved the data in the Windows Alternate Data Streams.  So that way the data would travel with the file.  At least, as long as it was on a NTFS drive.  Hard to tell now, as any search I do on "File Meta" always adds data to it.

This Voidtools forum post seems to confirm that.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Martin Z

Hi @StarGeek,

Apologies for the slow reply (remember reading this a while ago but lost track of it, however I wanted to take the time to reply), thanks for your info!...


RE: XMP (usually) when referring to tag names -- Cool, thanks for clarifying... again, I wasn't trying to be facetious or anything, mainly I thought I would ask in case there was a general rule in the metadata community (e.g.: "when people refer to a tag, always assume X unless they specify something else", etc) but if it's more just common sense / context-dependent, that's cool, think we should be on the same page from now on.

RE: Composite tags are a group unto themselves -- Ah / hmmm, OK cool... I would be lying if I said I was 100% clear (as in, I didn't realise MP3's had EXIF data in them, etc) but for the sake of brevity and not taking up any more of your time on this random thread then am happy to leave this for now (think I understand them enough to be going on with). Thanks

RE: IMatch -- Hmmmmm! Admittedly, I was sceptical but I checked out the page and IMatch looks quite promising!...
└  Just FYI #1 - to edit in-image metadata I have been using GeoSetter, and 'Microsoft Photo Gallery' (which is *OLD* and even discontinued by Microsoft), however it also does non-cloud face recognition, storing the face data in-image and helps manage system-wide tags (cleaning up un-used tags, renaming tag from X --> Y and updating all files, creating tag hierarchies, etc).
└  Purely FYI #2 - For metadata collection, I am using a few tools like...
  - Gallery-DL (with custom JSON➡️CSV➡️EXIFtool script)
  - WebTools-NG (with custom CSV-mapper script) for Plex
... although have to admit it is extremely time-consuming trying to wrangle all these tools and scripts and everything else together! -- maybe one day I will give up and have a nice, simple, stress-free life.... nah! 🤣

RE: PNG files can support XMP and EXIF [...] but you are correct in that software support is limited
└  That is cool, but for context in terms of my main aims/goals: one of the main underlying things I want to improve is, urgh, I guess I would call it "non-program-specific metadata searching for file discovery"
└  As in... Be able to search for, and files based on their metadata (ideally embedded) whether that's the person tagged it the photo, the location tag, the genre of the MP4, the Client referred to in the Excel file, the Author of the PDF document, or whatever it is... Now, yes I am using Windows (so Windows search is the key limiter), but if it can't read a EXIF/XMP tag in a PNG image, then yes it is great, BUT it does kind of limit it for me as I will have to do specific searches, in a specific app to find those images.

RE: FileMeta "I faintly recall looking at it and I thought that it saved the data in the Windows Alternate Data Streams"
└  WOW, so yes 100 bonus points to you good Sir!... TBH, I kept it slightly vague as I 'Alternate Data Streams' are beyond my understanding of files|filesystems|NTFS|file metadata/properties|ACLs|etc (and also, I didn't want to confuse potentially yourself or other future readers) but you are totally right!
└  NB: Again, without really knowing anything about 'Alternate Data Streams', my main takeaway from reading the various pages on FileMeta was essentially that the data was stored 'in system' so if it was emailed, uploaded to a website, directly copied to a non-NTFS USB drive, etc (i.e. basically anywhere where the direct NTFS-chain is broken) then the metadata would be lost... Assuming that's right, then again, overall a great idea and interesting project so kudos to the team! - but for me, it just seemed like there was too much of a chance of me spending tens/hundreds of hours adding loads of custom metadata to files, and then losing it all in say a Windows corruption, HDD failure, backup not actually capturing the 'Alternate Stream' info, etc - so I decided not to go down that route.


Cool, think that is everything.
Thanks once again for your help, info and time, I will try and check out IMatch sometime!

Cheers,
Martin

Martin Z

Quote from: StarGeek on April 23, 2023, 06:39:02 PM

PS -- I have to admit, I don't get a lot of xkcd jokes (I know, and can feel the shame/judgement already) but yeah think I have seen this one before and it is PARTICULARLY good! 🙂

... One of my other favourite ones is where the guy is on the phone to tech support, he has already done a lot of troubleshooting and limiting down the problem to 1-2 specific tech issues that he can't fix himself, is trying to explain this but is dealing with SUPER BASIC first line like "have you turned it on/off"... [WHICH I CAN 100% RELATE TO!! lol]... eventually gets put through to someone, she is amazing, sorts his problem in 2 seconds and tells him next time you phone up, use the secret password "foobar" (or whatever) to skip ALL the basic stuff and come through to me
└  Then he wakes up and was all a dream! 🤣😭

Will post the image below (although in fairness it is not quite as clearly painful (and so hysterical) as I first found it the first time I saw it, hence the extended description above)...


[Size limited for page readability, right-click and open in 'separate tab' to view fullsize.



PPS -- Also, this one is great just because it is actually REALLY. REALLY GOOD ADVICE (and explained really well, while also super simply)...


StarGeek

Quote from: Martin Z on May 13, 2023, 11:23:13 AMI didn't realise MP3's had EXIF data in them, etc

They don't.  The Composite:DateTimeOriginal for mp3s is comprised of
ID3:RecordingTime
ID3:Year
ID3:Date
ID3:Time
The Composite tags page shows what each composite tag reads.

QuoteJust FYI #1 - to edit in-image metadata I have been using GeoSetter

Geosetter's metadata editor is pretty good and can easily be used as a editor on its own without even touching the geotag options.

Quoteand 'Microsoft Photo Gallery' (which is *OLD* and even discontinued by Microsoft)

ehhh, it depends to much on Microsoft specific tags that nobody else use.

Quotehowever it also does non-cloud face recognition

I'm pretty sure that most software that does facial recognition is local, not cloud.  I haven't heard of a cloud based one.

Picasa had decent recognition for it's time.  Newer software, such as DigiKam and Imatch, have significantly better recognition. One of my favorite things is when I tag a cosplayer in Imatch and it then finds the same person, different costume, 10 years earlier, and as part of the background crowd.

Quote- Gallery-DL (with custom JSON➡️CSV➡️EXIFtool script)

I have this installed but it never works on any site I try it on. Though I've never dug too deeply into it.  Instead, I have a very old version of WaterFox installed that allows the use of legacy extensions so I can keep using a very powerful image downloading one.



Quote- WebTools-NG (with custom CSV-mapper script) for Plex

And instantly turned off because it doesn't describe what it actually does.  "WebTools-NG is designed to help users of Plex Media Server, maintain their server/s and media" is very vague and if I want to know more, I have to go digging for info.  This is always a pet peeve of mine where authors assume you already know what their program does.

Quote└  WOW, so yes 100 bonus points to you good Sir!... TBH, I kept it slightly vague as I 'Alternate Data Streams' are beyond my understanding of files|filesystems|NTFS|file metadata/properties|ACLs|etc

The analogy I use is that ADS are simply sidecar files, like XMP, and they automatically follow the file.  Except when you move the file to a non NTFS drive.  And you don't get to see them and have to jump through hoops in order do so.  My experience with that was with a digital comic organizing program.  Because RAR is not an open format, the program would add its data to the ADS, unlike zip files, which would get the data directly embedded.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

StarGeek

Quote from: Martin Z on May 13, 2023, 11:38:11 AM[Size limited for page readability, right-click and open in 'separate tab' to view fullsize.

And limiting the size is new to me.  Though just clicking on it toggles between big and small on my browsers.

QuotePPS -- Also, this one is great just because it is actually REALLY. REALLY GOOD ADVICE (and explained really well, while also super simply)...

Most of my passwords are 16 characters jibberish because I use a password manager.  On top of that, since I own my own domain, I can create a new email for every site to add to difficulty.  And the individual emails has alerted me a couple times that a site has had a breach because I started getting spam to those addresses that wasn't from the site it was connected to.

But for a few main passwords, I do use that xkcd option.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Martin Z

Quote from: Martin Z on May 13, 2023, 11:23:13 AMI didn't realise MP3's had EXIF data in them
StarGeek: They don't
Ah, OK I've realised where I went wrong now (thanks)... I had mis-learnt / remembered something you told me earlier about composites...

Quote from: StarGeek on April 23, 2023, 06:39:02 PMComposite tags don't exist outside of exiftool.  Exiftool creates them on the fly based upon other tags in the file.  I will also use the monospaced font for Composite tags
... I thought it was Composite tags don't exist outside of **EXIF** (i.e. the framework/data structure) rather than **EXIFTOOL** (i.e. the app), my bad - thanks once again for clarifying!


Quote from: StarGeek on May 14, 2023, 12:06:43 PMI'm pretty sure that most software that does facial recognition is local, not cloud.  I haven't heard of a cloud based one.
Ah OK, to be fair that may be entirely correct - I was conflating (and perhaps assuming) two things together, namely...
[1] Whether the face recognition is performed locally or in the cloud
[2] Whether the recognised faces are stored in the file or a separate database (and it is really the second option that I find more useful).

I fully appreciate your other comment about limited other apps using the Microsoft:People tags however the 2 other face-recognition apps I tried to use (Microsoft Photos, included in Windows 10) and Google Image's native face-detection... They both detected faces, I went in and named those people, but if I ever re-installed Windows (which I do every 12 months-ish) or took the image in/out of google, all the info was lost and I had to start from scratch... i.e. **RE-NAMING** people I had already named before... **RE-SPLITTING** results (where it had thought person A was person B, etc).

I recognise that Photo Gallery is old and far from the best but as it saves it's data into the images and saves me hours of tagging effort, re-tagging the same people in the same photos it's an acceptable evil I guess (at least for me).. PS -- GeoSetter has a nice interface for detected Microsoft:People, essentially the same as IMatch.

Quote from: StarGeek on May 14, 2023, 12:06:43 PMRE GalleryDL: I have this installed but it never works on any site I try it on
Yeah, in fairness I recognised from the offset it was a very powerful/cool utility (so I persevered with it) but yeah it was quite difficult to use/configure, especially in the beginning... I don't know quite what it is, as in there is *SOME* documentation but what is available just doesn't really cover all the parameters in anywhere near enough detail I think, or just assumes if you're configuring X you also know that you have to configure Y and Z and have done this (which obv. as a new user you don't) so the learning curve was pretty steep and unforgiving -- once you get rolling though it does become a lot easier to use!

Quote from: StarGeek on May 14, 2023, 12:06:43 PMRE WebTools-NG: And instantly turned off because it doesn't describe what it actually does.
To be fair, it is not the BEST tool... I used to use another tool that was designed solely for metadata extraction (and worked well) but at some point this was discontinued and rolled into WebTools-NG. From using WebTools-NG it seems like they want to developed it to be a multi-functional plugin but last time I checked it was essentially a wrap of the metadata extractor and just 1 other thing. So a WIP I guess, which I won't blame anyone for.

Quote from: StarGeek on May 14, 2023, 12:06:43 PMThe analogy I use is that ADS are simply sidecar files, like XMP, and they automatically follow the file.
OK. So do they exist as physical files on the O/S, as in can I copy them explicitly to a exFAT drive or attach them to an email (like I could a .xmp sidecar) or do they now quite work like that?

Quote from: StarGeek on May 14, 2023, 12:21:32 PMMost of my passwords are 16 characters jibberish because I use a password manager.  On top of that, since I own my own domain, I can create a new email for every site to add to difficulty.  And the individual emails has alerted me a couple times that a site has had a breach because I started getting spam to those addresses that wasn't from the site it was connected to. But for a few main passwords, I do use that xkcd option.
It's interesting... Not to get TOO far off topic or anything but the main point of the XKCD comic (at least considering the average person) is that making your password LONG is much better for stopping your account from being hacked as making your password COMPLEX (but short)...

I also use a password manager (shout out to: BitWarden, lol) but personally prefer PASSPHRASES rather than PASSWORDS as they both contain a mix of lowercase, uppercase, symbols, numbers - but when you have to type the password in, say if I am accessing the site from a friends computer or logging in from my TV, etc it's a lot easier to type "Elephant3.Tycoon.Mountain" than "^zPJ7662A@zPYUKescg%gG8Pb" (but both are 25 characters).

Lastly, just an acknowledgement that passphrases do have SOME vulnerability to dictionary-based attacks, especially with cracking tools getting better and things like password munging (so this advice may not age perfectly well). The main point I was trying to make really, was just that if you are anything like my mum (who uses "Catherine7" to login to all her websites) 🙈, PLEASE use a password manager, please make your passwords UNIQUE, and make them LONG! :-)

Quote from: StarGeek on May 14, 2023, 12:21:32 PMRE Manually shrinking images attached to posts: Though just clicking on it toggles between big and small on my browser
Ah, I did not know that!.. Nice tip, thanks!

Quote from: StarGeek on May 14, 2023, 12:06:43 PMGeosetter's metadata editor is pretty good and can easily be used as a editor on its own without even touching the geotag options
Good point, to be fair I probably use it 50% for geotagging and 50% for general metadata input/correction... It has a nice "copy and paste" feature where you can copy an image (mark it as a template/reference), hand-pick the 5, 10, 20 images you want to modify then "paste" the metadata, which bring up a sub-menu where you can select which tag groups you want to copy... So if you want to copy the keywords (only), or apply the same geo-location (but not keywords), etc - it's very handy for that... and FREE! (which is one of my favourite prices!) :-)

StarGeek

Quote from: Martin Z on May 16, 2023, 12:39:43 PM
Quote from: StarGeek on May 14, 2023, 12:06:43 PMThe analogy I use is that ADS are simply sidecar files, like XMP, and they automatically follow the file.
OK. So do they exist as physical files on the O/S, as in can I copy them explicitly to a exFAT drive or attach them to an email (like I could a .xmp sidecar) or do they now quite work like that?

It's complicated.  Yes, technically, they are another file, hidden behind the one they're attached to.  They take up drive space the same as any other file. You can copy them out into another file.  You can even read them directly into exiftool, as I did in this post which reads the "Zone.Identifier" ADS that chromium based browsers attach to any file it downloads.  But extracting them isn't really a simple thing.  In theory, you would just add a colon to the file name and then the name of the ADS.  But I really haven't found any program which can really read them easily. And even some of Windows command line programs won't read them.

Bah, I remember reading that the basic Notepad on Windows can read the ADS.  Well, not from the file select window.  You have to run it from the command line
Notepad file.txt:hidden.txt

So technically, yes, you could copy them to an exFAT or attach them to an email.  But you would have to extract them from the main file and copy/attach them separately.  They won't travel to a non-NTFS location.  If you try, you should get a popup like this


Though I just realized that I haven't seen that popup in a long time.  I have TeraCopy set as the default copy program because it re-reads and verifies the new file.  But it doesn't copy the ADS or give that warning.


QuoteI also use a password manager (shout out to: BitWarden, lol)

I also use it.  I used LastPass for the longest time, even paying for it when it was $12/year.  But then they got sold, the new owners jacked the price up and limited its usefulness.
* Did you read FAQ #3 and use the command listed there?
* Please use the Code button for exiftool code/output.
 
* Please include your OS, Exiftool version, and type of file you're processing (MP4, JPG, etc).

Martin Z

Cool, thanks for all the info on ADS files... I will have to give them a bit more thought on whether I want to make them part of my metadata paradigm (and possibly re-consider using something like FileMeta again). I'd be lying if I said I understood them 100% but I certainly understand them a lot better than I did a fortnight ago -- thanks!👍🏼

Quote from: StarGeek on May 16, 2023, 07:01:27 PMI also use it.  I used LastPass for the longest time, even paying for it when it was $12/year.  But then they got sold, the new owners jacked the price up and limited its usefulness

Lol, the similarity is getting even weirder!...  I also started used to use LastPass (think it was my first non-browser password manager actually)

I switched to BitWarden 2-3 years ago though, as:
•  Like you say though, they jacked up the prices and completely nerfed the free account (that's why I left)
•  They made it so unless you paid, you could only access your passwords by phone OR computer, not both
•  Most critically (although luckily I had already left and deleted by LastPass account) - they got hacked last year [backup link]
•  The hacker stole 'customer metadata, backups of all customer vault data, MFA Database (authenticator seeds + telephone numbers), API secrets, 3rd-party integration secrets'
•  Worst of all, everything except the customer vault data was unencrypted (or encrypted but the hacker also stole the relevant decryption keys)

-- Not exactly great for a digital security company! 🙈

lmiol

Quote from: Martin Z on May 23, 2023, 12:23:25 AMI switched to BitWarden
Try to use KeyPass. It is security strong, opensource and portable =)
Also you can keyfile backup to google drive. If anybody get this file - he get nothing =)