Is it possible to have ExifTool not automatically unhide Windows hidden files?

[mpegleg]

Hello.

I've searched the forum and FAQ for information on this issue, but wasn't able to find anything to solve my issue.

Whenever I use most ExifTool commands, (or GUI programs that make use of ExifTool), any Windows 10 files that have the "Hidden" attribute set, are changed to Unhidden status after the operations. [I generally use the -overwrite_original switch].

To keep things simple, I hide my original images (and any poor quality/uninteresting originals) in the same folder as my edits.

Via a resizer program, it only then sees the "unhidden" images, and thus only processes these particular images. Works perfectly for me.

So, what I wish for, is for ExifTool to leave any image files in the same Hidden/Unhidden state that they were in before they were processed. Is this at all possible? Is there an Exiftool switch that I'm not aware of, to enable this?

Note: I did write a short MSDOS batch file (to process multiple files via drag & drop) that did change the Hidden attribute back to it's original status after ExifTool was called, but I would much prefer to have ExifTool do this, so that any GUI programs will also leave hidden files as they were.

Thanks in advance.


[I'm using Windows 10 Pro with the Windows .exe version of ExifTool v11.44. I mainly use ExifMixer, or ExifTool GUI when I'm not using the command line directly].

[StarGeek]

Try using the -overwrite_original_in_place option instead of -overwrite_original.

It's not my favorite option to use, as it does what it says and since I actually have had the power go out while using exiftool.  But I know it's been mentioned for use on the Mac to preserve some OS attributes that exiftool can't edit.

I'm also actually a bit surprised that the FileAttributes tag isn't editable.  My first thought was that it might be preserved by using "-FileAttributes<FileAttributes". 

As an alternative, you could create a temp batch file to remember the files that are hidden before doing anything and then resetting them afterwards
exiftool >Temp.Bat -if "$FileAttributes=~/Hidden/" -p "attrib +h $FilePath" DIR

[mpegleg]

Thanks for your reply StarGeek.

I tried using the -overwrite_original_in_place option, but unfortunately it made no difference, which was a surprise. I assumed that writing to the same file would not cause a change in the file's "hidden" attribute status, but I was wrong.

I was also hoping that the FileAttributes tag would be editable, but I'd already tried that, without success.

The batch file method is what I'm currently using. It works, but it's clunky.

Unfortunately, as I said previously, ExifTool also changes the status of hidden files after operations where it's used in GUI programs. Thus it's always an annoying two-step process, and I've been caught out with files that were previously hidden, suddenly re-appearing. I would have thought a hidden file should remain just that... a hidden file. :/



-------------
ps. By the way, for my simple purposes there's only "one" tag that I always need to modify, (and preferably "not" have ExifTool change the hidden attribute status of my photo images), and that is... -comment  (ie. JPEG Comment).

Thus, I use ExifTool simply to copy across the easily modifiable Windows -XPComment tag to the -comment tag, (which can then be accessed via my photo resizer program, for superimposing this particular tag info onto the edited version of each photo).

This is the Windows command-line I use, which works, but unfortunately always changes any jpeg file attribute from hidden to unhidden...
exiftool -m -overwrite_original "-XPComment>comment" FILE

[Phil Harvey]

I have no idea why opening a file for rewrite should remove the hidden status in Windows.  I'll look into this when I get a chance (may be a while).

- Phil

[mpegleg]

Thanks Phil.

That will be much appreciated.

[StarGeek]

I did some testing on this at the time and today it came back and bit me as I couldn't write to the text file I was playing with.  This may actually be a Windows command line thing.

I often redirect long exiftool output into a text file, appropriately named Temp.txt.  I added the hidden attribute to it and tested it with Notepad++, which would rewrite the file without problem and save the hidden bit.  But on the command line, you can't redirect into a hidden file.  It will fail (see second paragraph).  And when I tried -overwrite_original_in_place on a hidden image file, exiftool returned Error opening y:/!temp/Test4.jpg for writing.

[Phil Harvey]

This makes more sense.  I didn't understand how -overwrite_original_in_place could change the file attributes.  The fact that the open fails is weird, but at least understandable based on the reference you gave.

- Phil

[mpegleg]

I have no idea why opening a file for rewrite should remove the hidden status in Windows.  I'll look into this when I get a chance (may be a while).

- Phil

I acknowledge that response,

but just to confirm what I already suspect about this problem that I've got with ExifTool,
where it's un-hiding all of my previously hidden Windows image files, upon write operations.

I'm assuming that at present, it's "currently" not possible to use any kind of option or command with ExifTool to re-hide an image?

Am I correct in that assumption?

I don't understand why System -FilePermissions are Writable, but -FileAttributes are not?
     _{Win32::File::SetAttributes()}?

I'm thinking that sure, it may not be possible to do any write operations on a "Hidden" file, but it should be possible to set the attributes on a new file copy?? No?

Most commonly the -FileAttributes option gives me the text:

"Regular; (none); Hidden, Archive" with a hidden image file,

and likewise:

"Regular; (none); Archive" when it's not hidden.


The only reason I'm asking, is that I was wondering whether (as a workaround for now) I could use the -FileAttributes option with some Regex code, to search for the "Hidden" text within this output string, (or even a binary match).

That would then allow me to use ExifTool to change the "Hidden" File Attribute of the actual image file back to "Hidden", after it has been written to by ExifTool.


So, what I'm effectively asking is... does ExifTool "currently" have the built-in capability to "modify" the status of the Hidden File Attribute (with Windows)?

_{I'm currently using the Windows MS-DOS attrib command in batch files to change the status back, but it would obviously be nicer/more efficient if ExifTool could do this itself.}

[mpegleg]

Hey, I just thought of a really easy way to perhaps achieve this.

Rename the output file as ".image.jpg" for example, and then immediately rename again back to the original filename.

When renaming a Windows file with a "." before the filename, it will always automatically force it into a "Hidden" state.

Problem solved. Too easy!


* EDIT: This is not a workaround fix that will work on standard Windows drives, only network drives. See following posts discussing this.

[obetz]

When renaming a Windows file with a "." before the filename, it will always automatically force it into a "Hidden" state.

Maybe other file managers or shells duplicate the unix dot file convention, but Explorer and command line "dir" of my Windows installations don't hide files starting with '.'.

BTW, I consider the Windows "hidden" attribute too fragile to use it as you are used to do.

[mpegleg]

Maybe other file managers or shells duplicate the unix dot file convention, but Explorer and command line "dir" of my Windows installations don't hide files starting with '.'.

Ah. Of course. Trust me to be using this on my mapped network drive of my QNAP NAS. (ie. Unix based). lol. I didn't even think of that. (Windows does not hide files based on filename).

For my situation, my photos will always be on the NAS so it's an (almost) perfect fix for me, yet I realize now, that isn't going to help the majority of folks that don't use a NAS to store their photos. Oh well. It was my topic of concern, and it's solved my dilemma, so until they find a full solution, it'll do for me.

BTW, I consider the Windows "hidden" attribute too fragile to use it as you are used to do.

You are not wrong there! It "is" fragile, as I've learnt, but for my situation, with just regular family photos, luckily it's not really an issue, and as long as I have a fairly robust tag with the Hidden attribute stored, I can easily recover things.

I'll never go back to having separated "working" Originals and Edits in separate directories. That was an organizational nightmare for me. Having then together makes it so much easier for me to manage the editing process. (I do have multiple archive & backup copies of all of my originals).

[mpegleg]

This code example, with the Copyright tag containing just "H":

Code Select Expand

-xpcomment="I am a new copied hidden file" execute -if "substr($Copyright,0,1) eq 'H' " -o "%d.%f.%e" -common_args -overwrite_original

gets my 'Hidden' "image.jpg" file processed, and renamed to ".image.jpg" (and thus is now also a "Hidden" file on my Unix based NAS mapped drive. It would "not" be Hidden on a standard Windows Drive).

Now, is there any way using ExifTool, to rename it back to the original filename? ie. image.jpg

Note: I don't want to be left with any extra files.

So close yet so far.  A measly "." away from solving my dilemma.

I'm of course worried that any ExifTool operation performed again, will simply "un-hide" the file all over again.

[mpegleg]

I really wish there was a test Regex generator like Regex101 in reverse? Does this exist?

For example, I could enter the base string="xyz", and it would give me the Regex code that would be needed to output just "yz".


Anyway, this will be very simple Regex for those that know, but I'm struggling to find it at the moment.

I just want the second character onwards returned.

ie. I have the string ".image.jpg", and I just want "image.jpg" returned. (ie. without the first ".")



Also, how do I use the value contained in xpcomment with the -o option?

eg).

I want this corrected:

-o "%d"$xpcomment

so that it will use the contents of xpcomment ("image.jpg") to complete the creation of the new filename.

At the moment, with my expression there, the $xpcomment is being treated as straight text, rather than using the "contents" of xpcomment.



Thanks,
-Paul

[Phil Harvey]

With a substitution:

exiftool "-filename<${filename;s/^\.//}" ...

or with substr():

exiftool "-filename<${filename;$_=substr($_,1)}" ...

- Phil

[mpegleg]

Thanks Phil. I can make use of both of those methods. Thanks for giving the alternate ways of doing it. That's always a great way of learning.

I'm going to relegate this issue to the too hard basket for the time being.

The problem is with the Windows OS, and how it doesn't like writing to Hidden files.

If you were able to find a way of remembering the original Hidden attribute status, and then re-applying it after all the ExifTool write operations, then that would be great.

[mpegleg]

I have no idea why opening a file for rewrite should remove the hidden status in Windows.  I'll look into this when I get a chance (may be a while).

- Phil

*BUMP*

Hi Phil. Just hoping that this problem hasn't been forgotten about? Exiftool is amazing, and I'm so happy with all the uses I have put it to. It's helped me tremendously with getting my archive photo/transparency collection organized.

However, (as previously explained here) whenever I do any Exiftool operations in Windows, I often forget that I need to then go and re-hide all of my 1000s of hidden edited jpg files again.

It really is a pain, esp. as Windows 10 Explorer currently has a bad Search bug that often prevents the correct function of the Explorer Search box, and usually requires a full system restart, for just a short-term temporary fix.  (So, it's not always an easy task to find and re-hide all the previously hidden files). edit-- Seems like Microsoft has "just" released a patch for this Search bug. Eagerly installing it now. Fingers crossed!

[Phil Harvey]

Sorry for the delay in getting around to this.  I can't reproduce this problem on my Windows 10 system.

I created a tmp directory with 2 identical files (a.jpg and b.jpg), then set the hidden attribute on one of them (a.jpg) from Windows Explorer.  Below is a screen capture from my cmd.exe window showing what happened.  It wouldn't let me write the hidden file when I used the -overwrite_original_in_place option.

- Phil

[mpegleg]

Thanks for your answer Phil.

Yes, I also get the same results as you.

I guess what I'm really hoping for, is a way for exiftool to initially read a file's attributes (and remember them), and then after doing any operation, there would be an internal way of then restoring the file's attributes to the way they were, even when writing a new file with the overwrite_original option.

I understand that I can use batch files to accomplish this, however there are many times when I use other shell programs that then also call exiftool. Thus it really isn't always practical to use batch files.

The way I have my photos now organised is as in the following simplified example:



This works very well for me. Any Original/mediocre images that have been edited can be hidden in situ, which is very convenient, and hidden/unhidden files can be viewed with a simple click. Also, I can tell exiftool to ignore anything in the "Other" hidden subdirectory, (containing other edits, videos and general info etc.) which will also not be seen by other programs.

Thus my other external programs (like Image Resizers) will generally only process unhidden files. Thus they only see the (unhidden) files that I want them to see and hence process only these.

Unfortunately if I run exiftool with -overwrite_original set, (which is what I always use) all the hidden files will be then get unhidden, and I have to then search for them all (I use the Copyright tag to indicate which images should always be hidden).

I understand that it's not exiftool's fault that they get unhidden when writing a new file, (it's a system thing) but I was really hoping that there may be something that could be done, to reset the original Hidden Attribute?

All up it's not a game changer, but something that is really annoying, as I will often forget to re-hide the images after using exiftool, and then process and resize them all. Then, I'll excitedly go and show my folks some new photos, only to realize that they are seeing all of my once hidden mediocre/rubbish images. 


I strongly believe that: If a file is designated as "Hidden"... Hidden it shall thus remain, for all eternity.

[Phil Harvey]

I'll see what I can do and post back when I know more.

- Phil

[ryerman]

I make use of the current behaviour.
For me, it is useful to expose hidden files.
Why?
Because when playing audio files, some applications create hidden JPG files near the audio files, which I don't want.
I want to expose those hidden files when tagging my own, desirable JPG files.

I test for hidden files in my tagging script so I will deal with any change in Exiftool behaviour, but perhaps others rely on the current behaviour for other reasons.

Another example of "One man's trash is another man's treasure". 

[mpegleg]

...perhaps others rely on the current behaviour for other reasons.

Another example of "One man's trash is another man's treasure". 

Perhaps, yet it is entirely unexpected behaviour. Hidden files are generally hidden for a good reason. Especially system files, which should not normally be played around with, moved or changed. I rely on that fact, as it is the correct accepted behaviour.


https://www.lifewire.com/what-is-a-hidden-file-2625898

For the best of both worlds, perhaps it would be be great if Phil could work out a way to give one the option to either keep the files's default prior attribute status, or to explicitly unhide a file (as is currently done).

Anyway, I won't go on about it, as Phil has said that he'll look into it. I'm just glad to have discussion about the issue, as it's very important to me. It's the only issue I have with ExifTool, as it causes a lot of extra work, to constantly have to go back and re-hide any hidden files (all of my original images) that were touched by ExifTool.

[mpegleg]

Ok. After some experimenting with other photo editing programs, it seems that when re-saving over the top of the same Hidden file (ie. not changing it's filename) most apps will either give an error that the file can't be written to, or they will write to another temporary file, delete the existing file, and then change it's name back to the original. Thus they exhibit the same behaviour as ExifTool in not preserving the Hidden file attribute. Grrrr.

Even so, as mentioned previously, I would still love for there to be an ExifTool option to preserve the Hidden File attribute. Here's hoping.

[Phil Harvey]

Unfortunately the Win32API::File module doesn't have a routine to set the file attributes, so I don't know how this could be done.  However, if it helps you can prevent ExifTool from writing hidden files or handle them separately with a command like this (as mentioned in here).

exiftool -api systemtags -if "$fileattributes !~ /Hidden/" ...

- Phil

[mpegleg]

Thanks Phil. I'll see whether I can make use of that.

I'm not pretending to understand the programming side of things, but are these references at all relevant to Perl? Just thought I'd reference them in case they are relevant. I have no idea, but they look interesting...

>>>> https://docstore.mik.ua/orelly/perl4/perlnut/ch22_05.htm <<<<

https://docs.microsoft.com/en-gb/windows/win32/api/fileapi/nf-fileapi-setfileattributesa

https://docs.microsoft.com/en-gb/windows/win32/api/fileapi/nf-fileapi-setfileattributesw

http://www.java2s.com/Code/Perl/Win32/TochangetheattributesyoucancallSetAttributes.htm

[Phil Harvey]

Thanks for the links, but Windows is more of a pain in the ass than you think.  Most Windows interfaces do not support wide-character file names, which is why I was using the Win32API module.  I suppose I could use Win32::File, but then I wouldn't be able to preserve the hidden attribute for any file with a special character in its name.  Also, ExifTool doesn't yet have a dependence on Win32::File, and don't want to add it just for this.

- Phil

[mpegleg]

Thanks again Phil. I totally understand where you are coming from in regards to programming for Windows being a PITA.

Anyway, I'm glad we had the discussion, as I needed to know if it was possible. I can live without it. Once I have my photo collection completely organised, (and it's getting close), I shouldn't have to worry too much about it, as long as I remember to keep all my files tagged internally with my own "Hidden" status tag. (Using the Copyright tag currently works for me, as long as I keep an eye that no other editing programs are messing with it). I've always got plenty of archived backups to fall back on.