Croatian character filenames problem + inserting a tab character on output

[ikiK]

Hello, I'm new with all this so please be gentle... Im using OSX system terminal.
I have read FAQ, and 40 topics here on this subject but it simply does not work for me. Im exporting pdf CreateDate into a tab delimited csv. I have two problems.

Code Select Expand

exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" ~/Dropbox/testmapa/mapa2/ >~/Dropbox/testmapa/mapa2/out.csv

1. The file name that contain special Croatian characters is on output messed up. I tried setting -charset on filename, I tried without it, I tried with -common_args, I tried Latin2 MacLatin2 UTF8 MacCroatian, noting works in all combinations. When the file name is "file2-test šžćč.pdf" I get output "file2-test sÌŒzÌŒćcÌŒ.pdf".

2. I tried to insert a tab character on place of _ between %d_%H (day and hour) because I need that information in separate tab delimited column in csv automatically.
I have tried to insert it in all possible ways with \t, in combination with "",'',& and no collapse, <tab>. It just wont work for me. Please take note that that line is in .command file. Not terminal itself. So keystrokes wont work, I need it written in code.

3. And this is just for fun, I'm trying to set this code to execute throe file, now I have relative path, but Can I set it to execute from where ever it is lunched "on click". So it does what it does on subfolders from wherever I start it? And that out.csv end up in folder where I started it?

Help me please and thank you in advance.

[Phil Harvey]

Your command works for me on OS X:

Code Select Expand

> ls tmp
file2-test šžćč.pdf
> exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" tmp
file2-test šžćč.pdf 2014.03.12_12.07.40

I don't have time to google right now to figure out how to add a tab to the command line, but I'm sure it could be done.  An alternative is to put your options in a -@ argfile to avoid the command-line problem.

- Phil

[ikiK]

Hi Phil, let me start by thanking you for making this great extension.
The code it self does read the filenames correctly but it does not write them correctly in .csv or .txt file. As its shown in screens below.
The hierarchy of files:


When scened;


and when I add  > ~/Dropbox/exiftool/testmapa/out10.csv to write it in .csv or .txt  file it does it wrong;


Reopening in some text editor with different encoding does not help ether. I might add that I tried with .txt files also. UTF8 must work on this character, but somehow he writes them wrong in those files.


And about tab, I have googled and searched before I got here, I even asked on stackoverflow, I got no useful answer or lets say none of them worked for me, or I did something wrong. As you can see in upper picture I have this simple line in csv.command file. If I understand correctly, making another argfile is not a option for me. As this one file will already be run by another custom program that I have. If someone could provide a correct add-on to my code that would be very useful as Im stuck with that one.

[Hayo Baan]

To add a tab character somewhere on the command-line is easy. Assuming you are using a standard Mac terminal, just press CTRL+v and then the TAB key, this will insert a literal tab character at the place of the cursor.

As to your problem, it looks like the editor/application you view the file in simply doesn't understand UTF-8 well enough as it works fine for me:

Code Select Expand

$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf
file2-test šžćč.pdf 2016.11.29_09.14.06
$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf > t.csv

I then opened the t.csv in both TextEdit and LibreOffice (see screenshot) and it is showing just fine.
Can you perhaps post your .csv file so we can have a look?

[ikiK]

To add a tab character somewhere on the command-line is easy. Assuming you are using a standard Mac terminal, just press CTRL+v and then the TAB key, this will insert a literal tab character at the place of the cursor.

Thank you for your replay. I have stated few times that keystrokes don't help me. I have file that will run this code, and file it self will be run by other program. I need a character in that code written in file...

As to your problem, it looks like the editor/application you view the file in simply doesn't understand UTF-8 well enough as it works fine for me:

Code Select Expand

$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf
file2-test šžćč.pdf 2016.11.29_09.14.06
$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf > t.csv

I then opened the t.csv in both TextEdit and LibreOffice (see screenshot) and it is showing just fine.
Can you perhaps post your .csv file so we can have a look?

Im using regular terminal yes, also I use office for mac 2011 as well as textwrangler witch is same as notepad plus or something similar for windows. Im using that for years and I can say with 100% that it understands encodings very well, I'm using it to edit php, html etc. EDIT: looks like sublime text app opens it right... This changes everything. And I'm confused now. textwrangler has an option ;reopen file using encoding, and non works with this files...
Update:I have just tried to open it on other machines and it works. Something is wrong with my system. Same version of file and text editor on other machine is opening it right...
Yes of corse I can post what I get out of this. Can I see yours if it's not a problem? Two files that are in attachment are fresh and unopened...

[Phil Harvey]

I have file that will run this code, and file it self will be run by other program. I need a character in that code written in file...

That is easy.  Just put the tab character in the file.  Make sure it is quoted if it is in a command.

I'm away on a business trip right now and don't have any more time to look into this in detail.

- Phil

[Hayo Baan]

To add a tab character somewhere on the command-line is easy. Assuming you are using a standard Mac terminal, just press CTRL+v and then the TAB key, this will insert a literal tab character at the place of the cursor.

Thank you for your replay. I have stated few times that keystrokes don't help me. I have file that will run this code, and file it self will be run by other program. I need a character in that code written in file...

Then I don't follow where do you want the TAB character to appear? If you want a TAB to appear in the file that creates the output, just put the TAB character where you want it. If for instance you need tabs in place of the "." in the output of your exiftool, just put them there (but then within the quotes). Like this:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y% %m% %d_%H% %M% %S" *.pdf
Ah, right, now I see what you mean: exiftool converts those tabs back to a "." when printing. Right, unless Phil knows a way to circumvent this, we'll have to work around that ourselves. The following will do this:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%##%m%##%d_%H%##%M%##%S" *.pdf | perl -pE 's/##/\t/g'
(I used ## as delimiter as that is extremely unlikely to be contained in e.g. the filename)

As to your problem, it looks like the editor/application you view the file in simply doesn't understand UTF-8 well enough as it works fine for me:

Code Select Expand

$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf
file2-test šžćč.pdf 2016.11.29_09.14.06
$ exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d_%H%"."%M%"."%S" *.pdf > t.csv

I then opened the t.csv in both TextEdit and LibreOffice (see screenshot) and it is showing just fine.
Can you perhaps post your .csv file so we can have a look?

Im using regular terminal yes, also I use office for mac 2011 as well as textwrangler witch is same as notepad plus or something similar for windows. Im using that for years and I can say with 100% that it understands encodings very well, I'm using it to edit php, html etc. EDIT: looks like sublime text app opens it right... This changes everything. And I'm confused now. textwrangler has an option ;reopen file using encoding, and non works with this files...
Update:I have just tried to open it on other machines and it works. Something is wrong with my system. Same version of file and text editor on other machine is opening it right...
Yes of corse I can post what I get out of this. Can I see yours if it's not a problem? Two files that are in attachment are fresh and unopened...

Both show up fine in my applications. Looks like you have textwrangler set up incorrectly. As the file is UTF-8 already you should NOT have it apply any extra encoding.

Hope this helps,
Hayo

[ikiK]

Then I don't follow where do you want the TAB character to appear? If you want a TAB to appear in the file that creates the output, just put the TAB character where you want it. If for instance you need tabs in place of the "." in the output of your exiftool, just put them there (but then within the quotes). Like this:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y% %m% %d_%H% %M% %S" *.pdf
Ah, right, now I see what you mean: exiftool converts those tabs back to a "." when printing. Right, unless Phil knows a way to circumvent this, we'll have to work around that ourselves. The following will do this:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%##%m%##%d_%H%##%M%##%S" *.pdf | perl -pE 's/##/\t/g'
(I used ## as delimiter as that is extremely unlikely to be contained in e.g. the filename)

Both show up fine in my applications. Looks like you have textwrangler set up incorrectly. As the file is UTF-8 already you should NOT have it apply any extra encoding.
Hope this helps,
Hayo

Thank you both for answering and trying to help me.
Here is example of first suggestion ( I tried that before) it doesn't work.

Code Select Expand

exiftool -T -r  -filename -CreateDate -d "%Y%"."%m%"."%d% %H%"."%M%"."%S" ~/Dropbox/exiftool/testmapa/mapa2/ > ~/Dropbox/exiftool/testmapa/out10.csv
I need tab between d% and %H, so that time gets in new  column in csv. So date and time in two separate columns. I need just one tab between for when importing in excel and such.
I Also tried this;

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d\t%H%"."%M%"."%S" ~/Dropbox/exiftool/testmapa/mapa2/  > ~/Dropbox/exiftool/testmapa/out10.csv | perl -pE 's/##/\t/g'


Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d%\t%H%"."%M%"."%S" ~/Dropbox/exiftool/testmapa/mapa2/  > ~/Dropbox/exiftool/testmapa/out10.csv | perl -pE 's/##/\t/g'


Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d%'\t'%H%"."%M%"."%S" ~/Dropbox/exiftool/testmapa/mapa2/  > ~/Dropbox/exiftool/testmapa/out10.csv | perl -pE 's/##/\t/g'

And around 20 more versions... Also nothing.
@Hayo Baan I need this to be a tab, because exiftool filename is separated with tab from date and time...
Also I'm not applying anything when I drag and drop that csv to excel for example. It wont open it right, and when i import in utf or any other encoding its still wrong. Same as textwrangler ( I just restore defaults, no help).

Im kind of lost at this point...

[Hayo Baan]

With my sample command using the perl pipe, just put ## where you want the tabs (like I showed you), then it will work. Also you need to put the redirection to your Dropbox file after the perl command, at the very end!

[ikiK]

With my sample command using the perl pipe, just put ## where you want the tabs (like I showed you), then it will work. Also you need to put the redirection to your Dropbox file after the perl command, at the very end!

Sir that worked, I wasn't kidding when I asked to be gentile , also I was thinking that ## would replace all tabs with ## delimiter, I'm sorry for that.

Too sum up now, I have my tabs in right place, Im at home currently on my Macpro and here I see all characters right in textwranlgler and sublimetext, as well import to excel works. Im sorry for taking up your time but I did come to the conclusion and finish line, the hard way to be fair.
For my 3d question I have half of solution I will post it later when finished if someone is interested how to run this from current folder without paths specified.
Thanks you for your help. Great tool!

[Hayo Baan]

You're welcome!

For your third question, instead of the full path (~/Dropbox/testmapa/mapa2/), just supply a dot: .
The dot stands for the current directory

Enjoy,
Hayo

[ikiK]

You're welcome!

For your third question, instead of the full path (~/Dropbox/testmapa/mapa2/), just supply a dot: .
The dot stands for the current directory

Enjoy,
Hayo

Actually after couple of hours trying, .  and some other things like pwd and whatnot didn't work for me, but I find out that $(dirname "$0") works. Heres my complete code;

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d%##%H%"."%M%"."%S" $(dirname "$0") | perl -pE 's/##/\t/g' > $(dirname "$0")/out.csv
exit;
Thanks so much for help, it was a life saver. Have a nice day.

[Hayo Baan]

I guess you wanted to have the command work on the directory your script was in (which I didn't understand to be the case), not your current directory? Otherwise the following should have worked too:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d%##%H%"."%M%"."%S" . | perl -pE 's/##/\t/g' > ./out.csv
exit;

[ikiK]

I guess you wanted to have the command work on the directory your script was in (which I didn't understand to be the case), not your current directory? Otherwise the following should have worked too:

Code Select Expand

exiftool -T -r -filename -CreateDate -d "%Y%"."%m%"."%d%##%H%"."%M%"."%S" . | perl -pE 's/##/\t/g' > ./out.csv
exit;

Yes, that was the case. And I did read the . description and I misunderstood that. So I lost some time, but learned in process. Thanks for the help nevertheless.
I will have to drop that file (well other application will) into folder user of computer picks, and that script runs on that folder. So specific paths wasn't good enough. Now its all good.
And sorry for my English, not my first language.