[SOLVED] Character enconding in konsole, terminal and ntfs3g

Grothendieck · Last edited by Grothendieck on Tue Jan 04, 2011 4:08 pm; edited 1 time in total

Hi guys,
trying to run Polish characters support in my Gentoo. I actually thought that they were up and running until I tried to use Konsole to copy some files to NTFS partition and got an error

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

I'd start with dropping '@euro' part, that's only for non-utf8 locales.
Also, if your ntfs3g is decently recent, it shouldn't have 'locale' option, but always mount as utf8.
And be consistent, don't go from utf8 to UTF-8.

Grothendieck · Posted: Sun Jan 02, 2011 8:53 pm Post subject:

First of all, thanks for the reply.
I dropped the euro part and it repaired the "locale: Cannot set LC_ALL to default locale: No such file or directory" error. I was aimed at this option by Gentoo Polish localization guide at http://www.gentoo.org/doc/pl/guide-localization.xml which in listing 3.1 states LANG="pl_PL.utf8@euro". I also corrected everything to UTF-8. Any chance for quick explanation of the difference between UTF-8 and utf8?
My ntfs-3g is 2010.8.8 and looks like still is able to recognize '-o locale=' because when I pass some random stuff instead of 'pl_PL.UTF-8' it reacts:

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

For utf8 and UTF-8, usually one is an alias for the other.
That output from ntfs3g means that 'locale=' is ignored, as that version is recent enough.

As for the filesystem, you just might be screwed - on linux, files are created with filenames of system encoding,
if those files were created while it was still latin2, you will need something like convmv and a lot of patience to fix it.
(a good test could be output of 'LANG=pl_PL ls <filename>', if that displays correctly, you need to convmv)

Be careful though, the guess algorithm of convmv isn't what you'd call flawless.

Grothendieck · Posted: Mon Jan 03, 2011 12:01 am Post subject:

Looks like I messed up with implementing the change suggested by VoidMage and a proper reboot revealed that removing '@euro' solved the fonts under konsole, mc and dolphin (env-update && source /etc/profile and opening new tab in konsole wasnt quite enough...). Now newly created files with Polish names display properly in those programs. What is more, running

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

There's no CONSOLEFONT issue, that var is exported only in startup script.
Post your full env, not just the vars printed by portage.
And drop that 'LC_ALL=""'.

Chiitoo · Posted: Mon Jan 03, 2011 1:36 am Post subject:

Grothendieck · Posted: Mon Jan 03, 2011 9:51 am Post subject:

Ok, here goes env:

Grothendieck · Posted: Mon Jan 03, 2011 10:24 am Post subject:

Ah, /etc/profile.env had a line which set TZ to GMT. Removing it solved the issue of GMT instead of CET. No idea, how GMT made its way to profile.env...
So, only fonts in terminal left to go.
Cheers!

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

What do you mean by "standard terminal" ? console, xterm ?

And you could at least 'env | sort', for easier reading.
Also, that LANGUAGE var is used somehow by KDE, so it should probably be set to a meaningful value or unset.

Grothendieck · Posted: Mon Jan 03, 2011 10:33 am Post subject:

I think its called the console. Not the konsole, nor xterm. You know, the old school thing, where you type 'startx'

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

Personally, when I use lat2a-16, I set consoletranslation="trivial" and unicodemap="lat2u",
as there's something broken about that font.

Mind I'm on baselayout2, I no longer fully recall how was I setting it on v1 (it was similar, but not the same).

Grothendieck · Posted: Mon Jan 03, 2011 10:52 am Post subject:

The only place I can find reference to LANGUAGE is /etc/make.conf:

Grothendieck · Posted: Mon Jan 03, 2011 11:02 am Post subject:

I set the unicodemap and consoletranslation as you said, but nothing changed. Maybe I am missing something trivial but why those varaibles dont show up in env?

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

As I said, these values work on baselayout 2.
Look at the script and see what's it using, then check the manpage of the tool.
IIRC on v1 I abused CONSOLETRANSLATION to pass both of these values.

Grothendieck · Posted: Tue Jan 04, 2011 11:06 am Post subject:

My /etc/init.d/consolefont uses setfont so I changed it to pass '-u ${UNICODEMAP}' and now using the lat2a-16 font together with CONSOLETRANSLATION="trivial" the console properly displays filenames created under konsole. However, polish characters typed in command line show up as blank characters and if I create a filename under console the polish characters display incorrectly in konsole or dolphin. Any ideas?
In addition I searched all directories that came to my mind in /etc/ and /home to track the LANGUAGE but I found nothing except /etc/make.conf setting its value to 48. What do you think about that?
Regards.

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

Is locale at console same as in X ?
See /etc/conf.d/keymaps too.

Grothendieck · Posted: Tue Jan 04, 2011 2:59 pm Post subject:

Yep, the results of 'locale' are the same under console and X.
Here goes /etc/conf.d/keymaps, I configured it according to the localization guide, at least I hope so.

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

Set keymap to pl2 and dumpkeys_charset to iso-8859-2.

Oh, and that comment is actually meant for x86/amd64.

Grothendieck · Posted: Tue Jan 04, 2011 4:07 pm Post subject:

And now its working! Thank you very much for your help!
Cheers!

miroR · l33t Joined: 05 Mar 2008 Posts: 826

miroR · l33t Joined: 05 Mar 2008 Posts: 826