I've found my way to get unicode and utf-8 working right

Message

daschapa · Post by **daschapa** » Wed Sep 06, 2006 6:29 pm

I know that my skills with english leaves much to be desired, so be patient with me.
I've read around about too many people getting troubles with Unicode and UTF-8 in consoles (specially with ncurses).
I don't tell you this is the right way, but it worked for me. Maybe you find it useful.
All this work is, in some or another way, inspirated by other posts or HOWTOs.
Okey, let's begin.

First step: check that you have set the "unicode" and "slang" use-flags
To check them you must look out the output of:

Code: Select all

#emerge --info

and also get a look of your /etc/make.conf

If you have it, then you are lucky! If not you may:

update your system's profile to 2006.1/desktop then check if one of them is missing yet.
add them manually to the USE in /etc/make.conf

I suggest you the first way.

Compiling a kernel supporting UTF-8
Update your kernel then configure it this way

Code: Select all

File systems --->
 Native Language Support --->
   (utf8) Default NLS Option
   <*>   NLS UTF8

If you use any FAT, vFAT or NTFS partitions, also check

Code: Select all

File systems --->
 DOS/FAT/NT Filesystems --->
   (850) Default Codepage for FAT
   (utf8) Default iocharset for FAT

Maybe you'd like to use another Codepage. Honestly, i don't know what is gonna happen if you do it.
Once you get your new kernel ready, it is time to begin its use. Add it to your lilo or grub.

Creating and setting a new locale
For example, if your locale if "es_AR" (as in my own case, cause i'm argentinian and wanna talk spanish), you will need a new one that supports UTF-8. It will be named "es_AR.UTF-8". Lets create this locale.

Code: Select all

# localedef -i -c es_AR -f UTF-8 es_AR.UTF-8

IMPORTANT: The parameter you put after "-c" must be a valid end existant locale. If you are in doubt read The gentoo localization guidehttp://www.gentoo.org/doc/en/guide-localization.xml.
VERY MUCH IMPORTANT: use the locale corresponding to YOUR country or you gonna end talking spanish with me

.
You must add this new locale to yor /etc/locale.gen. If you never did it before check this link http://www.gentoo.org/doc/en/gentoo-x86 ... ap2_sect17

Now, set your locales. If you don't know how to do it, follow this guide http://www.gentoo.org/doc/en/guide-localization.xml

Enabling Unicode and UTF-8 in your system
We need a terminal font supporting UTF-8. Terminus is quite ugly, but it works.

Code: Select all

# emerge terminus-font

Edit your /etc/rc.conf. Where it says

Code: Select all

UNICODE="no"

must be

Code: Select all

UNICODE="yes"

Now, let's visit /etc/conf.d/consolefont. Where it says:

Code: Select all

CONSOLEFONT=<anything>
#CONSOLETRANSLATION=""

Must say

Code: Select all

CONSOLEFONT="ter-v16b"
#CONSOLETRANSLATION=""

Now, it lefts a bit more of config. Edit /etc/profile. Then add the following lines:

Code: Select all

if test -t 1 -a -t 2 ; then
        echo -n -e '\033%G'
fi

I've heard it is problematic to put this code in /etc/profile. Well, I din't had any problem.

Upgdrading system to support Unicode
Smart ones do...

Code: Select all

# emerge -uDN world

Yeah, do it! Don't blame me! I know it would take a lot of time, but it is the best way. Well, at least update your base system with:

Code: Select all

# emerge -uDN system

then emerge with the same parameters all apps you wanna use under console. For example:

Code: Select all

# emerge -uDN mc

Maybe your gnome-terminal or your gnome environment becomes illegibles whitout unicode support on top of a Unicode-enabled system. In this case:

Code: Select all

[i](for gnome-terminal)[/i]
# emerge -uDN gnome-terminal

[i](for Gnome Desktop)[/i]
# emerge -uDN gnome

Stop watching me as a snake inside a pipeline! Do it if you wanna get full Unicode support.

Enjoying the unicode
For some strange reason, it doesn't work until you restart your system. So, until someone have any idea about how to avoid restartin you'll need to do it before see it work.

I'm expecting for good results by your side.
Have luck, pray and wear your talismans and amulets.

Again, check this guides. Study them:

Bye.

daschapa · Post by **daschapa** » Thu Sep 07, 2006 1:55 am

I've found some errors in my post. I'll be fixing them as I found them.
If you see somthing wrong or you are suspecting about something post it.
Thnx.

swimmer · Post by **swimmer** » Thu Sep 07, 2006 1:09 pm

I forgot why you need slang? Can you please enlighten me?

Gracias
swimmer

daschapa · Post by **daschapa** » Thu Sep 07, 2006 2:30 pm

swimmer wrote:I forgot why you need slang? Can you please enlighten me?

Gracias
swimmer

Note that "slang" isn't "The Slang Dictionary" but "The S-Lang Library"

. This library is some kind of ncurses, it is really similar, but it works fine with Unicode.
The BIG problem with Unicode in the console is that apps like Midnight Commander, that are using Ncurses becomes "ugly" with a "broken" look. This is due to a Ncurses malfunction with Unicode. I've heard around this was "solved", but apps using Ncurses must use another part of the library, so, until developers modify their apps, Ncurses will remain looking "ugly".
Sice the wide majority of programs I've seen that uses Ncurses has also support for S-Lang, and this works fine, this look like a good solution.
Maybe some apps still filing, but isn't anything you can't manage.
Be lucky.
By.

aries · Post by **aries** » Mon Nov 13, 2006 7:09 pm

Hello daschapa,

If I understand you correctly ncurses is the one causing the ungly look, is that rigth?
'make menuconfig' for instance has the same ugly look on my system.

daschapa · Post by **daschapa** » Mon Nov 13, 2006 7:29 pm

aries wrote:Hello daschapa,

If I understand you correctly ncurses is the one causing the ungly look, is that rigth?
'make menuconfig' for instance has the same ugly look on my system.

Yes, it looks like ncurses is the bad boy. I've been reading a little about this, and ncurses has some unicode suport, but the apps must change some includes.
However, with the "slang" useflag almost all apps will look "cute", but not absolutely all.
Have luck! And excuse my terrible english.

aries · Post by **aries** » Mon Nov 13, 2006 8:02 pm

Thanks daschapa.

So for make menuconfig we have to live with the ugly look.

Maybe I better config the console without unicode: it is only used in case of a problem with kernel / xorg.
Else I use xterm.

Pythagoras1 · Post by **Pythagoras1** » Sun Nov 26, 2006 9:14 pm

aries wrote:So for make menuconfig we have to live with the ugly look.

simply run unicode_stop and menuconfig will look nice again in the current console.

(using unicode_start, of course, reenables unicode support on the current console again)

i_magic · Post by **i_magic** » Tue Dec 26, 2006 10:04 pm

Pardon my stupidity, but why would one want UNICODE in the console?

Is this for locales other than English?

If I do unicode_stop, my mc and menuconfig both clear up. Also, if run mc
in kde konsole, it looks fine. It's only if I am in text mode that it looks ugly and stuff.

I haven't tried reading files with any foreign language characters, but once
in a blue moon I might need it.

Thanks in advance.

Doc Holiday

Pythagoras1 · Post by **Pythagoras1** » Wed Dec 27, 2006 2:38 pm

these tools principally support unicode terminals, but a few semi-graphical console toolkits are buggy and don't work correctly on linux (kernel) terminals in unicode mode. i don't know the reason. it should not happen, but it does

edit: what i mean is that i don't see the reason why there would be any difference between the kernel terminal in unicode mode and a unicode terminal in x11.

roki942 · Post by **roki942** » Wed Dec 27, 2006 4:28 pm

daschapa wrote:Yes, it looks like ncurses is the bad boy. I've been reading a little about this, and ncurses has some unicode suport, but the apps must change some includes.
However, with the "slang" useflag almost all apps will look "cute", but not absolutely all.

Way kool!!

Handled my problems ... thank you.