[SOLVED] UTF-8 character with leading circumflexed A

[Snowofice]

After changing my locales to UTF-8 I have the problem, that when I want to display certain characters in VI, they are shown with a leading  (A with circumflex). (same for cat, less, etc)

A few of them are: ° ; µ ; ² ; ³ ; ¶ ; »

Not that they are use frequently, but it is disturbing!

On the other hand, any try to write an "Â" (A with circumflex) concludes in a "Ã" (A with tilde). The same is for other letters resembling A - Unicode 00C0 and following!

I changed everything according to the official howto ( http://www.gentoo.org/doc/en/utf-8.xml ), including a full "emerge -p world; emerge -p system" and also "emerge --depclean" and "revdep-rebuild"


Can anyone give me a lead, what I might have missed?



# cat consolefont
CONSOLEFONT="lat9w-16"
CONSOLETRANSLATION="8859-15_to_uni"

# locale
LANG=en_US.UTF-8
LC_CTYPE=de_DE.UTF-8
LC_NUMERIC=de_DE.UTF-8
LC_TIME=de_DE.UTF-8
LC_COLLATE=de_DE.UTF-8
LC_MONETARY=de_DE.UTF-8
LC_MESSAGES=en_US.UTF-8
LC_PAPER=de_DE.UTF-8
LC_NAME=de_DE.UTF-8
LC_ADDRESS=de_DE.UTF-8
LC_TELEPHONE=de_DE.UTF-8
LC_MEASUREMENT=de_DE.UTF-8
LC_IDENTIFICATION=de_DE.UTF-8
LC_ALL=


# emerge --info
Portage 2.1.4.4 (default-linux/x86/vserver, gcc-4.1.2, glibc-2.5-r4, 2.6.8-022stab078.21 i686)
=================================================================
System uname: 2.6.8-022stab078.21 i686 AMD Athlon(tm) XP 2800+
Timestamp of tree: Tue, 11 Mar 2008 02:00:01 +0000
app-shells/bash: 3.2_p17-r1
dev-lang/python: 2.4.4-r9
dev-python/pycrypto: 2.0.1-r6
sys-apps/sandbox: 1.2.18.1-r2
sys-devel/autoconf: 2.61-r1
sys-devel/automake: 1.4_p6, 1.10
sys-devel/binutils: 2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool: 1.5.26
virtual/os-headers: 2.6.23-r3
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=athlon-xp -Os -pipe -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /opt/glftpd/etc /opt/glftpd/ftp-data"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-march=athlon-xp -Os -pipe -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="en_US.UTF-8"
LC_ALL=""
LINGUAS="de en_US en_GB"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_EXTRA_OPTS="--progress --exclude-from=/etc/portage/rsync_excludes"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="7zip apache2 bash-completion berkdb bzip2 cli cracklib crypt curl dri exif fortran ftp gpm hardened iconv isdnlog jpeg midi mudflap mysql mysqli ncurses nls openmp pam pcre perl png pppd python readline reflection session slang spell spl ssl tcpd unicode userlocales x86 xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="de en_US en_GB" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 i810 imstt mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LDFLAGS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTDIR_OVERLAY


# emerge -pv bash glibc vim
[ebuild R ] app-shells/bash-3.2_p17-r1 USE="nls -afs -bashlogger -plugins -vanilla" 0 kB
[ebuild R ] sys-libs/glibc-2.5-r4 USE="nls -build -debug -glibc-compat20 -glibc-omitfp (-hardened) (-multilib) -nptl -nptlonly -profile (-selinux)" 0 kB
[ebuild R ] app-editors/vim-7.1.123 USE="bash-completion gpm nls perl python -acl -cscope -minimal -ruby -vim-pager -vim-with-x" 0 kB


Unforunatly I cannot update glibc further, but I have experienced the same error even with fully updated glibc.


Kindly
Dennis

[cjubon]

Does this happen when you open already existing files? Or also, when you're typing into a fresh file?

In the first case, you might want to convert your old files, either using the iconv utility from sys-libs/glibc or app-text/recode, as described in the Gentoo unicode guide, http://www.gentoo.org/doc/en/utf-8.xm, code listing 3.4.
_________________
Mandrake Dec 2001 - Debian "Woody" Aug 2002 - Gentoo Jan 2004

[Snowofice]

This happens as well for existing files, as well for new created ones.

I also already tried to convert my files with iconv, but nothing changed. Even worst, a second A with circumflex showed up, but only in the console output. In vi again it's only one Â.

Kindly
Dennis

[R.Smith]

I've always experienced this, too, when compiling/browsing/viewing text files in TTYs despite following the unicode guide to the letter. Incidentally, by "always" I mean at least the last 3 years. I've never managed to pinpoint the cause, though.

[cjubon]

Strange, I never experienced this. I just tried it out with µ, », ¶ and the like, both in X and on the terminal, no problem. BTW, all these characters are part of Latin1 What's happening with German umlauts or ß, Snowofice? Or with some upper range unicode character such as स (Devanagari "s")?

These questions are just out of interest. Essentially, I'm sorry to be out of advice. I would double check my /etc/rc.conf and perhaps play around with different console fonts, but I guess you did so already...
_________________
Mandrake Dec 2001 - Debian "Woody" Aug 2002 - Gentoo Jan 2004

[Paapaa]

1. Is this in X or in console?
2. Which WM?
_________________
Paludis, the way packages are meant to be managed.

[yngwin]

You need to make sure you use a unicode aware shell (bash or zsh). Also, if this is in X, use a unicode capable terminal (xterm, rxvt-unicode, konsole, ...).
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF

[Snowofice]

Ok, I hereby declare myself for stupid!

One should know the software he uses!

Putty needed to be set to interpret incoming characters as UTF-8!

For everyone else:

Window - Translation - "Recieved data assumed to be in which character set:"
UTF-8
http://the.earth.li/~sgtatham/putty/0.58/htmldoc/Chapter4.html#config-charset

####
Guess the same is valid if you connect via ssh from a computer that is not capable to understand unicode, while your server outputs unicode. In that case you might experience the same!


Ps: It helped a lot to know that my settings seemed to be actually correct. Thx, guys!