GB UTF-8 localisation problems

Gentree · Posted: Tue Apr 28, 2015 1:24 pm Post subject: GB UTF-8 localisation problems

Hi,

this seems to be a recurrent problem and since I'm doing a clean installation, I want it right from the start.

From within the chroot shell:

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

You're asking the wrong question.
The right question is: why are you setting LC_ALL to en_GB ?

Actually, why are you setting it at all ?

Gentree · Posted: Tue Apr 28, 2015 3:56 pm Post subject:

Indeed.

I don't know when it got set and I have found that that was the problem.

I found a couple of places that said not to set it, but nothing saying that once it was set from a command line it got stored somewhere ( where is anyone's guess ) and over-rides all subsequent changes that all the other configs may or may not make.

I spent several hours screwing around with things that had no visible effect ( other than they did change LANG ) because the effects were masked by LC_ALL, that was presumably set manually during initial installation ( though I don't recall that and don't have it in my installation notes ). Well it only takes once apparently and it stuffs all later changes.

I set LC_ALL="" and the other changes were then effective and I got what I wanted to do.

The question remains, where is the definitive Gentoo doc on this . Are we now supposed to rely on wikipedia ?!

There is also the utf8 vs UTF-8 issue that seems very fuzzy and undefined, plus the comments in the header that were actually wrong.

Since correctly setting the locale is a pretty important requirement, this is very poor.
:roll:

Thanks for your reply.
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86

Gentree · Posted: Tue Apr 28, 2015 4:01 pm Post subject:

Fitzcarraldo · Posted: Tue Apr 28, 2015 5:08 pm Post subject:

Gentree · Posted: Tue Apr 28, 2015 5:51 pm Post subject:

Thanks, they are indeed "not bad".

One of those is the target of the redirection that I noted above. It's a wiki. One of the threads here pointed out it was wrong on some point and was told : well it's a user wiki not official.

Is this now the official developers documentation , or a user wiki all we get ?

Under "What are locales?" we find :

VoidMage · Watchman Joined: 14 Oct 2006 Posts: 6196

LC_ALL shouldn't be set at all under normal circumstances - by POSIX specs, it's meant to be master override for locale values.

As for UTF-8/utf8 suffixes they're more or less equivalent - later an alias of the former.

Gentree · Posted: Tue Apr 28, 2015 7:32 pm Post subject:

hey, thanks for the alias thing. It makes sense but where is that from? I've not seen any mention of that in all I've seen so far. Again, I ask : where's the doc?
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

Gentree · Posted: Wed Apr 29, 2015 7:52 am Post subject:

Thanks Charles.

So the wiki is now the official doc, including the errors. Great help.

Odd the way you have both UTF-8 and utf8, that seems to go against what VoidImage suggested above.
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

Seems that all LANG="de_DE.UTF-8" in handbook and https://wiki.gentoo.org/wiki/Localization/Guide are wrong as I am only getting the lower case version.

Gentree · Posted: Wed Apr 29, 2015 9:51 am Post subject:

Above I commented on another user saying that page was wrong. He was also german and I think this may be what he pointed to. I did not write it down.

This whole idea of the offical doc being on a wiki sucks big time. It means there is NO official doc. It's a mash of half educated users gems of what they think they know.

This will end up the way as WonkyPedia. Totally unreliable.

_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86

v_andal · Guru Joined: 26 Aug 2008 Posts: 543 Location: Germany

Gentree · Posted: Thu Apr 30, 2015 7:18 am Post subject:

thanks for your comment.

I was not referring to any "professional" status of devs. My point is that documentation should be written by those who code and implement a system, not by end users.

The UTF-8 vs utf8 thing seems undocumented. I don't see how anyone who has not written the code or spent hours pouring over the source to see how it is done can comment on that in any useful way.

A wiki could be a useful supplement, as are discussions here on the forum but the basic, definitive doc should come from developers, not users.
_________________
Linux, because I'd rather own a free OS than steal one that's not worth paying for.
Gentoo because I'm a masochist
AthlonXP-M on A7N8X. Portage ~x86

v_andal · Guru Joined: 26 Aug 2008 Posts: 543 Location: Germany

Fitzcarraldo · Posted: Thu Apr 30, 2015 12:04 pm Post subject:

Putting aside the issue of documentation per se for a moment, the 'UTF-8' versus 'utf8' issue has always made me wary. I have studiously avoided using 'utf8' or 'utf-8' in any file I create or edit, sticking to the official terminology 'UTF-8' (see Unicode 7.0.0). For example, when I recently installed Gentoo on my new laptop, after seeing that the command 'eselect locale set <n>' resulted in /etc/env.d/02locale containing LANG="en_GB.utf8", I edited the file to make it LANG="en_GB.UTF-8" instead. Interestingly, the result today is as follows (presumably as a consequence of entering 'env-update && source /etc/profile' for the first time after editing /etc/locale.gen and running 'locale-gen' when following the Handbook to install Gentoo):

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

Fitzcarraldo · Posted: Thu Apr 30, 2015 3:41 pm Post subject:

krinn · Watchman Joined: 02 May 2003 Posts: 7470

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

Fitzcarraldo · Posted: Sat May 02, 2015 10:27 am Post subject:

Well, I've been using Gentoo for quite a few years now and it's never caused me any trouble, but, as I wrote in my second post, I personally avoid using 'utf-8' and 'utf8' in any file I create or edit; I studiously stick to 'UTF-8'. Perhaps others who mix them have seen an impact; the first post by Gentree would appear to indicate he has a problem. When I installed Gentoo on my new laptop recently I edited the /etc/env.d/02locale created by the 'eselect locale' command and changed 'utf8' to 'UTF-8' and did not experience the result reported by Gentree in his first post.
_________________
Clevo W230SS: amd64, VIDEO_CARDS="intel modesetting nvidia".
Compal NBLB2: ~amd64, xf86-video-ati. Dual boot Win 7 Pro 64-bit.
OpenRC systemd-utils[udev] elogind KDE on both.
My blog

steveL · Posted: Sat May 02, 2015 11:46 am Post subject:

Yeah, but.. glibc defaults to labelling it "utf8". I agree we should just use "UTF-8" everywhere, fwtw.

Anyhow, here's my /etc/locale.gen:

charles17 · Advocate Joined: 02 Mar 2008 Posts: 3667

steveL · Posted: Sat May 02, 2015 1:00 pm Post subject:

I've never used eselect for locale, and I'm not about to now. (That's what ~/.bashrc is for, if a user wants to override.)

Can't recall the URL that helped the most, but there were several in any case; all from #bash on IRC: chat.freenode.net which is where I first learnt how the locale variables work.

If you want to chat to professional sysadmins, and see what they get up to in scripting, that's the place, ime.

Standard C and POSIX locale handling sure made a lot more sense after, for me.