Saving files as utf-8

fishbone · Posted: Thu Dec 08, 2005 9:28 pm Post subject: Saving files as utf-8

Hi,

I'm going nuts over this localisation stuff.

My system works pretty well with Danish characters. I can pretty much use special Danish characters in file names, console, applications etc.

However, when I edit files for web-use, I'm in trouble. If I open up a file in any editor (say, Gvim or Gnome's own text editor) I can edit the file and everything is well. If I upload the file to a webserver and open up the URL in a browser, special Danish characters are garbled. Why?

My locale settings are:

plasmagunman · l33t Joined: 07 Jun 2002 Posts: 604 Location: berlin

assure that the file was really saved in utf-8, "file -bi <filename>" should tell you the used character set
if you have ssh-access to the server do the same with the file there, assuring nothing changed the encoding while the transfer (don't know if that ever happens...)
check if the character encoding of your browser is set to utf-8, in firefox that's in the "view"-menu

if all these tests are positive i can't help you, sorry.
_________________
please, feel free to correct my english. - por favor, corrige mi español.

LemurFromTheId · n00b Joined: 22 Apr 2005 Posts: 64 Location: Finland

The special Danish characters are most likely garbled because your browser doesn't identify the page's encoding as UTF-8 and falls back to it's or server's default encoding, usually ISO-8859-1 or Windows-1252. (You can confirm this in your browser: in Opera check the Info panel -> Encoding from server - it also tells you if no encoding is supplied at all - and in Firefox right click -> View page info or something... ) There might be two possible reasons:

1) Your web server explicitly tells your browser to use some other encoding.

2) Your page doesn't provide encoding information. If you're playing with HTML, try something like <META http-equiv="Content-Type" content="text/html; charset=UTF-8"> in the Head section. For PHP you need to have this in the very top of the page: <? header("Content-Type: text/html; charset=UTF-8"); ?>.

fishbone · Posted: Wed Dec 14, 2005 8:57 pm Post subject:

Cintra · Advocate Joined: 03 Apr 2004 Posts: 2111 Location: Norway

Hei fishbone

I'm thinking of experimenting with your en_DK.utf8 setup, since there isn't an en_NO.utf8, and the difference between our keyboards is negligible..

I wonder what you have in your /etc/locales.build ?

Mvh
_________________
"I am not bound to please thee with my answers" W.S.

fishbone · Posted: Thu Dec 15, 2005 6:33 pm Post subject:

All I have is:

Cintra · Advocate Joined: 03 Apr 2004 Posts: 2111 Location: Norway

Thanks.. I'm in the middle of re-emerging glibc right now ;-)

Mvh

Edit - its looking good, but I do get a warning when opening tagtool:

fishbone · Posted: Thu Dec 15, 2005 7:57 pm Post subject:

Hmm... I get the same error every time I start an application through a terminal - but I haven't done anything about it and everything seems to work fine.

However, looking into stuff shows that I haven't compiled glibc with the userlocales USE-flag. I wonder if this has something to say. Did you include that flag before emerging glibc?

I'll look some more...
_________________
--
fishbone

Cintra · Advocate Joined: 03 Apr 2004 Posts: 2111 Location: Norway

Yes, but as you say, it doesn't seem to do any damage and at least I'm able to produce genuine utf-8 files now, so I'm not madly worried. I put a post at the end of the UTF-8 Howto thread about this, so hopefully someone has some ideas, though I have a feeling this requires 'upstream' attention..

Its been a good day anyway ;-)

Thanks for your help

Edit: I went back to LC_ALL="en_US.utf8" ripped the Portugese tracks afresh with audiocd:/ - the new files display accented characters correctly & tagtool sees both the files and tags correctly. What a game.. ;-)

_________________
"I am not bound to please thee with my answers" W.S.