man and utf8: problem with accents

[FrancoisVal]

Hello everybody,
I have switched to utf8 on gentoo. Everything seems to work correctly except man pages. I have read the guide of the page http://www.gentoo.org/doc/en/utf-8.xml and modified /etc/man.conf as suggested but it doesn't worked. In console or in a raphic terminal, the result is the same. In fact, all accents and other typical french characters are not displayed correctly.

Does anybody know a solution to the problem ?
Thanks for your help,

François Valenduc
_________________
François Valenduc

[chrroessner]

Check /etc/env.d/70less for the LESSCHARSET option (like in the doc described). env-update and check the /etc/man.conf NROFF thing. This should fix your problems, I hope.

Rössi

[FrancoisVal]

Thanks for your answer.

Unfortunately, that didn't solve the problem. I had already a line export LESSCHARSET=utf-8 in the ZSH config file (/etc/zsh/zshenv) and that doesn' work better. I had read in the Howto that ZSH didn't provide support for utf8 but it doesn't seem to be true anymore. I manage to display and manage utf8 characters correctly with ZSH (in console or in a grahical terminal). I have also tried with bash but I still have problems with manpages. Is it really possible to use man and utf8 correctly when you are native french speaker and want to read french with accents or other charaters (like é, è, à, ç...) ? All these characters are not reproduced correctly in man pages. For example, for "é", I get "é".
_________________
François Valenduc

[ecatmur]

"é" is what you get when you output UTF-8 text to an ISO-8859-1 terminal. Are you sure your terminal is in UTF-8? What happens in bash when you issue

[FrancoisVal]

I have tested and when I type echo $'\xc3\xa9', I get "é" in konsole, xterm or in a text console. If I use bash or zsh, the "é" is displayed correctly. When I use less to view a file, french characters are displayed correctly (in all the situations listed here above). So the problem is really due to man, groff or nroff. As explained in the howto, I have put the following line in the /etc/man.conf:
NROFF /usr/bin/nroff -mandoc -c

Is there a minimal version of man or groff required to use UTF8 encoding correctly ?
Thanks for your help
_________________
François Valenduc

[ecatmur]

The comments in man.conf suggest that in some circumstances nroff will "double convert" to utf8 i.e. it interprets a utf8 stream as iso8859 (latin1) and applies the latin1->utf8 conversion. That would give the results you are experiencing.

Try "/usr/bin/nroff -c -mandoc -Tlatin1".
_________________
No more cruft
dep: Revdeps that work
Using command-line ACCEPT_KEYWORDS?

[neysx]

With a fr_FR.utf8 locale and /usr/bin/nroff -c -mandoc -Tlatin1 in man.conf, it almost works but not quite as some characters are still borked. With /usr/bin/nroff -mandoc -Tlatin1 (remove -c), underlined é are displayed properly, but all à are show as <C3>.
Strangely enough, /usr/share/man/fr/man1/man.1.gz is utf-8 encoded. I am not familiar with man pages and *roff stuff, but it looks like there's some conversion to another encoding (latin?) and then another one back to utf8, and some chars are lost in the process.
If you recode /usr/share/man/fr/man1/man.1.gz to latin:

[FrancoisVal]

So it seems that different man pages have different character encoding. If a man page is encoded with latin9, using nroff -mandoc works even if the selected locale is fr_BE.utf-8 ? That looks me quite strange. Should't we open a bug on bugzilla to ask wheter all man pages should be encoded with a similar encoding ?

I will check on my computer when I am back from my work and see wheter I can make some progress.
Thanks for your help
_________________
François Valenduc

[FrancoisVal]

Indeed, converting the man pages of man to latin9 and using nroff -mandoc works for man and noano. However, I tried the trick with other man pages (ls, mount for example) and I didn't manage to display these pages correctly. Furthermore, if I have to recode all man pages into a suitable encoding, it is going to be very tiresome. I would like to find an easier solution (and not going back to iso8859-1), if possible !

Thanks for your help
_________________
François Valenduc

[buzz22]

Take a look at http://www.haible.de/bruno/packages-groff-utf8.html. It's a groff extension that allows to view UTF-8 encoded man pages.
It works for me. Just replace

[Dominique_71]

[grumblebear]

Sorry for waking up an old thread. But after years of ignoring the garbage displayed in localized man pages on a fully UTF-8 configured Gentoo, I have now found the correct solution.

The line in /etc/man.conf should read

[mike155]

Hi grumblebear, thanks for sharing this! This is indeed the right solution and it works.

I digged a little deeper into this. We live in 2017 and groff still doesn't support utf8 encoded input streams. What a shame! preconv does the right thing. It tries to detect the encoding of the input stream. If it detects utf8, it escapes all non-ASCII characters so that groff can't garble them.

There are other solutions. groff-utf8, for example. But I think the preconv-solution is currently the best we can do.

[charles17]

[grumblebear]

[charles17]