| View previous topic :: View next topic |
| Author |
Message |
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
 Posted: Thu Dec 29, 2016 1:48 pm Post subject: problems with non-english characters[solved] |
 |
|
this is a new gentoo install, any foreign character(accent, umlaut..etc)
for example:(using vim or cat)
<description>Mount a filesystem</description>
<description xml:lang="cs">P��ipojit souborov�� sysm</description>
<description xml:lang="da">Monter et filsystem</description>
<description xml:lang="de">Ein Dateisystem einh��ngen</description>
<description xml:lang="el">���������������������� ������</description>
I have followed the utf8 guide
Available targets for the LANG variable:
[1] C
[2] en_US.utf8 *
[3] POSIX
[ ] (free form)
and reemerged ncurses with unicode flag(if that is necessary, also rebuilt packages linking to ncurses
g50-80 Finntroll # locale
LANG=en_US.utf8
LC_CTYPE="en_US.utf8"
LC_NUMERIC="en_US.utf8"
LC_TIME="en_US.utf8"
LC_COLLATE="en_US.utf8"
LC_MONETARY="en_US.utf8"
LC_MESSAGES="en_US.utf8"
LC_PAPER="en_US.utf8"
LC_NAME="en_US.utf8"
LC_ADDRESS="en_US.utf8"
LC_TELEPHONE="en_US.utf8"
LC_MEASUREMENT="en_US.utf8"
LC_IDENTIFICATION="en_US.utf8"
LC_ALL=
it's also woth noting that thunar can display files just fine while ls cannot
ls:
2004 - Finntroll - Nattf��dd'
2004 - Finntroll - Nattfödd
thanks
Last edited by Adel Ahmed on Thu Jan 26, 2017 6:08 pm; edited 1 time in total |
|
| Back to top |
|
 |
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3664
|
| Posted: Thu Dec 29, 2016 4:17 pm Post subject: |
|
|
| Console or X? |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Thu Dec 29, 2016 5:29 pm Post subject: |
|
|
| both actually |
|
| Back to top |
|
|
Makubo
n00b
Joined: 28 Dec 2016
Posts: 5
Location: Russia, Orenburg/Japan, Sanda
|
| Posted: Thu Dec 29, 2016 8:15 pm Post subject: |
|
|
| Regarding X. Are you installed fonts which supports these languages? I had the same problem with the Japanese and Korean symbols. |
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Fri Dec 30, 2016 12:20 am Post subject: |
|
|
| Makubo wrote: | | Regarding X. Are you installed fonts which supports these languages? I had the same problem with the Japanese and Korean symbols. |
What he said. Even after following the Unicode support instructions, you need to have unicode fonts available which include the characters you want to display.
Also, not all packages will 'fallback' to the same fonts depending on USE flags etc.
In particular, one 'gotcha' is to set an application for a 'sans-serif' font while a different application is set for a 'serif' font from a different font family. The same type problem applies when you set a 'mono-spaced' font for one application while a different application is set to use 'proportional spaced' fonts. Their system default fallback fonts will be complete different.
Your example in particular looks like the results you get with an older font that doesn't include any Unicode characters. i.e. only the first 128 characters exist in the font {a standard ASCII support font for example}.
When I do a new installation for someone, I usually set one of the Android fonts as system default for whatever WM the user wants. They're UTF8 based, legible, have fairly extensive hinting and work well for most people. YMMV and all that.
At the CLI {command line} level, unless you've been really restrictive in USE flags and installed base level packages, the gentoo installed defaults should result in most characters being displayed. I still have a few {old video} files where the characters to be displayed are still a problem, but that's because the file names involved are not in utf8 but rather some obscure European code page promulgated from some early MS Windows installation.
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Fri Dec 30, 2016 7:11 am Post subject: |
|
|
unless you've been really restrictive in USE flags and installed base level packages,
yes that sounds like what I had done, how would i install the fonts now? |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3664
|
| Posted: Fri Dec 30, 2016 8:28 am Post subject: |
|
|
| Adel Ahmed wrote: | | both actually |
For console, have you checked both consolefont and keymaps?
What is in your output if you checked the rc log? | Code: | $ grep -E 'consolefont|keymaps' /var/log/rc.log
consolefont | * Setting console font [lat9w-16] ...
keymaps | * Setting keyboard mode [UTF-8] ...
keymaps | * Loading key mappings [de-latin1] ... |
|
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Fri Dec 30, 2016 12:06 pm Post subject: |
|
|
| I use systemd, and I have found nothing in journalctl pertaining to consolefont or keymaps |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3664
|
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Fri Dec 30, 2016 9:12 pm Post subject: |
|
|
| Adel Ahmed wrote: | | I use systemd, and I have found nothing in journalctl pertaining to consolefont or keymaps |
In reviewing the thread, I'm reminded of something I thought should be mentioned earlier but it slipped my mind while writing my earlier reply.
Your font problems break down into two unrelated and distinctly different problems. They are Console and X windows. Any problems you have with font display issues in Console originate with your kernel. Problems in this area are not 'font' problems but 'codepage' problems. Codepages are language specific character sets originally designed to support non-US English installations of mostly Microsoft DOS, Microsoft Windows and Apple Mac.
All other problems with proper font display are covered by userland programs outside of the kernel. This is what the Gentoo guides cover.
Just to be clear - "Console" means a direct tty login to the command line. i.e. Logging in via CLT-ALT-F2. It does not mean opening a terminal program while in an X windows manager to get to the command line. When you open a terminal window, the displayed font comes from your WM's default system fonts unless overriden in the terminal settings.
In your kernel under File Systems => Native language support, you should check: | Code: | CONFIG_NLS_DEFAULT
The default NLS used when mounting file system. Note, that this is
the NLS used by your console, not the NLS used by a specific file
system (if different) to store data (filenames) on a disk.
...
If you specify a wrong value, it will use the built-in NLS;
compatible with iso8859-1 |
Most normal installs set this to be 'utf8'. If you're not using 'utf8', then you're effectively telling the kernel to only display characters covered by the codepage you specified when you're logged in the Console. All other characters not covered within the codepage specified will be displayed as the diamonds you've been seeing. As kernel 'help' explains, if you have a wrong value, then the codepage the kernel will use is the US ASCII code page.
The example you gave using 'ls' earlier demonstrates this type of mismatch exactly.
The rest of that section in the kernel has to do with which code pages you want to have available to various file systems such as FAT, ext2 etc.
Most default installs for US, Canadian and European users set: | Code: | CONFIG_NLS_CODEPAGE_437
CONFIG_NLS_ASCII
CONFIG_NLS_ISO8859_1
CONFIG_NLS_UTF8 |
From the kernel help for "CONFIG_NLS_UTF8": | Code: | If you want to display filenames with native language characters
from the Microsoft FAT file system family or from JOLIET CD-ROMs
correctly on the screen, you need to include the appropriate
input/output character sets. Say Y here for the UTF-8 encoding of
the Unicode/ISO9646 universal character set. |
The kernel configuration settings mentioned are the miniumum. These are not 'exclusive or' settings. Rather, if you need to be able to properly list a filename orginally created on an Apple HFS filesystem by a typical European user, then you need to also turn on CONFIG_NLS_MAC_ROMAN.
Because I'm very interested in anime, I sometimes run across older filenames originally created on systems using Mircosoft's PacRim codepages. For these, I found it useful to set: | Code: | CONFIG_NLS_CODEPAGE_936 {Simplified Chinese GP936, GB2312}
CONFIG_NLS_CODEPAGE_936 {Traditional Chinese Big5}
CONFIG_NLS_CODEPAGE_932 {Japanese ShiftJIS, EUC-JP}
CONFIG_NLS_CODEPAGE_949 {Korean CP949, EUC-KR} |
Reading the individual help for each code page is informative as to which ones you may need.
I'd like to emphasize something the kernel help tries to make clear. Turning on these codepages are to be able to properly display filenames originally written on filesystems which were configured with specific native language support or to properly display filenames on such an already mounted filesystem. For most people, this means systems with non-US English Microsoft and Apple installs. These codepages have nothing to do with the files' contents.
HTH
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Sat Dec 31, 2016 2:11 am Post subject: |
|
|
well I have the console working now, thanks 
any tips on how I can get X working as well, I havae the dejavu fonts installed if that makes any difference |
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Sat Dec 31, 2016 11:08 am Post subject: |
|
|
| Adel Ahmed wrote: | | well I have the console working now, ... |
I assume this means that your CONFIG_NLS values are now correct. As part of [SOLVED], it's always a good idea to mention what specifically worked.
Regarding your font issues in X: Since you've followed the Guides for Unicode support and for 'locale', I'm assuming that your remaining problems are your selection of system fonts in the WM you're using.
The issue isn't so much as using one font over another, but to use fonts known to have utf8/unicode support. Early versions of fonts, including 'dejavu' 'arial' etc only had support for ASCII type codepage character sets. Later versions of these fonts are converted to supporting the utf8 codepage/unicode characters by extending the number of character glyphs in the font from their respective 127 glyphs to the full utf8 character (several thousands of characters) set. The latest versions of dejavu and arial should have full utf8support.
If you have a favorite font you've downloaded or copied from another system, it's a good bet that font has no utf8 support. A number of specialty fonts available in portage are of this nature. You need to go to the system setting for the WM you use in X and be sure that the system display fonts you've set are known to support Unicode character sets. Fonts in the portage tree known for their good utf8 support include the Android and Ubuntu font sets.
You'll need to emerge these fonts if you want to use them.
It is not enough to install the fonts. You also have to use your WMs "System Settings" facility to select and update the default fonts used by the WM.
I use KDE so under KDE System Settings ==> Fonts --> I have "Droid Sans Mono" and Arulent sans as my two system default font families.
Good luck!
Note: Any packages where you've overridden your system default fonts {like xterm, email, most text editors and web browsers} need to have their preferences revisited and checked to be sure you're using fonts with full utf8 support.
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Sat Dec 31, 2016 11:56 am Post subject: |
|
|
for the terminal, i rebuild all packages linking to ncurses(again), and recompiled the kernel, not sure which step solved this.
for X I emerged the droid fonts, tried them one by one in the xfce terminal, and nope nothing, still cannot read the characters
what was strange was that clementine was able to read those files(but not play them) and the correct characters were displayed
also thunar is able to display the names correctly(regardless of which system font I pick!!
smplayer is not able to play the files(and unable to display namees as well).
this is very strange |
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Sat Dec 31, 2016 1:25 pm Post subject: |
|
|
| Adel Ahmed wrote: | | recompiled the kernel, ... |
This is what fixed your Console problem.
| Adel Ahmed wrote: | for X I emerged the droid fonts, tried them one by one in the xfce terminal, and nope nothing, still cannot read the characters
what was strange was that clementine was able to read those files(but not play them) and the correct characters were displayed
also thunar is able to display the names correctly(regardless of which system font I pick!!
smplayer is not able to play the files(and unable to display namees as well). |
| dufeu wrote: | | It is not enough to install the fonts. You also have to use your WMs "System Settings" facility to select and update the default fonts used by the WM |
You still haven't mentioned which WM {Windows Manager} you're using. If it's KDE, then you need to go to KDE Settings. If it's lxqt or xfce4 then you need to go to the system settings for those. If you're using Gnome, then I don't know where you go because I gave up on Gnome a long time ago.
| dufeu wrote: | | Note: Any packages where you've overridden your system default fonts {like xterm, email, most text editors and web browsers} need to have their preferences revisited and checked to be sure you're using fonts with full utf8 support. |
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Sat Dec 31, 2016 2:56 pm Post subject: |
|
|
I'm using xfce and xfwm as my WM, I've changed the default font multiple times and am getting nothing, I tried all the fonts in the droid package in the xfce terminal,
ie changed to sans systemwide
and in xfce terminal im using monospace
the listing has now changed to:
'2004 - Finntroll - Nattf'$'\303\266''dd'
im using the droid fonts by defaullt and im getting nothing  |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Sun Jan 01, 2017 6:37 pm Post subject: |
|
|
| bump |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Wed Jan 04, 2017 1:35 pm Post subject: |
|
|
updates:
xfce4-terminal:
cannot display files,
can run files in smplayer
thunar:
can display files
cannot run files in smplayer
i switched to twm and the files are dispaying just fine, this is definitely a window manager issue |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Sat Jan 14, 2017 6:57 pm Post subject: |
|
|
| smplayer is still not working when launched from thunar, I've changed the system default font and no dice |
|
| Back to top |
|
|
Adel Ahmed
Veteran
Joined: 21 Sep 2012
Posts: 1523
|
| Posted: Thu Jan 26, 2017 6:08 pm Post subject: |
|
|
slim was starting xfce4 with a non-login shell, I change the line from
exec startxfce4
to
exec /bin/bash -login /usr/bin/startxfce4
and everything is being displayed just fine now.
thanks everyone for your help |
|
| Back to top |
|
|
|
Desktop Environments
