Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
dev-python/docutils Fails to Build: UnicodeDecodeError
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Kenji Miyamoto
Veteran
Veteran


Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

PostPosted: Tue Jun 08, 2010 7:06 pm    Post subject: dev-python/docutils Fails to Build: UnicodeDecodeError Reply with quote

While running a world upgrade, I ran into this error:
Code:
copying docutils/writers/s5_html/themes/small-white/framing.css -> build-3.1/lib/docutils/writers/s5_html/themes/small-white
copying docutils/writers/s5_html/themes/small-white/pretty.css -> build-3.1/lib/docutils/writers/s5_html/themes/small-white
PYTHONPATH=../build-3.1/lib python3.1 ../build-3.1/lib/tools/buildhtml.py --stylesheet-path=../html4css1.css --traceback ..
/// Processing directory: ..
    ::: Processing: BUGS.txt
Traceback (most recent call last):
  File "../build-3.1/lib/tools/buildhtml.py", line 244, in <module>
    Builder().run()
  File "../build-3.1/lib/tools/buildhtml.py", line 186, in run
    self.visit(recurse, root, dirs+files)
  File "../build-3.1/lib/tools/buildhtml.py", line 211, in visit
    prune = self.process_txt(directory, name)
  File "../build-3.1/lib/tools/buildhtml.py", line 237, in process_txt
    settings=settings)
  File "/var/tmp/paludis/dev-python-docutils-0.7_pre6334/work/docutils-0.7_pre6334/build-3.1/lib/docutils/core.py", line 357, in publish_file
    enable_exit_status=enable_exit_status)
  File "/var/tmp/paludis/dev-python-docutils-0.7_pre6334/work/docutils-0.7_pre6334/build-3.1/lib/docutils/core.py", line 641, in publish_programmatically
    output = pub.publish(enable_exit_status=enable_exit_status)
  File "/var/tmp/paludis/dev-python-docutils-0.7_pre6334/work/docutils-0.7_pre6334/build-3.1/lib/docutils/core.py", line 203, in publish
    self.settings)
  File "/var/tmp/paludis/dev-python-docutils-0.7_pre6334/work/docutils-0.7_pre6334/build-3.1/lib/docutils/readers/__init__.py", line 68, in read
    self.input = self.source.read()
  File "/var/tmp/paludis/dev-python-docutils-0.7_pre6334/work/docutils-0.7_pre6334/build-3.1/lib/docutils/io.py", line 258, in read
    data = self.source.read()
  File "/usr/lib64/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 177: ordinal not in range(128)

!!! ERROR in dev-python/docutils-0.7_pre6334::gentoo:
!!! In src_compile at line 6180
!!! buildhtml.py failed
Based on my experience with Python, this seems like an error on upstream's part. Does anybody else have any suggestions?
_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.
Back to top
View user's profile Send private message
savage
Apprentice
Apprentice


Joined: 01 Jan 2003
Posts: 161

PostPosted: Wed Jun 09, 2010 7:10 pm    Post subject: Reply with quote

What solved this problem for me was:

edit /etc/locale.gen:
Code:

en_US ISO-8859-1
en_US.UTF-8 UTF-8


then I ran
Code:
locale-gen


edit /etc/env.d/02.locale:
Code:

LANG="en_US.ISO-8859-1"


then I ran
Code:
env-update
source /etc/profile


then I was able to emerge docutils without the "'ascii' codec can't decode" error
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Thu Jul 01, 2010 12:55 pm    Post subject: Reply with quote

Any ideas on a workaround that doesn't involve changing the system's locale?

Because I don't consider having to switch away from en_US.UTF-8 as an acceptable thing to do to get docutils to build.
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Fri Jul 09, 2010 2:05 pm    Post subject: Reply with quote

Does anyone have any idea what is causing this problem?

It can't just be a compatibility issue between the docutils ebuild and the LANG env setting. Because I have numerous systems, all of which are set to LANG=en_US.UTF-8, and only one of them is bombing out with a UnicodeEncodeError on the docutils-0.6 build.


Here's an emerge info dump:

Code:
Portage 2.1.8.3 (default/linux/x86/10.0/desktop, gcc-4.3.4, glibc-2.11.2-r0, 2.6.31-gentoo-r10-20100225 i686)
=================================================================
System uname: Linux-2.6.31-gentoo-r10-20100225-i686-Intel-R-_Pentium-R-_4_CPU_2.66GHz-with-gentoo-1.12.13
Timestamp of tree: Fri, 09 Jul 2010 07:30:01 +0000
app-shells/bash:     4.0_p37
dev-java/java-config: 2.1.11
dev-lang/python:     2.4.6, 2.5.4-r3, 2.6.5-r2, 3.1.2-r3
dev-util/cmake:      2.6.4-r3
sys-apps/baselayout: 1.12.13
sys-apps/sandbox:    1.6-r2
sys-devel/autoconf:  2.13, 2.65
sys-devel/automake:  1.5, 1.7.9-r1, 1.8.5-r3, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.20.1-r1
sys-devel/gcc:       4.3.4, 4.4.3-r2
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.2.6b
virtual/os-headers:  2.6.30-r1
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-march=pentium4 -pipe -O2"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb /usr/share/config /var/lib/hsqldb"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/eselect/postgresql /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-march=pentium4 -pipe -O2"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests distlocks fixpackages news parallel-fetch protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X a52 aac acl acpi alsa apache2 apm bash-completion berkdb bluetooth branding bzip2 cairo cdr cli consolekit cracklib crypt cups cxx dbus dri dts dvd dvdr emboss encode exif fam flac fortran gd gdbm gif gpm graphviz gtk hal iconv imap imlib java jpeg kerberos lame lcms ldap libg++ libnotify libwww mad maildir mikmod mmx mng modules mozilla mp3 mp4 mpeg mudflap mysql mysqli ncurses nls nptl nptlonly nsplugin ogg openal opengl openmp pam pango pcre pdf perl php png posix postgres ppds pppd python qmail-spp qt3support qt4 readline reflection samba sdl session sharedmem sockets spell spl sse sse2 ssl startup-notification svg sysfs tcpd tiff truetype unicode usb valias vhosts vim-syntax vorbis wxwindows x264 x86 xcb xml xorg xulrunner xv xvid zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="vesa nvidia" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS
Back to top
View user's profile Send private message
Kenji Miyamoto
Veteran
Veteran


Joined: 28 May 2005
Posts: 1452
Location: Looking over your shoulder.

PostPosted: Sat Jul 24, 2010 2:28 am    Post subject: Reply with quote

After reading this, I did a couple of exports for an ISO 8859-1 locale and still got the error:
Code:
PYTHONPATH=../build-3.1/lib python3.1 ../build-3.1/lib/tools/buildhtml.py --stylesheet-path=../html4css1.css --traceback ..
/// Processing directory: ..
    ::: Processing: THANKS.txt
Traceback (most recent call last):
  File "../build-3.1/lib/tools/buildhtml.py", line 244, in <module>
    Builder().run()
  File "../build-3.1/lib/tools/buildhtml.py", line 186, in run
    self.visit(recurse, root, dirs+files)
  File "../build-3.1/lib/tools/buildhtml.py", line 211, in visit
    prune = self.process_txt(directory, name)
  File "../build-3.1/lib/tools/buildhtml.py", line 237, in process_txt
    settings=settings)
  File "/var/tmp/paludis/dev-python-docutils-0.7/work/docutils-0.7/build-3.1/lib/docutils/core.py", line 357, in publish_file
    enable_exit_status=enable_exit_status)
  File "/var/tmp/paludis/dev-python-docutils-0.7/work/docutils-0.7/build-3.1/lib/docutils/core.py", line 641, in publish_programmatically
    output = pub.publish(enable_exit_status=enable_exit_status)
  File "/var/tmp/paludis/dev-python-docutils-0.7/work/docutils-0.7/build-3.1/lib/docutils/core.py", line 203, in publish
    self.settings)
  File "/var/tmp/paludis/dev-python-docutils-0.7/work/docutils-0.7/build-3.1/lib/docutils/readers/__init__.py", line 68, in read
    self.input = self.source.read()
  File "/var/tmp/paludis/dev-python-docutils-0.7/work/docutils-0.7/build-3.1/lib/docutils/io.py", line 258, in read
    data = self.source.read()
  File "/usr/lib64/python3.1/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1546: ordinal not in range(128)

!!! ERROR in dev-python/docutils-0.7::gentoo:
!!! In src_compile at line 6290
!!! buildhtml.py failed

!!! Call stack:
!!!    * src_compile (/var/tmp/paludis/dev-python-docutils-0.7/temp/loadsaveenv:6290)
!!!    * ebuild_f_compile (/usr/libexec/paludis/2/src_compile.bash:56)
!!!    * ebuild_main (/usr/libexec/paludis/ebuild.bash:647)
!!!    * main (/usr/libexec/paludis/ebuild.bash:675)

diefunc: making ebuild PID 6958 exit with error
die trap: exiting with error.

Install error:
  * In program paludis --dl-upgrade as-needed -i docutils:
  * When performing install action from command line:
  * When executing install task:
  * When installing 'dev-python/docutils-0.7:0::gentoo' replacing { 'dev-python/docutils-0.6:0::installed' }:
  * When running an ebuild command on 'dev-python/docutils-0.7:0::gentoo':
  * Install failed for 'dev-python/docutils-0.7:0::gentoo'


*** Messages log: /var/log/paludis/1279938420-install-dev-python_docutils-0.7:0::gentoo.messages
*** Output log: /var/log/paludis/1279938420-install-dev-python_docutils-0.7:0::gentoo.out

_________________
[ Kawa-kun, new and improved!! ]

Alex Libman seems to be more of an anarchist than a libertarian.
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Wed Jul 28, 2010 2:49 pm    Post subject: Reply with quote

Not sure if it's related, but emerge says some interesting things on the machine exhibiting this problem.

If I'm updating world, emerge says the following about docutils:

Code:
!!! The following installed packages are masked:
- dev-python/docutils-0.6 (masked by: missing keyword)


missing? If I ask emerge about docutils itself, I get:

Code:
# emerge -vt --pretend docutils

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild   R   ] dev-python/docutils-0.6  USE="-emacs% -glep% (-iffon1%) (-le%)" 1,337 kB [?=>0]

Total: 1 package (1 reinstall), Size of downloads: 1,337 kB
Portage tree and overlays:
 [0] /usr/portage
 [?] indicates that the source repository could not be determined


Note the weird use flags. Compare to this, on a machine that isn't exhibiting the docutils unicode build error:

Code:
# emerge -vt --pretend docutils

These are the packages that would be merged, in reverse order:

Calculating dependencies... done!
[ebuild   R   ] dev-python/docutils-0.6  USE="-emacs -glep" 0 kB

Total: 1 package (1 reinstall), Size of downloads: 0 kB
Back to top
View user's profile Send private message
jancici
Apprentice
Apprentice


Joined: 27 Jan 2004
Posts: 284
Location: Slovakia

PostPosted: Thu Aug 19, 2010 7:23 pm    Post subject: Reply with quote

has anybody solution to fix this build ?
Back to top
View user's profile Send private message
Moriah
Advocate
Advocate


Joined: 27 Mar 2004
Posts: 2117
Location: Kentucky

PostPosted: Sat Aug 21, 2010 3:43 am    Post subject: Reply with quote

Ditto here. I am seeing this on only 2 out of 12 machines, both 32 bit boxes running 2.6.31-gentoo-r6 on i686 Intel(R) Celeron(R) CPU's.
_________________
The MyWord KJV Bible tool is at http://www.elilabs.com/~myword

Foghorn Leghorn is a Warner Bros. cartoon character.
Back to top
View user's profile Send private message
Dheath
Tux's lil' helper
Tux's lil' helper


Joined: 06 Aug 2006
Posts: 131

PostPosted: Sat Aug 21, 2010 11:09 am    Post subject: Reply with quote

See bug 333361.
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Mon Aug 23, 2010 6:56 pm    Post subject: Reply with quote

Sounds like a fix is winding its way down to us.

But just in case, here's a bit more evidence from ground zero. When I emerge anything, portage makes the following complaint at the end:

Code:
>>> Auto-cleaning packages...
portage: COUNTER for dev-python/docutils-0.6 was corrupted; resetting to value of 0
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Tue Aug 31, 2010 4:20 pm    Post subject: Reply with quote

How am I supposed to recover from this problem?

docutils-0.7 is stable, but I can't even upgrade because unmerging 0.6 bombs out with the unicode error.
Back to top
View user's profile Send private message
dahoste
Tux's lil' helper
Tux's lil' helper


Joined: 01 Dec 2005
Posts: 138
Location: Maryland, USA

PostPosted: Tue Aug 31, 2010 7:46 pm    Post subject: Reply with quote

I fixed the problem by just completely deleting:

Code:
/var/db/pkg/dev-python/docutils-0.6/


And then emerging docutils, which pulled in 0.7. No problem.

This leads me to believe that somehow the portage pkg db had gotten corrupted, or some bad locale state had gotten stuck into the db files (if indeed it was ever really locale-related).

Stomping on a directory like that probably isn't kosher, but it worked and things were good and broke otherwise.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum