| View previous topic :: View next topic |
| Author |
Message |
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
 Posted: Sat May 19, 2012 4:24 pm Post subject: Python upgrades failed today [SOLVED] |
 |
|
Today I had upgrades to both python-2.7 and python 3.2. On most systems all went well, on one system both failed.
A few lines out of "/var/log/portage/dev-lang:python-2.7.3-r1:20120519-121536.log"
| Code: | Python build finished, but the necessary bits to build these modules were not found:
bsddb185 dl imageop
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Failed to build these modules:
_elementtree pyexpat
|
And from "/var/log/portage/dev-lang:python-3.2.3:20120519-122756.log"
| Code: | Failed to build these modules:
pyexpat
|
I suspect that the second error is a subset of the first - I've really done nothing with python-3, so it's just there because Gentoo wants it to be. I've looked in setup.py as suggested, but it really doesn't help me. It appears that I'm obviously missing something, but I'm not sure what. I've got db installed, but it's db-4.8.something, so I presume the bsddb185 is some sort of virtual equivalence, and same for the rest.
Suggestions?
[SOLVED]
As suggested by Hu, I did indeed have some nasty stuff in /usr/local/lib*, and that was killing my python upgrades. I moved it temporarily aside so the upgrade could proceed, and for now have moved it back. Hu suggests that I could run the binary-only software that requires those libs in a partial chroot, to get around this problem in the future. I'll look at it, but for now am simply forewarned.
_________________
.sigs waste space and bandwidth
Last edited by depontius on Wed May 23, 2012 7:23 pm; edited 1 time in total |
|
| Back to top |
|
 |
teika
n00b
Joined: 19 Feb 2011
Posts: 54
Location: japan
|
| Posted: Sun May 20, 2012 1:12 am Post subject: |
|
|
Hi. Probably these messages have nothing to do with your build failure. My python-2.7 build log has these lines too, and it built successfully. You've got to spot the real error messages. (My 3.2 build log is lost. ;-)
_________________
Easy Shift / Ctrl / AltGr ... hack; save your pinkies, type without drudge: topic 865313 |
|
| Back to top |
|
|
VoidMage
Advocate
Joined: 14 Oct 2006
Posts: 4809
|
| Posted: Sun May 20, 2012 1:58 am Post subject: |
|
|
| depontius:, scroll a few lines back till the place pyexpat should have been built and search for the error there (_elementtree was most likely skipped, cause the former failed). |
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
| Posted: Sun May 20, 2012 2:59 am Post subject: |
|
|
The _elementree/pyexpat message was in what I posted. Looking back up the log file:
| Code: | building 'pyexpat' extension
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=native -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3 -c /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.c -o build/temp.linux-x86_64-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.o
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -march=native -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include build/temp.linux-x86_64-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.o -L/usr/local/lib -L. -lexpat -lpython2.7 -o build/lib.linux-x86_64-2.7/pyexpat.so
*** WARNING: renaming "pyexpat" since importing it failed: build/lib.linux-x86_64-2.7/pyexpat.so: undefined symbol: XML_SetHashSalt
building '_elementtree' extension
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=native -pipe -fwrapv -DNDEBUG -DUSE_PYEXPAT_CAPI -I. -IInclude -I./Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3 -c /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.c -o build/temp.linux-x86_64-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.o
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -march=native -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include build/temp.linux-x86_64-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.o -L/usr/local/lib -L. -lexpat -lpython2.7 -o build/lib.linux-x86_64-2.7/_elementtree.so
*** WARNING: renaming "_elementtree" since importing it failed: PyCapsule_Import could not import module "pyexpat" |
So this tells me that the problem is building pyexpat, but that appears to be an internal part of python, not a package.
A quick search, and this seems to be a common problem. A couple of thoughts... I last upgraded expat on April 21, but I also upgraded dev-python/lxml later in this session. I suspect that tomorrow I should try this upgrade again - maybe this is a dependency order problem?
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
VoidMage
Advocate
Joined: 14 Oct 2006
Posts: 4809
|
| Posted: Sun May 20, 2012 3:30 am Post subject: |
|
|
Not quite: XML_SetHashSalt was added in expat 2.1.0.
Thank you.  |
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
| Posted: Sun May 20, 2012 12:31 pm Post subject: |
|
|
Turned out that the emerge failure was on a different system, one that runs overnight. (my mythbackend) So I kicked off the emerge last night before heading to bed. Failed again.
But that system does have expat-2.1.0 on it, and apparently that must have been back on April 19, so now I'm still not understanding why this fails.
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
| Posted: Tue May 22, 2012 5:42 pm Post subject: |
|
|
I also tried manually installing elemettree, and python upgrades still fail.
I have a second system failing python upgrades now, an i686 at work, for the same reasons. The other day I tried installing dev-python/bsddb3, which didn't help. Today I installed elemettree here, and am rerunning. There are many upgrades happening, so I don't know when it will get back to the python packages. I seriously doubt that the combination of the 2 packages will do any good.
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
VoidMage
Advocate
Joined: 14 Oct 2006
Posts: 4809
|
| Posted: Tue May 22, 2012 6:42 pm Post subject: |
|
|
| I'm getting a feeling the problem lies along the lines of /usr/local/lib pollution. |
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
| Posted: Tue May 22, 2012 7:44 pm Post subject: |
|
|
| VoidMage wrote: | | I'm getting a feeling the problem lies along the lines of /usr/local/lib pollution. |
That's a very real possibility. I have had to keep a rather ugly /usr/local/lib on both systems, in order to run binaries from my employer. I'll move it out of the way temporarily and try a rebuild tonight.
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
depontius
Advocate
Joined: 05 May 2004
Posts: 2156
|
| Posted: Tue May 22, 2012 11:15 pm Post subject: |
|
|
That was it, at least for my system at home. I haven't move the /usr/local/lib* files back yet, but both pythons installed cleanly. My (ancient and slow i686) system at work was still building QT updates when I left today, but tomorrow I'll try the same fix, and mark this topic appropriately.
So I have to go off on a minor diatribe about "the platform". That's really the cause of my problems, because I had binary-only code from my employer, coded to "the platform", which so frequently means "in the most brittle fashion possible, may count of side-effects for proper operation, and has its best shot at working properly with only specific versions of every solib it uses." In the name of "low maintenance" they create a maintenance nightmare.
The philosophical opposite of Gentoo.
By the way, thanks for the suggestion.
_________________
.sigs waste space and bandwidth |
|
| Back to top |
|
|
Hu
Watchman
Joined: 06 Mar 2007
Posts: 7607
|
| Posted: Wed May 23, 2012 2:06 am Post subject: |
|
|
| If you only need to run a few programs with the custom libraries, you might be better served creating a partial chroot. Place the custom libraries in a non-standard location, such as /usr/local/platform-hack/usr/local/lib. Then, when you need to run those programs, run them in a chroot with bind mounts to provide /, /var, etc. This will avoid creating copies of any system managed libraries, let you run the fragile programs in a layout that they expect, and hide the custom libraries from the rest of the system. |
|
| Back to top |
|
|
RAPHEAD
Tux's lil' helper
Joined: 20 Jun 2003
Posts: 85
|
| Posted: Sun May 27, 2012 4:31 pm Post subject: |
|
|
Same Problem here:
86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -$
building 'ossaudiodev' extension
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I. -IInclude -I./Include -I/var/tmp/porta$
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -$
building '_ctypes' extension
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I/usr/lib64/libffi-3.0.11/include -I. -II$
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I/usr/lib64/libffi-3.0.11/include -I. -II$
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I/usr/lib64/libffi-3.0.11/include -I. -II$
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I/usr/lib64/libffi-3.0.11/include -I. -II$
x86_64-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -fwrapv -DNDEBUG -I/usr/lib64/libffi-3.0.11/include -I. -II$
x86_64-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -pipe -march=core2 -fomit-frame-pointer -$
Python build finished, but the necessary bits to build these modules were not found:
bsddb185 dl imageop
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Failed to build these modules:
_elementtree pyexpat
running build_scripts
creating build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/pydoc -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/idle -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/2to3 -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Lib/smtpd.py -> build/scripts-2.7
changing mode of build/scripts-2.7/pydoc from 644 to 755
changing mode of build/scripts-2.7/idle from 644 to 755
changing mode of build/scripts-2.7/2to3 from 644 to 755
changing mode of build/scripts-2.7/smtpd.py from 644 to 755
make: *** [sharedmods] Fehler 1
emake failed
Shouldn't this be reported as a bug rather than fixing it with complicated workarounds? |
|
| Back to top |
|
|
Hu
Watchman
Joined: 06 Mar 2007
Posts: 7607
|
| Posted: Sun May 27, 2012 4:45 pm Post subject: |
|
|
| RAPHEAD wrote: | | Shouldn't this be reported as a bug rather than fixing it with complicated workarounds? |
If there is a bug here, then by all means, please report it. The failure that depontius encountered is not a bug. It is a negative consequence of polluting the system with outdated and unmanaged files. He had a good reason for doing that, but since that is not an expected or supported configuration, there is nothing to be done there. It is an expected behavior that the compiler will search /usr/local. The only things Portage could do to protect you are to hide /usr/local from the build process or to warn you that you have files in /usr/local. |
|
| Back to top |
|
|
shur
Tux's lil' helper
Joined: 08 Aug 2004
Posts: 95
|
| Posted: Mon Jun 11, 2012 1:39 pm Post subject: |
|
|
I am having a similar problem with Python 2.7.3. I moved all files from /usr/local/lib and /usr/local/include
to my home directory, where hopefully they won't be used in the build. I also rebuilt all python dependencies (emerge -ev). Any suggestions?
Relevant cut and pastes follow :
[ebuild U ] dev-lang/python-2.7.3-r1 [2.7.2-r3] USE="gdbm ipv6 ncurses readline sqlite* ssl threads tk (wide-unicode) xml -berkdb -build -doc -examples -wininst" 0 kB
***** compiles normally up to here*******
Python build finished, but the necessary bits to build these modules were not found:
bsddb185 sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Failed to build these modules:
_elementtree pyexpat
running build_scripts
creating build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/pydoc -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/idle -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Tools/scripts/2to3 -> build/scripts-2.7
copying and adjusting /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Lib/smtpd.py -> build/scripts-2.7
changing mode of build/scripts-2.7/pydoc from 644 to 755
changing mode of build/scripts-2.7/idle from 644 to 755
changing mode of build/scripts-2.7/2to3 from 644 to 755
changing mode of build/scripts-2.7/smtpd.py from 644 to 755
make: *** [sharedmods] Error 1
emake failed
* ERROR: dev-lang/python-2.7.3-r1 failed (compile phase):
* emake failed |
|
| Back to top |
|
|
shur
Tux's lil' helper
Joined: 08 Aug 2004
Posts: 95
|
| Posted: Mon Jun 11, 2012 2:41 pm Post subject: |
|
|
getting a bit deeper into the compile log I found this:
building 'pyexpat' extension
i686-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=i686 -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include -I/usr/local/include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3 -c /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.c -o build/temp.linux-i686-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.o
i686-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -march=i686 -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include build/temp.linux-i686-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/pyexpat.o -L/usr/local/lib -L. -lexpat -lpython2.7 -o build/lib.linux-i686-2.7/pyexpat.so
*** WARNING: renaming "pyexpat" since importing it failed: build/lib.linux-i686-2.7/pyexpat.so: undefined symbol: XML_SetHashSalt
building '_elementtree' extension
i686-pc-linux-gnu-gcc -pthread -fPIC -fno-strict-aliasing -O2 -march=i686 -pipe -fwrapv -DNDEBUG -DUSE_PYEXPAT_CAPI -I. -IInclude -I./Include -I/usr/local/include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Include -I/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3 -c /var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.c -o build/temp.linux-i686-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.o
i686-pc-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,--as-needed -L. -Wl,-O1 -Wl,--as-needed -L. -fno-strict-aliasing -O2 -march=i686 -pipe -fwrapv -DNDEBUG -I. -IInclude -I./Include build/temp.linux-i686-2.7/var/tmp/portage/dev-lang/python-2.7.3-r1/work/Python-2.7.3/Modules/_elementtree.o -L/usr/local/lib -L. -lexpat -lpython2.7 -o build/lib.linux-i686-2.7/_elementtree.so
*** WARNING: renaming "_elementtree" since importing it failed: PyCapsule_Import could not import module "pyexpat"
***********
The problem seems to be related to undefined symbol: "XML_SetHashSalt" But I do have expat-2.1.0 installed. And no lib or include files in /usr/local.
One weird thing I noticed. The libexpat files in my /usr/lib directory are version 1.6.0 and they were just installed last night, seemingly from the ebuild for expat-2.1.0! Indeed that is the case... I did a reinstall and that is exactly what it did.
Googling led to the discovery that a very similar problem has been reported as a bug (and purportedly fixed) on Debian
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=665346 |
|
| Back to top |
|
|
shur
Tux's lil' helper
Joined: 08 Aug 2004
Posts: 95
|
| Posted: Mon Jun 11, 2012 3:32 pm Post subject: Solved |
|
|
something interesting... there were expat.h and expat_extern.h from a previous version (installed in 2010) in the /include directory. The current version installs these files in the /usr/include directory, and the old ones were obviously not removed. Likewise there were libexpat 1.5.2 files in the /lib directory while the current version 1.6.0 installs to /usr/lib.
Removing these old files solved the problem.
The numbering discrepancy between the expat distribution and the libexpat files occurs in the SourceForge tarball. I don't know why they did it, but they did, and anyway this was not the problem. |
|
| Back to top |
|
|
|
Portage & Programming
