trying to upgrade to kdelibs-3.5.7-r2: libexpat problems

Message

BitJam · Post by **BitJam** » Sat Aug 11, 2007 3:26 am

I'm running a stable amd64 system that has never been more than a week out of date.

I did an emerge sync and an emerge -av -uDN world. There were 150 new packages ready, mostly kde stuff. There were 3 blockers: kdelibs, kdesktop, and one other. I dropped down to the console and turned off X to give myself plenty of RAM. I unmerged the three blockers and and did an emerge -uDN world.

Emerging kdelibs bombed out with a message saying it couldn't find a version of QT >= 3.3 and < 4.0 even though I had the latest stable qt-3.3.8-r2 installed. I tried reinstalling QT and qt-4.3 installed but was of no use. I tried to re-install qt-3.3.8-r2 but it complained about missing libexpat. I re-emerged expat but this didn't solve anything. I got the same QT version error when I tried to re-emerge the version of kdelibs I had just unmerged. Revdep-rebuild spends about 15 minutes telling me almost every package on my system (including all of the old KDE packages) are missing libexpat.

I want my money back! Oh wait ... it was all free.

I'm posting this from Ubuntu. I can get to the command line on the Gentoo system but KDE was my only DE. Here is my emerge --info:

Code: Select all


 Portage 2.1.2.11 (default-linux/amd64/2007.0, gcc-4.1.2, glibc-2.5-r4, 2.6.20-gentoo-r7 x86_64)
=================================================================
System uname: 2.6.20-gentoo-r7 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 3600+
Gentoo Base System release 1.12.9
Timestamp of tree: Sat, 11 Aug 2007 01:30:01 +0000
ccache version 2.4 [enabled]
dev-java/java-config: 1.3.7, 2.0.33-r1
dev-lang/python:     2.4.4-r4
dev-python/pycrypto: 2.0.1-r6
dev-util/ccache:     2.4-r7
sys-apps/sandbox:    1.2.17
sys-devel/autoconf:  2.13, 2.61
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10
sys-devel/binutils:  2.17
sys-devel/gcc-config: 1.3.16
sys-devel/libtool:   1.5.23b
virtual/os-headers:  2.6.21
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=athlon64 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/gconf /etc/revdep-rebuild /etc/splash /etc/terminfo"
CXXFLAGS="-march=athlon64 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="ccache distlocks sandbox sfperms strict"
GENTOO_MIRRORS="http://cudlug.cudenver.edu/gentoo/ http://mirror.espri.arizona.edu/gentoo/ "
LANG="en_US.utf8"
LC_ALL=""
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --delete-after --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --filter=H_**/files/digest-*"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext X a52 aac acpi alsa amd64 apache2 bash-completion berkdb bitmap-fonts cddb cdparanoia cdr cli cracklib crypt cups dbus dri dvd dvdr dvdread encode exif fbcon ffmpeg firefox flac fortran gdbm gif gpm hal iconv ieee1394 imagemagick isdnlog java jpeg kde kdeenablefinal kdexdeltas libwww lm_sensors mad midi mmx mmxext mp3 mpeg mplayer mudflap musicbrainz ncurses nls nptl nptlonly nsplugin ogg opengl openmp pam pcre pdf perl plotutils png ppds pppd python qt3 quicktime readline reflection ruby session sockets spell spl sse sse2 ssl symlink tcltk tcpd tetex tidy tk truetype truetype-fonts type1-fonts unicode usb vim-pager vim-syntax vorbis xine xinerama xorg xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="nvidia"
Unset:  CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

roderick · Post by **roderick** » Sat Aug 11, 2007 3:55 am

You need to do a revdep-rebuild and then resume your emerge.

Some of the packages are in a broken state and need to be rebuilt against the correct version of libexpat. revdep-rebuild is part of the gentoolkit package. Emerge it and run the revdep-rebuild command.

This should fix things up.

didymos · Post by **didymos** » Sat Aug 11, 2007 4:11 am

I'm guessing you just updated to >=expat-2.0.0. I've been avoiding it precisely for this reason, and I'm not normally averse to running unstable (or even unkeyworded) packages. Still, I'm tempted to just get it over with.

BitJam · Post by **BitJam** » Sat Aug 11, 2007 4:28 am

revdep-rebuild fails in several ways. First, there is a conflict between two versions of kdelibs. I masked the newer version (since revdep-rebuild wants to rebuild ALL my old kde packages).

But revdep-rebuild then failed on the very first package, kdesktop, with libexpat errors.

Any suggestions on what version of expat I should try downgrading to?

EDIT: I downgraded to expat-1.95.8 and now things are starting to compile. Revdep-rebuild wants to rebuild all of my old kde so I will let it run overnight.

didymos · Post by **didymos** » Sat Aug 11, 2007 4:48 am

Seems kind of wasteful. Are you just planning to stick to the downgraded expat? I'd have just removed KDE and let revdep-rebuild fix the expat problems with non-KDE stuff, then re-emerge KDE.

roderick · Post by **roderick** » Sat Aug 11, 2007 4:55 am

If you run 2.0.0 expat you HAVE to rebuild a lot of KDE. version 2.0.0 of expat breaks the ABI and therefore everything which depends on it.

I'm pretty sure once kde 3.5.7 is all stablized (any day now for x86), expat 2.0.1, curl, and a couple of other packages will be stable as well and you will be faced with the inevitable rebuild all anyway.. might as well figure out the problem with the blocked kdelibs and move forward

nixnut · Post by **nixnut** » Sat Aug 11, 2007 12:16 pm

if you specify the -X option of revdep-rebuild it should take the latest available version and not merely rebuild the version that is actually installed on your system at that moment.

didymos · Post by **didymos** » Sat Aug 11, 2007 12:27 pm

nixnut wrote:if you specify the -X option of revdep-rebuild it should take the latest available version and not merely rebuild the version that is actually installed on your system at that moment.

I should have known that by now. Well, at least I never ended up having to follow my own advice on this.

Polynomial-C · Post by **Polynomial-C** » Sat Aug 11, 2007 1:34 pm

Unfortunately revdep-rebuild -X doesn't always do the trick...
On my amd64 server I used the -X switch and revdep-rebuild built dev-libs/apr{,util}-1.2.8 instead of dev-libs/apr{,util}-0.9.12 which left apache unuseable as apr is needed for startup of apache and the used apr-0.9.12 still linked to the old expat lib...

BitJam · Post by **BitJam** » Sat Aug 11, 2007 2:24 pm

Thanks for the help and suggestions.

Revdep-rebuild bombed out halfway through in the middle of the night (missing berkdb) but I was able to get KDE started anyway, probably because downgrading expat fixed most of the link problems revdep-rebuild was going to fix.

FWIW: I have been running an up-to-date stable system (not ~) system. Expat and KDE had both gone stable but the emerge went horribly, horribly wrong.

rickythesk8r · Post by **rickythesk8r** » Sat Aug 11, 2007 2:26 pm

roderick wrote:You need to do a revdep-rebuild and then resume your emerge.

Some of the packages are in a broken state and need to be rebuilt against the correct version of libexpat. revdep-rebuild is part of the gentoolkit package. Emerge it and run the revdep-rebuild command.

This should fix things up.

It doesn't, unfortunately

The first package in my revdep-rebuild list is pango. The ebuild looks for libexpat.so.0 which I surmise doesn't exist anymore.

Code: Select all

Writing a pango.modules file to use with tests/examples.
/var/tmp/portage/x11-libs/pango-1.16.4/work/pango-1.16.4/pango/.libs/lt-pango-querymodules: error while loading shared libraries: libexpat.so.0: cannot open shared object file: No such file or directory
make[3]: *** [pango.modules] Error 127
make[3]: Leaving directory `/var/tmp/portage/x11-libs/pango-1.16.4/work/pango-1.16.4/modules'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/var/tmp/portage/x11-libs/pango-1.16.4/work/pango-1.16.4/modules'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/tmp/portage/x11-libs/pango-1.16.4/work/pango-1.16.4'
make: *** [all] Error 2
 * 
 * ERROR: x11-libs/pango-1.16.4 failed.
 * Call stack:
 *   ebuild.sh, line 1654:   Called dyn_compile
 *   ebuild.sh, line 990:   Called qa_call 'src_compile'
 *   ebuild.sh, line 44:   Called src_compile
 *   ebuild.sh, line 1334:   Called gnome2_src_compile
 *   gnome2.eclass, line 71:   Called die
 * 
 * compile failure
 * If you need support, post the topmost build error, and the call stack if relevant.
 * A complete build log is located at '/var/tmp/portage/x11-libs/pango-1.16.4/temp/build.log'.

I have dozens of broken packages now.

Edit: emerge '<dev-libs/expat-2.0.1' has temporarily set matters right. I wonder how one will know it is safe to allow portage to upgrade expat without examining individual ebuilds that might be affected.

BitJam · Post by **BitJam** » Sat Aug 11, 2007 2:54 pm

I downgraded to expat-1.95.8 and this helped get past the libexpat errors. IMO there is something about expat and portage that is horribly broken ATM.

nixnut · Post by **nixnut** » Sat Aug 11, 2007 3:12 pm

expat is a major pita, but there's not much portage can do about it. revdep-rebuild -X followed by revdep-rebuild (for older slotted versions) should take care of all the breakage, but no guarantees (unless you show us your support contract

)

BitJam · Post by **BitJam** » Sat Aug 11, 2007 3:23 pm

If you read my previous posts in this thread, you will see it didn't work for me. No emerge requiring libexpat worked with the latest stable expat-2.0.1. Revdep-rebuild would spend a very long time coming up with a list of 160 packages that needed libexpat and then the very first emerge would fail with a libexpat error.

I had unmerged kdelibs because it was blocking the new kde stuff and I was unable to emerge any version of kdelibs with the new expat. I was dead in the water until I downgraded expat.

rickythesk8r · Post by **rickythesk8r** » Sat Aug 11, 2007 3:51 pm

BitJam wrote:If you read my previous posts in this thread, you will see it didn't work for me. No emerge requiring libexpat worked with the latest stable expat-2.0.1. Revdep-rebuild would spend a very long time coming up with a list of 160 packages that needed libexpat and then the very first emerge would fail with a libexpat error.

I had unmerged kdelibs because it was blocking the new kde stuff and I was unable to emerge any version of kdelibs with the new expat. I was dead in the water until I downgraded expat.

I thought the whole purpose behind the versioning system of shared objects (in Unix generally, not just Linux or Gentoo) was to allow multiple versions to coexist. Somehow the ebuild of expat-2-0.1 takes it upon itself to remove prior versions upon which other executables, and other ebuilds, depend.

BitJam · Post by **BitJam** » Sat Aug 11, 2007 4:03 pm

My guess is that expat wasn't slotted which would make it a Gentoo problem not an expat problem.

Carlo · Post by **Carlo** » Sat Aug 11, 2007 4:07 pm

BitJam wrote:I downgraded to expat-1.95.8 and this helped get past the libexpat errors. IMO there is something about expat and portage that is horribly broken ATM.

It is what you always have to expect, when using a distro building from source and not some easy binary distro taking care for such incompatibilities for you. Even if Portage had reverse dependency support built in, the rebuild orgy is inevitable (but less painful having built your system with the unsupported --as-needed linker flag).

rickythesk8r wrote:I thought the whole purpose behind the versioning system of shared objects (in Unix generally, not just Linux or Gentoo) was to allow multiple versions to coexist.

It's not that simple. There can be issues with other files (headers, config, etc.) which cannot be installed side by side or even odd runtime issues of packages depending on each other built against different incompatible library versions. Also retaining the older libs is security-wise problematic and raises maintenance costs.

BitJam wrote:My guess is that expat wasn't slotted which would make it a Gentoo problem not an expat problem.

No. Slotting isn't your saviour. It's a tool when libs are meant to live side by side, not when one should replace the other.

rickythesk8r · Post by **rickythesk8r** » Sat Aug 11, 2007 4:26 pm

Carlo wrote:
BitJam wrote:I downgraded to expat-1.95.8 and this helped get past the libexpat errors. IMO there is something about expat and portage that is horribly broken ATM.
It is what you always have to expect, when using a distro building from source and not some easy binary distro taking care for such incompatibilities for you. Even if Portage had reverse dependency support built in, the rebuild orgy is inevitable (but less painful having built your system with the unsupported --as-needed linker flag).

rickythesk8r wrote:I thought the whole purpose behind the versioning system of shared objects (in Unix generally, not just Linux or Gentoo) was to allow multiple versions to coexist.
It's not that simple. There can be issues with other files (headers, config, etc.) which cannot be installed side by side or even odd runtime issues of packages depending on each other built against different incompatible library versions. Also retaining the older libs is security-wise problematic and raises maintenance costs.

BitJam wrote:My guess is that expat wasn't slotted which would make it a Gentoo problem not an expat problem.
No. Slotting isn't your saviour. It's a tool when libs are meant to live side by side, not when one should replace the other.

Now we're going in circles.

Nothing in the foregoing addresses the question: How do you rebuild? orgiastically

or otherwise?

The libpango ebuild is broken because it insists on libexpat.so.0 and that .so is no longer available. Every ebuild will have to be examined by hand for compatibility with the expat upgrade.

BitJam · Post by **BitJam** » Sat Aug 11, 2007 5:10 pm

Carlo wrote:
BitJam wrote:I downgraded to expat-1.95.8 and this helped get past the libexpat errors.

BitJam wrote:My guess is that expat wasn't slotted which would make it a Gentoo problem not an expat problem.
No. Slotting isn't your saviour. It's a tool when libs are meant to live side by side, not when one should replace the other.

I didn't say that expat should be slotted. I said it was a Gentoo problem and not an expat problem. Although IMO, expat should be slotted because the new version breaks just about everything. By "breaks" I don't mean a revdep-rebuild is required, I mean emerge (and hence revdep-rebuild) fail.

Isn't this exactly what the slots are for? When some packages need a new version of a library that will break most of the other packages in Portage? The core problem is that most packages won't build with the new expat. I think it needs to get ~ masked until either this is fixed or it gets slotted.

Polynomial-C · Post by **Polynomial-C** » Sat Aug 11, 2007 5:36 pm

Sorry, it's not that easy. As you can see in this thread, the breakage started for ~arch users over a year ago. And because the devs knew about the breakage, they already held >=expat-2.0.0 back from becoming stable until now. I'm quite sure, the devs would have chosen a better upgrade procedure if there would be any...

But to be honest... LDFLAGS="-Wl,--as-needed" is still unsupported though this could make this kind of expat-upgrade so much easier to handle...

roderick · Post by **roderick** » Sat Aug 11, 2007 6:07 pm

I agree. I have --as-needed in per package LDFLAGS for kde only, and I never had any expat issues that a simple revdep-rebuild couldn't fix.

I think the KDE ebuilds could be configured by default with as-needed at this point (unless there really is some reason to hold this back).

I guess with the looming kde 4, it's probably a moot point though.

BitJam · Post by **BitJam** » Sat Aug 11, 2007 6:07 pm

Polynomial-C wrote:Sorry, it's not that easy. As you can see in this thread, the breakage started for ~arch users over a year ago. And because the devs knew about the breakage, they already held >=expat-2.0.0 back from becoming stable until now. I'm quite sure, the devs would have chosen a better upgrade procedure if there would be any...

From the last page of the thread you linked to:

eean wrote: I "fixed" it by editing the ebuild of expat-2.0 and changing its slot from 0 to 1. So now I have both emerged and subversion runs fine.

I'm not jumping with joy at having to rebuild most of my system but that's not even the issue. The problem is that I can't revdep-rebuild because expat-2.0 seems to break just about everything. The existing versions of packages are broken due to the missing libexpat and re-emerging also fails with libexpat errors so I can't rebuild anything.

Carlo · Post by **Carlo** » Sat Aug 11, 2007 6:52 pm

rickythesk8r wrote:The libpango ebuild is broken because it insists on libexpat.so.0 and that .so is no longer available. Every ebuild will have to be examined by hand for compatibility with the expat upgrade.

Either a bug in revdep-rebuild or a bug in your package database or you did install pango sidestepping Portage.

BitJam wrote:I didn't say that expat should be slotted. [...] Although IMO, expat should be slotted because the new version breaks just about everything.

Please stop playing words. You didn't say it, but implied already in your previous post and I tell you you're wrong.

BitJam wrote:From the last page of the thread you linked to:
eean wrote: I "fixed" it by editing the ebuild of expat-2.0 and changing its slot from 0 to 1. So now I have both emerged and subversion runs fine.

Don't. It's just stupid.

beso_1717 · Post by **beso_1717** » Sat Aug 11, 2007 7:05 pm

the problem is that expat 2.0.1 has changed libexpat.so.0 name, so you have to do a revdep-rebuild -X --library libexpat.so.0 as suggested by the expat 2.0.1 infos, which will fix the broken links and recompile about 120 packages... if it doesn't recompile you'll have to run a revdep-rebuild after the first revdep-rebuild command and that will reinstall the broken packages.... among these packages are shared-mime, gconf, kde, amarok, and practically all the packages. i hope that the next time someone does such a big idi*** of changing names they would advise before getting the users mad.
the new curl also need a revdep-rebuild.

i don't know if portage prints the output messages of expat, but my paludis did so... here are the lines:

* Please note that the soname of the library changed!
* If you are upgrading from a previous version you need
* to fix dynamic linking inconsistencies by executing:
* revdep-rebuild -X --library libexpat.so.0

i'll still run another revdep after the first one has ended, so that to be sure everything is ok....

BitJam · Post by **BitJam** » Sat Aug 11, 2007 7:22 pm

Carlo wrote:
BitJam wrote:I didn't say that expat should be slotted. [...] Although IMO, expat should be slotted because the new version breaks just about everything.
Please stop playing words. You didn't say it, but implied already in your previous post and I tell you you're wrong.

If I emerge the latest stable expat (2.x) it breaks my system and I can no longer re-emerge things so revdep-rebuild is of no use. IMO, this is a Gentoo problem independent of whether slotting expat is the correct solution or not.

I try to be careful with my words and not play with them. There are two separate concepts here (1) is the problem in Gentoo or Expat, and (2) what is the proper solution. My initial comment was addressing the first concept and not the second.

Since Gentoo has the slots mechanism and from my simple understanding of how Emerge works, ISTM that it is Gentoo that is removing the older version of Expat after the new version is installed. The poster I was responding to thought it was expat that was removing the older version.

You keep saying slots are not the correct solution for this problem. I have no idea. But if we don't use slots then don't we need to either fix (or mask) expat or fix all the ebuilds that no longer emerge when the new expat is installed?