WARNING: prelink and glibc-2.13 update

Message

vdracula · Post by **vdracula** » Sun Feb 06, 2011 11:11 am

Hello!

Just wanted to warn all those who are just updated to glibc-2.13 and using prelink.

I got everywhere a segfault...

more here:

https://bbs.archlinux.org/viewtopic.php?pid=885429

seqizz · Post by **seqizz** » Sun Feb 06, 2011 1:02 pm

Thank you very much to give this info. I was f*** up. I didn't see something like this before. I can't even chroot to fix "whatever it is".

Yuu · Post by **Yuu** » Sun Feb 06, 2011 1:12 pm

Hi,

thank you for the information. Hopefully, I'm still in amd64 stable branch, but I hope that'll be fixed soon, because I like prelink :]
Also, I noticed that there's an [bug=353814]opened bug here[/bug].

@seqizz : maybe you should see that [bug=353814]opened bug[/bug], there's instructions to undo prelinking from the liveCD.

seqizz · Post by **seqizz** » Sun Feb 06, 2011 1:39 pm

Thanks @Yuu,

Even if i had to force my one friend to compile a binary prelink package with -march=atom cflags. After a successful prelink -ua, i guess everything is ok.

Odysseus · Post by **Odysseus** » Sun Feb 06, 2011 1:59 pm

If you're running a ~ arch, DO NOT update to glibc-2.13 if you have prelink installed!!! It will completely hose your box!!! I just spent several hours trying to recover from this bug http://bugs.gentoo.org/show_bug.cgi?id=353814

If you want the new glibc, you must first uninstall prelink or if you want to keep prelink, you need to mask the new glibc until one or the other is fixed.

When I updated yesterday morning glibc was in the queue. It installed without issues. I then 'emerge -eavND @system' to rebuild against it. Halfway through the emerge, portage crashed, bash segfaulted and my KDE DE ceased functioning. I tried to ssh into my box, but it was impossible because all of my network connections ceased functioning. The only way to get out of X was a hard reboot, but then all three of my kernels panicked just after leaving the grub splash.

I was finally able to regain control of my machine by booting into a rescue disc and following the instructions posted in comment #3 in the bug report I linked to. I had to chroot into my box to uninstall prelink, before I was able to regain control and reboot back into my system.

Please be aware that this is a serious issue.

You have been forewarned!

Ciao

Edit: apparently this isn't the only distro to be affected by this bug. Arch Linux is as well, as evidenced by these posts: https://bbs.archlinux.org/viewtopic.php?id=112510 and https://bbs.archlinux.org/viewtopic.php ... 97#p884397

hielvc · Post by **hielvc** » Mon Feb 07, 2011 2:11 am

Thanks for the heads up and leading the way

FireBurn · Post by **FireBurn** » Mon Feb 07, 2011 6:24 am

I tried to get the committer of the new glibc to mask the package immediately before I figured out it was caused by the interaction with prelink

It's more than a day on and still nothing has been done to stop people from hosing their systems

All it would take is a !sys-devel/prelink in the DEPEND of the new glibc or a !sys-lib/glibc-2.13 in the prelink builds

It's not rocket science - I wonder how many folks have hosed their system because of this?

furanku · Post by **furanku** » Mon Feb 07, 2011 7:29 am

Wow --- that was scary!

Thanks you so much for giving the link to the workaround so quickly.

SoLoR · Post by **SoLoR** » Mon Feb 07, 2011 8:16 am

glibc-2.13 also breaks postfix for me...

Feb 7 08:02:18 mih postfix/master[11799]: daemon started -- version 2.8.0, configuration /etc/postfix
Feb 7 08:02:18 mih postfix/qmgr[11803]: A82474006E: from=<root@xxxx.xxx.xxx>, size=14500, nrcpt=1 (queue active)
Feb 7 08:02:18 mih postfix/local[11805]: fatal: unable to determine open file limit
Feb 7 08:02:19 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 11805 exit status 1
Feb 7 08:02:19 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:03:19 mih postfix/local[11826]: fatal: unable to determine open file limit
Feb 7 08:03:20 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 11826 exit status 1
Feb 7 08:03:20 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:04:20 mih postfix/local[11841]: fatal: unable to determine open file limit
Feb 7 08:04:21 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 11841 exit status 1
Feb 7 08:04:21 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:05:21 mih postfix/local[11861]: fatal: unable to determine open file limit
Feb 7 08:05:22 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 11861 exit status 1
Feb 7 08:05:22 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:06:22 mih postfix/local[11868]: fatal: unable to determine open file limit
Feb 7 08:06:23 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 11868 exit status 1
Feb 7 08:06:23 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:07:23 mih postfix/local[12215]: fatal: unable to determine open file limit
Feb 7 08:07:24 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 12215 exit status 1
Feb 7 08:07:24 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:08:24 mih postfix/local[21471]: fatal: unable to determine open file limit
Feb 7 08:08:25 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 21471 exit status 1
Feb 7 08:08:25 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:09:25 mih postfix/local[22152]: fatal: unable to determine open file limit
Feb 7 08:09:26 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 22152 exit status 1
Feb 7 08:09:26 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:10:26 mih postfix/local[23152]: fatal: unable to determine open file limit
Feb 7 08:10:27 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 23152 exit status 1
Feb 7 08:10:27 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:11:27 mih postfix/local[23861]: fatal: unable to determine open file limit
Feb 7 08:11:28 mih postfix/master[11799]: warning: process /usr/lib/postfix/local pid 23861 exit status 1
Feb 7 08:11:28 mih postfix/master[11799]: warning: /usr/lib/postfix/local: bad command startup -- throttling
Feb 7 08:11:55 mih postfix/postfix-script[24050]: stopping the Postfix mail system
Feb 7 08:11:55 mih postfix/master[11799]: terminating on signal 15

reverting glibc back to 2.12.2 and problem is gone.

Etrans · Post by **Etrans** » Mon Feb 07, 2011 9:33 am

Read this post too late..now I have a kernel panic with some mentioning of ld.so-2.13

FireBurn · Post by **FireBurn** » Mon Feb 07, 2011 10:15 am

Etrans wrote:Read this post too late..now I have a kernel panic with some mentioning of ld.so-2.13

Don't panic (unlike your kernel)

1) Boot from CD
2) mount your root partition to /mnt/gentoo
3) run: /mnt/gentoo/usr/sbin/prelink -u /mnt/gentoo/bin/bash
4) chroot into your install: chroot /mnt/gentoo /bin/bash
5) Switch of prelink: prelink -au
6) Unmerge prelink: emerge --unmerge prelink

Alternatively instead of 6) you could downgrade glibc and mask 2.13

grozin · Post by **grozin** » Mon Feb 07, 2011 10:48 am

I run a pure ~x86 box. After upgrade to glibc-2.13, skype segfaults immediately at start-up. Please help! My wife is worried!

Is it possible to downgrade glibc in any way? Higher computer-related things can wait, I urgently need skype.

SoLoR · Post by **SoLoR** » Mon Feb 07, 2011 10:57 am

grozin wrote:I run a pure ~x86 box. After upgrade to glibc-2.13, skype segfaults immediately at start-up. Please help! My wife is worried!

Is it possible to downgrade glibc in any way? Higher computer-related things can wait, I urgently need skype.

there is: here

im trying to recompile whole system with new glibc to see if it fixed my postfix issue, but it will take a while...

vdracula · Post by **vdracula** » Mon Feb 07, 2011 11:04 am

Even for testing it's a very bad QA, in particular for glibc.

FireBurn · Post by **FireBurn** » Mon Feb 07, 2011 11:12 am

vdracula wrote:Even for testing it's a very bad QA, in particular for glibc.

The ebuild was marked ~unstable for x86 amd64 ppc ppc64 spark ia64 sh & s390

I very much doubt if all these targets were tested especially the latter; I doubt very many people have access to a mainframe to do testing

tnt · Post by **tnt** » Mon Feb 07, 2011 11:21 am

FireBurn wrote:
Etrans wrote:Read this post too late..now I have a kernel panic with some mentioning of ld.so-2.13
Don't panic (unlike your kernel)

1) Boot from CD
2) mount your root partition to /mnt/gentoo
3) run: /mnt/gentoo/usr/sbin/prelink -u /mnt/gentoo/bin/bash
4) chroot into your install: chroot /mnt/gentoo /bin/bash
5) Switch of prelink: prelink -au
6) Unmerge prelink: emerge --unmerge prelink

Alternatively instead of 6) you could downgrade glibc and mask 2.13

thx

vdracula · Post by **vdracula** » Mon Feb 07, 2011 11:27 am

FireBurn wrote:
vdracula wrote:Even for testing it's a very bad QA, in particular for glibc.
The ebuild was marked ~unstable for x86 amd64 ppc ppc64 spark ia64 sh & s390

I very much doubt if all these targets were tested especially the latter; I doubt very many people have access to a mainframe to do testing

That's true, but at least for the very popular archs it could be tested. For instance, x86, amd64 (and if, ppc/ppc64).

Jaglover · Post by **Jaglover** » Mon Feb 07, 2011 3:04 pm

Yes, three of my boxes were running overnight and were screwed. There is no need to unmerge prelink, though, just make sure you have

Code: Select all

PRELINKING=""

in your /etc/conf.d/prelink.

I assume most of us want to keep prelink despite this little @#$%.

tomk · Post by **tomk** » Mon Feb 07, 2011 6:52 pm

Merged from [post=6568519]here[/post].

Simon196405 · Post by **Simon196405** » Mon Feb 07, 2011 8:08 pm

I just wonder if prelink is still usefull.
Nowedays we some linker optimizations like: --as-needed, hash-style=gnu, O1.
Also there was something like a useflag kdeenablefinal.

These basically make sure there is less to link and that linking goes faster.
So the befit of prelink is smaller these days.

I just wonder if prelink is still worthwhile?

SamuliSuominen · Post by **SamuliSuominen** » Mon Feb 07, 2011 8:20 pm

Simon196405 wrote:I just wonder if prelink is still usefull.
Nowedays we some linker optimizations like: --as-needed, hash-style=gnu, O1.
Also there was something like a useflag kdeenablefinal.

These basically make sure there is less to link and that linking goes faster.
So the befit of prelink is smaller these days.

I just wonder if prelink is still worthwhile?

No. It's not worth it on modern systems.

http://lwn.net/Articles/341244/

js08 · Post by **js08** » Mon Feb 07, 2011 8:24 pm

Jaglover wrote:Yes, three of my boxes were running overnight and were screwed. There is no need to unmerge prelink, though, just make sure you have
Code: Select all
PRELINKING=""
in your /etc/conf.d/prelink.

I assume most of us want to keep prelink despite this little @#$%.

thanx this helped.

rhill · Post by **rhill** » Mon Feb 07, 2011 10:13 pm

vdracula wrote:Even for testing it's a very bad QA, in particular for glibc.

Yes it sucks, but prelink is run by a cron job. Until prelink is run, everything appears to work. No amount of testing would catch it unless the cron job happened to trigger.

robinmarlow · Post by **robinmarlow** » Tue Feb 08, 2011 12:04 am

Cripes! That was scary.

Thanks for the solution. I'd have never figured this one out.

Robin

NightMonkey · Post by **NightMonkey** » Tue Feb 08, 2011 12:19 am

Welcome to unstable, everyone.

WARNING: prelink and glibc-2.13 update

WARNING: prelink and glibc-2.13 update

Omg

WARNING: updating to glibc-2.13 w/prelink hoses system