| View previous topic :: View next topic |
| Author |
Message |
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
 Posted: Mon Sep 01, 2014 7:31 am Post subject: PLEASE ADVISE: I did something stupid |
 |
|
I just read about a recent exploitable bug in glibc, and without thinking through carefully, upgraded my glibc (from 2.17 to 2.19-r1).
I realized the potential stupidity of this right after the emerge finished, and thought maybe I can try to downgrade, even though I now remembered this is a big no-no, but portage "protected" me from this further mistake.
What can I do, or what do I need to do at this point, to save myself from a reinstall? Things still seem to be working fine, and I'm not shutting my machine down yet!
Thanks! |
|
| Back to top |
|
 |
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3664
|
| Posted: Mon Sep 01, 2014 7:42 am Post subject: Re: PLEASE ADVISE: I did something stupid |
|
|
| Why do you think having glibc-2.19-r1 is bad? Isn't it stable enough for the major arches? |
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 7:48 am Post subject: |
|
|
I don't think it's bad... it seems to be the version that has the bugfix that I wanted. However...
OK... I actually didn't tell the whole story  I did this upgrade at the same time on my wife's computer, and really discovered that something was wrong because things started to break on that machine.... Firefox crashed and when I tried to restart, it gave a segfault. Then I tried starting up Chromium and it segfaulted too. That's when I started breaking out in a cold sweat... 
Oh yes, and emerge is also throwing a segfault
At this point what I really need to fix is my wife's machine (not the one I originally referred to, that is still working fine, it seems). I'm just worried that things will begin to fall apart...
Last edited by Punchcutter on Mon Sep 01, 2014 7:56 am; edited 1 time in total |
|
| Back to top |
|
|
charles17
Advocate
Joined: 02 Mar 2008
Posts: 3664
|
| Posted: Mon Sep 01, 2014 7:55 am Post subject: |
|
|
| Sometimes running revdep-rebuild after an upgrade may be of help. Would Firefox/Chromium still segfault after recompiling them? |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Mon Sep 01, 2014 7:58 am Post subject: |
|
|
You should be fine, it definitely is stable: http://packages.gentoo.org/package/sys-libs/glibc
You may want to emerge binutils, and then, some say this should be followed by 'emerge -e @world'. Shouldn't hurt but perhaps not necessary. |
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 7:59 am Post subject: |
|
|
Thanks... I edited my previous post to note that emerge is also segfaulting. I just tried revdep-rebuild, and it runs, but it gives:
| Code: | * Configuring search environment for revdep-rebuild
* No search defined -- this is a bug.
|
|
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 8:03 am Post subject: |
|
|
| I appreciate the reassurance that the version I installed is stable, and so therefore there should be no reason to worry about upgrading it in isolation. It makes me worry less about my own machine, but without being able to use emerge, I feel kinda screwed on my wife's machine (also not sure why they are behaving differently) |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Mon Sep 01, 2014 8:07 am Post subject: |
|
|
Hmm. I hadn't seen the post about emerge seg-faulting.
I hesitate to give advice without seeing what you are seeing.
You should keep checking the logs and dmesg as you proceed.
May want to :
| Code: | # . /etc/profile
# env-update
# emerge --info |
If that works you may be out of the woods. |
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 8:12 am Post subject: |
|
|
russK, thanks. env-update also segfaults My /var/log/messages (is that what you meant?) looks like this:
| Code: | Aug 31 21:51:42 linux7 kernel: python2.7[10609]: segfault at 596e ip 0000596e sp bfabf82c error 4 in python2.7[8048000+1000]
Aug 31 21:53:02 linux7 kernel: python2.7[10611]: segfault at 596e ip 0000596e sp bfa09bec error 4 in python2.7[8048000+1000]
Aug 31 21:53:02 linux7 kernel: gawk[10640]: segfault at 596e ip 0000596e sp bfe3fc4c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:02 linux7 kernel: gawk[10644]: segfault at 596e ip 0000596e sp bfe4274c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:02 linux7 kernel: gawk[10647]: segfault at 596e ip 0000596e sp bfab5adc error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:02 linux7 kernel: gawk[10650]: segfault at 596e ip 0000596e sp bf9487ec error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:10 linux7 kernel: python2.7[10654]: segfault at 596e ip 0000596e sp bffedecc error 4 in python2.7[8048000+1000]
Aug 31 21:53:10 linux7 kernel: gawk[10682]: segfault at 596e ip 0000596e sp bfc96c6c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:10 linux7 kernel: gawk[10685]: segfault at 596e ip 0000596e sp bff3673c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:10 linux7 kernel: gawk[10688]: segfault at 596e ip 0000596e sp bfcfd30c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:53:10 linux7 kernel: gawk[10691]: segfault at 596e ip 0000596e sp bfecf2cc error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:54:33 linux7 kernel: python2.7[10695]: segfault at 596e ip 0000596e sp bfd5b27c error 4 in python2.7[8048000+1000]
Aug 31 21:54:33 linux7 kernel: gawk[10725]: segfault at 596e ip 0000596e sp bfb3610c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:54:33 linux7 kernel: gawk[10728]: segfault at 596e ip 0000596e sp bf8b744c error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:54:33 linux7 kernel: gawk[10731]: segfault at 596e ip 0000596e sp bff1b9fc error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:54:33 linux7 kernel: gawk[10734]: segfault at 596e ip 0000596e sp bfb7dfec error 4 in gawk-3.1.6[8048000+4d000]
Aug 31 21:56:34 linux7 kernel: python2.7[10737]: segfault at 596e ip 0000596e sp bfddfb8c error 4 in python2.7[8048000+1000]
Aug 31 21:59:01 linux7 cron[10741]: (root) CMD (rm -f /var/spool/cron/lastrun/cron.hourly)
Aug 31 22:00:01 linux7 cron[10745]: (root) CMD (test -x /usr/sbin/run-crons && /usr/sbin/run-crons)
Aug 31 22:00:01 linux7 kernel: find[10752]: segfault at 596e ip 0000596e sp bfa55b6c error 4 in find[8048000+21000]
Aug 31 22:00:01 linux7 kernel: find[10754]: segfault at 596e ip 0000596e sp bfed812c error 4 in find[8048000+21000]
Aug 31 22:00:01 linux7 kernel: find[10755]: segfault at 596e ip 0000596e sp bfb269cc error 4 in find[8048000+21000]
Aug 31 22:00:01 linux7 kernel: find[10757]: segfault at 596e ip 0000596e sp bfbdb73c error 4 in find[8048000+21000]
Aug 31 22:00:01 linux7 sSMTP[10753]: Unable to locate mail
Aug 31 22:00:01 linux7 sSMTP[10753]: Cannot open mail:25
Aug 31 22:00:01 linux7 cron[10744]: (root) MAIL (mailed 508 bytes of output but got status 0x0001
)
Aug 31 22:05:07 linux7 kernel: python2.7[10762]: segfault at 596e ip 0000596e sp bfa0237c error 4 in python2.7[8048000+1000]
Aug 31 22:05:11 linux7 kernel: python2.7[10765]: segfault at 596e ip 0000596e sp bf97f77c error 4 in python2.7[8048000+1000]
Aug 31 22:05:27 linux7 kernel: python2.7[10766]: segfault at 596e ip 0000596e sp bfde79ac error 4 in python2.7[8048000+1000]
Aug 31 22:06:32 linux7 kernel: python2.7[10774]: segfault at 596e ip 0000596e sp bf808cfc error 4 in python2.7[8048000+1000]
Aug 31 22:06:54 linux7 kernel: python2.7[10779]: segfault at 596e ip 0000596e sp bfd34c9c error 4 in python2.7[8048000+1000]
|
And dmesg looks very similar. |
|
| Back to top |
|
|
xaviermiller
Bodhisattva
Joined: 23 Jul 2004
Posts: 8708
Location: ~Brussels - Belgique
|
| Posted: Mon Sep 01, 2014 8:22 am Post subject: |
|
|
what kernel version do you have ? Is there any elog message telling there is a missing kernel option ?
_________________
Kind regards,
Xavier Miller |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Mon Sep 01, 2014 8:24 am Post subject: |
|
|
Yes that's the right file.
Looks like it's having pretty much the same problem with gawk, python, and find. If it was me, at this point I might just reboot and hope it comes up OK. If not, it might be time to repair with a CD and getting some good binaries. Sorry I don't have any better advice. |
|
| Back to top |
|
|
xaviermiller
Bodhisattva
Joined: 23 Jul 2004
Posts: 8708
Location: ~Brussels - Belgique
|
| Posted: Mon Sep 01, 2014 8:30 am Post subject: |
|
|
since the segfault is in pyhton, try to re-emerge python
| Code: | | emerge -1av python:2.7 |
_________________
Kind regards,
Xavier Miller |
|
| Back to top |
|
|
VoidMage
Watchman
Joined: 14 Oct 2006
Posts: 6196
|
| Posted: Mon Sep 01, 2014 8:44 am Post subject: |
|
|
| XavierMiller wrote: | since the segfault is in pyhton, try to re-emerge python
| Code: | | emerge -1av python:2.7 |
|
...if even python is segfaulting, emerge might not be an option... |
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 8:54 am Post subject: |
|
|
I found the following in /var/log/portage/elog/summary.log. This is what I recall seeing as the output of emerge when I did the glibc upgrade on that machine (although I can not recover it directly now). Interestingly, the build.log is now non-existent. If the emerge failed, I wonder why the "live" library was affected? Anyway, if this gives anyone any clues... I'm gonna hold off rebooting just a bit longer 
| Code: | >>> Messages generated by process 13094 on 2014-08-31 20:51:01 HST for package sys-libs/glibc-2.15-r2:
ERROR: prerm
The ebuild phase 'prerm' has exited unexpectedly. This type of behavior
is known to be triggered by things such as failed variable assignments
(bug #190128) or bad substitution errors (bug #200313). Normally, before
exiting, bash should have displayed an error message above. If bash did
not produce an error message above, it's possible that the ebuild has
called `exit` when it should have called `die` instead. This behavior
may also be triggered by a corrupt bash binary or a hardware problem
such as memory or cpu malfunction. If the problem is not reproducible or
it appears to occur randomly, then it is likely to be triggered by a
hardware problem. If you suspect a hardware problem then you should try
some basic hardware diagnostics such as memtest. Please do not report
this as a bug unless it is consistently reproducible and you are sure
that your bash binary and hardware are functioning properly.
ERROR: sys-libs/glibc-2.15-r2::gentoo failed (prerm phase):
filter-bash-environment.py failed
Call stack:
ebuild.sh, line 480: Called __preprocess_ebuild_env
phase-functions.sh, line 156: Called __filter_readonly_variables '--filter-features' '--filter-locale' '--filter-path' '--filter-sandbox'
phase-functions.sh, line 137: Called die
The specific snippet of code:
"${PORTAGE_PYTHON:-/usr/bin/python}" "${PORTAGE_BIN_PATH}"/filter-bash-environment.py "${filtered_vars}" || die "filter-bash-environment.py failed"
If you need support, post the output of `emerge --info '=sys-libs/glibc-2.15-r2::gentoo'`,
the complete build log and the output of `emerge -pqv '=sys-libs/glibc-2.15-r2::gentoo'`.
The complete build log is located at '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/temp/environment'.
Working directory: '/usr/lib/portage/pym'
S: '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/work/glibc-2.15'
ERROR: postrm
The ebuild phase 'postrm' has exited unexpectedly. This type of behavior
is known to be triggered by things such as failed variable assignments
(bug #190128) or bad substitution errors (bug #200313). Normally, before
exiting, bash should have displayed an error message above. If bash did
not produce an error message above, it's possible that the ebuild has
called `exit` when it should have called `die` instead. This behavior
may also be triggered by a corrupt bash binary or a hardware problem
such as memory or cpu malfunction. If the problem is not reproducible or
it appears to occur randomly, then it is likely to be triggered by a
hardware problem. If you suspect a hardware problem then you should try
some basic hardware diagnostics such as memtest. Please do not report
this as a bug unless it is consistently reproducible and you are sure
that your bash binary and hardware are functioning properly.
The 'postrm' phase of the 'sys-libs/glibc-2.15-r2' package has failed
with exit value 1.
The problem occurred while executing the ebuild file named
'glibc-2.15-r2.ebuild' located in the '/var/db/pkg/sys-
libs/glibc-2.15-r2' directory. If necessary, manually remove the
environment.bz2 file and/or the ebuild file located in that directory.
Removal of the environment.bz2 file is preferred since it may allow the
removal phases to execute successfully. The ebuild will be sourced and
the eclasses from the current portage tree will be used when necessary.
Removal of the ebuild file will cause the pkg_prerm() and pkg_postrm()
removal phases to be skipped entirely.
ERROR: sys-libs/glibc-2.15-r2::gentoo failed (postrm phase):
filter-bash-environment.py failed
Call stack:
ebuild.sh, line 480: Called __preprocess_ebuild_env
phase-functions.sh, line 156: Called __filter_readonly_variables '--filter-features' '--filter-locale' '--filter-path' '--filter-sandbox'
phase-functions.sh, line 137: Called die
The specific snippet of code:
"${PORTAGE_PYTHON:-/usr/bin/python}" "${PORTAGE_BIN_PATH}"/filter-bash-environment.py "${filtered_vars}" || die "filter-bash-environment.py failed"
If you need support, post the output of `emerge --info '=sys-libs/glibc-2.15-r2::gentoo'`,
the complete build log and the output of `emerge -pqv '=sys-libs/glibc-2.15-r2::gentoo'`.
The complete build log is located at '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/temp/environment'.
Working directory: '/usr/lib/portage/pym'
S: '/var/tmp/portage/._unmerge_/sys-libs/glibc-2.15-r2/work/glibc-2.15'
>>> Messages generated by process 13094 on 2014-08-31 20:51:01 HST for package sys-libs/glibc-2.19-r1:
LOG: preinst
Defaulting /etc/host.conf:multi to on
ERROR: postinst
The ebuild phase 'postinst' has exited unexpectedly. This type of
behavior is known to be triggered by things such as failed variable
assignments (bug #190128) or bad substitution errors (bug #200313).
Normally, before exiting, bash should have displayed an error message
above. If bash did not produce an error message above, it's possible
that the ebuild has called `exit` when it should have called `die`
instead. This behavior may also be triggered by a corrupt bash binary or
a hardware problem such as memory or cpu malfunction. If the problem is
not reproducible or it appears to occur randomly, then it is likely to
be triggered by a hardware problem. If you suspect a hardware problem
then you should try some basic hardware diagnostics such as memtest.
Please do not report this as a bug unless it is consistently
reproducible and you are sure that your bash binary and hardware are
functioning properly.
ERROR: sys-libs/glibc-2.19-r1::gentoo failed (postinst phase):
filter-bash-environment.py failed
Call stack:
ebuild.sh, line 714: Called __ebuild_main 'postinst'
phase-functions.sh, line 891: Called __filter_readonly_variables '--filter-path' '--filter-sandbox' '--allow-extra-vars'
phase-functions.sh, line 137: Called die
The specific snippet of code:
"${PORTAGE_PYTHON:-/usr/bin/python}" "${PORTAGE_BIN_PATH}"/filter-bash-environment.py "${filtered_vars}" || die "filter-bash-environment.py failed"
If you need support, post the output of `emerge --info '=sys-libs/glibc-2.19-r1::gentoo'`,
the complete build log and the output of `emerge -pqv '=sys-libs/glibc-2.19-r1::gentoo'`.
The complete build log is located at '/var/tmp/portage/sys-libs/glibc-2.19-r1/temp/build.log'.
The ebuild environment file is located at '/var/tmp/portage/sys-libs/glibc-2.19-r1/temp/environment'.
Working directory: '/usr/lib/portage/pym'
S: '/var/tmp/portage/sys-libs/glibc-2.19-r1/work/glibc-2.19'
ERROR: other
The ebuild phase 'other' has exited unexpectedly. This type of behavior
is known to be triggered by things such as failed variable assignments
(bug #190128) or bad substitution errors (bug #200313). Normally, before
exiting, bash should have displayed an error message above. If bash did
not produce an error message above, it's possible that the ebuild has
called `exit` when it should have called `die` instead. This behavior
may also be triggered by a corrupt bash binary or a hardware problem
such as memory or cpu malfunction. If the problem is not reproducible or
it appears to occur randomly, then it is likely to be triggered by a
hardware problem. If you suspect a hardware problem then you should try
some basic hardware diagnostics such as memtest. Please do not report
this as a bug unless it is consistently reproducible and you are sure
that your bash binary and hardware are functioning properly.
|
|
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Mon Sep 01, 2014 9:50 am Post subject: |
|
|
| Rebooted. Surprisingly, the machine came up and not much of the system init was disturbed by segfaults. But what was broken before is still broken. |
|
| Back to top |
|
|
lagalopex
Guru
Joined: 16 Oct 2004
Posts: 562
|
| Posted: Mon Sep 01, 2014 11:52 am Post subject: |
|
|
What kernel version? What version of gcc?
Can you run a failing command in gdb to debug?
Can you get a x86 binary copy of glibc 2.17 and unpack it somewhere? You could then try to set LD_LIBRARY_PATH to load those instead of the broken 2.19. |
|
| Back to top |
|
|
i92guboj
Bodhisattva
Joined: 30 Nov 2004
Posts: 10315
Location: Córdoba (Spain)
|
| Posted: Mon Sep 01, 2014 12:08 pm Post subject: |
|
|
This kind of massive breakage is often caused by a glibc downgrade. So, my question is: what glibc version was installed before?
If that's not the case, then you either did something very strange or you have hit quite a bug.
You should be able to recover your system by untarring a working toolchain. Someone with a similar setup should be able to quickpkg you glibc, gcc, binutils and the basic tools. If tar doesn't work in your system, then you will also need a bootable media that will allow you to mount your system and untar the packages. _Once you have a working toolchain, re-emerging the system should be easy. |
|
| Back to top |
|
|
russK
l33t
Joined: 27 Jun 2006
Posts: 665
|
| Posted: Mon Sep 01, 2014 4:07 pm Post subject: |
|
|
| Might be interesting to boot with a recent install CD and chroot into the system, see if the segfaults occur then. If they don't, you may just need to build a new kernel. |
|
| Back to top |
|
|
Jaglover
Watchman
Joined: 29 May 2005
Posts: 8291
Location: Saint Amant, Acadiana
|
|
| Back to top |
|
|
Punchcutter
Guru
Joined: 11 Feb 2007
Posts: 354
|
| Posted: Thu Sep 25, 2014 8:07 am Post subject: |
|
|
| Well, in the end I decided that trying to do as little as possible, debugging with gdb, etc, was probably more trouble than just doing a fresh reinstall, which I finally found time for, and did, and now we're back mostly to normal. At least I had a backup of /boot, /etc, and other config information (kernel .config etc), and have /home on its own partition, so most of the trouble was waiting for emerge -e world to finish. Thanks for the responses, everyone. |
|
| Back to top |
|
|
|
Portage & Programming
