GCC 3.2 is here: Now is the time to rebuild everything. How?

[IWBCMAN]

oh welll the error messages which posted earlier makes me think that I /grep/gawk/gettext/libtools/perl etc. got b0rked....

oh well
mv /etc /oldetc
tar -xvjpf stage-3-i686-1.4.tbz2(ie. the 3.2_pre version)
mv /etc /newetc
mv /oldetc /etc
emerge rsync
passwd
emerge gcc
emerge -e system
then much, much later(perhpas tommorow morning)
./rmerge -f (with aggresive settings again.....)
---it would all take a lot longer if I didn't already have ALL the distfiles on my machine (emerge -fe world)
if I just knew how to script(perl/bash/ etc) I would draft up a script to automate this "reinstall and recompile every damned thing" process


to jackp:
I unfortunately cannot really answer your question-what I do know is that 3.2 works fine with -msse2(pentium4 stuff) whereas 3.2_pre notified me constantly that sse was being disabled....secondly 3.2 should be the final version ie. there should not need be any changes made to it-it implements the complete new C++ ABI, which will become more and more important as the precompiled stuff(flash,java etc) we get is compiled using the new ABI- 3.2_pre probably is not complete, yet I suspect the differences are rather small...in all likelyhood you will have to upgrade at some point, right now it is probably not necessary....

(BUT I WANT MY SSE2 OPTIMIZATIONS!!!!!!!!!-why the hell did I fork out so dmaned much money for a processor which isn't being utilized? the fp unit on the P4 is the wimpiest damned things since 8087)

[jackp]

I just recently set up gentoo 1.4beta with gcc-3.2_pre. For the most part things went pretty smoothly. Now, my question is, is there a drastic change between 3.2_pre and 3.2 that requires me to recompile ALL my packages, including my kernel? AFAIU the C++ ABI changed from 3.1.1 to 3.2, so would that be pretty much stabilized now (even with that, that only affects C++ code, so I can track down the C++ packages and recompile those if necessary)? Shouldn't the two compiler versions be compatible with each other at least to generate similar enough binary code such that my old packages would run in a "gcc 3.2 environment"?

Or do you think I should play it safe and do an 'emerge -e world' anyway?

EDIT: I think my first question is better rephrased as: are there significant library incompatibilities between 3.2_pre and 3.2? Will things immediately break after I update?

answered my own question. I decided to take a stab at it and upgrade just GCC to 3.2 to see what that would do. and things are starting to yell out "recompile me recompile me!". For ex., check out galeon:

Code: Select all

kungfoo portage # galeon
/usr/bin/galeon-bin: /usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/libstdc++.so.5: version `CXXABI_1' not found (required by /usr/bin/galeon-bin)
/usr/bin/galeon-bin: /usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/libstdc++.so.5: version `GLIBCPP_3.1' not found (required by /usr/bin/galeon-bin)
/usr/bin/galeon-bin: /usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/libstdc++.so.5: version `GLIBCPP_3.1' not found (required by /usr/lib/mozilla/libgtkembedmoz.so)
/usr/bin/galeon-bin: /usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/libstdc++.so.5: version `CXXABI_1' not found (required by /usr/lib/mozilla/libxpcom.so)
/usr/bin/galeon-bin: /usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/libstdc++.so.5: version `GLIBCPP_3.1' not found (required by /usr/lib/mozilla/libxpcom.so)

So I guess the C++ ABI did change after all. It looks as if the 3.2_pre was using the 3.1 ABI spec or something, I'm not quite sure. What does "GLIBCPP_3.1" mean to you?

so I guess it's time to recompile everything

[Beattie]

So, to rebuild my system with gcc 3.2, I need to:

Log in as root

emerge rsync # to make sure everything is up to date

emerge -e world

run some bootstrap script (I installed from stage 2 the first time)

2 questions. (one is a 2 parter)
Is this right?
What exactly does bootstrapping do? Cant you just recompile gcc?

[rac]

What exactly does bootstrapping do? Cant you just recompile gcc?

Bootstrapping (oversimplifying a bit) compiles glibc -> gcc (C only) -> glibc -> gcc.

The safest thing to do here is run bootstrap.sh. Stop reading now.

This is not an official opinion. Actually, almost nothing that I say is an official opinion.

I have not tried this with GCC 3.2 yet. I did use this process to bootstrap GCC 3.1 by hand on both x86 and PPC, and it worked then.

OK, disclaimed enough?

If you have a running Gentoo system already, I think it is theoretically OK to skip the first glibc step, and just emerge gcc, emerge glibc, emerge gcc again. In order to get the first gcc to skip building other languages such as C++ and java, which are not needed just to compile glibc and gcc, you can temporarily turn on the "bootstrap" USE flag. This will speed up the first gcc compile considerably. Make sure that the "bootstrap" USE flag is off by the time you emerge gcc the second time, or you will not have a C++ compiler, and that will hurt you when it comes time to recompile the rest of your system.

Especially if you are upgrading directly from 2.95.3 to 3.2, there are lots of new compiler flags that you won't be able to use until GCC3 is installed. This means that you will have to run bootstrap.sh twice, if you want your new flags to take effect for glibc. That is one more advantage of the manual bootstrap method - you can switch to your new GCC3 flags right after the C-only gcc 3.2 has been emerged, then emerge glibc (once), and then gcc itself again to compile it against the new glibc, and to build support for the other languages.

It is probably a good idea to have a .tbz binary package of a known good compiler available if you run into problems.

[Dolio]

Okay, here's the deal:

I tweaked the script to allow for backup CFLAGS. What you'll have to do is do the following before running rmerge:

export BCFLAGS="..."

Where '...' is whatever you want your backup CFLAGS (and CXXFLAGS) to be. The script assumes that you want your CFLAGS and CXXFLAGS to be the same (which most people do. If this is really a problem, I can change it). I tried making a --bcflags=... command line parameter, but I was unable to do so (bash doesn't seem to let you have spaces in a parameter, even when surrounded by quotes. I don't think escaping it would work either, but if someone can tell me how to do this, I'd be glad to change it).

My main problem is that I don't have a place to host this. I don't think it'd be good to repost to the forum every time (it's very long), and I don't want to submit a new bug for every single fix that goes on over the next few days. Can anyone suggest a free place to host this script so that people will be able to get to it easily?

[Dolio]

P.S.: If anyone wants this stuff right away, they can go hack the script. It's pretty short.

Go find the merge() function in the script. It should look like this:

Code: Select all

merge() {
     if [ $QUIET -gt 1 ]; then
          emerge --oneshot ${1}
     else
          emerge --oneshot ${1} 2>&1 >/dev/null
     fi
}

Now you need to add this to the end (longer than the original function ):

Code: Select all

retval=$?

if [ $retval -eq 1 ] && [ -n "${BCFLAGS}" ]; then
     debug 1 || echo -e "${LR}Merge failed. Using backup CFLAGS${CL}"
     OLDCFLAGS="${CFLAGS}"
     OLDCXXFLAGS="${CXXFLAGS}"
     export CFLAGS="${BCFLAGS}"
     export CXXFLAGS="${BCFLAGS}"
     if [ $QUIET -gt 1 ]; then
          emerge --oneshot ${1}
     else
          emerge --oneshot ${1} 2>&1 >/dev/null
     fi
     retval=$?
     if [ -n "${OLDCFLAGS}" ]; then
          export CFLAGS="${OLDCFLAGS}"
     else
          unset CFLAGS
     fi

     if [ -n "${OLDCXXFLAGS}" ]; then
          export CXXFLAGS="${OLDCXXFLAGS}"
     else
          unset CXXFLAGS
     fi
fi

return $retval

That "will work." It's what I have in the script right now. I'm going to clean up the code a little bit (make functions for duplicated code so it looks nicer ).

Hope that helps.

P.S.: I entered that code by hand. Hopefully I didn't make a typo. Let me know if I did.[/code]

[Beattie]

What exactly does bootstrapping do? Cant you just recompile gcc?

Bootstrapping (oversimplifying a bit) compiles glibc -> gcc (C only) -> glibc -> gcc.

The safest thing to do here is run bootstrap.sh. Stop reading now.

So, this can be done without changing that make.conf or whatever pointer people keep talking about? what about emerging gcc3 in the first place? when I type emerge -p gcc I get version 2.95. What's the proper way to emerge gcc3 itself?

[rac]

So, this can be done without changing that make.conf or whatever pointer people keep talking about? what about emerging gcc3 in the first place? when I type emerge -p gcc I get version 2.95. What's the proper way to emerge gcc3 itself?

There are several profiles in /usr/portage/profiles. They tell portage what version of key programs should be installed. /etc/make.profile is usually a symlink pointing to one of them. In order to emerge GCC 3.1, you need to use the default-1.0-gcc3 profile. To use GCC 3.2, you would want the default-x86-2.0 profile. You would need to change the /etc/make.profile symlink to point to one of these, and then emerge portage should pick up the newer packages.

[echeslack]

I wanted to point people to this thread http://forums.gentoo.org/viewtopic.php? ... highlight= , in case its of any help (though jackp seems to have figured out precisely what I did).

-ewen

[IWBCMAN]

ok ... slowly working my way through an upgrade from gcc-3.2_pre to gcc-3.2 ....just wanted to pass something along. don't know if this is specific to my config but several packages failed to build when I ((re(re-statrted my install. when I typed emerge -e world the emerge failed at gawk(I alo had used rmerge -f ..it failed at gawk, perl, gettext ....and then my perl bailed out....) this time (!) I looked at the error output clearly and figured out that certain programs were being called but not being found. I did a whereis foo found the source and the overkill link approach liinking to bot /bin and /usr/bin and retrying emerge gawk

here are the sym links I had to set...
ln -s /usr/bin/X11/xmkmf /usr/bin/
ln -s /usr/bin/X11/xmkmf /bin/
ln -s /usr/bin/X11/imake /usr/bin/
ln -s /usr/bin/X11/imake /bin/
ln -s /usr/bin/X11/gccmakedep /bin/
ln -s /usr/bin/X11/gccmakedep /usr/bin/

I don't know why I need to do this and I can't say if links need to be made in both directories...and of course this could be a fluke of my config.....but alll these files were available in the PATH.....

[Dolio]

Okay:

I made a little geocities account to store the rmerge tarball. Any changes will be put up there.

Currently, the script on there has the BCFLAGS chage, as well as the skipping option. adding the command line parameter --skip=NUM will skip NUM packages from the start of an rmerge. So if you emerge xfree yourself after it bombs, you can just add --skip=1 and it will skip xfree and go on to the next package.

The current url is:
www.geocities.com/gargamel205/rmerge-2.1.tar.gz

I'll put up an index.html that links to the available versions of the tarball.

If anyone has any further suggestions, I'll make the changes and put them up there.

Cheers.

[Dolio]

Sorry for making so many posts.

A bug got uncovered on the ebuild submission bug report. I fixed it, and uploaded the updated version. So you can get it from two places:

http://bugs.gentoo.org/show_bug.cgi?id=5667: the bug report.
http://www.geocities.com/gargamel205/ the makeshift rmerge homepage.

Make sure you get the 2.2 version. It's the only one that's completely correct as far as anyone knows.

Note also that you can now put your BCFLAGS and BCXXFLAGS (support for both of them, instead of just one set) in /etc/make.conf, as specified in the manpage and the text displayed by the --help option.

Enjoy.

P.S.: note that I changed my second post on this forum, because --fix is not the option I meant (--fix allows you to resume from when you forcibly exit the program without letting it clean up). --lock is actually the option I meant for you to use (otherwise you'll probably end up emerging crap packages). However, as of 2.2 --lock is now enabled by default (because it's 'safe' now), so you no longer need to worry about it.

[tebers]

I was so brave emerge -e world after upgradinmg to gcc3.2
and my system stopped here.
So i think this package is broken.

Code: Select all

compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
version.c: In function `init_Linux_version':
version.c:35: warning: implicit declaration of function `exit'
sysinfo.c: In function `meminfo':
sysinfo.c:255: warning: unsigned int format, different type arg (arg 3)
sysctl.c: In function `ReadSetting':
sysctl.c:343: warning: suggest parentheses around assignment used as truth value
sysctl.c:347: warning: suggest parentheses around assignment used as truth value
sysctl.c: In function `DisplayAll':
sysctl.c:401: warning: suggest parentheses around assignment used as truth value

!!! ERROR: The ebuild did not complete successfully.
!!! Function src_compile, Line 1, Exitcode 2
!!! (no error message)

!!! emerge aborting on  /usr/portage/sys-apps/procps/procps-2.0.7-r6.ebuild .

[dizzy]

Dolio: Could you explain what "backup CFLAGS" and "--skip" do? Sorry I'm in my windows box otherwise I would d/l it and take a look myself.

[Daganoth]

OK, I've been searching the gcc3 forum for an answer that works. I compilied gcc3.2 without any problems, but whenever I do a gcc -v I get 2.95.3.

I changed /etc/env.d/05gcc to:

Code: Select all

LDPATH=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2

...and /etc/make.profiles to point to /usr/portage/default-1.0-gcc3 (or should it be default-x86-2.0?)

Also, is a bootstrap or clean install necessary? or will a emerge system or emerge world -e do the same?

Thanks!

[Dolio]

This is going to be quick, because I have to go soon, but:

if you set BCFLAGS and BCXXFLAGS, then if rmerge detects that an ebuild failed, then it sets CFLAGS=$BCFLAGS and CXXFLAGS=$BCXXFLAGS, and retries the emerge. If it fails then, then rmerge does whatever normally happens when a merge normally fails (either stops or goes on to the next package). If you don't set BCFLAGS or BCXXFLAGS then nothing extra will happen.

--skip=# skips the next # packages in the emerge. So, if you're starting anew, and you use --skip=1, then it will skip the first package in the sequence. If you're resuming, then it will skip the next package to be merged.

So, if your rmerge dies on X, you can tweak the flags, get it to emerge, and then do --skip=1 to resume, skip X and continue on with subsequent packages.

Hope that helps.

[IWBCMAN]

Dagonath

you should set your link to default-x86-2.0
I recommend making a back up of your /etc(and any other important ones(root etc.) directory if you plan to do this upgrade, if something bombs you may need to unpack the stage3-1.4 tarball to get things working again-and this will overwrite all your config files(and your passwd!)

you shouldn need to bootstrap or clean install, but it might be easier just staarting from scratch...

I would:
first emerge gcc3.2
then change /etc/env.d/05gcc
then set link to default-x86-2.0
then emerge gcc3.2 again(recompiling 3.2 with 3.2-try USE="bootstrap"-maybe quicker)
then emerge glibc
then emerge -e world
and use conservative optimizations ie. no more than -march=i686 -O3 -pipe -fomit-frame-pointers)
later you can recompile specific things with more aggressive settings....
patience, is here, a virtue....if your optimizations are conservatice enough you might make it all the way through emerge -e world without any packages failing!

[Daganoth]

I still had the problem of having the wrong gcc versoin as default. I unmerged both versions of gcc2.95.3 (which were both protected ) and tried a gcc -v. The symlink for gcc was deleted, but there was an executable called gcc-3.2. Thus I symlinked it too gcc, the version is now correct, and I'm compiling gcc3.2 now.

Hopefully is cool sailing from now on. IWCBMAN, thanks!

[tebers]

Solved it by changing my makeoptions.
I had "-j8 -s", I comment them and remerged procps.
I think the -s flag was the cause for that.

I was so brave emerge -e world after upgradinmg to gcc3.2
and my system stopped here.
So i think this package is broken.

Code: Select all

compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
version.c: In function `init_Linux_version':
version.c:35: warning: implicit declaration of function `exit'
sysinfo.c: In function `meminfo':
sysinfo.c:255: warning: unsigned int format, different type arg (arg 3)
sysctl.c: In function `ReadSetting':
sysctl.c:343: warning: suggest parentheses around assignment used as truth value
sysctl.c:347: warning: suggest parentheses around assignment used as truth value
sysctl.c: In function `DisplayAll':
sysctl.c:401: warning: suggest parentheses around assignment used as truth value

!!! ERROR: The ebuild did not complete successfully.
!!! Function src_compile, Line 1, Exitcode 2
!!! (no error message)

!!! emerge aborting on  /usr/portage/sys-apps/procps/procps-2.0.7-r6.ebuild .

[Kumba]

Okay, so reading through all of this, on a Gentoo 1.3b system, running only console stuff (none of that X garbage), and following what rac said, I need to do the following:


1. Symlink /etc/make.profile to /usr/portage/profiles/default-x86-2.0
2. unmask gcc-3.2 in the /usr/portage/profiles/package.mask
3. edit /etc/make.conf and add "bootstrap" to the USE variable
4. emerge /usr/portage/sys-devel/gcc
5. emerge /usr/portage/sys-libs/glibc
6. edit /etc/make.conf and remove "bootstrap" from the USE variable
7. emerge /usr/portage/sys-devel/gcc (Second time)
8. emerge -e world
9. reboot (to be safe)


Am I close by chance? This seems to me to be the logical steps neccessary to update an existing/running system of this nature. I also use a kernel I compiled by hand, outside of Gentoo's emerge system, I take it that would also need recompiling, just to play it on the safe side?


--Kumba

[Daganoth]

*Looks over the death and destruction and sighs* Its sad when you have to use Window$ XP as a backup. I believe I'll wait for 1.4

[Tuxisuau]

i686-pc-linux-gnu-g++ -DGSL_WANT_GLIB_WRAPPER -DGSL_WANT_ARTS_THREADS -DNDEBUG -DNO_DEBUG -O2 -march=athlon -O3 -pipe -fno-exceptions -fno-check-new -ftemplate-depth-99 -o .libs/gslffttest gslffttest.o dummy.o ./.libs/libgsl.a /var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/mcop/.libs/libmcop.so -L/lib -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2 -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/../../../../i686-pc-linux-gnu/lib -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.2/../../.. -L/usr/i686-pc-linux-gnu/bin -L/usr/i686-pc-linux-gnu/lib -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/gcc-lib/i686-pc-linux-gnu/3.1 -L/usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/../../../../i686-pc-linux-gnu/lib -lresolv -ldl /usr/lib/gcc-lib/i686-pc-linux-gnu/3.1/libstdc++.so -lm -lc -lgcc_s -Wl,--rpath -Wl,/usr/kde/3/lib -Wl,--rpath -Wl,/usr/lib/gcc-lib/i686-pc-linux-gnu/3.1
creating gslffttest
make[3]: Leaving directory `/var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/flow/gsl'
make[3]: Entering directory `/var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/flow'
../mcopidl/mcopidl -t ../flow/artsflow.idl
lt-mcopidl: error while loading shared libraries: libstdc++.so.4: cannot open shared object file: No such file or directory
make[3]: *** [artsflow.h] Error 127
make[3]: Leaving directory `/var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/flow'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/var/tmp/portage/arts-1.`/var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/flow'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2'
make: *** [all] Error 2

!!! ERROR: The ebuild did not complete successfully.
!!! Function kde_src_compile, Line -7414, Exitcode 2
!!! died running emake, kde_src_compile:make

Notice that ../mcopidl/mcopidl its /var/tmp/portage/arts-1.0.2-r1/work/arts-1.0.2/mcopidl/mcopidl

So, what the fuck is going on? I need that to be able to rebuild KDE3.0.2

[dizzy]

*Looks over the death and destruction and sighs* Its sad when you have to use Window$ XP as a backup. I believe I'll wait for 1.4

Agreed. Why put yourself through the pain if you can't solve half of the error that comes up? Wait for the final release and be happy. People are so impatient these days.

Having said all that I'm d/ling the 1.4 testing stuff right now. Can't wait!

[samppa]

I had the same problem. I had -j3 on make.conf, commented that out, and it worked after that. So -s should not be the problem here.

Solved it by changing my makeoptions.
I had "-j8 -s", I comment them and remerged procps.
I think the -s flag was the cause for that.

I was so brave emerge -e world after upgradinmg to gcc3.2
and my system stopped here.
So i think this package is broken.

Code: Select all

compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:143:17: warning: pasting "->" and "resident" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
compare.c:144:14: warning: pasting "->" and "share" does not give a valid preprocessing token
version.c: In function `init_Linux_version':
version.c:35: warning: implicit declaration of function `exit'
sysinfo.c: In function `meminfo':
sysinfo.c:255: warning: unsigned int format, different type arg (arg 3)
sysctl.c: In function `ReadSetting':
sysctl.c:343: warning: suggest parentheses around assignment used as truth value
sysctl.c:347: warning: suggest parentheses around assignment used as truth value
sysctl.c: In function `DisplayAll':
sysctl.c:401: warning: suggest parentheses around assignment used as truth value

!!! ERROR: The ebuild did not complete successfully.
!!! Function src_compile, Line 1, Exitcode 2
!!! (no error message)

!!! emerge aborting on  /usr/portage/sys-apps/procps/procps-2.0.7-r6.ebuild .

[IWBCMAN]

Dagonath

sorry to here that the process bombed on you....
I myself ave been working on this update for 4 days now, and I have thought I actually had it completeted at least a half dozen times only to find out that I was wrong....but I personaly enjoy the challenge, my system proficiency has reached a level I never dreamed of and I feel like I am begginning to understand how all of the parts of this big puzzle (linux) fit together...I have lost nothing serious in this process, I made backups of key directories in case something happens, and I have unpacked the stage3-1.4 tarball on top of my running systems several times(ie. when python, portage, gcc or glibc bailed out on me) -yet I have not lost my world-file, my configuration files are still there, and I now have %70 percent of my machine successfully recompiled-

luckily I have suse8.0 installed on another partition- it is a rreeal life saver -ie. if I forget to reset my passwd after unpacking the stage3 tarballl!!!!
or forget to copy over my old fstab!!!!!!!!!!!!!!!!!!