GCC 4.1.1 and glibc 2.4 stable on x86 and amd64

[literfizzer]

That post doesn't explain why it's necessary to emerge system, then world. On my system, at least, emerge -e world recompiles everything that was already recompiled in the previous emerge -e system step. This makes no sense to me. The recompile is because of a C++ ABI change. It has nothing to do with other dependent libs, kernel headers, or anything else, as some of the other replies have said. So recompiling more than once is a waste. It will generate exactly the same object code the second time.

Code: Select all

$ emerge --pretend --emptytree system | grep '^\[ebuild' | sed 's/^.*] //' | sort >/tmp/system
$ emerge --pretend --emptytree world | grep '^\[ebuild' | sed 's/^.*] //' | sort >/tmp/world
$ diff -u /tmp/system /tmp/world | grep '^-'

It's the same reason fresh installs are bootstrapped, compiling the toolchain multiple times. Assuming you start with gcc 3.4.6 and follow the guide:

emerge -u gcc gives you a gcc 4.1.1 compiled with gcc 3.4.6.
emerge -e system gives you a toolchain partially compiled with a gcc 4.1.1 that was compiled with gcc 3.4.6, and partially compiled with a gcc 4.1.1 that was compiled with gcc 4.1.1 (including gcc itself)
emerge -e world gives you everything compiled with a gcc 4.1.1 that was compiled with gcc 4.1.1

Not that I would call all that "necessary."

You're right, it's not. The gcc build itself does a three-stage build: once against the current system gcc, a second time against the new version of gcc just built (but not yet installed), and a third time, to sanity-check that the resulting binaries are the same as in stage 2.

In any case, gcc has already been rebuilt this way even before doing `emerge -e system`. If you then do `emerge -e world`, gcc has been re-merged three times, for a total of nine complete builds! This seems crazy.

Where did this bit of tribal knowledge come from? Whoever first pronounced 'you must re-merge system, then world' must have had a reason.

[depontius]

won't nptlonly break binary packages that don't support it? what are the ups and downs in enabling/not enabling it?

NPTL is fully backwards compatible with LinuxThreads. At least I've never had a problem with binary only stuff.

Then perhaps my problems weren't with nptlonly, but with some other aspect of my system. I had a system built with nptlonly, but found that there were several binary-only CAD (Computer Aided Design - Silicon CAD, more specifically) that I simply couldn't run. Once things got to "LD_ASSUME_KERNEL=2.4.1 (or 2.4.19) it was as if the system got amnesia, and couldn't run spit in that session, with ld.so complaining about not being able to find libc.so.5.

I'd love it if someone could tell me what went wrong after LD_ASSUME_KERNEL=2.4.x, so I could go ahead and do the full upgrade. Otherwise it looks like staying at glibc-2.3 is going to make Gentoo into a real pain. I plan to upgrade my home systems as I get time, but won't touch my work systems until this is resolved.

Actually, I need OpenAFS at work, and at present it says in the ebuild that it simply won't work with glibc-2.4, so I'm in a pickle on my work machines. The incompatibility is flagged in the ebuild, but I see nothing about it in the FORUMs. Is this on the radar screen?

[chtephan]

Yes, but there are also some glibc - gcc cross-dependencies which are very important when moving from a non-nptlonly to an nptlonly system. In order get this right when both are upgrade, they both should be compiled at least twice. By following the official upgrade guide everything should work.
First, gcc is compiled. This makes the c++ library symbols it would like to use available. It also makes all subsequent builds use the gcc 4 ABI. Then system is reemerged. This means that gcc is rebuilt against dependencies that were compiled using gcc 4 and you get a glibc using gcc 4 features. Then, when emerging world, gcc is rebuilt a third time and can now use features provided by a gcc4-compiled and nptl enabled glibc.

You can probably get away with even fewer recompiles but you really have to know what you are doing. emerge -e system and emerge -e world twice is really overkill. This should only be required if some ABI has really completely changed and it is required to have dependencies of dependencies linked against the correct version and stuff.

[drphibes]

what i do is first nail down the toolchain with two passes:

Code: Select all

emerge --oneshot linux-headers glibc binutils-config binutils gcc-config gcc
emerge --oneshot linux-headers glibc binutils-config binutils gcc-config gcc

then i build the system and save packages simultaneously:

Code: Select all

emerge --emptytree --buildpkg system

and then i build world, but tell it to use system packages already built:

Code: Select all

emerge --emptytree --usepkg world

this has worked reliably for me and I have the last step in progress now.

[oldefortran]

Is there any programs that have problems with the new compiler?

[drphibes]

the only thing i am noticing is many more warnings due to -fstrict-aliasing now being implied with -O2
and higher optimization. of course any program built with -Werror will not compile with such warnings,
but i haven't run into one yet.

[geniux]

GCC upgraded without errors, but the nvidia module wont work anymore. I've reinstalled, unmerged and remerge, even tried the unstable versions of both media-video/nvidia and x11-drivers/nvidia-driver but it gives me the same message:

Code: Select all

FATAL: Error inserting nvidia (/lib/modules/2.6.17-beyond2.2/video/nvidia.ko): Invalid module format

Anyone else who got this problem? Everything else are working just fine

[drphibes]

GCC upgraded without errors, but the nvidia module wont work anymore. I've reinstalled, unmerged and remerge, even tried the unstable versions of both media-video/nvidia and x11-drivers/nvidia-driver but it gives me the same message:

Code: Select all

FATAL: Error inserting nvidia (/lib/modules/2.6.17-beyond2.2/video/nvidia.ko): Invalid module format

Anyone else who got this problem? Everything else are working just fine

did u rebuild the kernel with the new toolchain (you should)?

[CRV§ADER//KY]

-Os is for embeded system. It depend also of the arch, but on x86, it is not recommended for a desktop.

nonsense. I've been using -Os for over a year on my AthlonXP 2000+ desktop AND my celeron 466 laptop and I'm very happy with it. it's stable, slightly faster than -O3 (which in turn is INSANELY bigger), and much faster than -O2. As a plus, I save disk space, RAM and loading times.


insight-6.1 failed. 563 packages to go....

[CRV§ADER//KY]

GCC upgraded without errors, but the nvidia module wont work anymore. I've reinstalled, unmerged and remerge, even tried the unstable versions of both media-video/nvidia and x11-drivers/nvidia-driver but it gives me the same message:

Code: Select all

FATAL: Error inserting nvidia (/lib/modules/2.6.17-beyond2.2/video/nvidia.ko): Invalid module format

Anyone else who got this problem? Everything else are working just fine

has your kernel been compiled with the new toolchain? I bet not

[geniux]

did u rebuild the kernel with the new toolchain (you should)?

has your kernel been compiled with the new toolchain? I bet not

My kernel were compiled with the new toolchain, I followed the steps I've read at the forum, but I choose the wrong kernelimage at boot Sorry about that guys. Thank you much anyway

[gami]

Am I the only one here who has seen a broken compiler after the upgrade? This is strange because the methods I used on two systems worked for one but hosed the other.....

Any one with:

Code: Select all

checking for C compiler default output file name... configure: error: C compiler cannot create executables

or

Code: Select all

/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/../../../../i686-pc-linux-gnu/bin/ld:/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/../../../libc.so: file format not recognized; treating as linker script
/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/../../../../i686-pc-linux-gnu/bin/ld:/usr/lib/gcc/i686-pc-linux-gnu/3.4.6/../../../libc.so:5: syntax error
collect2: ld returned 1 exit status

please speak up (there is power in numbers)...

p.s. I get this when selecting gcc 4.0, 4.1, 3.3 or 3.4. And as it happened only after I emerged the new glibc package that is where I am pointing my finger.

I had this issue, too, that halfway through emerge -eav system it suddenly could no longer compile more packages. A little bit of digging around on the system showed that there were some older slotted versions of binutils still installed The latest version (2.16.*) had not overwritten the symbolic links installed by the 2.15.* version(s) of binutils. I uninstalled the older versions (emerge -Cav "<binutils-2.16*") and could continue the interrupted emerge (emerge --resume).

[literfizzer]

My kernel were compiled with the new toolchain, I followed the steps I've read at the forum, but I choose the wrong kernelimage at boot Sorry about that guys. Thank you much anyway

That makes sense. I believe that only the C++ ABI changed, so C programs don't need to be recompiled when upgrading to gcc-4.1.1.

[thorak]

Brilliant, thanks gami, that fixed it for me!
If you're ever in Oslo, let me know and I'll buy you some beers.

[dave-gallagher]

[sonicbhoc]

won't nptlonly break binary packages that don't support it? what are the ups and downs in enabling/not enabling it?

NPTL is fully backwards compatible with LinuxThreads. At least I've never had a problem with binary only stuff.

My glibc has (-nptl) and (-nptlonly) and I can't change it <_<

Without nptl I can't have a bootsplash... I was hoping I could put one on there for my pastor, who's computer is nearly perfect now, aside from this last issue.

So what am I doing wrong?

[STrRedWolf]

So, the main problem is that many packages are rebuilt up to three times, and that chews up time... but if we do some filtering, we can cut this out.

Code: Select all

# Build the listings...
emerge -p gcc glibc libtool | perl -nle 'print $1 if(/\]\s+(\S+)\s+\[/);' > gcc.txt
emerge -ep system | perl -nle 'print $1 if(/\]\s+(\S+)\s+\[/);' > system.txt
emerge -ep world | perl -nle 'print $1 if(/\]\s+(\S+)\s+\[/);' > world.txt

# Filter already built ebuilds out.  We need sort, of course, because uniq only compares two lines at a time.
cat gcc.txt system.txt | sort | uniq -u | xargs echo emerge -av > build.system
cat gcc.txt system.txt world.txt | sort | uniq -u | xargs echo emerge -av > build.world

Then "sh build.system" instead of "emerge -eav system" and the same for world.

Now only if they can put in a --readlistfrom option in emerge...[/code]

[depontius]

Yes, but there are also some glibc - gcc cross-dependencies which are very important when moving from a non-nptlonly to an nptlonly system. In order get this right when both are upgrade, they both should be compiled at least twice. By following the official upgrade guide everything should work.
First, gcc is compiled. This makes the c++ library symbols it would like to use available. It also makes all subsequent builds use the gcc 4 ABI. Then system is reemerged. This means that gcc is rebuilt against dependencies that were compiled using gcc 4 and you get a glibc using gcc 4 features. Then, when emerging world, gcc is rebuilt a third time and can now use features provided by a gcc4-compiled and nptl enabled glibc.

You can probably get away with even fewer recompiles but you really have to know what you are doing. emerge -e system and emerge -e world twice is really overkill. This should only be required if some ABI has really completely changed and it is required to have dependencies of dependencies linked against the correct version and stuff.

Were you answering me or someone else?

Are you saying that the problems I had with nptlonly and LD_ASSUME_KERNEL=2.4.x killing ld.so were not strictly related to nptlonly, and that it might have been my build sequence?

I guess I'm still stuck with glibc-2.4 problems though, because OpenAFS won't build with it, and I need OpenAFS.

[hielvc]

STrRedWolf youir method will screw your system up. Your totally throwing out the build time dependencies. Look at my script
Note This isnt in the upgrade guide
An emerge wrapper for more correctly building the toolchain. If you choose to use emwrap.sh you need to

Code: Select all

 USE="-fortran" emerge -1O gcc 
gcc-config  -l   <<< lower case L
gcc-config  #    <<< number of gcc-4.1.1
emwrap.sh -seb1  << builds TC, then system files minus the TC
emwrap.sh -S     <<< this is to be safe. Rebuilds the system minus the TC files. If I have probs I do it and a 2nd "-W"
emwrap.sh -W     <<< builds the world files minus the system files minus the TC

This also a good time to update your /etc/make.profile link.

Code: Select all

cd /etc
rm make.profile
ln -s /usr/portage/profiles/default-linux/x86/2006.1/desktop /etc/make.profile

Then do the above.

EDIT Fixed details

[icb410]

Hi,

I started upgrading to gcc 4.1.1 and am now doing the 'emerge -e system' step. However, I realized that instead of doing:

fix_libtool_files.sh 3.4.6

I did:

fix_libtool_files.sh 4.1.1

Is this going to screw me over? Should I stop my 'emerge -e system' and go back to the previous steps?

Thanks,
Ian

[Nicias]

Ok, I have seriously screwed up my system I think.

A few of my friends share emerge jobs by doing a central emerge using distcc and then distributing the binaries and using -k.

So I when gcc4 when stable, I did my part though rebuilding libtool. Then his machine did the emerge -e system's and world's.

I copied the binary packages, and then *I think here is my error* did a emerge -ek world.
(maybe I should have done system first?)

Now nothing will emerge. Worse then that, things like "rm", "install", and "mv" don't work. I get these kinds of errors:

Code: Select all

rm: /lib/tls/libc.so.6: version `GLIBC_2.4' not found (required by rm)

How in the world do I fix this? I guess I should reboot from a CD etc, but I don't want to try and just muck around until I fix it somehow. I would rather have a plan I know will work.

My idea is this, put in an install CD, boot into that. And then install they binary packages I already have for system by changin $ROOT and $PORTAGE_CONFIGROOT to be /mnt/gentoo/ (If I chroot I will have the same problems)

Will this work?


Update:
It worked, I also had to manually set PORtDIR and PKGDIR, but then an emerge -k1 glibc worked.

[Gergan Penkov]

you should find a way to install glibc from your friends bin pkgs, it seems that you are missing it.
the correct order would have been, emerge -ek glibc and after that emerge -ek system (or probably just world)

[Isaiah]

I want to upgrade from gcc 3.4 to gcc 4.1 - which instructions do i use?
Please follow the general instructions.

Well done people - this was perfect (no runs, drips, or errors) - thank you very much

[Orific]

During my emerge -e world, libdv broke my up to now flawless gcc upgrade .
So after investigation, I found out that libdv build fails with gcc-4 without a certain patch (see here and here).
Now that patch is used in the latest ~x86 version (libdv-0.104-r1), but not in the stable version (0.102), which I am using.
My question is: could I add "media-lib/libdv ~x86" to my /etc/portage/package.keywords now without breaking --resume ?
Thanx.

[DrSnoegler]

why sys-libs/glibc-2.4-r3 has no glibc-compat20 useflag? i really really use this to run matlab!

I've wrongly updated glibc to 2.4-r3 and now I can't downgrade back to sys-libs/glibc-2.3.6-r4.
I've still sys-devel/gcc-3.4.6-r1 installed and really have to downgrade glibc!
can I edit the ebuild to do this anyway?

@gian:
have you still keep the old version to save 'your application' ?