Stage 1/3 Installation Support - Gentoo 2004.3 & GCC 343

Message

Bob P · Post by **Bob P** » Wed Mar 23, 2005 4:59 am

you know, kimchi_sg, sometimes you can be a real pain in the butt.

i finished another Stage 1/3 install today on the same pentium classic

computer that i'd been using as one of the testbeds for the Stage 1/3 Guide. much to my surprise, that PC got caught by the /etc/mtab problem, even though my logs show that it has never been caught by that problem on the 5 previous installs i had performed on it. the problem also cropped up on another testbed that finished the install today, and that PC has never had the problem before either. so it seems that something has changed in some of the ebuilds between the time that i developed the Guide and now, as machines that had never been effected before are now being effected by the /etc/mtab problem. i updated the Guide earlier today and just finished posting the results.

performing a test installation on one of my test PCs takes a solid week of compile time to get through the Stage 1/3 install. finding a problem, changing the install, and starting over takes two solid weeks. if i'm not working fast enough for you guys, you could always send me a bunch of faster PCs to work with.

until that happens, there's no point in complaining.

96140 · Post by **96140** » Wed Mar 23, 2005 6:52 am

Bob P wrote:so it seems that something has changed in some of the ebuilds between the time that i developed the Guide and now, as machines that had never been effected before are now being effected by the /etc/mtab problem.

Ahh, /etc/mtab vindication at last! Wish it didn't have to be the hard way, though. I knew the fix I wrote up a few pages ago in this thread had to solve problems a lot of users were having, as it didn't seem related to the newness of one's hardware (or lack of newness). From what you're saying, Bob, it sounds like most users would have been affected by this if indeed something important was changed in an ebuild after you wrote the guide. But, which one? I wouldn't have thought that the stage 3 tarballs would have been updated, so it must occur somewhere in an emerge sync? Some sneaky package lying in wait with slight changes that result in a lot of grief for the end user. *sigh*

And if someone's sending out complimentary PCs, feel free to send one my way!

Bob P · Post by **Bob P** » Wed Mar 23, 2005 7:15 am

nightmorph wrote:And if someone's sending out complimentary PCs, feel free to send one my way!

read this.

IcEr3K · Post by **IcEr3K** » Wed Mar 23, 2005 10:39 am

df -h gives me the following:

Code: Select all

Filesystem		Size  Used Avail Use% Mounted on
rootfs			7.9G 1010M  7.0G  13% /
tmpfs			7.9G 1010M  7.0G  13% /
/newroot/dev/cdroms/cdrom0
			7.9G 1010M  7.0G  13% /mnt/cdrom
df: `/mnt/livecd': No such file or directory
none			7.9G 1010M  7.0G  13% /sys
none			7.9G 1010M  7.0G  13% /dev
none			7.9G 1010M  7.0G  13% /dev/pts
none			7.9G 1010M  7.0G  13% /dev/shm
df: `/mnt/livecd/usr/lib/hotplug/firewire': No such file or directory
/dev/hda3		7.9G 1010M  7.0G  13% /
/dev/hda1		40M    33M  7.2M  82% /boot

Sorry if it's not aligned. I had to re-type it all manually.

Turns out that the last four lines of my compile error gives me the following:

Code: Select all

VM: killing process make
make[1]: *** [iconvdata/others] Killed
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4'
make: *** [all] Error 2

Needless to say, this was during the compiling of glibc (emerge -e system).

Thanks in advance!

torchZ06 · Post by **torchZ06** » Wed Mar 23, 2005 6:26 pm

hielvc wrote:Im on a 899 via c3 samuel 2 right and if you figure out make this as snapy as my athlon-xp 2000 Id be real happy

hi hielvc

cool! those via's arent bad little chips.... ive done stage 1 builds targeting the via on my p4s and they always worked fine except i never used that 3dnow flag until now ARRGH!

i re-tried the stage1-on-3 last night but this time on the via board instead of the p4 and it just successfully completed that step 7.1! so i guess you've got 2 options:

* if you're on a intel system, remove the 3dnow flag and do the build on a zippy box

* or if you've got a amd that has 3dnow support you're in luck! the build should work fine on that box, then transfer your image down to the via board... or play it safe and be patient while you do build on the via itself.

bob,

you're right, i was 95% sure it was not related to your method and should have made a separate post. but, when i get this one finished, you'll be able to say there's a 1-on-3 running on a very cool mobile robot!

cheers

Bob P · Post by **Bob P** » Wed Mar 23, 2005 8:52 pm

kimchi_sg may like to poke fun at me for using pentium classic as the default architecture for the Stage 1/3 guide -- i think that some of this is attributable to the fact that the's a ricer at heart -- but believe it or not, i did not choose "pentium" by accident, or because i was working exclusively with old hardware.

there's a good reason to specify "pentium" as the default architecture, especially if you're interested in performing installs on a wide variety of boxes -- it provides cross-platform compatability for all of the chips that have descended from the pentium architecture, and it effectively sidesteps the flag compatability problems that you've encountered.

for example, my P3 box has a spare HD and a spare gentoo installation. its built with flags for "pentium" even though its pentium 3. why? because when its time to install gentoo on a PPro, a K6, Pentium-MMX, or an Athlon, i'm not going to spend up to a week building each new system. all that i have to do is clone the hard disk, and stick it in the new box. at worst, i may have to make a few changes to the kernel. the new box is up and running in an hour or so.

on the other hand, if you want to trudge through the entire process of rebuilding glibc and gcc redundantly for every system you want to install, you can, but imho that amounts to doing things the hard way.

believe it or not, there's not THAT big of a difference in performance on most code between most processors running pentium code and their native code. the amount of incremental performance that is gained by generating optimized native code is lost many times over on the front-side as installation time. if you want to be up and running quickly, disk cloning from an ancestral platform is the way to go. if you're adamant that machine specific optimizations are necessary, its always more expedient and convenient to recompile on the machine after the OS is installed than it is to go through the trouble of building an optimized OS for each machine from the ground up.

i think that you've caught onto the right idea after perfoming a couple of the installs. the reason that you weren't successful is that you were compiling for too modern an arch to make the OS portable. if you're planning on performing the Stage 1/3 install on a large number of PCs, your best approach is to determine the most recent ancestral platform that all of your machines will share in common, perform the installation ONCE, and then clone the installation to your other PCs. then if you feel the need, rebuild each PC afterward.

just to make this clear -- the ancestral platform doesn't have to be pentium if you've 100% intel or 100% amd. it just needs to be the most recent platform that all of your boxes are derived from. i just used pentium in the guide and on my boxes because its the least common denominator for me.

hielvc · Post by **hielvc** » Thu Mar 24, 2005 1:06 am

Isnt that most common denominator

kimchi_sg · Post by **kimchi_sg** » Thu Mar 24, 2005 1:09 am

hielvc wrote:Isnt that most common denominator

hielvc · Post by **hielvc** » Thu Mar 24, 2005 1:24 am

MCD and LCD if I rememeber my algebra Humm then again I might have that backwards you dastardly pointer outer

See thats how you call peeps names and dont get kicked to hard, I HOPE

sciwhiz007 · Post by **sciwhiz007** » Thu Mar 24, 2005 2:35 am

I'm about to take the plunge and try out this really cool installation, but I have a question. I like running hardened Gentoo, so has anyone tried putting "hardened pie pic" in their USE flags when using this method? Or am I going to have to be the guinea pig?

Is hardened known to break with NPTL and GCC 3.4.3?

If this question has been answered elsewhere, I'm afraid I couldn't find it. I apologize in advance and will gladly accept a link to a post.

Cheers, and two thumbs up to Bob P!

IcEr3K · Post by **IcEr3K** » Thu Mar 24, 2005 3:15 am

Through logical reasoning, I have figured out that my error only occured with the emerging of glibc. During the make process, the iconvdata/others somehow causes some sort of memory problem. I only have 128MB of RAM on the laptop that I'm running gentoo on. I've been following the guide and up until "emerge -e system" i've been pretty much smooth sailing.

Help me please.
Thanks in advance,
Benny

kimchi_sg · Post by **kimchi_sg** » Thu Mar 24, 2005 4:23 am

sciwhiz007 wrote:I'm about to take the plunge and try out this really cool installation, but I have a question. I like running hardened Gentoo, so has anyone tried putting "hardened pie pic" in their USE flags when using this method? Or am I going to have to be the guinea pig? Is hardened known to break with NPTL and GCC 3.4.3?

USE="hardened" will work fine during the install.

sciwhiz007 · Post by **sciwhiz007** » Thu Mar 24, 2005 4:44 am

Hmm, sorry if I'm being a little overtly careful, but don't I need "pie pic" as well in my use flags for a hardened toolchain? Or will just putting "hardened" alone do the job? Also, is it possible to have distcc set up to help out an old pc with compilation, as long as I have an identical toolchain? My setup is as follows:
- A workhorse AMD64 (running in 32-bit legacy mode, no x86_64) win32 box that runs virtual Gentoo boxen on VMware (dual-boot is too much of a hassle, and this box is fast enough to run both simultaneously

) The Gentoo VM serves as a distcc server for my less "able" clients.
- A relatively old P4 that's being transformed into a full-time Gentoo server/workstation. Currently, it runs a plain 2.6 hardened installation with virtually no ~x86 packages.

As I have a few days off, I'd like to try out this method, but first on the VM as I can then use it to give other boxes a hand with compiling. (The only requirement for successful distcc'ing is that all compilation servers possess the same toolchain, right?) Once I get the virtual box running, I'll upgrade the P4 and any other boxes I can salvage safely

Do you think I will have any issues with my plans? My only requirements are a hardened system and preferably, distcc (and of course, NPTL and all the other goodies that are promised by this installation

)

Cheers, and thanks in advance!

(EDIT: P.S. - One more thing, I won't have any trouble with using hardened-dev-sources and GRSecurity/PaX, right?)

kimchi_sg · Post by **kimchi_sg** » Thu Mar 24, 2005 4:52 am

sciwhiz007 wrote:One more thing, I won't have any trouble with using hardened-dev-sources and GRSecurity/PaX, right?)

That's enough "one more thing"s for a day already.

I'm not sure if a lot of people are using hardened stuff with this tutorial. Therefore, I can only say: Please be a guinea pig, and test out the hardened toolchain with this tutorial yourself. As Bob P would say, what you're doing is "unsupported" according to this tutorial.

sciwhiz007 · Post by **sciwhiz007** » Thu Mar 24, 2005 5:09 am

Alright, thanks for the help. Heck, it's only a VM anyway.

Sorry if I got a little too many questions crammed in there.

southpaw · Post by **southpaw** » Thu Mar 24, 2005 6:40 pm

Hey guy's,

I have an installation question in reference to section 6.5 Configure USE Flags....
I'm using this method to install gentoo over a previous standard gentoo installation.
Are there any other options I should consider adding to /etc/make.conf so that I could accomplish a clean install without having to zero out my drive???

Any suggestion's are appreciated thanx

Bob P · Post by **Bob P** » Thu Mar 24, 2005 8:37 pm

regarding the use of distcc:

the first thing to do is to get the syntax right. i used to refer to fast boxes as "distcc servers," but it turns out that i was wrong in doing so. technically, fast boxes that assist with distcc compilations are supposed to be referred to as "compiling assistants." technically, the PC that is hosting the compile operation is the server. it does the preprocessing and assigns compiling tasks to the assistants. i'm not trying to be an arse by pointing this out -- its just that the one person referring to the host as the "server" and the other person referring to the assistant as the "server" can make things pretty confusing. to make sure we're on the same page, the best way to talk about this is to refer to the PC that's doing the preprocessing as the "host" or "server", and the other PCs on the LAN as "assistants."

i have a hobby farm of 6 PCs that serve as compiling assistants under distcc. one would think that this really helps to speed things up, but in reality it does not. your ability to distribute tasks is subject to the ability of the host machine to preprocess and distribute, so most of your assistants end up in a wait state. i won't belabor this here, as we've discussed this on many threads in the forum already.

personally, i think its fine to use distcc to assist with the rebuild of the world files, but i would NOT recommend using distcc for rebuilding the toolkit and system files. IME its more trouble than its worth.

Bob P · Post by **Bob P** » Thu Mar 24, 2005 8:38 pm

hielvc wrote:Isnt that most common denominator

wow, i'm glad that 3/23/05 is over. it was a good day for everyone to poke me with a sharp stick.

hielvc wrote:... See thats how you call peeps names and dont get kicked to hard, I HOPE

don't worry -- i won't go cry to @anme.

Bob P · Post by **Bob P** » Thu Mar 24, 2005 8:49 pm

kimchi_sg wrote:
sciwhiz007 wrote:One more thing, I won't have any trouble with using hardened-dev-sources and GRSecurity/PaX, right?)
That's enough "one more thing"s for a day already.

I'm not sure if a lot of people are using hardened stuff with this tutorial. Therefore, I can only say: Please be a guinea pig, and test out the hardened toolchain with this tutorial yourself. As Bob P would say, what you're doing is "unsupported" according to this tutorial.

I've never done it. One obvious thing that you should plan for is that after emerging the new toolkit you will need to change the command "gcc-config 2" to whatever is appropriate for your situation:

Code: Select all

gentoo linux # gcc-config -l
[1] i386-pc-linux-gnu-3.3.4
[2] i686-pc-linux-gnu-3.4.3-20050110 *
[3] i686-pc-linux-gnu-3.4.3-20050110-hardened
[4] i686-pc-linux-gnu-3.4.3-20050110-hardenednopie
[5] i686-pc-linux-gnu-3.4.3-20050110-hardenednossp

when you've successfully finished the hardened installation, report back to us! that kind of information would be a helpful addition to the other thread.

now there! i didn't use the word "unsupported" in my post!

Bob P · Post by **Bob P** » Thu Mar 24, 2005 9:22 pm

a gentle reminder:

I will not repond to any Personal Messages requesting support for the Stage 1 on 3 Installation Guide.

All Personal Messages sent to me will be deleted without a reply, so please post your support requests in this thread.

Thanks, and have fun!

this policy includes help requests for implementing distcc with the Stage 1/3 guide. the use of distcc to build your toolkit is subtly dangerous. it is unsupported and i advise against it! if you choose to do it, then proceed at your own risk!

sciwhiz007 · Post by **sciwhiz007** » Fri Mar 25, 2005 1:14 am

Bob P wrote:a gentle reminder:

I will not repond to any Personal Messages requesting support for the Stage 1 on 3 Installation Guide.

All Personal Messages sent to me will be deleted without a reply, so please post your support requests in this thread.

Thanks, and have fun!

this policy includes help requests for implementing distcc with the Stage 1/3 guide. the use of distcc to build your toolkit is subtly dangerous. it is unsupported and i advise against it! if you choose to do it, then proceed at your own risk!

Hmm, I don't think I PMed you or asked for help with distcc did I? No, I'm not getting antsy or anything, just checking..

Anyway, I've decided not to touch distcc for a while, this being a new method and all.

I enabled the "hardened" USE flag and tried installing on the VM using this method, but ran into the same problem twice. To be more specific, emerging glibc at step 7.1 fails at the exact same place. As I am not sure if this is a hardened related issue, I'd appreciate it if someone can tell me if they've encountered this error when not using hardened. I have included as many lines of the error as possible, and also the output from emerge info. For the record, I followed the given instructions practically word-for-word.

Code: Select all

   
   usr/lib/gconv/ISO-2022-CN-EXT.so
   usr/lib/gconv/ISO_10367-BOX.so
   usr/sbin/zic
   usr/sbin/nscd
   usr/sbin/nscd_nischeck
   usr/sbin/zdump
   usr/sbin/rpcinfo
   usr/sbin/iconvconfig
   sbin/sln
   sbin/ldconfig
shell-init: could not get current directory: getcwd: cannot access parent directories: No such file or directory
 * Installing info pages...
make -r PARALLELMFLAGS="-j2 -j1" CVSOPTS="" -C /var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4 objdir=`pwd` info
make[1]: Entering directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4'
make -j2 -j1 -C manual info
make[2]: Entering directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4/manual'
make[2]: Nothing to be done for `info'.
make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4/manual'
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4'
 * Installing locales in /etc/locales.build...
make -r PARALLELMFLAGS="-j2 -j1" CVSOPTS="" -C /var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4 objdir=`pwd` localedata/install-locales
make[1]: Entering directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4'
make -j2 -j1 -C localedata install-locales
make[2]: Entering directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4/localedata'
.././scripts/mkinstalldirs /var/tmp/portage/glibc-2.3.4.20050125-r1/image//usr/lib/locale
mkdir /var/tmp/portage/glibc-2.3.4.20050125-r1/image/usr/lib/locale
en_US.ISO-8859-1... done
en_US.UTF-8... done
make[2]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4/localedata'
make[1]: Leaving directory `/var/tmp/portage/glibc-2.3.4.20050125-r1/work/glibc-2.3.4'
 * Installing man pages...
ls: relocation error: /var/tmp/portage/glibc-2.3.4.20050125-r1/image//lib/libc.so.6: symbol ___tls_get_addr, version GLIBC_2.3 not defined in file ld-linux.so.2 with link time reference

!!! ERROR: sys-libs/glibc-2.3.4.20050125-r1 failed.
!!! Function toolchain-glibc_src_install, Line 489, Exitcode 127
!!! simple run test (ls) failed
!!! If you need support, post the topmost build error, NOT this status message.

Code: Select all

=================================================================
System uname: 2.6.9-gentoo-r1 i686 AMD Athlon(tm) 64 Processor 3500+
Gentoo Base System version 1.4.16
Autoconf: sys-devel/autoconf-2.59-r5
Automake: sys-devel/automake-1.8.5-r1
Binutils: sys-devel/binutils-2.14.90.0.8-r1
Headers:  sys-kernel/linux26-headers-2.6.8.1-r2
Libtools: sys-devel/libtool-1.5.2-r5
ACCEPT_KEYWORDS="x86"
AUTOCLEAN="yes"
CFLAGS="-march=pentium3 -O2 -pipe"
CHOST="i686-pc-linux-gnu"
COMPILER=""
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=pentium3 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoaddcvs autoconfig candy ccache distlocks sandbox sfperms"
GENTOO_MIRRORS="ftp://mirror.iawnet.sandia.gov ftp://gentoo.risq.qc.ca http://www.ibiblio.org/pub/Linux/distributions/gentoo"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="x86 X alsa apm arts avi berkdb bitmap-fonts crypt cups emboss encode font-server foomaticdb fortran gdbm gif gpm gtk2 hardened imlib ipv6 ithreads jpeg kde libg++ libwww mad mikmod motif mp3 mpeg ncurses nls nptl oggvorbis opengl pam pdflib perl png pthreads python qt quicktime readline sdl spell ssl svga tcpd truetype truetype-fonts type1-fonts xml2 xmms xv zlib"

Once I get it working I will certainly post what I did here. However, right now, to see if I screwed up somewhere else, I am going to try disabling the hardened flag and emerging glibc. All hardware used is pretty standard and new. Processor as you can see is an AMD64. The machine itself is a VMWare virtual machine that "has" an Intel 440BX chipset with 256 MB ram and a 8.0 GB hard disk. If I didn't mention earlier, the system runs strictly in 32-bit mode, no emulation or x86_64 to complicate matters.

Thanks in advance!

d9169037 · Post by **d9169037** » Fri Mar 25, 2005 2:47 am

i have exactly the same problem as yours, and the differences is that i also enabled the nptlonly useflag.

sciwhiz007 · Post by **sciwhiz007** » Fri Mar 25, 2005 3:02 am

Hmm, wasn't it said earlier that enabling nptlonly may b0rk your system? The aim of this HOWTO is to build an ultimately stable system, isn't it? If you're interested in only have support for NPTL, I think you should take a look at http://gentoo-wiki.com/HOWTO_Gentoo_200 ... 6_and_NPTL

That said, it is very strange that you get the _exact_ same error as I do, as I don't have nptlonly in my USE flags, so there's something definitely afoot. Anyone else care to shed some light on this matter?

sciwhiz007 · Post by **sciwhiz007** » Fri Mar 25, 2005 3:13 am

Alright, I removed the hardened USE flag, and I get the exact error once again. I guess the problem is with glibc 20050125-r1. Is it worth a try to revert to 20050125?

Bob P · Post by **Bob P** » Fri Mar 25, 2005 5:40 am

sciwhiz007 wrote:
Bob P wrote:a gentle reminder:

I will not repond to any Personal Messages requesting support for the Stage 1 on 3 Installation Guide.

All Personal Messages sent to me will be deleted without a reply, so please post your support requests in this thread.

Thanks, and have fun!

this policy includes help requests for implementing distcc with the Stage 1/3 guide. the use of distcc to build your toolkit is subtly dangerous. it is unsupported and i advise against it! if you choose to do it, then proceed at your own risk!
Hmm, I don't think I PMed you or asked for help with distcc did I? No, I'm not getting antsy or anything, just checking..

no, its not about you. i get PMs and questions all of the time by people who want me to give them free, personalized technical support. most of the time is related to the Guide, but much of the time its related to taking the guide in some unsupported direction, like distcc, nptlonly, or in this case, hardened. unfortunately, i don't have enough time to support the guide as it is written, and i certainly don't have the time to develop personalized/custom solutions for people who want to change it, but can't do it on their own.

the mere mention of distcc in this thread has resulted in a barrage of personal messages to me from people who seek an expedited method of installing on older hardware. unfortunately, there is no free lunch, and people who aren't satisfied with the time that the installation will take need to review the warning that is written at the beginning of the guide:

Stage 1/3 Installation Guide wrote:WARNING: This is an ADVANCED installation method. The amount of time required for you to complete this type of installation will equal or exceed that of any other Gentoo installation method. Your time will be well invested, though, as the result will be a very stable Gentoo system. This installation method may prove to be somewhat difficult and cumbersome for users who are new to Gentoo. It will prove especially painful for users who plan to install Gentoo old hardware.

periodically, i have to post an announcement like the one you quoted to discourage people from continuously nagging me with PMs. i have tried to disable PM for my user profile, but unfortunately, i cannot turn it off.

Guys, do me a favor. if you decide that you want to follow a different method of installing gentoo, don't ask for help in this support thread. if you choose to undermine the integrity of your system by using "nptlonly" or you choose to harden it by going with hardened GCC, please ask for help in another thread. such propositions are sufficiently deviant from the Stage 1/3 Guide that they really deserve to be discussed in their own threads. THIS thread is intended to support the Stage 1/3 guide as it is written, and nothing else. if it gets hijacked i'm just not going to read it anymore.

to be fair to everyone, if you're doing a hardened GCC project and you have problems, then your problems are GCC issues, not Guide issues. similarly, if you decide to use "nptlonly", then you're not following the guide and you really should take your support questions to another thread. its not fair to the people who are trying to read this thead and follow the guide to have a bunch of irrelevant stuff interspersed with the thread.

i hope that you understand why i feel the need to ask for this. thanks.