Fiordland Stage 1 Install Guide Ver. 2.0 (updated)

Message

warrens · Post by **warrens** » Thu Jan 11, 2007 2:23 am

Damn, I forgot how big a PITA building LFS is!! The last time I built LFS was about 3 years ago.

warrens · Post by **warrens** » Tue Jan 16, 2007 12:46 am

While this guide works great, for me anyways

, I am working on making this into a drop in replacement for bootstrap.sh. This will make Fiordland install a much easier process.

warrens · Post by **warrens** » Sun Feb 04, 2007 10:18 pm

Here is a working copy of the shell script of Fiordland. Be aware that the script is not commented as of yet. I first tried modifying bootstrap.sh but that did not work so good, so I just made a script that does what is done in the guide and so far is working great. Here is the script:

Fiordland.sh

Code: Select all

#!/bin/bash

# Fiordland.sh
# Copyright 2007
# Distributed under the terms of the GNU General Public License v2

# Protect /etc/make.conf as it will be overwritten
cp -f /etc/make.conf /etc/make.conf.build

# Feel free to change CHOST, CFLAGS, and CXXFLAGS to match your arch
# Use -O2 -pipe and, depending on arch -fomit-frame-pointer for this part of the build
# where CHOST and various tool chain (gcc, glibc, binutiils, linux-headers) items will 
# likely change from what is on the stage 1 tarball

# ACCEPT_KEYWORDS setting [i]arch[/i] or [i]~arch[/i] depending on your preference

# Everything else should be as showen

cat > /etc/make.conf << "EOF"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
CXXFLAGS=${CFLAGS}
ACCEPT_KEYWORDS="~x86"
MAKEOPTS="-j2"
PORTAGE_NICENESS=3
AUTOCLEAN="yes"
FEATURES="distlocks sandbox userpriv usersandbox"
USE="nptl nptlonly bootstrap -fortran"
EOF

emerge --oneshot portage &&

emerge --oneshot --nodeps binutils gcc-config libstdc++-v3 gcc &&

# Make sure that we will be using the correct gcc for the rest of the build

gcc-config -l

echo "Please select the correct compiler for your system.  "

read opt

gcc-config $opt

emerge --prune gcc &&


env-update
source /etc/profile

emerge --oneshot --nodeps linux-headers unifdef glibc &&

emerge timezone-data &&

# Set your timezone setting here, this listing show my timezone

rm /etc/localtime
ln -s /usr/share/zoneinfo/CST6CDT /etc/localtime

# Here your CFLAGS and CXXFLAGS should be set to match those in your /etc/make.conf file
cat > /etc/make.conf << "EOF"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium4 -fforce-addr -fomit-frame-pointer -ftracer -pipe"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"
ACCEPT_KEYWORDS="~x86"
MAKEOPTS="-j2"
PORTAGE_NICENESS=3
AUTOCLEAN="yes"
FEATURES="distlocks sandbox userpriv usersandbox"
USE="nptl nptlonly -fortran -X -tk"
EOF



emerge --oneshot --nodeps =sys-devel/autoconf-2.13 autoconf autoconf-wrapper automake gcc binutils gawk bzip2 gzip diffutils findutils make grep sed gettext gpm ncurses patch tar sys-apps/texinfo bash bash-completion m4 bison flex util-linux gdbm perl &&

emerge --oneshot --nodeps linux-headers unifdef glibc binutils gcc &&

emerge -e --oneshot man-pages coreutils zlib findutils gawk ncurses sys-libs/readline m4 bison less groff sed flex gettext perl sys-apps/texinfo autoconf automake bash file libtool bzip2 diffutils kbd e2fsprogs grep gzip hotplug man make module-init-tools patch procps psmisc shadow sysvinit tar udev util-linux &&

# Restore original /etc/make.conf

cp -f /etc/make.conf.build /etc/make.conf

If you are not comfortable with the CFLAGS that I use feel free to change them both in the script and your make.conf. The CFLAGS that I use here have served me well over the past year since I first developed the Fiordland Install. I have not had one crash for the past year with these flags, I think that has more to do with the fact that Fiordland properly builds the tool chain. Go ahead and give this a try and most of all have fun.

[/b]

edit: added --oneshot to the emerges in the script.
edit 2: added copyright and comments.

slycordinator · Post by **slycordinator** » Mon Feb 05, 2007 12:57 am

For the packages that are already gonna be in the system profile, shouldn't you change the command from

Code: Select all

emerge list of packages

to

Code: Select all

emerge -1 list of packages

?

Because if I put a bunch of packages into my world file that are already in the system profile, I'd end up calculating dependencies for the whole system profile (which would include the packages in question) and then calculate the dependencies of those packages again. Essentially, that would clutter up your world file.

warrens · Post by **warrens** » Mon Feb 05, 2007 1:39 am

slycordinator wrote:For the packages that are already gonna be in the system profile, shouldn't you change the command from
Code: Select all
emerge list of packages
to
Code: Select all
emerge -1 list of packages
?

Because if I put a bunch of packages into my world file that are already in the system profile, I'd end up calculating dependencies for the whole system profile (which would include the packages in question) and then calculate the dependencies of those packages again. Essentially, that would clutter up your world file.

Actually I had

Code: Select all

emerge --oneshot list of packages

as part of the script, but it was causing me all kinds of problems later after

Code: Select all

emerge -e system

that went away when I removed the --oneshot flag. The one change to the script that I have been considering is dropping the last

Code: Select all

emerge -e list of packages

altogether, as all those packages are part of the system profile and would be picked up by

Code: Select all

emerge -e system.

That I think would save about an hour of compile time without hurting the overall stability of the final system, have to try it to be certain though.

slycordinator · Post by **slycordinator** » Mon Feb 05, 2007 8:11 pm

So you had problems doing

Code: Select all

emerge -1 list of packages

then later did

Code: Select all

emerge -e system

then reran the previous command without the -1 (and it worked) then you assumed the problem was the -1.
What?! All "--oneshot" does is make it so nothing is written to the world file. So if you were having problems they were most likely fixed by doing the

Code: Select all

emerge -e system

And this is kinda debatable but some devs (both gentoo and non-gentoo ones) said "-fvisibility-inlines-hidden" shouldn't be in your CXXFLAGS. It's enabled automatically on the packages that actually support that functionality. Then again, you'd be relying on the dev to enable an option you want to support.

vipernicus · Post by **vipernicus** » Mon Feb 05, 2007 8:25 pm

slycordinator wrote:And this is kinda debatable but some devs (both gentoo and non-gentoo ones) said "-fvisibility-inlines-hidden" shouldn't be in your CXXFLAGS. It's enabled automatically on the packages that actually support that functionality. Then again, you'd be relying on the dev to enable an option you want to support.

Yeah, it's not worth enabling global, when it is already applied to the applications that will use it.

warrens · Post by **warrens** » Tue Feb 06, 2007 1:43 am

The problems were occurring while adding packages after running

Code: Select all

emerge -e system

I know that it is crazy, but the fact the the problems went away after removing all the --oneshot flags still remains.

On every successful run of the script I ran

Code: Select all

emerge -e system

and then continued to build the system towards a complete desktop system. At the end of the script the system is basically a LFS system ready for bootloader, system logger, and a kernel. It is also somewheres between a Gentoo stage 2 and stage 3. [/code]

warrens · Post by **warrens** » Thu Feb 08, 2007 5:35 pm

I have put the --oneshot flag back into the script. Now your world file will be clean until you start building your "world".

I am typing this from a fresh install using the script in its current configuration.

slycordinator · Post by **slycordinator** » Fri Feb 09, 2007 1:20 am

warrens wrote:I have put the --oneshot flag back into the script. Now your world file will be clean until you start building your "world".

I am typing this from a fresh install using the script in its current configuration.

How about adding it to the first post as well since that'll be the first one people will see?

warrens · Post by **warrens** » Fri Feb 09, 2007 2:00 am

Ok, done.

Now to get fiordland.sh fault tolerent, which in its current state it is not, and also it needs to be able to use the -p and -r options. Right now if one of the emerges fail the script will just carry on with the next emerge and this is not acceptable for building the guide around the script at this time. However my scripting skill are lacking in this area, so any help wourd be apprieciated.

slycordinator · Post by **slycordinator** » Fri Feb 09, 2007 3:26 am

Add " &&" at the end of every command that absolutely must finish and exit correctly (except for your final command).

Code: Select all

#!/bin/bash



cp -f /etc/make.conf /etc/make.conf.build



cat > /etc/make.conf << "EOF"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
CXXFLAGS=${CFLAGS}
ACCEPT_KEYWORDS="~x86"
MAKEOPTS="-j2"
PORTAGE_NICENESS=3
AUTOCLEAN="yes"
FEATURES="distlocks sandbox userpriv usersandbox"
USE="nptl nptlonly bootstrap -fortran"
EOF

emerge --oneshot portage &&

emerge --oneshot --nodeps binutils gcc-config libstdc++-v3 gcc &&

gcc-config -l

echo "Please select the correct compiler for your system.  "

read opt

gcc-config $opt

emerge --prune gcc &&


env-update &&
source /etc/profile

emerge --oneshot --nodeps linux-headers unifdef glibc &&

emerge timezone-data &&

rm /etc/localtime
ln -s /usr/share/zoneinfo/CST6CDT /etc/localtime


cat > /etc/make.conf << "EOF"
CHOST="i686-pc-linux-gnu"
CFLAGS="-O3 -march=pentium4 -fforce-addr -fomit-frame-pointer -ftracer -pipe"
CXXFLAGS="${CFLAGS} -fvisibility-inlines-hidden"
ACCEPT_KEYWORDS="~x86"
MAKEOPTS="-j2"
PORTAGE_NICENESS=3
AUTOCLEAN="yes"
FEATURES="distlocks sandbox userpriv usersandbox"
USE="nptl nptlonly -fortran -X -tk"
EOF



emerge --oneshot --nodeps =sys-devel/autoconf-2.13 autoconf autoconf-wrapper automake gcc binutils gawk bzip2 gzip diffutils findutils make grep sed gettext gpm ncurses patch tar sys-apps/texinfo bash bash-completion m4 bison flex util-linux gdbm perl &&

emerge --oneshot --nodeps linux-headers unifdef glibc binutils gcc &&

emerge -e --oneshot man-pages coreutils zlib findutils gawk ncurses sys-libs/readline m4 bison less groff sed flex gettext perl sys-apps/texinfo autoconf automake bash file libtool bzip2 diffutils kbd e2fsprogs grep gzip hotplug man make module-init-tools patch procps psmisc shadow sysvinit tar udev util-linux &&

cp -f /etc/make.conf.build /etc/make.conf

warrens · Post by **warrens** » Tue Feb 20, 2007 2:37 am

Updated the script. Getting closer to building the guide around fiordland.sh.

@slycordinator: Thanks for the help.

Koreth · Post by **Koreth** » Mon Mar 19, 2007 10:55 pm

First, I wanted to say thanks for the great guide. I did however have a few questions.

Since some of the things that need to be built, like glibc or gcc take a while, is there a way that the Fiordland install could be set up to use distcc? Using distcc would be a great way for persons like me who are installing on older hardware to speed things up. In my case, I have 5 Dell Poweredge 2400s with P3 Coppermine CPUs on hand. Since the LiveCD comes with distcc installed, I think all I'd have to do is burn enough copies of the live CD to boot all the Dell servers, and turn distccd on, but I understand that distcc would still need to be present in order to use all the distcc servers. Looking at the install guide and the Fiordland script, it doesn't look like it is set up to use distcc.

At what point of the install could distcc be added to speed the rest of the install up? I'm personally thinking during step 8.2, as it looks that most of distcc's dependencies (as per emerge -p distcc) are installed then. I'm not sure how step 8.2 would need to be modifed to allow the install of distcc however.

So, what do you think? How would one go about using distcc with Fiordland to speed things up?

warrens · Post by **warrens** » Tue Mar 20, 2007 10:16 am

@koreth
I never have used distcc so I am not sure, but theoretically it should work. How one would set distcc up to work with Fiordland I have no clue.

slycordinator · Post by **slycordinator** » Wed Mar 21, 2007 3:07 am

http://www.gentoo.org/doc/en/distcc.xml

Look at section 4, the part called "Distcc to bootstrap"

Should be pretty easy to combine this with the guide.

Koreth · Post by **Koreth** » Wed Mar 21, 2007 4:50 am

I did see the part about using distcc to boostrap. I did have a concern about it though, because the offical guide has one run the emerge with --oneshot and --nodeps. Since distcc normally pulls in a few deps, I was worried that emerging distcc in such a manner could cause distcc to break and in turn break the stuff it compiles in the later stages of the install. That was all speculation though. I suppose the only way to really find out would be to actually try it and post the results.

slycordinator · Post by **slycordinator** » Wed Mar 21, 2007 5:03 am

Koreth wrote:I did see the part about using distcc to boostrap. I did have a concern about it though, because the offical guide has one run the emerge with --oneshot and --nodeps. Since distcc normally pulls in a few deps, I was worried that emerging distcc in such a manner could cause distcc to break and in turn break the stuff it compiles in the later stages of the install. That was all speculation though. I suppose the only way to really find out would be to actually try it and post the results.

1) There is no mention of --oneshot in the guide.

2) The only way installing distcc in the manner mentioned in the guide would cause it to break is the possibility of the package failing to build. And the reason --nodeps is used is because you want to install the whole system using distcc. If I did "emerge distcc", all of its dependencies would get built (without using distcc as it's not installed), thus defeating the purpose of installing distcc.

3) If you're really worried about it, replace

Code: Select all

USE="-*" emerge --nodeps distcc

with

Code: Select all

emerge distcc

and then continue with the guide as mentioned. This method, though, will take more time as you'll be installing some of the same packages twice (though the second time they get installed will most likely be quicker as you're using the newly-installed distcc)

Koreth · Post by **Koreth** » Wed Mar 21, 2007 11:19 pm

slycordinator wrote:
Koreth wrote:I did see the part about using distcc to boostrap. I did have a concern about it though, because the offical guide has one run the emerge with --oneshot and --nodeps. Since distcc normally pulls in a few deps, I was worried that emerging distcc in such a manner could cause distcc to break and in turn break the stuff it compiles in the later stages of the install. That was all speculation though. I suppose the only way to really find out would be to actually try it and post the results.
1) There is no mention of --oneshot in the guide.

2) The only way installing distcc in the manner mentioned in the guide would cause it to break is the possibility of the package failing to build. And the reason --nodeps is used is because you want to install the whole system using distcc. If I did "emerge distcc", all of its dependencies would get built (without using distcc as it's not installed), thus defeating the purpose of installing distcc.

3) If you're really worried about it, replace
Code: Select all
USE="-*" emerge --nodeps distcc
with
Code: Select all
emerge distcc
and then continue with the guide as mentioned. This method, though, will take more time as you'll be installing some of the same packages twice (though the second time they get installed will most likely be quicker as you're using the newly-installed distcc)

Actually, you are correct, --oneshot isn't used, just --nodeps. I'm not sure why I thought the initial emerge of distcc used --oneshot.

Anyway, I did install distcc to be used during boostrap, and set up one of my servers with a live CD and got its disccd running before proceeding with the Fiordland guide and emerging the toolchain. Thus far, everything has worked and nothing is broken. Everything has completed faster than it did without using distcc. How much, I'm not sure, on the PIII 733Mhz equipment I'm using I've gotten to step 9 in less then 12 hours (if I'm guestimating correctly, about 6). Without using Distcc, It took me a at least a day to get this far. I'm using ccache in my make.conf as well, and I'm sure that is speeding things up a bit too.

Now if I could figure out how to integrate the use of distcc into the shell script posted in this thread so it runs as automatically as everything else, then that would be awesome. I'll have to poke at it and see what I can come up with.

Koreth · Post by **Koreth** » Thu Mar 22, 2007 6:07 am

I've got a new question regarding Use flags and /etc/make.profile. The install guide for the Fiordland install uses the nptl and nptlonly USE flags. However, it looks like the curent stage1 tarball defaults /etc/make.profile to /usr/portage/profiles/default-linux/x86/no-nptl. Looking at the guide to USE flags at http://www.gentoo.org/doc/en/handbook/h ... t=2&chap=2, the make.profile is not mentioned in the section on flag precedence. If I am understanding profiles correctly (probably not), wouldn't this mean that even though the nptl and nptlonly flags were set in make.conf, the flags are masked and thus disregarded when set in make.conf?

I can change my make.profile to something else, but I am wondering if I am going to have a problem. Am I going to need to redo my Fiordland install (again) so as to make use of the nptl and nptl only libraries? Should I even be caring? Can anyone recommend which profile would be ideal if this install is going to be a workstation for a small business (mostly office apps and email)?

Since the current stage 1 tarball defaults to the no-nptl profile, shouldn't the Fiordland guide include a section on selecting a profile as well?

slycordinator · Post by **slycordinator** » Thu Mar 22, 2007 5:52 pm

Koreth wrote:I've got a new question regarding Use flags and /etc/make.profile. The install guide for the Fiordland install uses the nptl and nptlonly USE flags. However, it looks like the curent stage1 tarball defaults /etc/make.profile to /usr/portage/profiles/default-linux/x86/no-nptl. Looking at the guide to USE flags at http://www.gentoo.org/doc/en/handbook/h ... t=2&chap=2, the make.profile is not mentioned in the section on flag precedence. If I am understanding profiles correctly (probably not), wouldn't this mean that even though the nptl and nptlonly flags were set in make.conf, the flags are masked and thus disregarded when set in make.conf?

Of course those flags are ignored! The stage1 tarball is a generic x86 tarball. Generic x86 means that it is built to do anything that a 386 can do. And you have to have AT LEAST A 486 to do nptl.

So the stage1 tarball is built with CHOST of i386 and since it can't do nptl it has to be built with a profile that doesn't do nptl (because all other profiles default to nptl).

Since the current stage 1 tarball defaults to the no-nptl profile, shouldn't the Fiordland guide include a section on selecting a profile as well?

You could actually just download a different tarball that is for your CHOST. A stage2 or stage3 would work. Even if you download a stage3 tarball you can still run this install guide just fine.

edit: And an advantage of using the method I described is that if you download a tarball that's for your CHOST it won't have random crap sitting around for the old CHOST. If you download the stage1 tarball and make the CHOST be i686-pc-linux-gnu there will be files left over from the i386-pc-linux-gnu.

The disadvantage, though, is that the other tarballs are bigger.

Nsane457 · Post by **Nsane457** » Wed Apr 04, 2007 3:25 am

slycordinator wrote:And an advantage of using the method I described is that if you download a tarball that's for your CHOST it won't have random crap sitting around for the old CHOST. If you download the stage1 tarball and make the CHOST be i686-pc-linux-gnu there will be files left over from the i386-pc-linux-gnu.

I've heard this before and I don't quite understand. On a stage3 build there is always an i386 gcc build left over in the /usr directory that never unmerges when GCC is pruned. And all sorts of binutils related symlinks won't unmerge. There are all sorts of old gcc binaries left in the /usr/bin directory that also don't unmerge.

Nsane457 · Post by **Nsane457** » Wed Apr 04, 2007 4:02 am

Between Step 7 and Step 8 there should be the following:

cd /etc
rm make.profile
ln -s ../usr/portage/profiles/default-linux/x86/2006.1 make.profile
cd ..

slycordinator · Post by **slycordinator** » Wed Apr 04, 2007 6:07 am

Nsane457 wrote:Between Step 7 and Step 8 there should be the following:

cd /etc
rm make.profile
ln -s ../usr/portage/profiles/default-linux/x86/2006.1 make.profile
cd ..

Assuming you're using a 2006.1 tarball isn't make.profile already set to that?

edit: nm, forgot that the stage1 x86 uses no-nptl.

slycordinator · Post by **slycordinator** » Wed Apr 04, 2007 6:11 am

Nsane457 wrote:
slycordinator wrote:And an advantage of using the method I described is that if you download a tarball that's for your CHOST it won't have random crap sitting around for the old CHOST. If you download the stage1 tarball and make the CHOST be i686-pc-linux-gnu there will be files left over from the i386-pc-linux-gnu.
I've heard this before and I don't quite understand. On a stage3 build there is always an i386 gcc build left over in the /usr directory that never unmerges when GCC is pruned. And all sorts of binutils related symlinks won't unmerge. There are all sorts of old gcc binaries left in the /usr/bin directory that also don't unmerge.

All that's in my /usr/i386-pc-linux-gnu is a directory called i686-pc-linux-gnu (which just contains the 686 libraries).