Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo 'Parasite' Install Guide - NFS Portage & Distcc
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
DOSBoy
Tux's lil' helper
Tux's lil' helper


Joined: 26 Jun 2005
Posts: 84

PostPosted: Sun Oct 09, 2005 12:04 pm    Post subject: Gentoo 'Parasite' Install Guide - NFS Portage & Distcc Reply with quote

0. Introduction
===============

The contents of this document are licensed under the Creative Commons
- Attribution / Share Alike license.

This document uses information from several others:
http://gentoo-wiki.com/HOWTO_Using_a_shared_portage_via_NFS
http://www.gentoo.org/doc/en/distcc.xml
http://www.gentoo.org/doc/en/handbook/index.xml

If you're trying something like this, it is assumed that you know what
you're doing, and so some details are glossed over.

Now that that's out of the way, what's this guide about? It describes
an installation method where one machine (the parasite) builds its
packages (and possibly the toolchain, if necessary) using one or more
other machines to help with compilation. It also uses a shared portage
tree, to conserve hard drive space.

The case study of my home network will be used to demonstrate samples
of commands. My network consists of 3 (relevant) machines:

- Urr, an 800MHz PIII that holds the portage tree and will be a distcc
host,

- Povtek, a 366MHz PII that will be a distcc host, and

- Nentunchul, another 366MHz PII that will be getting gentoo installed
upon it.

(Don't ask about the hostnames, it's a long story.)

1. Preparations
===============

By the end of this section, we will have:

- One machine exporting the portage tree,

- One or more machines running distccd.

First, the easy bit. NFS.

Check your kernel configuration. We need NFS support, either builtin
or modular. I'm a fan of monolithic kernels, so mine looks like this:
Code:

File systems --->
Network File Systems --->
<*> NFS file system support
[*]   Provide NFSv3 client support
<*> NFS server support
[*]   Provide NFSv3 server support

If you needed to change anything, do the 4 Rs of kernel updates:

- Rebuild

- Reinstall

- Remerge (if you use packages like alsa-driver, nvidia-kernel etc.)

- Reboot (if you built NFS into the kernel, not as a module.)

For those that built NFS as modules:
Code:

urr # modprobe nfs

Building NFS as modules means that you probably should add the
following to your /etc/modules.autoload.d/kernel-version file:
Code:

nfs

Merge NFS utilities:
Code:

urr # emerge -avt nfs-utils

Add NFS to the default runlevel:
Code:

urr # for x in portmap nfs; do rc-update add ${x} default; done

And export the portage tree. Add this to your server's /etc/exports
file:
Code:

/usr/portage   ip_range/subnet(sync,no_root_squash,rw)

For my machine (urr):
Code:

/usr/portage   10.1.1.0/24(sync,no_root_squash,rw)

For those that wonder why we export rw and not ro, it will mean that
we don't have to manually put the distfiles in place on the server
using (e.g. emerge -f --oneshot --nodeps package) or similar. Just be
careful who can access the export.

Now, the fun part, DistCC.

- Don't mix GCC versions. Update all distcc hosts to the same version
(i.e. copy over the relevant parts of one machine's
/etc/portage/package.keywords and /etc/portage/package.unmask and
remerge, using gcc-config and binutils-config as necessary (though
I've never seen binutils get updated, yet). Don't forget to emerge
-e system && emerge -e system && emerge -e world && emerge -e world
if you had to upgrade). All my boxes are currently running GCC
3.4.4-r1, so I'm good to go.

- Don't mix hardened and non-hardened GCC.

- Don't mix CHOSTS. (Apparantly you can get around this, but I say
this to cover myself.)

That said, let's begin.

Merge DistCC:
Code:

urr # emerge -avt distcc

povtek # emerge -avt distcc

On all machines, edit /etc/conf.d/distccd. Near the bottom, there's a
line detailing the --allow option. Check the distccd manpage for valid
options here. As a reference, my line is (on both machines):
Code:

DISTCCD_OPS="${DISTCCD_OPTS} --allow 10.1.1.0/24"

Add DistCC to the default runlevel:
Code:

urr # rc-update add distccd default

povtek # rc-update add distccd default

Start DistCC on the relevant machines:
Code:

urr # /etc/init.d/distccd start

povtek # /etc/init.d/distccd start

These machines are now DistCC servers. Configuring them as clients is
beyond the scope of this guide. It's not difficult and nentunchul is
configured as a distcc client later on.

2. LiveCD Preparations
======================

By the end of this section, we will have:

- Our parasite machine (which will become nentunchul) booted from
LiveCD,

- Functioning network and basic services.

Grab yourself a minimal LiveCD (I'm using 2005.0 for x86) and fire it
up in your parasite machine. I got sandbox violations trying to
compile with 2005.1, which went away once I switched to 2005.0. YMMV.

Make your network go, one way or another. Details are varied that I
can't really help here.

Start some basic services, so we can work on the machine from
elsewhere. (Although we will be getting assisted compilation, the
parasite has to preprocess everything, and some ebuilds don't work
with parallell makes or DistCC):
Code:

livecd # for x in sshd portmap nfs; do /etc/init.d/${x} start; done && passwd && screen

If you don't know how good screen is, read the manpage.

3. Hard Disk Preparations
=========================

By the end of this section, we will have:

- Hdparm will have been run,

- The disk will be partitioned,

- Filesystems and swap will be prepared,

- Filesystems will be mounted,

- The stage tarball will be put in place,

- The remote portage tree will be prepared.

So, hdparm. Nentunchul has IDE disks, so this is going to help us out:
Code:

livecd # hdparm -d1A1m16u1a64 /dev/hda

Next, partition the disk:
Code:

livecd # fdisk /dev/hda

Details are left to the reader.

Prepare filesystems. Substitute the FS making command of your choice:
Code:

livecd # for x in 1 3; do mkreiserfs /dev/hda${x}; done && mkswap /dev/hda2 && swapon /dev/hda2

Mount filesystems:
Code:

livecd # mount /dev/hda3 /mnt/gentoo && mkdir /mnt/gentoo/boot && mount /dev/hda1 /mnt/gentoo/boot

Note that you may need -t ext3 in your mount commands if you are using
ext3.

Stage tarball:

We need a stage3, so we can add users without messing in /etc/passwd
by hand. This guide covers toolchain rebuilds, since my DistCC hosts
are running gcc 3.4.4.

WGET your stage from your favourite mirror:
Code:

livecd # cd /mnt/gentoo && wget ftp://mirror.pacific.net.au/linux/Gentoo/releases/x86/2005.1/stages/x86/stage3-x86-2005.1.tar.bz2 -O - | tar -xjvpf -

The output looks kind of wierd, since you get the output of tar and
wget going to the screen at once. Find something to amuse yourself for
a while. Or watch some scrolling text.

Attach the portage tree:
Code:

livecd # mkdir /mnt/gentoo/usr/portage && chown portage:portage /mnt/gentoo/usr/portage && mount -t nfs -o rw nfs_server_addr:/usr/portage /mnt/gentoo/usr/portage

Some network cards can choke on the NFS mount unless you specify
altered rsize and wsize parameters. If this is the case with your
card, try mounting with the options -o rw,rsize=8192,wsize=8192. You
can ignore 'mount version older than kernel' messages.

4. Prepare to compile
=====================

By the end of this section, we will have:

- make.conf set up for the first toolchain build,

- All portage directories in place,

- The relevant toolchain packages will be unmasked,

- Glibc user locales will be enabled,

- Chroot complete,

- Timezone set,

- Clock set,

- Portage cache updated.

Make.conf:

Here is a sample that will suit most Australians in terms of
mirrors. I plan on rebuilding the toolchain, since not only do the
DistCC hosts run GCC 3.4.4, but I used an x86 tarball and I plan to
build with -march=pentium2 and change the CHOST from i386-pc-linux-gnu
to i686-pc-linux-gnu.
Code:

CHOST="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium2 -pipe -fomit-frame-pointer"
CXXFLAGS=${CFLAGS}
ACCEPT_KEYWORDS="x86"
PORTAGE_TMPDIR=/var/tmp
PORTDIR=/usr/portage
DISTDIR=${PORTDIR}/distfiles
PKGDIR=${PORTDIR}/packages
PORT_LOGDIR=/var/log/portage
PORTDIR_OVERLAY=/usr/local/portage
GENTOO_MIRRORS="ftp://mirror.pacific.net.au/linux/Gentoo http://gentoo.llarian.net/ http://mirror.espri.arizona.edu/gentoo/"
SYNC="rsync://rsync.au.gentoo.org/gentoo-portage"
RSYNC_RETRIES="3"
RSYNC_TIMEOUT=180
MAKEOPTS="-j2"
PORTAGE_NICENESS=3
AUTOCLEAN="yes"
FEATURES="distlocks sandbox userpriv usersandbox"
CCACHE_SIZE="2G"
#RSYNC_EXCLUDEFROM=/etc/portage/rsync_excludes
USE="nptl -nptlonly"

Note that I plan to build an NPTL-capable system. Change it if you
want.

Note also that since I will be rebuilding the toolchain, I don't
bother going overboard on the CFLAGS. Rice them up later if you so
desire.

Portage directories:
Code:

livecd # mkdir /mnt/gentoo/usr/local/portage /mnt/gentoo/var/log/portage

Unmask a new GCC:

If your DistCC servers run a masked GCC, you'll want to copy over
relevant parts of their /etc/portage/package.keywords.

Glibc locales:
Code:

livecd # touch /mnt/gentoo/etc/portage/package.use && echo 'sys-libs/glibc userlocales' >> /mnt/gentoo/etc/portage/package.use

And edit /mnt/gentoo/etc/locales.build to taste. Mine:
Code:

en_AU/ISO-8859-1
en_AU.UTF-8/UTF-8

Chroot:
Code:

livecd # cp -L /etc/resolv.conf /mnt/gentoo/etc/resolv.conf && mount -t proc none /mnt/gentoo/proc && cp /proc/mounts /mnt/gentoo/etc/mtab && chroot /mnt/gentoo /bin/bash
livecd # env-update && source /etc/profile

Date and time:
Code:

livecd # rm /etc/localtime && ln -s /usr/share/zoneinfo/your_timezone_here /etc/localtime && date MMDDHHMMYYYY

Update portage cache:
Code:

livecd # emerge --metadata

5. Toolchain, Pass One
======================

By the end of this section, we will have:

- The first pass of the toolchain compiled,

- The new GCC version set as our compiler, if we updated.

Compile the toolchain:
Code:

livecd # emerge -avt gcc-config binutils glibc libstdc++-v3 gcc

Will do the trick for GCC 3.4.4. I'm not sure about GCC 3.3.5, perhaps
Code:

livecd # emerge -avt binutils glibc gcc

will work, but no guarantees. We'll get DistCC in for the second pass
of the toolchain, fear not (although some toolchain parts ignore
DistCC).

If you upgraded GCC, switch to the new version:
Code:

livecd # gcc-config -l
livecd # gcc-config 6 && env-update && source /etc/profile

Check the number that you need to supply to gcc-config. It may vary.

6. DistCC
=========

By the end of this section, we will have:

- DistCC installed,

- Make.conf updated to take advantage of a new compiler and distcc,

- DistCC configured.

Install DistCC:
Code:

livecd # USE="-gnome -gtk" emerge -avt distcc

DistCC has graphical monitors, and having the gnome or gtk use flags
will pull in lots of packages we want to avoid building, for now.

Update Make.conf:

Now's the time to rice up your CFLAGS, if you want. More importantly,
you need to set MAKEOPTS="-jN", where some recommend to use N=2X+1,
where X is the number of available CPUs. I'm going to try out N=3X
(i.e. 9), and see what happens. The last thing to do here is to add
'distcc' to your FEATURES= line.

Configure DistCC:
Code:

livecd # distcc-config --install && distcc-config --set-hosts "10.1.1.4 10.1.1.6 localhost" && distcc-config --get-hosts

Make sure that you list the hosts in descending order of power. I
don't have a nameserver, so I have to list by IP. This can be changed
later, once you get some kind of name lookup services.

7. Toolchain, Pass Two
======================

By the end of this section, we will have:

- Portage updated,

- The system toolchain compiled with itself,

- The old toolchain removed.

Update portage:
Code:

livecd # emerge -uavt portage

And etc-update if necessary.

Recompile the toolchain:

Run the same emerge command used to build the toolchain the first
time, without gcc-config. Mine was:
Code:

livecd # emerge -avt glibc binutils gcc libstdc++-v3

Start the distcc monitor, so we can get some more lovely scrolling
text to watch while we compile:

Press Ctrl-a Shift-s to make screen split the window in two.

Press Ctrl-a TAB to switch to the bottom pane.

Press Ctrl-a Ctrl-c to start a new shell.

In this shell:
Code:

livecd # chroot /mnt/gentoo /bin/bash
livecd # DISTCC_DIR=/var/tmp/portage/.distcc distccmon-text 1

Then press Ctrl-a TAB to switch back to the top screen. Sit back,
relax and watch some scrolling text :).

8. System Packages
==================

By the end of this section, we will have:

- System packages compiled,

- Old GCC removed.

Compile system packages:

Now you'll really get to see DistCC do its thing, since most of the
system packages don't filter it out.
Code:

livecd # emerge -avte system && emerge -e system

We do this twice for safety's sake. If you think you know what you're
doing, you might be able to get away with one or zero
invocations. Just be careful and don't complain if it breaks ;).

Etc-update here, if necessary.

Prune out the old GCC, if you updated it:
Code:

livecd # emerge -aP gcc

9. Additional Packages
======================

By the end of this section, we will have:

- Almost finished emerging things,

- Have the default runlevel set up,

- Clock skew eliminated, thanks to NTP,

- Have this machine able to help out compilation with DistCC.

Emerge the last of the packages. (I had to add USE="-X" before the second emerge command, to get apmd to build. Just FYI):
Code:

livecd # emerge -avt your_syslogger your_crondaemon your_bootloader xinetd hotplug coldplug your_filesystem_package sysfsutils nfs-utils udev dhcpcd your_kernel_sources && emerge -avtO your_power_management_daemon ntp

Update the default runlevel:
Code:

livecd # for x in your_syslogger your_crondaemon your_power_management_daemon net.eth0 xinetd sshd hotplug coldplug ntp-client distccd nfs domainname gpm; do rc-update add ${x} default; done && /sbin/depscan.sh

Clock skew fix:
Code:

livecd # ntpdate -b -u pool.ntp.org

Done.

DistCC host:

Edit /etc/conf.d/distccd like you did at the start of the document.

10. Kernel / Bootloader:
========================

By the end of this section, we will have:

The kernel built and installed,

The bootloader installed.

Kernel:

Configure:

N is the same N you put into MAKEOPTS= line when you edited make.conf.
Code:

livecd # cd /usr/src/linux && make -jN CC=distcc menuconfig

Build / Install:
Code:

livecd # make -jN CC=distcc && make -jN CC=distcc modules modules_install install

You can check the distcc monitor for kernel builds, by Ctrl-C ing the
current running one, and starting it again without the DISTCC_DIR= bit
in front.

After you're done compiling, add any relevant modules to
/etc/modules.autoload.d/kernel-version.

Bootloader:

The example here is grub. Here's a sample /boot/grub/grub.conf to
modify. Note that the sample machine had vesafb-tng issues, so I'm
using vesafb:
Code:

# Grub boot menu configuration file
#
# Boot automatically after 10 secs.
timeout 10

# By default, boot the third entry.
default 2

# Fallback to the second entry.
fallback 1

# Use default Grub Splash image
# splashimage=(hd0,0)/grub/splash.xpm.gz
#
# Use custom (downloaded) Gentoo Splash Image
splashimage=(hd0,0)/grub/gentoo.xpm.gz

# Boot Gentoo Linux (failsafe)
title Gentoo 2.6.12-gentoo-r10 (failsafe)
root (hd0,0)
kernel (hd0,0)/vmlinuz ro root=/dev/hda4

# Boot Gentoo Linux (no framebuffer)
title Gentoo-2.6.12-gentoo-r10
root (hd0,0)
kernel (hd0,0)/vmlinuz ro root=/dev/hda4 vga=791

# Boot Gentoo Linux at 1024x768 framebuffer resolution
title Gentoo-2.6.12-gentoo-r10, 1024x768
root (hd0,0)
kernel (hd0,0)/vmlinuz ro root=/dev/hda4 vga=791 splash=verbose,fadein,theme:emergence quiet CONSOLE=/dev/tty1
initrd (hd0,0)/fbsplash-emergence-1024x768

# For installing GRUB into the hard disk
title Install GRUB into the hard disk
root (hd0,0)
setup (hd0)

Some more merging etc, to get the flashy framebuffer and grub splash:
Code:

livecd # emerge -avt splashutils splash-themes-gentoo && splash_geninitramfs -v -g /boot/fbsplash-emergence-1024x768 -r 1024x768 emergence && rc-update add splash default && cd /boot/grub && wget http://www.schultz-net.dk/downloads/grub/gentoo.xpm.gz

Now, install grub:
Code:

livecd # grub
grub> root (hd0,0)
grub> setup (hd0)
grub> quit

11. FINAL final packages
========================

By the end of this section, we will have:

- Really finished with emerge, for the installation.

Some packages can't be built until you have a ready kernel source
tree. Now would be the time to merge such packages - nvidia-kernel,
thinkpad and alsa-driver are examples of this. You might want to do
this with USE="-X" somewhere, though. Switch your distcc monitor back
to monitoring portage's compiles, and:
Code:

livecd # USE="-X" emerge -avt some_stuff

I'd only do this for really critical packages to get the system going
e.g. network related stuff.

12. Users
=========

By the end of this section, we will have:

- Reset the root password,

- Additional user accounts.

Root password:
Code:

livecd # passwd

Enter root's new password.

New users:
Code:

livecd # useradd user_name -m -G groups && passwd user_name

Replace 'groups' with relevant groups for said user, and set a password for you new accounts.

13. Config Files
================

By the end of this section, we will have:

- Set host / domain name,

- Set up the network config,

- Set up /etc/fstab,

- Set up udev support in /etc/conf.d/rc,

- Set up hdparm options.

Host and domain names:

Nothing suprising here. Edit /etc/conf.d/hostname and
/etc/conf.d/domainname to taste.

Network config:

Edit /etc/conf.d/net to taste. Provided example is for DHCP:
Code:

iface_eth0="dhcp"
dhcpcd_eth0="-t 10 -N"

/etc/fstab:

A very solid sample file is provided, and the handbook covers this in
detail, so I'll omit the details in the interest of brevity. The only
extra thing you need here is a line of the form:
Code:

nfs_host:/usr/portage        /usr/portage  nfs   rw   0 0

Change nfs_host to a real NFS host, and you might need to add the rsize=8192,wsize=8192 as before.

Udev:

Edit /etc/conf.d/rc and ensure that you have:
Code:

RC_DEVICES="udev"
RC_DEVICE_TARBALL="no"

Hdparm options:

Edit /etc/conf.d/hdparm:
Code:

livecd # nano -w /etc/conf.d/hdparm && rc-update add hdparm boot

Sample:
Code:

disc0_args="-d1A1m16u1a64"
cdrom0_args="-d1u1"

14. Clean up
============

By the end of this section, we will have:

- Closed screen,

- Unmounted all filesystems.

Closing screen:

Nothing too fancy, just exit the chroot and the terminal in both the
distcc monitor shell and the main shell.

Unmount filesystems:
Code:

livecd # cd && umount /mnt/gentoo/usr/portage && umount /mnt/gentoo/proc && umount /mnt/gentoo/boot && umount /mnt/gentoo && swapoff /dev/hda2

15. Finale
==========

By the end of this section, you will have:

- Rebooted into your completed parasite.

Reboot:
Code:

livecd # shutdown -r now


And it is done. Thanks for reading.
Back to top
View user's profile Send private message
DOSBoy
Tux's lil' helper
Tux's lil' helper


Joined: 26 Jun 2005
Posts: 84

PostPosted: Mon Oct 10, 2005 5:20 am    Post subject: Reply with quote

The main weakness with this install method is the several toolchain rebuilds, to get your GCC to match those on the DistCC servers. It only just hit me, you could probably use a Jackass! tarball and avoid most of the delay.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum