My Gentoo diskless install howto

Message

rogeroberholtzer · Post by **rogeroberholtzer** » Mon Nov 03, 2003 10:59 pm

I have done everything to set up a diskless client from the current 1.4 live cd. I have exported the directory containing the gentoo client is to use with rw permissions (as in the description at the top of this thread). However, when the diskless client boots, it claims / is mounted read-only. A local 'mount' lists it as 'rw'. And the server's /proc/fs/nfs/exports file lists it as 'rw' by my client.

So, why the read-only errors on writes?

I have a suspicion of the 'checkroot' file in the rc directory. There was no mention in the diskless info on doing anything with this file. But it does seem to be odd in that it complains about the nfs mounted / not being an ext2 file system. Well, of course it isn't. Why the complaint?

I am so close I can taste it.

Any pointers?

rogeroberholtzer · Post by **rogeroberholtzer** » Tue Nov 18, 2003 12:38 pm

I am trying to use swap via NFS. I seem to need it for a big compile. I only plan to use it as needed.

I made the swap file on the NFS server:

dd if=/dev/zero of=swp count=1000 bs=1M

Then, on the diskless client, I ran:

mkswap /full/path/to/swp

No complaint. 'file /full/path/to/swp' reports that it is a modern swap file. It has the proper permissions (0600) and belongs to root. However, when I run:

swapon /full/path/to/swp

I get the following error:

swapon: /full/path/to/swp: Invalid argument

The file exists. If I use some name that does not exist, I get an appropriate error.

The NFS mount is like this (listed by mount):

acme:/full/path on /full/path type nfs (rw,noatime,bg,soft,
rsize=8192,wsize=8192,addr=10.1.6.131)

The swap file is:

-rw------- 1 root root 1048576000 Nov 18 13:15 swp

Is there some NFS mount option I have overlooked?

(The diskless client is a VME-based PC to which adding additional RAM is not an option).

patsonrt1 · Post by **patsonrt1** » Wed Nov 19, 2003 2:17 am

Just a guess Rogeroberholtzer, but did you add "no_root_squash" in the servers exports file?

darkjedi521 · Post by **darkjedi521** » Wed Feb 04, 2004 6:16 am

I have the system booting with the kernel on a floppy. I followed the howto almost exactly in setup (subsituted floppy for PXE/TFTP). The kernel boots fine and then gentoo starts booting. It gets to the point where it attempts to mount local filesystems, and then I get 5-6 screens worth of errors regarding /var/lib before being dumped at a login prompt. None of my other rc scripts run. /usr doesn't mount, and pam returns fatal errors after a username is entered. Setting console one in inittab to be /bin/bash instead of agetty let me poke at the running system. doing the following:

Code: Select all

source /etc/profile
env-update
source /etc/profile
rc

ran the rest of the rc-scripts, but pam was still having fatal errors. If anyone knows where to poke at, it would be much appreciated. Thanks in advance.

hms · Post by **hms** » Sun Feb 08, 2004 12:32 pm

I have succesfully managed to set up a diskless system but when I start xine I get stalls in the movie. I'm guessing this is because the xfree server is run over nfs. Is there any way to improve xfree´s performance for a diskless system.
I don't have a powerfull enough server to run X there and with my experience a network isn't fast enough to export a fullscreen movie in real time over a network either so this isn't an option.

I have the same system running on a local disk and there xine plays movies smothless. During the tests with the diskless system the actual movie-file being played is on a local disk. My network is a 100Mbit so it shouldn't be the bottleneck.

I read the post about NBD and wonder if that could help me get the performance needed to run xfree over nfs.

/Magnus

GlennM · Post by **GlennM** » Thu Mar 11, 2004 11:13 pm

EDIT: Whoops, fixed it. Turns out you need to remove the local-only line from /etc/xinetd.conf. Doh!

I was wondering if someone could give me some help with TFTP...it seems that my test workstation can't get the kernel image. When the workstation starts, I get:

Code: Select all

Searching for server (DHCP)...
..Me: 192.168.1.110, Server 192.168.1.2, Gateway 192.168.1.1
Loading 192.168.1.2:/lts/vmlinuz-2.4.24-ltsp-1......

That looks right, except that it sits there forever, with no network traffic.

I double-checked to make sure that the kernel is in /tftpboot/lts, and it is, and named correctly. The server and gateway are correct. I think I've got dhcp, xinet, and tftp configured correctly. The relevant files are as follows:

/etc/dhcp/dhcpd.conf:

Code: Select all

# Some general options
default-lease-time            21600;
max-lease-time                21600;
use-host-decl-names           on;
ddns-update-style             ad-hoc;

# Bootp options
allow booting;
allow bootp;

# Network Options
option subnet-mask            255.255.255.0;
option broadcast-address      192.168.1.255;
option routers                192.168.1.1;
option domain-name-servers    192.168.1.2;
option log-servers            192.168.1.2;
option domain-name            "domain-removed.net";

# LTSP Path Options
option root-path              "192.168.1.2:/opt/ltsp/i386";
filename                      "/lts/vmlinuz-2.4.24-ltsp-1";

shared-network WORKSTATIONS {
  subnet 192.168.1.0 netmask 255.255.255.0 {
    # Distribute dynamic IPs to the workstations
    range dynamic-bootp 192.168.1.110 192.168.1.111;
    # Workstation specific configuration for PXE booting
    #host ws001 {
    #  hardware ethernet     00:E0:06:E8:00:84;
    #  fixed-address         192.168.0.1;
    #}
  }
}

/etc/xinetd.d/tftp:

Code: Select all

service tftp 
{ 
	disable = no 
	socket_type = dgram 
	protocol = udp 
	wait = yes 
	user = root 
	server = /usr/sbin/in.tftpd 
}

/etc/conf.d/in.tftpd:

Code: Select all

# Config file for /etc/init.d/in.tftpd
# Remove the -l if you use [x]inetd

INTFTPD_PATH="/tftpboot"
INTFTPD_OPTS="${INTFTPD_PATH}"

I really don't have any more ideas. It just looks like tftpd isn't sending the kernel to the workstation. I've tried changing the "filename" line in dhcpd.conf to "/tftpboot/lts/vmlinuz-2.4.24-ltsp-1", but that doesn't fix the problem. Are there any other files I can adjust?

akmolloy · Post by **akmolloy** » Sun Mar 21, 2004 4:21 am

I've currently got a mythfrontend machine that I've spent quite a bit of time configuring, and it works well. I want to convert the exisiting installation (settings, programs, etc.) to a diskless install.

I've got a server setup and running, and sucessfully booted into a small kernel.

Any tips on how to do this? Could it be as simple as copying over the directories on the exisiting machine to the master server, and then modifying a few files like fstab?

Ben2040 · Post by **Ben2040** » Fri Mar 26, 2004 5:44 pm

Hi

When compiling the grub image, both 0.93 and 0.92 fail with the following. I have tried without CFLAGS too....

Code: Select all

gcc -DHAVE_CONFIG_H -I. -I. -I.. -I../stage1  -Wall -Wmissing-prototypes -Wunused -Wshadow -Wpointer-arith -malign-jumps=1 -malign-loops=1 -malign-functions=1 -Wundef -Os -fno-builtin -nostdinc -I../netboot -DSUPPORT_NETBOOT=1 -DSUPPORT_SERIAL=1 -DSUPPORT_HERCULES=1 -DFSYS_EXT2FS=1 -DFSYS_FAT=1 -DFSYS_FFS=1 -DFSYS_MINIX=1 -DFSYS_REISERFS=1 -DFSYS_VSTAFS=1 -DFSYS_JFS=1 -DFSYS_XFS=1 -DUSE_MD5_PASSWORDS=1 -DFSYS_TFTP=1 -g -c -o pre_stage2_exec-fsys_reiserfs.o `test -f fsys_reiserfs.c || echo './'`fsys_reiserfs.c
cc1: warning: -malign-loops is obsolete, use -falign-loops
cc1: warning: -malign-jumps is obsolete, use -falign-jumps
cc1: warning: -malign-functions is obsolete, use -falign-functions
fsys_reiserfs.c:115: error: long, short, signed or unsigned used invalidly for `j_mount_id'
fsys_reiserfs.c: In function `next_key':
fsys_reiserfs.c:761: warning: operation on `depth' may be undefined
make[2]: *** [pre_stage2_exec-fsys_reiserfs.o] Error 1
make[2]: Leaving directory `/usr/portage/distfiles/grub-0.92/stage2'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/usr/portage/distfiles/grub-0.92'
make: *** [all] Error 2

Also note that I have done this before on a 32bit Athlon, and it worked fine, now I'm in [32 bit mode] gentoo on a 64 bit Athlon 64, if that makes a difference.

Ben

[EDIT]
Just adding that I'm configuring with:

Code: Select all

./configure --enable-diskless --enable-rtl8139 --enable-3c90x

hygge · Post by **hygge** » Mon Apr 12, 2004 11:57 am

thx for the great guide!

what do you think of adding options to the client's /etc/fstab?
maybe nfsvers=3,hard,rsize=8192 and wsize=8192?
will this speed something up or is it just uneccesary?

markedmann · Post by **markedmann** » Wed Apr 14, 2004 3:19 pm

Hey all:
I'm having a lot of trouble with configuring dhcpd.conf... Could someone please post a (working) sample for me to follow? I've used the one in the howto, but it appears to be simply a snippet.
Thanks in advance,
-Mark

tiny · Post by **tiny** » Thu Apr 22, 2004 11:37 am

This worked for me.

Code: Select all

option option-150 code 150 = text ;
ddns-update-style ad-hoc;
subnet 192.168.254.0 netmask 255.255.255.0 { }
group {
        host nova {
        hardware ethernet 00:08:9b:1e:69:7a;
        fixed-address 192.168.254.57;
        option option-150 "/nova/boot/grub.lst";
        filename "/nova/boot/pxegrub";
        }
}

Donpasquale · Post by **Donpasquale** » Tue May 11, 2004 2:29 pm

Hi.

fisrt of all: thanks for the great howto.

But i still got some problems with that. If i boot up my client it all works fine till the point where the kernel wants to start the init it just stops and stays there doing nothing. If i mount the nfs share in another pc which runs from hd it does it in not even a sec.

I tried every option in the kernel i think. Please help me fix that problem. Im going crazy bout this

Here is my config:

dhcpd.conf:

Code: Select all

host eta {
        hardware ethernet ::::;
        fixed-address 192.168.0.160;
        option option-150 "/eta/grub.lst";
        filename "/eta/boot/pxegrub";
}

and my grub.lst:

Code: Select all

default 0
timeout 30

title=Gentoo Networkboot
root (nd)
kernel /eta/bzImage ip=dhcp root=/dev/nfs nfsroot=192.168.0.211:/raid/diskless/eta

Hopefully someone can help me. It will be an installation server to do automated gentoo installations as i finish it.

Thx

Donpasquale

dmitrio · Post by **dmitrio** » Tue Jun 29, 2004 11:30 pm

I have copied this HOWTO, with permission of Alexandre "Swiss.Mage" Georges, to gentoo-wiki.com
http://gentoo-wiki.com/HOWTO_Gentoo_Diskless_Install
If you see anything that should be added or changed, feel free to do so.

Thank you for a great HOWTO.

lschiere · Post by **lschiere** » Tue Aug 17, 2004 4:08 pm

Can anyone give me any pointers on how this diskless howto would need to be modified to be able to pxe boot (for instance) amd64 clients with an x86 server?

someguy · Post by **someguy** » Wed Aug 18, 2004 11:17 am

Aug 18 06:22:06 horus dhcpd: DHCPDISCOVER from 00:e0:c5:68:22:a7 via eth0
Aug 18 06:22:06 horus dhcpd: DHCPOFFER on 192.168.2.15 to 00:e0:c5:68:22:a7 via eth0
Aug 18 06:22:08 horus dhcpd: Dynamic and static leases present for 192.168.2.15.
Aug 18 06:22:08 horus dhcpd: Remove host declaration ibis or remove 192.168.2.15
Aug 18 06:22:08 horus dhcpd: from the dynamic address pool for 192.168.2.0/24
Aug 18 06:22:08 horus dhcpd: DHCPREQUEST for 192.168.2.15 (192.168.2.4) from 00:e0:c5:68:22:a7 via eth0
Aug 18 06:22:08 horus dhcpd: DHCPACK on 192.168.2.15 to 00:e0:c5:68:22:a7 via eth0
Aug 18 06:22:08 horus in.tftpd[16988]: RRQ from 192.168.2.15 filename /mnt/diskless/ibis/boot/pxegrub
Aug 18 06:22:08 horus in.tftpd[16988]: sending NAK (1, File not found) to 192.168.2.15
Aug 18 06:22:08 horus in.tftpd[16989]: RRQ from 192.168.2.15 filename /mnt/diskless/ibis/boot/pxegrub
Aug 18 06:22:08 horus in.tftpd[16989]: sending NAK (1, File not found) to 192.168.2.15

im getting this too any help would be mucho appreciado

Skywacker · Post by **Skywacker** » Wed Sep 22, 2004 10:03 pm

the ftp program is getting "File not found" for filename /mnt/diskless/ibis/boot/pxegrub

look in /etc/conf.d/in.tfpd for your initial ftp directory. chances are your pxegrub file and kernel are not located in /diskless/mnt/diskless/ibis....

basically you need to edit your grub.lst until you have the correct location for your kernel, etc

I had to change my kernel to be -/b8/boot/kernel-3.2.6-gentoo-r8 because of my installation scripts.

corn · Post by **corn** » Sat Sep 25, 2004 4:30 pm

It wasnt mentioned here but the init scripts seem to look for a variable called CDBOOT by setting this in /etc/rc.conf the startup scripts will skip stuff like fsck etc... Im not sure what it should be set to but all the scripts just check if it has a length.

I was wondering about shared resources such as a sysklogd. I heve them turned off right now. I was curious because I will be running 10-20 machines off my server.

Skywacker · Post by **Skywacker** » Mon Sep 27, 2004 2:46 pm

I built 1 diskless node thanks to these instructions, but I'm wondering the best way to roll this out to my other 30+ machines. I tried just copying /diskless/b8 to/diskless/b9 and modifying fstab, hostname, and a couple other files. This gives me a booting, mostly working Gentoo system with one catch: I can't lamboot (lam-mpi) my cluster if I include b9.

b2 is my master
b8 is my first working diskless node
b9 is ia copy of the b8 dir with a few config files changed.

I can lamboot b2 and b8, but not b9.

Any ideas?

All other applications seem to work: ganglia, nfs, ntp, ldap authentication.

Thanks,
Skywacker

riksta · Post by **riksta** » Sat Oct 02, 2004 4:28 am

just a quick tip to consolidate the howto commands a little

# mkdir /diskless
# mkdir /diskless/eta
# mkdir /diskless/eta/boot

can all be done with

# mkdir -p /diskless/eta/boot

Skywacker · Post by **Skywacker** » Wed Nov 10, 2004 10:57 pm

No offense to this thread as it was very helpful, but once I had a working 8 node diskless cluster I ran a comparison test versus a diskfull 8 node cluster and the diskless cluster was about 400% slower. They were not identical clusters, and the master node on the diskfull cluster was faster, but I ended up reverting back to a diskfull cluster. Just something to think about if anyone out there is in the planning stage still.

-Skywacker

kakakoka · Post by **kakakoka** » Sat Nov 13, 2004 4:20 pm

The dhcpd server i am using (the freebsd dhcpd server on my lan) wont start with this:

option option-150 code 150 = text ;

In the header of the dhcpd.conf.

Whats could I be doing wrong?

rkrenzis · Post by **rkrenzis** » Fri Nov 19, 2004 8:26 pm

option option-150 code 150 = text ;

Remove the space between the word "text" and the ";" symbol.

option option-150 code 150 = text;

rkrenzis · Post by **rkrenzis** » Fri Nov 19, 2004 8:27 pm

Everything works with the nfs-boot. emerge sync and adding packages is a breeze. *BUT* when I'm a regular user (non-root or uid 0), I can't resolve hostnames. Any thoughts?

TIA.

kakakoka · Post by **kakakoka** » Tue Nov 23, 2004 1:23 pm

Removing the space seems to make no difference.

Code: Select all

 /usr/sbin/dhcpd xl0
Internet Software Consortium DHCP Server 2.0pl5-OpenBSD
Copyright 1995, 1996, 1997, 1998, 1999 The Internet Software Consortium.
All rights reserved.

Please contribute if you find this software useful.
For info, please visit http://www.isc.org/dhcp-contrib.html

/etc/dhcpd.conf line 19: expecting string or hexadecimal data.
option option-150 code
       ^
Configuration file errors encountered -- exiting
exiting.

dsf · Post by **dsf** » Sat Mar 05, 2005 8:30 pm

I've updated the wiki for the HOWTO by simplifying several manual steps because of an improvement in the grub ebuild.

Code: Select all

As of grub 0.94-r1 (and later), it now builds pxegrub properly. It enables diskless booting, and compiles in all supported GRUB network drivers.

# cp /usr/lib/grub/pxegrub /diskless/eta/boot/pxegrub

My Gentoo diskless install howto

diskless '/' gets mounted read-only

NFS swap

Wierd NFS error

Diskless xfree performance

Converting an install to diskless?

:. copied to gentoo-wiki.com

server and clients of different types

/etc/dhcpd.conf fix

Can't resolve as non-root user...