My Gentoo diskless install howto

[kucrut]

First off, thanks for the original howto, as well as pdf howto from wimalopaan. one thing left tho, does the client use server's resources or does it use its own?

[wimalopaan]

If you have only one client-configuration, all is well with my description in

http://mozart.informatik.fh-kl.de/downl ... ss2_en.pdf

But if have several groups of clients with slightly different configurations it is wasteful, if you use a different full client-root. It would be much easier if we could use filesystem unification. This can be done by unionfs

http://www.fsl.cs.sunysb.edu/project-unionfs.html

but there are some issues on using unionfs over nfs on unionfs. I wrote a patch for the actual unionfs-cvs-snapshot to overcome this limitation.

The patch disables the lookup of whiteouts in the right-most branch and
disables the lookup of whiteouts in ro-branches before the check if the
whiteout must be renamed. You can't use unionfs directly as an underlying fs
for unionfs, but you can use a nfs-exported unionfs-filesystem as a
ro-branch. Now, unionfs sees a nfs-branch, which is allowed, and the unneeded
lookups aren't done anymore. This was a major drawback, because the
whiteout-lookup in the ro-branches causes the underlying unionfs to return an
'Operation not permitted'. With the patch, this doesn't happen anymore.

This problem is typical in the case of diskless-clients with different
configurations. In this scenario you have a base client-root-directory
(say /tftproot/client_A) and some derived client-roots
(e.g. /tftproot/client_B), which are something like

/tftproot/client_B dirs=/tftproot/client_Bdiff=rw:/tftproot/client_A=ro.

These /tftproot/client_* can be ro-exported via nfs. On the client-side you
can use them as a rightmost ro-branch for the client-root and the other
branches tmpfs, e.g.
 
mount -t tmpfs none /tmpfs/etc
mount -t unionfs none /etc -o dirs=/tmpfs/etc=rw:/etc=ro

We found this very useful in our environment.

[wimalopaan]

Sorry, but the link to the patch wasn't included in my previous posting:

http://mozart.informatik.fh-kl.de/downl ... /gdxs.html

[kyron]

First, obvious thanks go to wimalopaan for his document.

I jsut wanted to tell you all that I am trying to sum up a wiki version of this document. I wrote an e-mail to wimalopaan for explicit permission to do so. As I stated in my e-mail, I am only using my personnal wiki as a "sandbox" before posting a more complete version of the document on the (non-) official Gentoo Wiki

My motives for wikifying the document:
1- easy update
2- easy copy-paste of scripts

[simon78]

Thanks for the howto! I installed a mythfrontend using this guide a year or so ago. I updated the server (dhcp/nfs/tftp) to the new baselayout and udev. Now it seems like tftp doesn't work. The diskless recieves its IP from dhcp:

Code: Select all

Jan 25 12:55:39 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2
Jan 25 12:55:39 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2
Jan 25 12:55:41 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2
Jan 25 12:55:41 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2
Jan 25 12:55:45 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2
Jan 25 12:55:45 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2
Jan 25 12:55:54 simons dhcpd: DHCPDISCOVER from 00:04:61:50:3b:06 via eth2
Jan 25 12:55:54 simons dhcpd: DHCPOFFER on 192.168.2.2 to 00:04:61:50:3b:06 via eth2

But, it does not get te kernel image. When i try to manully download it works, and i get log entries from tftp that says that a file was downloaded. But from the diskless, nothing!

tftpd is running.

Any hints?

[kyron]

[...snip...]

Any hints?

Check the LTSP wiki, these guys are my inspiration to all this and it's LTSP that gave me a good start for such a platform.

[BonesToo]

I've been running a diskless gentoo client now for 2 months with help from this thread and the diskless doc on gentoo.org. All is running good. Although I have one issue that I haven't worked out yet and I can't seem to find an answer for it anywhere.

When I reboot/shutdown the machine with "reboot", "shutdown -h now", or "halt" everything shuts down correctly except at the very end of the process my hardware is still running. It never actually powers off or reboots to the bios, it just waits with the following text on the screen:

Code: Select all

 * Unmounting filesystems ...                                        [ok]
 * Remounting remaining filesystems readonly ...                     [ok]

 * fsck will be skipped on next startup
Power down.
/etc/init.d/shutdown.sh: line 8: /sbin/halt: Network is unreachable

My theory is that since the root NFS mount was unmounted it can't get to the actual halt command to turn the hardware off? I could be completely wrong. I need some input on what is going on here. Is anyone else seeing something like this???

Could it be that my hardware just doesn't support software shutdown? Im running a PC/104 single board computer with VIA C3 Nehemiah CPU. I can post more specs if needed.

Any ideas?

[kyron]

[...snip...]
My theory is that since the root NFS mount was unmounted it can't get to the actual halt command to turn the hardware off? I could be completely wrong. I need some input on what is going on here. Is anyone else seeing something like this???

Could it be that my hardware just doesn't support software shutdown? Im running a PC/104 single board computer with VIA C3 Nehemiah CPU. I can post more specs if needed.

Any ideas?

sounds like a fair assumption. Try mdifying the disklesse's /etc/init.d/shutdown.sh scritp to:

Code: Select all

/sbin/poweroff -f

Note that this is a "nasty" shutdown and should only be used on "stateless" machines (ie: all processes and their temporary information rely on RAM to store their states). This usually impliest that /etc and /var is mostly in some sort of (RAM|UNION)fs

[BonesToo]

I'll try changing script to /sbin/poweroff -f. But I don't know how this will be any different.


You bring up a good point. Using a ramfs to store the files needed during the shutdown process. This is what I've been spending most of my time on. But I am having trouble getting my initramfs cpio image compiled into the kernel. I followed gentoo doc instructions and did the following:

1) Copied my initramfs cpio compressed image to /usr/src/linux/usr/initramfs_data.cpio.gz

2) execute: touch /usr/src/linux/usr/initramfs_data.cpio.gz

3) make the kernel as normal

But it still seems to make a new cpio image from the initramfs_list. Any help here? I'm going to look around on the forums for more info on this.

Code: Select all

...
  HOSTCC  usr/gen_init_cpio
  CHK     usr/initramfs_list
  CPIO    usr/initramfs_data.cpio
  GZIP    usr/initramfs_data.cpio.gz
  AS      usr/initramfs_data.o
  LD      usr/built-in.o
...

But the question is, is this the correct approach?
Will unpacking a cpio image into a ramfs with /sbin/halt in it be what I need to shutdown the machine correctly?
Does a root ram fs maintain itself after the nfs root is mounted?
Will this ramfs be available at shutdown?

I'm at a lose here.

EDIT: My pxelinux config looks like this? Am I missing a /dev/ram0 in here?

Code: Select all

DEFAULT kernel15.nofb

LABEL kernel15.nofb
KERNEL bzImage.15.nofb
APPEND ip=dhcp root=/dev/nfs nfsroot=192.168.1.102:/root/diskless/smokey,rsize=8192,wsize=8192,timeo=14

[Per Olav]

If you are happy, like me, with this, post a message here to gratulate me

Thank you! Worked great for me!

I ran into one problem only.. but it's not really related to the Howto I guess.
Here goes:

I needed to connect some samba shares on my diskless client, and then use the netmount init script. When I shut down the client,
it umounts them but also the root filesystem which is connected via NFS. This causes a lot of errors during the rest of the shutdown.
Is there a simple solution to this?

Edit: typo

[emuller]

When I shut down the client, it umounts them but also the root filesystem which is connected via NFS. This causes a lot of errors during the rest of the shutdown. Is there a simple solution to this?

On my machine it doesn't unmount root but mounts it read-only upon failing its unmount attempt (umount -r option used in netmount implies this behavior) ... which causes a lot of errors for the rest of the shutdown ...

I put a "return 0" as the first line in stop() of the netmount script. This effectively disables stopping netmount. Since I have /usr, /opt and /home as seperate NFS mounts (appart from /) I didn't feel like figuring out any other clean way to keep them from unmounting...

I guess one could clean up the stop code in netmount:

Code: Select all

stop() {
        local ret
        ebegin "Unmounting network filesystems"
        [ -z "$(umount -art ${NET_FS_LIST// /,} 2>&1)" ]
        ret=$?
        eend ${ret} "Failed to simply unmount filesystems"
        ...

to fit your needs.

Any other ideas? I'd be interested to know what solution you arrive at.

Now that I think about it a "return 1" might be better as the first line of stop() ... so that it always errors?

e.

[kyron]

FYI, there is some work being done specifically for the NFS root mount/unmount issues:
http://bugs.gentoo.org/show_bug.cgi?id=99682
And the following is also interesting:
http://bugs.gentoo.org/show_bug.cgi?id=112059

[wimalopaan]

Hi,

for anyone who is interested:

as described in a previous posting our diskless setup includes the use of unionfs on the client and on the server. On the server we can then maintain relatively simple a set of diskless client roots for different purposes, e.g. one setup for a diskless rich client (the user logs into the client and the KDE-session starts on the client) and one setup for a diskless thin client (the user logs into the server and the KDE-session starts on the server), and so on.

Especially if you have diskless thin clients it is neccessary to transport the local devices of the thin client to the server. We experimented with several solutions (nfs back to the server, samba, ...) but we came up with the adoption of the LTSP solution what is called ltsp-fs.

We provide ebuilds for the ltsp-fs and some (little) documentation on

http://mozart.informatik.fh-kl.de/downl ... /gdxs.html

and

http://mozart.informatik.fh-kl.de/downl ... tspfs.html

Please note that you need to use the provided overlay ebuild for pmount as well (mainly for security reasons). Also note that you have to setup two kde autostart scripts as described in the output of the ebuilds.

[wescott]

Thanks for the howto. This would work great for the set-top boxes I want to develop (for expandability, extra boxes that boot this way).

[cloc3]

I have made a new version of the diskless gentoo client guide (see below).

I've also write a unionfs gentoo diskless guide, before reading this topic, and I've put it on the wiki, to be modify by someone more practiced then me.

If you like, have a look to it.

[petrasl]

Hi,

I'm trying to setup Gentoo on a diskless laptop following the wiki, but I have some trouble:

The server is ready and the laptop boot on « pxe » mode ; the grub's menu is printed.
The image of the kernel is loaded, but the system hangs when once the kernel gives the hand to « init ».

With the last kernel's log, I see that the interface « eth0 » is ready and has the good settings: IP / rootserver / rootpath
- The nfs root file system is mounted correctly
- devfs is mounted on /dev (kernel-2.4)
- INIT: version 2.86 booting

This is part of my kernel configuration:

Code: Select all

CONFIG_PACKET=y
CONFIG_UNIX=y
CONFIG_INET=y
CONFIG_IP_MULTICAST=y
CONFIG_IP_PNP=y
CONFIG_IP_PNP_DHCP=y
CONFIG_IP_PNP_BOOTP=y

CONFIG_NFS_FS=y
CONFIG_NFS_V3=y
CONFIG_NFS_DIRECTIO=y
CONFIG_ROOT_NFS=y
CONFIG_NFSD=y

It's a 2.4 series kernel, so I done:

Code: Select all

emerge --unmerge udev ; emerge devfsd

/etc/fstab :

Code: Select all

/dev/cdroms/cdrom0 /mnt/cdrom iso9660 noauto,ro 0 0
proc /proc proc defaults 0 0
tmpfs /dev/shm tmpfs nodev,nosuid,noexec 0 0

Finally I emerged the lasted staffs like syslog-ng and wixie-cron.

Anybody konw what should'I try?

[erik258]

I am doing the same install method. All seems to go OK at boot, until

Code: Select all

CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
CLIENT IP: 192.168.10.100       MASK: 255.255.255.0     DHCP IP: 192.168.10.1
GATEWAY IP: 192.168.10.1
Probing...[RTL8139] - The PCI BIOS has not enabled this device!
Updating PCI command 0006->0007. pci_bus 00 pci_device_fn 00
PCI latency timer (CFLT) is unreasonably low at 0. Setting to 32 clocks.
ioaddr 0x0000, addr 00:80:40:55:AA:2A 10Mbps full-duplex

comes up on the screen. the computer halts there, crtl+alt+del ineffective.

Does anyone recognize this?

The logs from the root show...

Code: Select all

Dec 10 15:32:55 [dhcpd] DHCPDISCOVER from 00:50:70:56:2e:ca via eth1
Dec 10 15:32:55 [dhcpd] DHCPOFFER on 192.168.10.100 to 00:50:70:56:2e:ca via eth
1
Dec 10 15:33:03 [dhcpd] DHCPREQUEST for 192.168.10.100 (192.168.10.1) from 00:50
:70:56:2e:ca via eth1
Dec 10 15:33:03 [dhcpd] DHCPACK on 192.168.10.100 to 00:50:70:56:2e:ca via eth1
Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_
Dec 10 21:33:03 [in.tftpd] tftp: client does not accept options_
Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_

It seems that dhcpd and in.tftpd are working, if misconfiguredly.

Any help is greatly appreciated

[mikek]

I am doing the same install method. All seems to go OK at boot, until

Code: Select all

CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
CLIENT IP: 192.168.10.100       MASK: 255.255.255.0     DHCP IP: 192.168.10.1
GATEWAY IP: 192.168.10.1
Probing...[RTL8139] - The PCI BIOS has not enabled this device!
Updating PCI command 0006->0007. pci_bus 00 pci_device_fn 00
PCI latency timer (CFLT) is unreasonably low at 0. Setting to 32 clocks.
ioaddr 0x0000, addr 00:80:40:55:AA:2A 10Mbps full-duplex

comes up on the screen. the computer halts there, crtl+alt+del ineffective.

Does anyone recognize this?

The logs from the root show...

Code: Select all

Dec 10 15:32:55 [dhcpd] DHCPDISCOVER from 00:50:70:56:2e:ca via eth1
Dec 10 15:32:55 [dhcpd] DHCPOFFER on 192.168.10.100 to 00:50:70:56:2e:ca via eth
1
Dec 10 15:33:03 [dhcpd] DHCPREQUEST for 192.168.10.100 (192.168.10.1) from 00:50
:70:56:2e:ca via eth1
Dec 10 15:33:03 [dhcpd] DHCPACK on 192.168.10.100 to 00:50:70:56:2e:ca via eth1
Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_
Dec 10 21:33:03 [in.tftpd] tftp: client does not accept options_
Dec 10 21:33:03 [in.tftpd] RRQ from 192.168.10.100 filename pxegrub_

It seems that dhcpd and in.tftpd are working, if misconfiguredly.

Any help is greatly appreciated

I got that same message trying to use pxegrub. I think it came from grub not having support for my ethernet hardware. I switched to pxelinux (part of syslinux) and was able to get the kernel booted.

Now, I'm having problems with the kernel just "hanging" after mounting the rootfs. It's still alive (I can ping it), but no progress is being made. If you hit that and figure out a solution, let me know!

mike

[erik258]

I think i may have had a similar problem. i will try to help if you like; i'd like to see the kernel boot command line and the last line on the screen, though. My guess would be that either the mount is failing but thinks its succeeding or succeeds but the directory is specified incorrectly and there isn't a system rooted there, or perhaps you forgot to add support for root fs on nfs? I wanted to quote menuconfig to help you out but couldn't find it ; (. Sorry, you'll have to consult an online howto.

I think you're right about the pxegrub thing; looks like emerge doesn't encorporate these configuration flags yet. I mean to get it working some time, but since none of my diskless machines have bootable NICs, it seems pointless to bother now. At the moment, I have them booting from floppy instead. ; )

[erik258]

I got a box of nice PXE-enabled 3C905s and am back on the project now. I also got a whole new message after booting grub! This time i dowloaded grub-0.97 and patched (won't compile w/o), then, copied grub over and booted as previously.

Here's the last lines.

Code: Select all

CLIENT MAC ADDR: 00 50 70 56 2E CA      GUID: FFFFFFFF-FFFF-FFFF-FFFF-FFFFFFFFFFFF
CLIENT IP: 192.168.10.100       MASK: 255.255.255.0     DHCP IP: 192.168.10.1
GATEWAY IP: 192.168.10.1 
Probing pci nic...
[3c905c-tpo]
3C90X Driver 2.00 Copyright 1999 LightSys Technology Services, Inc. 
Portions Copyright 1999 Steve Smith
Provided with ABSOLUTELY NO WARRANTY
---------------------------------------------------------------------------------------------------------
MAC Address =
Connectors present: 10Base-T / 100Base-TX

that's all. It looks a lot more promising, but nothing actually loads (i left it sitting overnight just to be sure).

I'd much prefer pxegrub to pxelinux... anyone have any suggestions for me?

[erik258]

evidently PXE booting with grub is now broken. I recommend PXELinux as a replacement. Looks like i should have listened to everyone else, my kernel is booting fine now.

here's the howto i found useful:
http://www.gentoo.org/doc/en/diskless-howto.xml

[jesse_kahtava]

Hello.
I've followed this howto as well as http://www.gentoo.org/doc/en/diskless-howto.xml and everything but nfs as root seems to be working. Here's the bottom bit of output:

Code: Select all

eth0: link up, 100Mbps, full-duplex, lpa 0x45E1
Sending DHCP requests ., OK
IP-Config: Got DHCP answer from 192.168.1.10, my address is 192.168.1.11
IP-Config: Complete:
        device=eth0, addr=192.168.1.11, mask=255.255.255.0, gw=192.168.1.1,
        host=192.168.1.11, domain=hornpipe, nis-domain=(none),
        bootserver=192.168.1.10, rootserver=192.168.1.10, rootpath=
Looking up port of RPC 100003/2 on 192.168.1.10
Looking up port of RPC 100005/1 on 192.168.1.10
VFS: Mounted root (nfs filesystem) readonly.
Freeing unused kernel memory: 144k freed
nfs_stat_to_errno: bad nfs status return value: 45
Warning: unable to open an initial console.
nfs_stat_to_errno: bad nfs status return value: 45
nfs_stat_to_errno: bad nfs status return value: 45
nfs_stat_to_errno: bad nfs status return value: 45
nfs_stat_to_errno: bad nfs status return value: 45
Kernel panic - not syncing: No init found. Try passing init= option to kernel.

Also, here's my kernel command line options:

Code: Select all

default 0
timeout 30
title=Diskless Gentoo
root (nd)
kernel /mystic/boot/linux-2.6.17-beyond4 ip=dhcp root=/dev/nfs nfsroot=192.168.1.10:/diskless/mystic

Has anyone else encountered this error? I'm able to mount nfs fine inside of a working system.
I tried a different kernel, but got the same problem. I know for certain that NFS for root is enabled.


Oh, and in response to the above post; PXE Grub is working well for me

[erik258]

I've encountered that error once. It means either that init itself (/sbin/init) can't be found, or can't be executed, or else (this was my problem) the libraries it needs can't be found.

NFS4 still fails, thus adding the kernel boot option: nfsroot=...,nfsver=3 helped.

try nfs v 3? Although, i didn't need to do this. Only thing I can think of, you haven't installed nfs-utils. Do you have nfs software installed?

[jesse_kahtava]

I've already completely disabled NFSv4 support in the kernel so I expect it's using NFSv3.
Also, I checked that init is in the right place and it executes within a chroot.
The error value of 45 is mainly what's concerning me since it spits out a bunch of them before finally giving a kernel panic.
I don't think it's able to read from the NFS mount at all.
Oh, and nfs-utils is installed. As I said though, i'm under the impression that it's completely unable to read from NFS.

[erik258]

Also, I checked that init is in the right place and it executes within a chroot.

within a chroot? It needs access to /lib to properly run!

although, i agree with your concern about the NFS error being more severe. But perhaps the chroot explains that too.