Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Gentoo Diskless with ATA over Ethernet (AoE)
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Thu Feb 16, 2006 9:04 am    Post subject: HOWTO: Gentoo Diskless with ATA over Ethernet (AoE) Reply with quote

LAST EDITED: 24-04-06
24-04-06: Added the note about using initramfs instead of initrd. Now the boot time and application execution on the diskless PC even faster then on my local Gentoo installation!

I'm using diskless PC's with Gentoo Linux for quite a while. It works just fine and you even don't notice performance differences for light DE's like XFCE4, but if you'd like to switch to KDE... that's another story. So, I've decided to open the topic here in order to accumulate all performance tricks for diskless desktops.
The most basic setup is like bellow:
- PC boots using PXE and downloading kernel + initrd
- the boot process set all parameters using DHCP and mount all necessary shares using NFS
In order to use heavy DE (KDE) I have to do some adjustments.
1) make squashfs separately for /usr, /opt and mount them via loopback
2) mount /tmp and /var using tmpfs
With the two adjustments on diskless PC I already can't notice performance degree even with KDE :)
What could be optimized/tuned more? I tried using Network Block Devices (NBD) for mounting /usr and /opt, but got just a lot of troubles. At the moment I'm thinking about mounting /usr and /opt using ATA over Ethernet (AoE) instead loopback and NFS. In my opinion it can make system even faster with less load on the server and network.
If somebody has any ideas/experiences please share them!

P.S. If somebody needs a detailed "how to optimize" I can post requested details.

EDIT:AoE works like a charm... and with combination with squashfs I'm getting even quicker boot now.

P.S.S. Who wants step-by-step HOWTO!?

HOWTO
!!!Change net.home to you domain and bsd to your server name bellow in config files.
1) Configure your server (probably Gentoo as well :) )
If you need more information how to configure server look at http://www.faqs.org/docs/Linux-mini/Remote-Boot.html#ss4.4. Here I'll explane only "my way".

  • configure DHCP, I'm using dnsmasq, which can act as DNS server for small/middle network as well.
    Example of dnsmasq.conf.
    Code:
    domain-needed
    bogus-priv
    filterwin2k
    local=/net.home/
    interface=eth0
    expand-hosts
    domain=net.home
    dhcp-range=192.168.1.50,192.168.1.100,12h
    dhcp-option=1,255.255.255.0
    dhcp-option=3,192.168.1.1
    dhcp-option=6,192.168.1.1
    dhcp-option=17,192.168.1.1:/mnt/data/diskless/
    dhcp-option=28,192.168.1.255
    dhcp-option=44,192.168.1.1
    dhcp-option=58,300
    dhcp-option=59,300
    dhcp-option=135,menu
    #dhcp-option=252,"http://proxy/proxy.pac"
    dhcp-boot=/pxelinux.0,bsd.net.home

  • Configure tftp server. Under Gentoo I'm using atftp.
    Example of /etc/conf.d/atftp
    Code:
    TFTPD_ROOT="/mnt/local/tftproot"
    TFTPD_OPTS="--daemon --user nobody --group nobody --bind-address 192.168.1.1"

  • Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory:
    Code:

    PROMPT 0
    DEFAULT menu.c32
    TIMEOUT 50
    MENU TITLE Boot menu

    label local
        MENU LABEL ^Windows XP
        LOCALBOOT 0

    label gnet-nfsroot
        MENU LABEL Gentoo Network NFS root
        kernel kernel/bzImage-2.6.15-ck2-rr4
        append init=/sbin/init ramdisk_size=32000 root=/dev/nfs ip=dhcp nfsroot=bsd:/mnt/data/diskless/net.home/root video=vesafb,mtrr vga=0x31A

    label gnet-aoe
        MENU LABEL Gentoo Network AoE
        MENU DEFAULT
        kernel kernel/bzImage-2.6.15-ck2-rr4-aoe
        append initrd=kernel/initrd.gz ramdisk_size=32000 root=/dev/ram0 rw ip=dhcp video=vesafb,mtrr vga=0x31A

    In the example above all kernel and initrd images are kept under kernel subdirectory of tftp root directory.
  • Code:
    emerge –av sys-block/vblade


    The server part is finished at the moment, so now the clien (diskless) PC. Let assume you have Gentoo Desktop successfully installed, which we will use as "prototype" of a diskless system. Firstly let have some notes of the kernel configuration. Be sure that you have:
    - Network driver compiled in the kernel (not as module)
    -
    Code:
     [*]   IP: kernel level autoconfiguration           
     [*]     IP: DHCP support                           
     [*]     IP: BOOTP support

    - ATA ove Ethernet support compiled as module
    Code:
    <M> ATA over Ethernet support

    - Initial ramdisk support compiled in the kernel
    Code:
     <*> RAM disk support                 
    (16)  Default number of RAM disks     
    (32000) Default RAM disk size (kbytes)


Now the optimization part begins.
1) I would suggest creating a separate directory for the “diskless snapshot” somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
Code:
mkdir /mnt/tmp
mount –o bind / /mnt/tmp
cd /mnt/tmp
find . –print | cpio –pamd /mnt/diskless

At this point we have the snapshot under /mnt/diskless.
2) You can easy remove /mnt/diskless/usr/src, /mnt/diskless/usr/include, (/mnt/diskless/usr/kde/kde3.x/include), /mnt/diskless/usr/doc directories and /mnt/diskless/boot. Optionally /mnt/diskless/usr/man and /mnt/diskless/usr/info could be removed as well.
3) You should adjust come config files in the /etc directory for the diskless workstation as you wish. Bellow is only example of fstab for usling with AoE:
Code:
 /dev/ram0                               /       ext2            rw,noatime                      0 0
/dev/etherd/e0.0                        /usr    squashfs        ro,defaults                     0 0
/dev/etherd/e0.1                        /opt    squashfs        ro,defaults                     0 0
bsd:/mnt/data/diskless/net.home         /mnt/diskless   nfs     rw,hard,intr,nolock,nfsvers=3   0 0
bsd:/mnt/data/diskless/net.home/home    /home   nfs             rw,hard,intr,nolock,nfsvers=3   0 0

tmpfs                                   /tmp     tmpfs          size=64m,mode=1777              0 0
tmpfs                                   /var/tmp tmpfs          size=64m,mode=1777              0 0
tmpfs                                   /var/log tmpfs          size=16m                        0 0

proc                    /proc                   proc            defaults                        0 0
shm                     /dev/shm                tmpfs           nodev,nosuid,noexec             0 0

4) add
Code:
aoe aoe_iflist="eth0"
to the kernel-2.6 file in the /etc/modules.autoload.d directory.
5) Now we will make squashfs of /usr and /opt, just
Code:
cd /mnt/diskless && mksquashfs ./usr  /path/to/usr.sqfs
mksquashfs ./opt  /path/to/opt.sqfs

6) Now we can easily remove content of /usr and /opt under diskless snapshot (/mnt/diskless) and make initrd image of the rest:
Code:
#!/bin/bash
WORKDIR=/path/to/workdir
cd $WORKDIR
dd if=/dev/zero of=initrd count=64000 bs=512
mke2fs initrd
tune2fs -c0 -i0 initrd
mount -o loop initrd /mnt/initrd && \
cd /mnt/diskless && cp -ar . /mnt/initrd
umount /mnt/initrd && cd $WORKDIR && gzip -9 -c initrd > initrd.gz

7) copy initrd.gz and corresponding bzImage for diskless workstation to tftpfoot/kernel

At the moment we need one more step in order to complete server setup:
Copy opt.sqfs and usr.sqfs somewhere to the server and edit /etc/conf.d/vblade:
Code:
 config_vblade0="0 0 eth0 /path/to/usr.sqfs"
config_vblade1="0 1 eth0 /path/to/opt.sqfs"

Code:
cd /etc/init.d
ln –s  vblade.vblade0 vblade.vblade1
./ vblade.vblade0 start
./ vblade.vblade0 start


So, it’s time now try bootig your client ;)


EDIT: For using initramfs (I'm preferring this way ;) )
Adjust the kernel patch for using tmpfs instead of initramfs (found on http://www.tux.org/pub/people/kent-robotti/looplinux/rip)
There's a problem with hotplug during early boot, so it should be disabled temporarly (chmod 644 /sbin/hotplug), then I modified /etc/ini.d/bootmisc and place the code bellow in the start section:
Code:
# Workaround for udevsend early boot error messages
        ebegin "Enabling hotplug"
        /bin/chmod 755 /sbin/hotplug
        eend 0

3) fstab should be adjusted
Code:
 /dev/ram0                               /       tmpfs           size=64m                        0 0
/dev/etherd/e0.0                        /usr    squashfs        ro,defaults                     0 0
/dev/etherd/e0.1                        /opt    squashfs        ro,defaults                     0 0
bsd:/mnt/data/diskless/net.home         /mnt/diskless   nfs     rw,hard,intr,nolock,nfsvers=3   0 0
bsd:/mnt/data/diskless/net.home/home    /home   nfs             rw,hard,intr,nolock,nfsvers=3   0 0

tmpfs                                   /tmp     tmpfs          size=64m,mode=1777              0 0
tmpfs                                   /var/tmp tmpfs          size=64m,mode=1777              0 0
tmpfs                                   /var/log tmpfs          size=16m                        0 0

proc                    /proc                   proc            defaults                        0 0
shm                     /dev/shm                tmpfs           nodev,nosuid,noexec             0 0


6) instead of creating initrd we should use initramfs:
Code:
cd /mnt/diskless
ln -s sbin/init init
find . | cpio -v -o -H newc | gzip -9 > ../initramfs.cgz

Finally we have to adjust boot loader configuration:
[codfe]label gnet-nfsroot
MENU LABEL Gentoo Network NFS root
kernel kernel/bzImage-2.6.15-ck2-rr4
append quiet initrd=kernel/initramfs.cgz root=/dev/ram0 rw ip=dhcp video=mtrr,vesa vga=0x31A[/code]


Last edited by VoVaN on Mon Apr 24, 2006 8:45 am; edited 10 times in total
Back to top
View user's profile Send private message
VStrider
Apprentice
Apprentice


Joined: 27 Jun 2005
Posts: 244
Location: 1 to Rule All way, Moria Gate, Middle Earth, SAU 70N

PostPosted: Thu Feb 16, 2006 12:07 pm    Post subject: Reply with quote

Hi. I'm also using diskless clients. Are you using ltsp or something else?

VoVaN wrote:
In order to use heavy DE (KDE) I have to do some adjustments.
1) make squashfs separately for /usr, /opt and mount them via loopback
2) mount /tmp and /var using tmpfs

Interesting, could you give some more details on your setup? Why do you need to mount /usr/ and /tmp?

I'm using gnome on the diskless clients and works great as long as there's no mono apps running (like beagle) and as long as I'm not using FF-1.5 (using FF1.5 will hang the clients after a few mins of use. I suppose it has something to do with the heavy memory use of FF1.5 which drains the res on the server. Btw I read this a couple of days ago, so it looks like mem cache can be disabled or reduced.)
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Thu Feb 16, 2006 12:24 pm    Post subject: Reply with quote

VStrider wrote:

Interesting, could you give some more details on your setup? Why do you need to mount /usr/ and /tmp?

As I mentioned I'm using initrd, so root at /dev/ram0 (in memory). The initrd is the same for all PC's, so all specific PC's parameters done on at boot time based on DHCP info. I didn't leave a lot of free space in initrd, so I mounted /tmp as tmpfs, which is basically designed for things like that. Secondly, I don't think it's smart to use ram for /usr, /opt..., so you have to mount it somehow later.
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Fri Mar 03, 2006 11:49 am    Post subject: Reply with quote

Who wants step-by-step HOWTO!?
Back to top
View user's profile Send private message
playahater
Guru
Guru


Joined: 02 Jul 2005
Posts: 382
Location: Serbia

PostPosted: Fri Mar 03, 2006 12:13 pm    Post subject: Reply with quote

VoVaN wrote:
Who wants step-by-step HOWTO!?


i would like one, please .. :)

Cheers
_________________
http://droopia.net
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Fri Mar 03, 2006 1:48 pm    Post subject: Reply with quote

playahater wrote:
VoVaN wrote:
Who wants step-by-step HOWTO!?


i would like one, please .. :)

Cheers


You got it ;)
Back to top
View user's profile Send private message
playahater
Guru
Guru


Joined: 02 Jul 2005
Posts: 382
Location: Serbia

PostPosted: Fri Mar 03, 2006 2:58 pm    Post subject: Reply with quote

Gr8 job man ... :) .. this is very usefull :)
finally the old Pentium I without hdd will have its usage :)

one more question .. i need PXE chip on my eth card, right? .. do i need PXE on both, client and server or .. ?
i do not have any eth card for PI but i`m planning to buy it in couple of days .. so can you recommend one ? ..
i don`t know wich cards do have PXE :(

10x in advace

Cheers
_________________
http://droopia.net
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Fri Mar 03, 2006 3:08 pm    Post subject: Reply with quote

playahater wrote:
Gr8 job man ... :) .. this is very usefull :)
finally the old Pentium I without hdd will have its usage :)

one more question .. i need PXE chip on my eth card, right? .. do i need PXE on both, client and server or .. ?
i do not have any eth card for PI but i`m planning to buy it in couple of days .. so can you recommend one ? ..
i don`t know wich cards do have PXE :(

10x in advace

Cheers

You have to have PXE card only for the client PC. Most Intel and 3Com cards have it. I'm personly using Intel cards. Succes with the setup... later you probably would try to boot Windows95/98 diskless as well as me (for gaming :) )
Back to top
View user's profile Send private message
playahater
Guru
Guru


Joined: 02 Jul 2005
Posts: 382
Location: Serbia

PostPosted: Fri Mar 03, 2006 3:26 pm    Post subject: Reply with quote

excellent :)

10x man ..

Cheers .. :)
_________________
http://droopia.net
Back to top
View user's profile Send private message
cazze
Apprentice
Apprentice


Joined: 26 Mar 2003
Posts: 155
Location: Brussels - Belgium

PostPosted: Mon Mar 06, 2006 8:53 pm    Post subject: Reply with quote

Well, i would like to have a howto too :)

this sounds very nice for a silent htpc, so i can skip those noisy hdd's

kammicazze
_________________
Required: Windows 95 or better, so i installed Linux!!!
Back to top
View user's profile Send private message
playahater
Guru
Guru


Joined: 02 Jul 2005
Posts: 382
Location: Serbia

PostPosted: Mon Mar 06, 2006 8:57 pm    Post subject: Reply with quote

kammicazze wrote:
Well, i would like to have a howto too :)

this sounds very nice for a silent htpc, so i can skip those noisy hdd's

kammicazze


1st post .. :wink:
_________________
http://droopia.net
Back to top
View user's profile Send private message
JeliJami
Veteran
Veteran


Joined: 17 Jan 2006
Posts: 1086
Location: Belgium

PostPosted: Tue Mar 07, 2006 8:43 am    Post subject: Re: HOWTO: Gentoo Diskless with ATA over Ethernet (AoE) Reply with quote

VoVaN wrote:
Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory:


any reason you don't use the syslinux ebuild?
Code:
# emerge -s syslinux
Searching...   
[ Results for search key : syslinux ]
[ Applications found : 1 ]
 
*  sys-boot/syslinux
      Latest version available: 3.09
      Latest version installed: [ Not Installed ]
      Size of downloaded files: 1,159 kB
      Homepage:    http://syslinux.zytor.com/
      Description: SysLinux, IsoLinux and PXELinux bootloader
      License:     GPL-2


Quote:
1) I would suggest creating a separate directory for the “diskless snapshot” somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
Code:
mkdir /mnt/tmp && mount –o bind / /mnt/tmp && cd /mnt/tmp && find . –print | cpio –pamd /mnt/diskless


I'm probably missing something here.
the series of commands
Code:
mkdir /mnt/tmp
mount -o bin / /mnt/tmp
cd /mnt/tmp

would result in / being accessible in /mnt/tmp
i guess you need a copy of your install into /mnt/tmp first (or a fresh install)
then
Code:
# chroot /mnt/tmp /bin/bash

and remove stuff in this chroot environment


Quote:
2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.

you're removing /usr/src, ..., /boot from the / system (your server basically)?
_________________
Unanswered Post Initiative | Search | FAQ
Former username: davjel
Back to top
View user's profile Send private message
FormerSlacker
Guru
Guru


Joined: 11 Mar 2003
Posts: 340
Location: Toronto, ON. Canada

PostPosted: Tue Mar 07, 2006 9:11 am    Post subject: Reply with quote

I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.

I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.

Such a neat thing. :)
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Tue Mar 07, 2006 9:18 am    Post subject: Re: HOWTO: Gentoo Diskless with ATA over Ethernet (AoE) Reply with quote

davjel wrote:
VoVaN wrote:
Download syslinux (http://syslinux.zytor.com/) and copy chain.c32 menu.c32 memdisk pxelinux.0 to tftp root directory. Create pxelinux.cfg dir and create the default file under pxelinux.cfg directory:


any reason you don't use the syslinux ebuild?
Code:
# emerge -s syslinux
Searching...   
[ Results for search key : syslinux ]
[ Applications found : 1 ]
 
*  sys-boot/syslinux
      Latest version available: 3.09
      Latest version installed: [ Not Installed ]
      Size of downloaded files: 1,159 kB
      Homepage:    http://syslinux.zytor.com/
      Description: SysLinux, IsoLinux and PXELinux bootloader
      License:     GPL-2


That's possible as well, but I would preffer downloading: 1) the latest version there is 3.11; 2) I need only 4 files and nothing else.

davjel wrote:

Quote:
1) I would suggest creating a separate directory for the “diskless snapshot” somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
Code:
mkdir /mnt/tmp && mount –o bind / /mnt/tmp && cd /mnt/tmp && find . –print | cpio –pamd /mnt/diskless

I'm probably missing something here.
the series of commands
Code:
mkdir /mnt/tmp
mount -o bin / /mnt/tmp
cd /mnt/tmp

would result in / being accessible in /mnt/tmp
i guess you need a copy of your install into /mnt/tmp first (or a fresh install)


you aren't right here....
Code:
mount -o bind / /mnt/tmp
makes root accesseble under /mnt/tmp.
I'm doing thet in order to get rid of all unnecessary files under /proc /sys /dev directories.

davjel wrote:
then
Code:
# chroot /mnt/tmp /bin/bash

and remove stuff in this chroot environment

Sorry, didn't get your point here :(

davjel wrote:

Quote:
2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.

you're removing /usr/src, ..., /boot from the / system (your server basically)?

Thanx for this comment, I'll correct it in the original post. I meant /usr/src, /boot, etc. from the snapshot directory (in our case /mnt/diskless) so it should be /mnt/diskless/usr/src, /mnt/diskless/boot...
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Tue Mar 07, 2006 9:23 am    Post subject: Reply with quote

2 davjel: Thanx for reviewing :) I corrected what I promissed.
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Tue Mar 07, 2006 9:31 am    Post subject: Reply with quote

FormerSlacker wrote:
I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.

I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.

Such a neat thing. :)


First of all, I'm glad to hear that I'm not along with the "diskless workstation" idea ;)
Even in your case you'll benefit if you're doing AoE and squashfs. In my case KDE loads twice quicker just with squashfs, then over NFS.
As for NFS root: If you have only one computer "NFS root" is just fine, but if you have a lot of them.... it' better to have single initrd with boottime setup.
Finally I'd like to mention, that NFS under heavy load takes a lot of CPU time in case of AoE it's 5 times as less in my setup.
Back to top
View user's profile Send private message
JeliJami
Veteran
Veteran


Joined: 17 Jan 2006
Posts: 1086
Location: Belgium

PostPosted: Tue Mar 07, 2006 9:56 am    Post subject: Re: HOWTO: Gentoo Diskless with ATA over Ethernet (AoE) Reply with quote

davjel wrote:

any reason you don't use the syslinux ebuild?

VoVaN wrote:
That's possible as well, but I would preffer downloading: 1) the latest version there is 3.11; 2) I need only 4 files and nothing else.

and i would prefer to have my packages maintainable by my package manager, in this case portage
but that's just me, i guess :)

VoVaN wrote:
1) I would suggest creating a separate directory for the “diskless snapshot” somewhere and copy all necessary parts there. Let assume that the snapshot directory is /mnt/diskless
Code:
mkdir /mnt/tmp && mount –o bind / /mnt/tmp && cd /mnt/tmp && find . –print | cpio –pamd /mnt/diskless

davjel wrote:
I'm probably missing something here.
...
i guess you need a copy of your install into /mnt/tmp first (or a fresh install)

you aren't right here....
Code:
mount -o bind / /mnt/tmp
makes root accesseble under /mnt/tmp.
I'm doing thet in order to get rid of all unnecessary files under /proc /sys /dev directories.

what i missed was
Code:
find . –print | cpio –pamd /mnt/diskless

i didn't see that one as a copy command :oops:
VoVaN wrote:
davjel wrote:
then
Code:
# chroot /mnt/tmp /bin/bash

and remove stuff in this chroot environment

Sorry, didn't get your point here :(

what i meant was: use the alternative installation guide to install gentoo in a chrooted environment (your new 'snapshot') using an installed linux system (your 'server')
but never mind, your copy operation (see above) is definitely faster :)
davjel wrote:

Quote:
2) You can easy remove /usr/src, /usr/include, (/usr/kde/kde3.x/include), /usr/doc directories and /boot. Optionally /usr/man and /usr/info could be removed as well.

you're removing /usr/src, ..., /boot from the / system (your server basically)?

VoVaN wrote:
Thanx for this comment, I'll correct it in the original post. I meant /usr/src, /boot, etc. from the snapshot directory (in our case /mnt/diskless) so it should be /mnt/diskless/usr/src, /mnt/diskless/boot...

you're welcome 8)
_________________
Unanswered Post Initiative | Search | FAQ
Former username: davjel
Back to top
View user's profile Send private message
FormerSlacker
Guru
Guru


Joined: 11 Mar 2003
Posts: 340
Location: Toronto, ON. Canada

PostPosted: Tue Mar 07, 2006 11:46 pm    Post subject: Reply with quote

VoVaN wrote:
FormerSlacker wrote:
I've got a setup like that myself. Old P100 with no hard drive, booting off the network via etherboot and tftp, and it's root is exported via NFS off the server, and then it launches X and grabs the nice little KDM screen via XDMCP.

I've used Debian for the diskless client OS, seems to work well once you strip away most of the startup services. I'm always amazed to see KDE running perfectly on such a krappy old computer.

Such a neat thing. :)


First of all, I'm glad to hear that I'm not along with the "diskless workstation" idea ;)
Even in your case you'll benefit if you're doing AoE and squashfs. In my case KDE loads twice quicker just with squashfs, then over NFS.
As for NFS root: If you have only one computer "NFS root" is just fine, but if you have a lot of them.... it' better to have single initrd with boottime setup.
Finally I'd like to mention, that NFS under heavy load takes a lot of CPU time in case of AoE it's 5 times as less in my setup.


With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.

It's really a dumb thin client, that does nothing but boots linux and starts an X server.

NFS load is minimal, after the inital bootup and starting X (~45s-1min), the nfs is rarely accessed by the client.

What I have been thinking about is getting rid of the NFS server completely, and sending the clients a tiny squashfs initrd and kernel via tftp that contains just enough to connect to an XDMCP X server. I've tinkered with pxes, but its kernel just seg faults right on boot.
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Wed Mar 08, 2006 8:25 am    Post subject: Reply with quote

FormerSlacker wrote:

With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.

It's really a dumb thin client, that does nothing but boots linux and starts an X server.

NFS load is minimal, after the inital bootup and starting X (~45s-1min), the nfs is rarely accessed by the client.

What I have been thinking about is getting rid of the NFS server completely, and sending the clients a tiny squashfs initrd and kernel via tftp that contains just enough to connect to an XDMCP X server. I've tinkered with pxes, but its kernel just seg faults right on boot.


Could you tell more, probably I can help if you'd like. I've a lot of diskless computers and they all are using pxe + kernel + initrd.
Back to top
View user's profile Send private message
olger901
l33t
l33t


Joined: 17 Mar 2005
Posts: 612

PostPosted: Wed Mar 08, 2006 8:44 am    Post subject: Reply with quote

Shouldn't this be moved over to Documentation, Tips & Tricks...?
Back to top
View user's profile Send private message
FormerSlacker
Guru
Guru


Joined: 11 Mar 2003
Posts: 340
Location: Toronto, ON. Canada

PostPosted: Wed Mar 08, 2006 11:32 pm    Post subject: Reply with quote

VoVaN wrote:
FormerSlacker wrote:

With the setup the way I have it, each client would require its own NFS mount, with Debian and X its about 300 megs, not too bad. KDE is loaded on the server, the client just loads a X session and connects to the server.

It's really a dumb thin client, that does nothing but boots linux and starts an X server.

NFS load is minimal, after the inital bootup and starting X (~45s-1min), the nfs is rarely accessed by the client.

What I have been thinking about is getting rid of the NFS server completely, and sending the clients a tiny squashfs initrd and kernel via tftp that contains just enough to connect to an XDMCP X server. I've tinkered with pxes, but its kernel just seg faults right on boot.


Could you tell more, probably I can help if you'd like. I've a lot of diskless computers and they all are using pxe + kernel + initrd.


Such friendly folk around here. :)

What I was trying to do initially is send out Pxes to the diskless clients to boot. I've managed to do this, but the Pxes Kernel, 2.4 and 2.6, seems to segfault on boot; the keyboard lights just blink on the clients with no error displayed.

When that didn't work out, I just settled with Debian on an NFS root.

What I'd like to do is just send a tiny distro, Pxes or something else, in an initrd and the kernel to all the clients that would initiate an X session with the server.

Worse comes to worse, it may be time to build a tiny distro on my own.

If you have any pointers to any other tiny Linux distros, with just bare X to connect locally with XDMCP and nothing else, or any help it'd be much appreciated.
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Thu Mar 09, 2006 7:06 am    Post subject: Reply with quote

FormerSlacker wrote:

Such friendly folk around here. :)

;)

FormerSlacker wrote:

When that didn't work out, I just settled with Debian on an NFS root.

Could you tell more details about your setup? I'm using PXE, not pxes distro you mentioned.

FormerSlacker wrote:

What I'd like to do is just send a tiny distro, Pxes or something else, in an initrd and the kernel to all the clients that would initiate an X session with the server.

Worse comes to worse, it may be time to build a tiny distro on my own.

You can build relly tiny distro based on Gentoo as well. I think it's better if you have some experiences, look at sys-apps/baselayout-lite in portage ;)

FormerSlacker wrote:

If you have any pointers to any other tiny Linux distros, with just bare X to connect locally with XDMCP and nothing else, or any help it'd be much appreciated.

I've seen the floppy based distro exactly for things you mentioned, but can't remember anymore :( Look at http://www.linux.org/dist/list.html in "embedded" category, probably you'll fing it, but personally I would preffer to do it with Gentoo.
Back to top
View user's profile Send private message
FormerSlacker
Guru
Guru


Joined: 11 Mar 2003
Posts: 340
Location: Toronto, ON. Canada

PostPosted: Thu Mar 09, 2006 11:11 pm    Post subject: Reply with quote

VoVaN wrote:
Could you tell more details about your setup? I'm using PXE, not pxes distro you mentioned.

Sure can. Here's the setup. Perhaps this may help some others out there.

1) The client boots up and makes a dhcp request for the boot files. Since the computers in question are real old (P100 to 166, 32 to 128m) and do not support network booting, I had to use a bootdisk for the clients using Etherboot rom-o-matic. Tape the floppy drive shut and you have ghetto network booting. :wink:

2) DHCP server gives the client the necessary information for it to boot up.
Code:
 host george {
      hardware ethernet 00:80:C6:EA:RB:53;
      fixed-address 192.168.0.4;       # clients address
      option host-name "george"
      filename "/tftpboot/clients/george/vmlinuz.nbi"; # network boot image, kernel + initrd
      next-server 192.168.0.3; # instruct the client that dhcp server is not the host, this ip is
      option root-path "/diskless/clients/george/"; # root nfs path
   }


3) TFTP and NFS are setup on the server using the above paths for booting and root repsectively.

4) The NBI image is a custom kernel for the client with the "ROOT on NFS" option enabled, made with the mknbi package in portage.

5) The clients NFS root is just a bare debian distro, that was installed then copied over to the clients NFS export directory, configured to query the server for an xsession on boot: "X -query 192.168.0.3"

6) Login manager, KDM, is configured to listen for remote login requests by editing /usr/kde/3.5/share/config/kdm/Xaccess and adding the ip of the client and setting "Enable=true" under "[Xdmcp]" in /usr/kde/3.5/share/config/kdm/kdmrc.

Thats the long and the short of it. Remote, fast, KDE goodness on ancient machines.

VoVaN wrote:
You can build relly tiny distro based on Gentoo as well. I think it's better if you have some experiences, look at sys-apps/baselayout-lite in portage ;)

I've seen the floppy based distro exactly for things you mentioned, but can't remember anymore :( Look at http://www.linux.org/dist/list.html in "embedded" category, probably you'll fing it, but personally I would preffer to do it with Gentoo.


I've debated whether sending the client distro as a initrd and having it mounted from ram is a good idea. I'm not sure if the 32mb clients can afford to do that. I think the only thing left is, as you mentioned, roll out a tiny version of gentoo for this purpose.

olger901 wrote:
Shouldn't this be moved over to Documentation, Tips & Tricks...?

I suppose since the end goal of all this is getting a desktop environment on diskless workstations, this is also relavant here. Then again what do I know, I'm not a mod. 8)
Back to top
View user's profile Send private message
VoVaN
l33t
l33t


Joined: 02 Jul 2003
Posts: 667
Location: The Netherlands

PostPosted: Fri Mar 10, 2006 9:28 am    Post subject: Reply with quote

FormerSlacker wrote:


1) The client boots up and makes a dhcp request for the boot files. Since the computers in question are real old (P100 to 166, 32 to 128m) and do not support network booting, I had to use a bootdisk for the clients using Etherboot rom-o-matic. Tape the floppy drive shut and you have ghetto network booting. :wink:

2) DHCP server gives the client the necessary information for it to boot up.
Code:
 host george {
      hardware ethernet 00:80:C6:EA:RB:53;
      fixed-address 192.168.0.4;       # clients address
      option host-name "george"
      filename "/tftpboot/clients/george/vmlinuz.nbi"; # network boot image, kernel + initrd
      next-server 192.168.0.3; # instruct the client that dhcp server is not the host, this ip is
      option root-path "/diskless/clients/george/"; # root nfs path
   }


3) TFTP and NFS are setup on the server using the above paths for booting and root repsectively.

4) The NBI image is a custom kernel for the client with the "ROOT on NFS" option enabled, made with the mknbi package in portage.

5) The clients NFS root is just a bare debian distro, that was installed then copied over to the clients NFS export directory, configured to query the server for an xsession on boot: "X -query 192.168.0.3"

6) Login manager, KDM, is configured to listen for remote login requests by editing /usr/kde/3.5/share/config/kdm/Xaccess and adding the ip of the client and setting "Enable=true" under "[Xdmcp]" in /usr/kde/3.5/share/config/kdm/kdmrc.

Thats the long and the short of it. Remote, fast, KDE goodness on ancient machines.


One aspect of your setup looks strange: if ypu're using NFS root why you need to create initrd or kernel+initrd(nbi)? Just single kernel with right config is enough to mount root over NFS.

FormerSlacker wrote:
I've debated whether sending the client distro as a initrd and having it mounted from ram is a good idea. I'm not sure if the 32mb clients can afford to do that. I think the only thing left is, as you mentioned, roll out a tiny version of gentoo for this purpose.


Ayway in you case and for the hardware you have the best solution is there -> http://thinstation.sourceforge.net/wiki/index.php/. This is the distro I've been talking about and I've recently found it for you ;)
Back to top
View user's profile Send private message
FormerSlacker
Guru
Guru


Joined: 11 Mar 2003
Posts: 340
Location: Toronto, ON. Canada

PostPosted: Fri Mar 10, 2006 11:28 am    Post subject: Reply with quote

VoVaN you are correct, I'm only sending out the kernel at the moment no initrd, sorry if I got it mixed up in one of my previous posts.

I must thank you kindly for the thinstation link, it is exactly what I'm looking for! :o I'm going to give it a spin and see how well it works.

Cheers mate! :D

**EDIT**

Just a FYI, it works great. The kernel and initrd come up to 3.5 megs, and it works great. Thanks again VoVan. 8)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum