View previous topic :: View next topic |
Author |
Message |
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Thu Nov 14, 2013 9:03 pm Post subject: Framework for mass installation and maintenance of gentoo |
|
|
I am a teacher at a school for vocational education and besides that responsible for the maintenance of about 140 workstations. To make this possible without running crazy, between 2005 and 2010 a former student of me, Manuel Mommertz, and I developed a framework to mass install and maintain the whole installation (well, in fact, my part had been mainly requestig for features and testing). Since 2010 I am mostly on my own now, maintainig the whole system and slowly increasing my insight in how it works.
Unfortunately the headmaster now decided to migrate the entire school to Microsoft-Software whithin the next two years. So Manuel and I decided to hand the system over to the interested public, as long as I have a running system at hand to help getting it started elsewhere.
How it works
Maintaining 140 workstations as source code installed boxes is not possible. We run a chroot environment on one box which creates binary packages, which then get installed on the workstation. We in fact create a binary distribution, sort of, tailored to our specific needs. For the base installation of the boxes we boot the bare new box over the network and run a script which is automatically doing all things you would do by hand, installing gentoo. As there are partitioning the harddrive, create swap and filesystem, install the stage3, get a homebrew package with the kernel and install grub. Finally a package with the update script gets installed which on a running sytem later would do the usual regular upates. This script then is run to install our system in its final state. The whole thing can be configured such that a new box gets installed by just choosing pxe-boot as a boot device on startup.
The whole emerge world thing and providing the binary packages in in an ftp-archive is done automatically. There is need for manual intervention though, when upstream is doing some fundamental work on portage, packages need new USE-flags or things like that.
The creation of our "distribution" is done by a local overlay whose ebuilds pull in the upstream ebuilds as dependencies. All local customization is done by local ebuilds or by modify-scripts for base ebuilds.
I am able to install different kinds of boxes: ordinary workstations for students, workstations for teachers, terminalservers, notebooks and so on. The decision what gets installed is based on the DNS-name of the box in question.
I hope you got the picture so far. The whole thing had not been created with publication in mind. So it will not be an easy task to hand it over to someone else. First of all, I will have to remove some things specific to our installation, password-hashes for instance. Then there are a lot of things which have to be cleaned up which are leftovers from some experiments in the past and so forth.
Therefor it will not make sense to put it all in a tarball on an ftp-server and forget about it. I am looking for knowledgeable developers which are willing to replicate our environment, with my help clean things up and eventually put the whole thing into one or two ebuilds. There will have quite amount of documentation to be done too.
First thing to do will be to discuss how work can be organized. I have some mailing list in mind and a place where to put the software. Later on may be some cvs repository or the like will be needed. I don't know, I am not used to collaborative developement.
Despite that I have the whole system running for some years now, there is quite a lot of work to be done. But I think it would be worth it. As, to my understanding, some kind of such system is the only chance to use gentoo in larger installations efficiently.
As you may have noticed already, english is not my native language. So bear with me as I may ocassionally not be very responsive. I will need some spare time for proper answers when things get demanding. |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Sun Nov 17, 2013 3:59 am Post subject: |
|
|
I'm sorry to hear they decided to step over to Windows. Seems like the framework you guys build received a lot of love over the years.
I certainly would not mind deployment software tailored for Gentoo. There are lot of different tools out there (like chef) but they are not quite Gentoo specific. Now there are already feature requests popping to mind, but that is for later concern.
Be sure to edit out all hardcoded password and hostnames and all that.
I guess the first thing is to be able to correctly replicate your setup. (Get all "hacks" back to proper ebuilds and patches and conventions etc.) |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Sun Nov 17, 2013 9:31 am Post subject: |
|
|
Maitreya wrote: | I'm sorry to hear they decided to step over to Windows. Seems like the framework you guys build received a lot of love over the years.
I certainly would not mind deployment software tailored for Gentoo. There are lot of different tools out there (like chef) but they are not quite Gentoo specific. Now there are already feature requests popping to mind, but that is for later concern.
Be sure to edit out all hardcoded password and hostnames and all that.
I guess the first thing is to be able to correctly replicate your setup. (Get all "hacks" back to proper ebuilds and patches and conventions etc.) |
At first, I just planned to put all things into a tarball, hand it over and help others getting it running. Eventually I realise that this would raise quite some security concerns nowadays. Besides that there is at least the remote boot system which would be better created from scratch anyway, with me just providing the essential shell scripts needed for installing new boxes. Unfortuantely the remote boot environment is the part of which I have the least knowledge.
The script, which is installing new boxes is an init-script. I have a rewrite of this script which will run from any live system by hand. But you definitely will get the most out of the build-system with the remote boot system at hand. One of our usual IT-classrooms has 25 client box. Installing them new by hand is not fun. With the remote boot system, after registering them in DNS and DHCP it is a matter of ten minutes work (though the installation itself will run for about an hour or so, but there is nothing interesting to be seen).
The response to my post hasn't been too enthusiastic so far. May be I should tell something about the prerequisites to replicate the build-system. To do that you will need:
- DHCP server: All boxes get identified by their MAC address
- DNS server: All boxes have individual names, following some naming convention, which serve to decide what should be installed and how they should get configured
- NFS server: Provides the root filesystem of the remote boot system and the place, where the stage3 archive can be found.
- TFTP server: Provides the remote boot kernel
- a box with some space to hold the build-system in an chroot environment. Inside this environment will run a FTP server too, which provides the binary packages to the client boxes. I just have an old server with Pentium4 processor. The rebuild of the whole system with about 900 packages needs about three or four days to finish. I had to do this only once fortunately.
Of course, you will need at least one spare client box for testing.
Last edited by AgBr on Thu Dec 12, 2013 3:56 pm; edited 1 time in total |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Sun Dec 08, 2013 4:38 pm Post subject: |
|
|
Considering the lack of interest in this topic it came to my mind, that obviously maintainig large installations with Linux-desktops may be usaually better be done with some binary distribution, providing the necessary maintanance tools, which do not exist for gentoo. So, regarding gentoo in large installations, I may have some kind of a hen-egg problem here for which I am about to provide the egg. Therefore disregard of the meager interst in my topic, with your courtesy and patience I will procede in describing what we have done, just in case someone, some day is looking for such a solution.
I will start with describing the setup of our remote boot environment. This may have been done somewhere else already but we did not find anything which fully suites our needs so we brew our own. I can keep thing short here as most of it is quite straight foreward and follows a standard gentoo installation.
Instead of installing gentoo into a fresh partition you unpack a stage3 into a directory which will later get exported via NFS. Chroot into this directory as described in the handbook and follow the istructions of the handbook. To get a bootable system with network running. There are just a few minor differences:
The kernel must get built with support for root residing on NFS: "CONFIG_ROOT_NFS=y"
Options for tmpfs and devfs are needed to be set too:
CONFIG_DEVTMPFS=y
CONFIG_DEVTMPFS_MOUNT=y
CONFIG_TMPFS=y
All support for your network hardware must of course be hard coded into the kernel as the kernel needs to know about network configuration before it gets access to the filesystem.
Build the kernel and modules as usual and install the kernel modules. The kernel gets copied to /tftpboot on your TFTP-server.
You don't need grub obviously but you need a pxe bootloader. The one I use is pxelinux.0. It is part of syslinux as provided by kernel.org:
https://www.kernel.org/pub/linux/utils/boot/syslinux/
Download the tarball extract it somewhere and call make. You can do a lot of fancy things with syslinux but we just need bios/core/pxelinux.0 (note, the last character is a zero) Copy the file to /tftpboot too.
Further in /tftpboot you need a directory(!) pxelinux.cfg and within that directory a file named 'default' with a contents like this:
Code: |
default Rescue
prompt 1
timeout 1
label Rescue
kernel /kernel-3.10.17-gentoo
append ip=dhcp root=/dev/nfs rootfstype=nfs nfsroot=172.16.20.8:/usr/local/util/netsys2 init=/linuxrc
|
You will have to tailor the IP-address and path to the NFS-root to your needs obviously.
All files and directories inside /tftpboot must be world readable to work properly
After loading the kernel and before openrc gets over, there have to be made some arrangements which linuxrc takes care of
I will just present my linuxrc here:
Code: |
#!/bin/sh
export PATH=/bin
# store the kernel CMDLINE
mount -t proc none /proc
CMDLINE=$(cat /proc/cmdline)
umount /proc
mount -t tmpfs none /tmp
mkdir /tmp/root /tmp/var /tmp/mnt
cp -rp /etc /tmp/
cp -rp /run /tmp/
mount -o bind /tmp/var /var
mount -o bind /tmp/root /root
mount -o bind /tmp/etc /etc
mount -o bind /tmp/mnt /mnt
mount -o bind /tmp/run /run
mkdir -p /var/lib/xkb /var/log /var/spool\
/var/lib/dhcpcd /var/lib/syslog-ng /var/lib/nfs
ln -s /run /var/run
ln -s /run/lock /var/lock
case $(hostname) in
sn07-01) ln -snf rescue /etc/runlevels/default ;;
sn07-??) ln -snf rescue /etc/runlevels/setup ;;
legolas) ln -snf rescue /etc/runlevels/default ;;
dragon) ln -snf rescue /etc/runlevels/terminal ;;
* ) ln -snf rescue /etc/runlevels/default ;;
esac
exec /sbin/init ${CMDLINE}
|
The root-filesystem gets mounted ro. To make sure the system can write to certain parts of the filesystem, we create /tmp in memory and bind all parts of the root-filesystem which need to be writable to directories therein. Then we link the desired runlevel-target to default, based on the DNS-name of the starting machine and finally start init with the initial kernel commandline as parameter.
In order to get this working you will have to set 'wipe_tmp="NO"' within /etc/conf.d/bootmisc. Otherwise /etc/init.d/bootmisc would delete you your sophistically created virtual system within /tmp.
The 'rescue' target is the usual 'default' runlevel which just starts a machine without graphical login. The 'terminal' target will start an X-display manager. The setup target starts an init-script which installs a new machine onto the harddisk. You easily can define other targets to your liking. Just chroot into your netsystem on the NFS-Server and install the needed software.
You will have to make sure, the client machines get their hostname via dhcp and the machines will keep their addresses.
In /etc/conf.d/net you should set: 'dhcpcd_eth0=" -n -p "'
I'll show you how my /etc/runlevels is looking:
Code: |
eowyn / # ls /etc/runlevels/
boot rescue setup shutdown sysinit
eowyn / # ls /etc/runlevels/boot
bootmisc keymaps modules net.eth0 procfs sshd syslog-ng tmpfiles.setup
hwclock localmount mtab net.lo root sysctl termencoding urandom
eowyn / # ls /etc/runlevels/sysinit/
devfs dmesg sysfs udev udev-mount
eowyn / # ls /etc/runlevels/rescue/
local netmount sshd
eowyn / # ls /etc/runlevels/setup/
install netmount
eowyn / # ls /etc/runlevels/shutdown/
killprocs mount-ro savecache
eowyn / #
|
/etc/init.d/install will get its own post in the next days as soon I'll find the time write it up.
In dhcp.conf you need some entries to provide the location of the root filesystem and the kernel for the client machines:
Code: |
option root-path "172.16.20.8:/usr/local/util/netsys2";
next-server 172.16.20.8;
filename "/pxelinux.0";
|
Restart the dhcp server.
You can now fire up tftp and nfs and run your test machine into pxe on bootup.
EDIT: added some minor details
Last edited by AgBr on Thu Dec 12, 2013 3:57 pm; edited 1 time in total |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Tue Dec 10, 2013 12:33 pm Post subject: |
|
|
In which runlevel netsys is running is controled by the DNS name of the machine and the configuration in linuxrc
Code: |
case $(hostname) in
sn07-01) ln -snf rescue /etc/runlevels/default ;;
sn07-??) ln -snf rescue /etc/runlevels/setup ;;
legolas) ln -snf rescue /etc/runlevels/default ;;
dragon) ln -snf rescue /etc/runlevels/terminal ;;
* ) ln -snf rescue /etc/runlevels/default ;;
esac
|
In runlevel 'setup' the init-script 'install' gets started, which in its first part is the doing all steps you would do manually installing a fresh machine running from a live system.
Code: |
# /etc/init.d/install
#!/sbin/runscript
DRIVE=""
# leave STAGE empty to use the newest available
STAGE="stage3-i686-20130827.tar.bz2"
#STARGE=""
DISTDIR="/home/.newinstall" #nfs-directory exported rw, mounted at /home
WINDOWS_IMAGE="$DISTDIR/windows.img"
# to create and install windows images you will need ntfsclone on netsys
ROOTPW='<put the password hash for the root password of the new machine here>'
FTPHOST="ftp://packages/stable/"
WINDOWS_INSTALL=false
LINUX_INSTALL=true
MAX_SIZE=76 # largest disk size for system partitions in Gigabyte
MAX_SIZE=$((MAX_SIZE*1024*1024*2)) # sam as above in blocks of 512 Byte
FREE_PART="" # just declaration for util partition
depend() {
need net netmount
}
getDrive() {
DRIVES_SCSI=$(ls /sys/bus/scsi/drivers/sd/*/block) # new kernels
DRIVE=""
SIZE=0
for cur in $DRIVES_SCSI; do
cursize=$(cat /sys/block/$cur/size)
if [ "$cursize" -gt "$SIZE" ]; then
DRIVE=/dev/$cur
SIZE=$cursize
fi
done
# if disk is larger than MAX_SIZE only use MAX_SIZE
if [ "$SIZE" -gt "$MAX_SIZE" ]; then
SIZE="$MAX_SIZE"
FREE_PART="true"
fi
if [ ! "$DRIVE" ]; then
eerror "No harddisk found"
fi
einfo "Selected $DRIVE for installation"
}
part() {
if [ ! "$DRIVE" ]; then
eerror "No drive selected"
fi
if [ "$WINDOWS_INSTALL" = "true" ]; then
WINDOWS_SIZE=$((5*1024))
else
WINDOWS_SIZE=0
fi
if [ "$LINUX_INSTALL" = "true" ]; then
RAM=$(cat /proc/meminfo |grep MemTotal|grep -Eo "[0-9]+")
SWAP=$((RAM*2))
SWAP_SIZE=$((SWAP/1024))
LINUX_SIZE=$((10*1024))
else
SWAP_SIZE=0
LINUX_SIZE=0
fi
# all space larger than minimum
REST=$((SIZE/2/1024-LINUX_SIZE-WINDOWS_SIZE-SWAP_SIZE))
i=0
echo -n > /tmp/partitiontable
if [ "$WINDOWS_INSTALL" = "true" ]; then
# 7/10 go to Windows if it gets installed at all
FOR_WIN=$((REST*7/10))
WINDOWS_SIZE=$((WINDOWS_SIZE+FOR_WIN))
# subtract space allocated to Windows from REST
REST=$((LINUX_SIZE+REST-WINDOWS_SIZE))
WINDOWS_PART=${DRIVE}$((++i))
echo ",$WINDOWS_SIZE,7" >> /tmp/partitiontable
fi
if [ "$LINUX_INSTALL" = "true" ]; then
echo ",$SWAP_SIZE,S,*" >> /tmp/partitiontable
SWAP_PART=${DRIVE}$((++i))
# Linux-partition gets minimum + REST
LINUX_SIZE=$((LINUX_SIZE+REST))
echo ",$LINUX_SIZE,L" >> /tmp/partitiontable
LINUX_PART=${DRIVE}$((++i))
# every space left goes to the free partition
if [ "$FREE_PART" != "" ];then
echo ",,L" >> /tmp/partitiontable
FREE_PART=${DRIVE}$((++i))
fi
fi
cat /tmp/partitiontable | sfdisk -uM -D $DRIVE
BOOT_FROM=${DRIVE}
if [ "$LINUX_INSTALL" = "true" ]; then
einfo "Making swap"
mkswap ${SWAP_PART}
einfo "Making ext3-filesystem"
mkfs.ext3 ${LINUX_PART}
# if there is a free partition make a filesystem
if [ "$FREE_PART" != "" ]; then
mkfs.ext3 ${FREE_PART}
fi
fi
}
grubdev() {
local STRING=$(echo "$*"|sed 's/\/dev\/.d./hd0,/')
local PART=$(echo "$STRING"|cut -d "," -f2-)
local DEV=$(echo "$STRING"|cut -d "," -f1)
if [ "$PART" ]; then
PART=",$((PART-1))"
fi
echo "($DEV$PART)"
}
mksys() {
if [ ! "$LINUX_PART" ]; then
eerror "No drive selected"
fi
mkdir -p /mnt/gentoo
mount ${LINUX_PART} /mnt/gentoo
test "${SWAP_PART}" && swapon ${SWAP_PART}
einfo "Extracting stage-archiv"
local STAGEDIR STAGEFILE
STAGEFILE="$(basename "$STAGE")"
STAGEDIR="$(dirname "$STAGE")"
if [ "$STAGEDIR" = "." ]; then
STAGEDIR="$DISTDIR"
fi
tar xjpf "$STAGEDIR/$STAGEFILE" -C /mnt/gentoo
sed -i -e "s#^root:\*:#root:$ROOTPW:#" /mnt/gentoo/etc/shadow
mkdir -p /mnt/gentoo/usr/portage
mkdir -p /mnt/gentoo/etc/portage
echo "PORTAGE_BINHOST=$FTPHOST" > /mnt/gentoo/etc/make.conf
echo "PORTDIR=/usr/portage" >> /mnt/gentoo/etc/make.conf
echo "PORTAGE_BINHOST=$FTPHOST" > /mnt/gentoo/etc/portage/make.conf
echo "PORTDIR=/usr/portage" >> /mnt/gentoo/etc/portage/make.conf
|
In the second part of the install three special packages get installed:
portage.tar.gz is an archive of our own /usr/portage which contains info to the profile 'binclient' and local meta ebuilds, which later on pull in all other packages to be installed in this particular machine, based on its DNS name.
gentoo sources is sys-kernel/gentoo-sources enhanced with the binary kernel (a modify script is doing this on the build-server)
update-client is a package which installs shell scripts to do the regular updates on the machine automatically. As our machine to be installed is at this point a fully capable linux envirnonment, which by profile is configured as a 'binclient', running the script update-install in the chroot-environment will pull in all software needed for this particular machine.
Code: |
local TMPFILE="$(mktemp -t portage.tar.bz2.XXXXXX)" &&
wget -q "${FTPHOST}portage.tar.bz2" -O "$TMPFILE" &&
tar xjf "$TMPFILE" -C "/mnt/gentoo/usr/portage" &&
rm -f "$TMPFILE"
cp -L /etc/resolv.conf /mnt/gentoo/etc/
mount -o bind /dev /mnt/gentoo/dev
mount -t proc none /mnt/gentoo/proc
(
echo "${LINUX_PART} / auto noatime 0 1"
echo "${SWAP_PART} none swap sw 0 0"
echo "home:/home /home nfs defaults,nolock 0 0"
echo "shm /dev/shm tmpfs nodev,nosuid,noexec 0 0"
) > /mnt/gentoo/etc/fstab
if [ "$FREE_PART" != "" ]; then
echo "${FREE_PART} /usr/local/util auto noatime 0 1" >> /mnt/gentoo/etc/fstab
mkdir -p /mnt/gentoo/usr/local/util
fi
einfo "Updating environment"
chroot /mnt/gentoo /usr/sbin/env-update
chroot /mnt/gentoo eselect profile set 1
einfo "selecting python 2.7"
chroot /mnt/gentoo emerge -G1 dev-lang/python:2.7
chroot /mnt/gentoo eselect python set 1
einfo "Installing Grub"
chroot /mnt/gentoo emerge -Gn1 grub:0
(
echo "default 0"
if [ "$WINDOWS_SIZE" -gt 0 ]; then
echo "timeout 8"
else
echo "timeout 4"
fi
echo "title=Linux"
echo "root $(grubdev ${LINUX_PART})"
echo "kernel /boot/vmlinuz root=${LINUX_PART}"
if [ "$WINDOWS_SIZE" -gt 0 ]; then
echo "title=Windows"
echo "rootnoverify $(grubdev ${WINDOWS_PART})"
echo "chainloader +1"
fi
)> /mnt/gentoo/boot/grub/grub.conf
chroot /mnt/gentoo grep -E "/dev/[hs]d[a-z][0-9]+ " /proc/mounts > /mnt/gentoo/etc/mtab
chroot /mnt/gentoo grub-install --no-floppy "${BOOT_FROM}"
einfo "Running complete update"
chroot /mnt/gentoo emerge -G1 portage gentoo-sources update-client
chroot /mnt/gentoo /usr/local/sbin/update-install
umount -fl /mnt/gentoo/dev /mnt/gentoo/proc /mnt/gentoo
test "${SWAP_PART}" && swapoff ${SWAP_PART}
if [ "$WINDOWS_SIZE" -gt 0 ]; then
einfo "Installing Windows"
test "$WINDOWS_IMAGE" && ntfsclone -r -O "$WINDOWS_PART" "$WINDOWS_IMAGE" && ntfsresize -f "$WINDOWS_PART"
fi
}
getStage() {
if [ "$STAGE" ]; then return; fi
einfo "Getting stage"
mkdir -p "$DISTDIR"
cd "$DISTDIR"
VERSION="$(basename "$(echo -e "user anonymous 123\ncd /gentoo/releases/x86/autobuilds/current-stage3\npwd"|ftp -np de-mirror.org|tail -n 1| cut -d '"' -f2)")"
einfo "Current Version: $VERSION"
if [ -f "stage3-i686-$VERSION.tar.bz2" ] &&
grep "bz2$" "stage3-i686-$VERSION.tar.bz2.DIGESTS" | md5sum -c &> /dev/null; then
einfo "Found in $DISTDIR"
else
einfo "Not Found - trying to fetch..."
rm -f "$DISTDIR/stage3-i686-"*
wget "ftp://de-mirror.org/gentoo/releases/x86/autobuilds/$VERSION/stage3-i686-$VERSION.tar.bz2"
wget "ftp://de-mirror.org/gentoo/releases/x86/autobuilds/$VERSION/stage3-i686-$VERSION.tar.bz2.DIGESTS"
if ! grep "bz2$" "stage3-i686-$VERSION.tar.bz2.DIGESTS" | md5sum -c &> /dev/null; then
eerror "Checksums don't match"
return 1
else
einfo "Fetched stage3-i686-$VERSION.tar.bz2 to $DISTDIR"
fi
fi
STAGE="stage3-i686-$VERSION.tar.bz2"
}
start() {
ebegin "Installing Gentoo"
getStage &&
getDrive &&
part &&
mksys
eend $? "Failed to install Gentoo"
}
|
You may notice that grub is still used here. I have tried to convert the script to grub2 but grub2 and I don't get friends easy. I gave up by now.
We stay with python 2.7 as updating to 3 on the build-server is a hazzle and python 2.7 works.
If no one stops me, I will procede to present the installation environment. I will show you the update-install script as soon as I'll find some spare time. But eventually I will have to offer a tarball with the essential parts of the build server as the environment is too complex to present it in full length here. Please pm me and give me an eMail address if you are interested.
Last edited by AgBr on Tue Dec 10, 2013 11:47 pm; edited 1 time in total |
|
Back to top |
|
|
Maitreya Guru
Joined: 11 Jan 2006 Posts: 441
|
Posted: Tue Dec 10, 2013 12:59 pm Post subject: |
|
|
Thank you for your effort, don't mistake lack of posts as non-interest.
Please go on! |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Tue Dec 10, 2013 5:50 pm Post subject: |
|
|
Just a note on installing Windows in a second partition. Maintaining windows boxes in large numbers, without proper tools is a PITA. These tools are expensive and at least german schools mostly aren't lying on a bed of roses regarding their financial situation.
If there were a need to (we, until now, do not use Windows) I could maintain Windows boxes by image based updates on the cheap, just by creating an ebuild which uses ntfsclone to copy a fresh image onto its partition. If you know what you are doing, you even may be able to install all the necessary registry hacks to individualize these clones afterwards. You can copy *reg files into the partition and have the files installed on bootup by some autoexec script which looks for *reg files and have regedit suck them in.
If you have the financial ressources, you may be better off with some commercial msi-package based tool though.
Last edited by AgBr on Wed Dec 11, 2013 12:14 pm; edited 1 time in total |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Wed Dec 11, 2013 12:11 pm Post subject: |
|
|
Next step should be to explain, what makes a box an update-client. Essential for this is the contents of /etc/portage and /usr/portage. Note the profile 'binclient'
Code: |
rh12-04 ~ # ls -l /etc/portage/
insgesamt 24
drwxr-xr-x 2 root root 4096 26. Nov 15:15 bin
-rw-r--r-- 1 root root 62 26. Nov 14:43 make.conf
-rw-r--r-- 1 root root 544 27. Aug 09:24 make.conf.catalyst
lrwxrwxrwx 1 root root 36 10. Dez 06:16 make.profile -> ../../usr/portage/profiles/binclient
drwxr-xr-x 2 root root 4096 26. Nov 15:15 postsync.d
drwxr-xr-x 2 root root 4096 26. Nov 14:47 repos.conf
drwxr-xr-x 4 root root 4096 26. Nov 15:08 savedconfig
rh12-04 ~ # ls -l /etc/portage/bin/
insgesamt 4
-rwxr-xr-x 1 root root 190 8. Nov 19:48 post_sync
rh12-04 ~ # ls -l /etc/portage/postsync.d/
insgesamt 4
-rw-r--r-- 1 root root 68 8. Nov 19:48 q-reinitialize
rh12-04 ~ # ls -l /etc/portage/repos.conf/
insgesamt 0
rh12-04 ~ # cat /etc/portage/bin/post_sync
#!/bin/sh
# Copyright 2006-2009 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
for f in /etc/portage/postsync.d/* ; do
[ -x "${f}" ] && "${f}"
done
:
rh12-04 ~ # cat /etc/portage/make.conf
PORTAGE_BINHOST=ftp://packages/stable/
PORTDIR=/usr/portage
rh12-04 ~ # cat /etc/portage/postsync.d/q-reinitialize
#!/bin/sh
[ -x /usr/bin/q ] && /usr/bin/q -r ${PORTAGE_QUIET:+-q}
:
rh12-04 ~ # ls -l /usr/portage/
insgesamt 24
drwxr-xr-x 10 root root 4096 9. Dez 17:55 bbs1-config
drwxr-xr-x 4 root root 4096 9. Dez 17:56 bbs1-meta
drwxr-xr-x 2 root root 4096 6. Dez 2011 eclass
drwxr-xr-x 2 root root 4096 9. Dez 17:55 metadata
drwxr-xr-x 4 root root 4096 10. Dez 06:19 packages
drwxr-xr-x 10 root root 4096 9. Dez 17:56 profiles
|
bbs1-config contains ebuilds which do configuration tasks on the client machine
Code: |
rh12-04 ~ # ls -l /usr/portage/bbs1-config/
insgesamt 32
drwxr-xr-x 3 root root 4096 9. Dez 17:55 fstab
drwxr-xr-x 3 root root 4096 9. Dez 17:55 fstab-notebook
drwxr-xr-x 3 root root 4096 9. Dez 17:55 gasthome-setup
drwxr-xr-x 3 root root 4096 9. Dez 17:55 gast-login
drwxr-xr-x 2 root root 4096 9. Dez 17:55 make-conf
drwxr-xr-x 3 root root 4096 9. Dez 17:55 netsetup
drwxr-xr-x 3 root root 4096 9. Dez 17:55 netsetup-notebook
drwxr-xr-x 3 root root 4096 9. Dez 17:56 xsession
|
For instance, we had to add 'nfsvers=3' to our machines
Code: |
rh12-04 ~ # cat /usr/portage/bbs1-config/fstab/fstab-0-r20130417203500.ebuild
# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
EAPI="2"
SRCDIST=true
DESCRIPTION="add nfs-option nfsvers=3 to fstab "
HOMEPAGE=""
SRC_URI=""
LICENSE=""
SLOT="0"
KEYWORDS="x86"
IUSE=""
DEPEND=""
RDEPEND=""
pkg_postinst()
{
if [[ ! -s /etc/fstab ]] ;then
cat >> /etc/fstab <<END
/dev/sda2 / auto noatime 0 1
/dev/sda1 none swap sw 0 0
home:/home /home nfs defaults,nfsvers=3,nolock 0 0
shm /dev/shm tmpfs nodev,nosuid,noexec 0 0
END
fi
sed -i -e "s#defaults,nolock#defaults,nfsvers=3,nolock#" /etc/fstab
}
|
bbs1-meta/base is what makes the update-client pull in all necessary packages. On the build-server there are some other ebuilds like 'notebook' which are not needed on the clients. (Why 'notebook' made its way on ordinary clients which are not notebooks, is one of the miracles which I am working on actually)
Code: |
rh12-04 ~ # ls -l /usr/portage/bbs1-meta/
insgesamt 8
drwxr-xr-x 2 root root 4096 9. Dez 17:56 base
drwxr-xr-x 2 root root 4096 9. Dez 17:56 notebook
rh12-04 ~ # cat /usr/portage/bbs1-meta/base/base-0-r20131127111300.ebuild
# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
EAPI="2"
inherit hostdeps
DESCRIPTION="Basispaket - Installiert alles was benötigt wird"
HOMEPAGE=""
SRC_URI=""
LICENSE=""
SLOT="0"
KEYWORDS="x86"
IUSE=""
DEPEND=""
RDEPEND="
sys-boot/grub:0
sys-boot/grub:2
app-admin/syslog-ng
app-admin/logrotate
sys-kernel/gentoo-sources
sys-kernel/linux-firmware
net-misc/dhcpcd
net-fs/nfs-utils
sys-apps/netplug
sys-process/vixie-cron
net-misc/ntp
net-dns/bind-tools
sys-apps/smartmontools
sys-apps/less
net-misc/openssh
net-dns/avahi
app-editors/vim
sys-fs/reiserfsprogs
sys-apps/preload
sys-power/acpid
bbs1-config/make-conf
bbs1-config/fstab
bbs1-config/netsetup
mail-mta/postfix
"
KLASSENRAUM="..[0-9]{2}[a-z]?-[0-9]{2}|legolas"
LEHRER="..[0-9]{2}[a-z]?-01|legolas|rh12-04"
TERMINALSERVER="eowyn"
NOTEBOOK="ww13-[0-9]{2}"
BRENNER="legolas|eomer|video"
VIDEOSERVER="video"
if_host_add_rdepend "$KLASSENRAUM" bbs1-meta/klassenraum
if_host_add_rdepend "$LEHRER" bbs1-meta/lehrer
if_host_add_rdepend "$TERMINALSERVER" bbs1-meta/terminal-server
if_host_add_rdepend "$NOTEBOOK" bbs1-meta/notebook
if_host_add_rdepend "$VIDEOSERVER" bbs1-meta/video-server
if_host_add_rdepend "$BRENNER" app-cdr/k3b
|
As you can see, base pulls in some essential packages for all machines, the rest depends on the hostname of that particular machine.
Code: |
rh12-04 ~ # ls -l /usr/portage/eclass/
insgesamt 4
-rw-r--r-- 1 root root 646 6. Dez 2011 hostdeps.eclass
|
The eclass hostdeps is of special interest here. It enables emerge to do installations based on the hostname of the machine
Code: |
h12-04 ~ # cat /usr/portage/eclass/hostdeps.eclass
# Copyright 1999-2010 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
#
# Original Author: Manuel Mommertz
# Purpose: Choose dependencies by hostname
#
SRCDIST=true
_if_host()
{
REGEX="$1"
shift
if [[ $(hostname) =~ ^$REGEX$ ]]
then
echo "$*"
else
echo "build-server? ( $* )"
fi
}
_add_buildserver()
{
has build-server $IUSE || IUSE+=" -build-server "
}
if_host_add_depend()
{
DEPEND+=" $(_if_host "$@") "
_add_buildserver
}
if_host_add_rdepend()
{
RDEPEND+=" $(_if_host "$@") "
_add_buildserver
}
if_host_add_pdepend()
{
PDEPEND+=" $(_if_host "$@") "
_add_buildserver
}
|
layout.conf defines our own repository
Code: |
rh12-04 ~ # ls -l /usr/portage/metadata/
insgesamt 4
-rw-r--r-- 1 root root 58 9. Dez 17:56 layout.conf
rh12-04 ~ # cat /usr/portage/metadata/layout.conf
repo_name=bbs1
masters=gentoo
profile-formats = portage-1
|
The profile binclient is a child of
Code: |
rh12-04 ~ # cat /usr/portage/profiles/binclient/parent
../default/linux/x86/13.0/desktop/kde
|
Next step will be, to show you what the script 'update-install' makes of this, before I can procede to the build-server.
Last edited by AgBr on Thu Dec 12, 2013 8:16 am; edited 1 time in total |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Thu Dec 12, 2013 7:56 am Post subject: |
|
|
This will be the last post relating the update client. In executing the updates there are two scripts involved. The first, update-check, checks whether there are updated packeges on the server. The second pulls and installes them. Update-check is called by update-install
Code: |
rh12-04 ~ # cat /usr/local/bin/update-check
#!/bin/bash
source /etc/portage/make.conf
if [[ ! "$PORTAGE_BINHOST" ]]
then
echo "You have to set PORTAGE_BINHOST in /etc/portage/make.conf"
exit 255
fi
while [[ "${1:0:1}" == "-" ]]
do
case "$1" in
--) break ;;
-q|--quiet) QUIET="true" ;;
-u|--update) INSTALL="true" ;;
-v|--verbose) VERBOSE="true" ;;
*) echo "Parameter "$1" is not supported"
exit 128 ;;
esac
shift
done
show()
{
[[ "$QUIET" == "true" ]] || echo "$@"
}
show_verbose()
{
[[ "$VERBOSE" == "true" ]] && show "$@"
}
LOCAL_TIMESTAMP="${1-/var/db}/.timestamp"
REMOTE_TIMESTAMP="$(wget -q -O - "${PORTAGE_BINHOST%/}/timestamp")" ||
{
show "Could not fetch timestamp from binhost"
exit 255
}
if [[ "$INSTALL" == "true" ]]
then
if echo "$REMOTE_TIMESTAMP" > "$LOCAL_TIMESTAMP"
then
show "Updated timestamp"
else
show "Could not write timestamp"
false
fi
else
if [[ -f "$LOCAL_TIMESTAMP" ]]
then
show_verbose "Installed version: $(date -d @$(<"$LOCAL_TIMESTAMP"))"
if [[ "$REMOTE_TIMESTAMP" -le "$(<"$LOCAL_TIMESTAMP")" ]]
then
show "Up to date"
false
fi
fi && (
show_verbose "Available version: $(date -d @$REMOTE_TIMESTAMP)"
show "Update available"
)
fi
|
Update-install is run by cron every 10 minutes. If we have to alter the configuration of our clients, I make the changes on the build-server, run packages-update there and within 10 minutes after packages-update has finished successfully, the updates get installed on the clients. The variable PORTAGE_BINHOST usually is 'ftp://packages/stable'. All new updates go into unstable. Stable is just a link to unstable. I have one client configured to unstable to test the update. Larger updates, for instance world updates, go then to the link testing with some more clients configures to this target. These will run, if needed, for some days or weeks in every day use. If all goes well I create the link to stable, to update all other machines. Before I make world updates, I copy the old successful unstable to stable.backup, to make sure that I have a working copy, just in case my world update fails. In this case I can at least reinstall broken clients. Unfortuantely during such times I am not able to make smaller changes just in configuration, as unstable only gets updated if the packages-update run finished successfully. Up till now there is no way to roll back to an earlier state after a world update. To find a solution for this would be a large improvement of the system as especially changes in the way portage works occasionally breaks the system and I need some time to figure out, what went wrong.
Code: |
rh12-04 ~ # cat /usr/local/sbin/update-install
#!/bin/bash
# The developers at one point removed PORTDIR from make.globals. Further down an instruction relies on the existence
# in an 'rm -rf' - instruction. If PORTDIR is not set the instruction reduces to 'rm -rf /'. /home is nfs-mounted. Guess how I found out.
: ${PORTDIR:=/usr/portage}
source /usr/share/portage/config/make.globals
source /etc/portage/make.conf
if [[ ! "$PORTAGE_BINHOST" ]]
then
echo "You have to set PORTAGE_BINHOST in /etc/make.conf"
exit 255
fi
PORTAGE_PACKAGE="${PORTAGE_BINHOST%/}/portage.tar.bz2"
export CONFIG_PROTECT="-*"
export CLEAN_DELAY="0"
export EMERGE_WARNING_DELAY="0"
export EPAUSE_IGNORE=true
EMERGE_OPTS=()
EMERGE_DEFAULT_OPTS=( -Dugq --binpkg-respect-use=n --rebuilt-binaries y )
while [[ "${1:0:1}" == "-" ]]
do
case "$1" in
--) shift
EMERGE_OPTS=( "$@" )
break ;;
-q|--quiet) QUIET="true" ;;
-p|--pretend) PRETEND="true" ;;
-f|--fetchonly) FETCHONLY="true" ;;
-c|--check) CHECK="true" ;;
*) echo "Parameter "$1" is not supported"
exit 128 ;;
esac
shift
done
UPDATE_CHECK="/usr/local/bin/update-check"
GCC_CONFIG="/usr/bin/gcc-config"
update-needed()
{
$UPDATE_CHECK --quiet "$@"
case $? in
0) return 0 ;;
1) return 1 ;;
255) show "Could not reach server"
exit 255 ;;
*) show "Unkown Error"
exit 254 ;;
esac
}
show()
{
[[ "$QUIET" == "true" ]] || echo "$@"
}
update-portage()
{
if update-needed "$PORTDIR"
then
local TMPFILE="$(mktemp -t portage.tar.bz2.XXXXXX)" &&
show "Fetching portage..." &&
wget -q "$PORTAGE_PACKAGE" -O "$TMPFILE" &&
show "Installing portage..." &&
rm -rf "${PORTDIR%/}/"* &&
tar xjf "$TMPFILE" -C "$PORTDIR" &&
rm -f "$TMPFILE" &&
update-needed --update "$PORTDIR" &&
eselect profile set 1 &&
show "Portage updated successfully" &&
show
else
true
fi
}
update-system()
{
if update-needed
then
$GCC_CONFIG 1
show "Searching for packages..."
if [[ "$PRETEND" == "true" ]]
then
emerge "${EMERGE_DEFAULT_OPTS[@]}" "${EMERGE_OPTS[@]}" -p world
elif [[ "$FETCHONLY" == "true" ]]
then
emerge "${EMERGE_DEFAULT_OPTS[@]}" "${EMERGE_OPTS[@]}" -f world
else
emerge "${EMERGE_DEFAULT_OPTS[@]}" "${EMERGE_OPTS[@]}" world &&
#
# gcc-config ausführen um sicherzustellen, dass ein gültiges gcc-Profil gewählt ist
#
show "Searching for old packages..." &&
emerge -q --depclean 2>/dev/null &&
(
$GCC_CONFIG 1
eselect java-vm set system 1
eselect java-nsplugin set 1
eselect python set 1
newaliases
true
) &&
update-needed --update &&
show &&
show "System updated successfully"
fi
else
show "System is up to date"
fi
}
if flock --nonblock 200
then
if [[ "$CHECK" == "true" ]]
then
show "Update is not running"
exit 1
fi
update-portage &&
update-system
else
show "Update is already running"
fi 200<$0
|
|
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Fri Dec 13, 2013 5:28 pm Post subject: |
|
|
The following post will be dedicated to the build-server. Other than the netsys environment I described further up, I did never set it up myself and I do not have the time to make a test setup now. But I think I am able to describe it sufficiently exact to reproduce it. My environment carries a heritage of configuration ebuilds, package.use files and modify-scripts I would have to sort out first. So it will be better to present the essentials here and build up the system step by step nearly from scratch. I will copy and paste the basically needed scripts here and outline how it works though.
First you need the build-server. It is a chroot environment set up just like the netsys environment described further up. Just mkdir build-server somewhere where you have some disk space put in there the stage3, bind the necessary filesystems to the appropriate directories and step in.
To my understanding, with the stage3 you have all necessary developement tools you need. app-portage/eix would be handy. There may be some other things but you will not install them directly. And you do not want anything in your world file. My world file only contains things needed for the developement environment or the build-server itself and I am not sure if even these are really needed there. My world file contains:
app-portage/eix
app-portage/gentoolkit
app-portage/layman
dev-util/lafilefixer
net-ftp/vsftpd
sys-apps/less
sys-devel/automake
sys-apps/less may not be needed here as it will get installed as part of our bbs1-meta/base.ebuild anyway later.
vsftp is needed as the ftp-server providing the packages for the client is running inside the chroot-envirnment. Its home is /home/ftp. It gets started outside from the envirnment by a startup-script:
Code: |
faramir ~ # cat /etc/conf.d/local.start
# /etc/conf.d/local.start
# This is a good place to load any misc programs
# on startup (use &>/dev/null to hide output)
chroot /usr/local/build-server/ vsftpd
faramir ~ # cat /etc/fstab
/dev/sda2 / auto noatime 0 1
/dev/sda1 none swap sw,pri=0 0 0
shm /dev/shm tmpfs nodev,nosuid,noexec 0 0
/dev /usr/local/build-server/dev none bind 0 0
shm /usr/local/build-server/dev/shm tmpfs nodev,nosuid,noexec 0 0
proc /usr/local/build-server/proc proc defaults 0 0
|
The build envirnoment consists of three parts which will be explained bit by bit:
- make.conf
- /etc/portage, especially the scripts in portage.modify
- /usr/local/portage with our local ebuilds
The idea is to never touch the client-machines manually. All configuration has to be done on the build-server. On the client-machines all that is done by emerge is unpacking the tar files as long as these would not try to overwrite files belonging to other packages. So configuration files belonging to regular ebuilds would have to be modified before they get packed. All files belonging to openrc emediately come to mind, as these determain most of the behavior of the client.
Other configuration files do not belong to any regular ebuild. /etc/fstab for example is created manually during installation. These configuration files will have to be created or modified by local ebuilds which lie in /usr/local/portage.
My make.conf looks like this:
Code: |
faramir / # cat /etc/portage/make.conf
CFLAGS="-O2 -march=i686 -pipe -fomit-frame-pointer -ggdb"
CXXFLAGS="$CFLAGS"
# WARNING: Changing your CHOST is not something that should be done lightly.
# Please consult http://www.gentoo.org/doc/en/change-chost.xml before changing.
CHOST="i686-pc-linux-gnu"
MAKEOPTS="-j5"
FEATURES="splitdebug"
LINGUAS="de"
INPUT_DEVICES="evdev synaptics"
VIDEO_CARDS="intel radeon vesa"
USE="-gnome -gtk -eds -semantic-desktop nfs icu zeroconf"
PORTDIR="/usr/portage"
LOCAL_OVERLAY="/usr/local/portage"
source /usr/local/layman/make.conf
GRUB_PLATFORMS="efi pc"
|
As far as I can tell there isn't anything fancy in there. Layman is only needed for the overlay nx, as we installed an nx-server on one of our terminal-servers. The overlay is set up in the usual way, so I don't have anything to report here.
My next post will be dedicated to /etc/portage but I need to clear some details for myself first before I can present it to you. |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54216 Location: 56N 3W
|
Posted: Fri Dec 13, 2013 6:43 pm Post subject: |
|
|
AgBr,
A page at wiki.gentoo.org will probably get more interest than the forums.
Its also much easier to collaborate there.
An email to pr AT gentoo.org would be good too. That may lead to something in the Gentoo Monthly Newsletter or even an interview/podcast.
I'm not a member of pr.
The forums are read by a small subset of developers, so more publicity will be better. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Fri Dec 13, 2013 7:13 pm Post subject: |
|
|
NeddySeagoon wrote: | AgBr,
A page at wiki.gentoo.org will probably get more interest than the forums.
Its also much easier to collaborate there.
An email to pr AT gentoo.org would be good too. That may lead to something in the Gentoo Monthly Newsletter or even an interview/podcast.
I'm not a member of pr.
The forums are read by a small subset of developers, so more publicity will be better. |
Thank you for the hint. It was my impression that the wiki is a place for documentation of a more definite state and I never thought of the wiki to get more traffic than the forums. My initial idea was to find one/some people interested in reproducing the installation to share the files with. I will think about transferring my little 'essay'. But may be I should finish it here first, just to make it complete. Who is pr by the way? |
|
Back to top |
|
|
NeddySeagoon Administrator
Joined: 05 Jul 2003 Posts: 54216 Location: 56N 3W
|
Posted: Fri Dec 13, 2013 7:24 pm Post subject: |
|
|
AgBr,
pr is the Gentoo Public Relations team.
The wiki is open to all, provided they register. There are many skilled Gentoo people who are not Gentoo developers.
That does not matter. For your protect to live, you only need people with an interest and the skills to contribute. _________________ Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
Back to top |
|
|
AgBr Apprentice
Joined: 06 Nov 2010 Posts: 195
|
Posted: Thu Feb 13, 2014 6:59 am Post subject: |
|
|
Just in case anyone is wondering, I am still on it. Unfortunately right now my time is occupied by some other things. If meanwhile anyone should be interested in looking into our system on his own, just pm me. I have a tarball with all the essential parts of the system, which just can be unpacked into the chroot environment of the build-server outlined above. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|