If you're interested in full details of Xen please visit the Xen website:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/
Briefly, however, Xen is a virtualization technique which allows several (Linux and BSD at this time) Operating Systems to run on a single piece of hardware. Xen is actually a para virtualization technique (read the real documentation for the full description), but is most similar to VMWare ESX (the full Enterprise Edition) in operation. Xen is not an emulation technique like Wine which uses a software layer to allow (in the case of Wine), MS Windows DLL's to "talk" with Linux programs. Xen is a specialized version of the Linux kernel which "grabs" all the hardware from the system during bootup and then allocates abstracted access to that hardware for the virtual Operating Systems (domains). (No flame wares meant to be started here, I simply want to categorize where Xen fits in the grand scheme).
I highly recommend the FAQ and Architecture papers for an overview and the Users' Guide at
http://www.cl.cam.ac.uk/Research/SRG/ne ... /user.html
for general installation instructions (non-Gentoo specific) and the full list of domain parameters and command/gui domain control
features. It's fascinating reading, but there is a lot to learn and I've tried to distill the essence of Xen and Gentoo-ify it here.
Happy Xen-ing and I hope this helps at least one person.
----
Prerequisites
1. Gentoo on x86 (or x86_64 installed 32 bit). The July '05 Xen release will be available for x86_64 full 64 bit and will clean and thin the Xen code. My understanding is that the current Xen code is off a 2.4 kernel or very early 2.6 kernel and that v3.0 will update much of
the code and thin it out.
2. (WARNING) glibc: Xen currently doesn't play well with NPTL (tls emulation is very slow), so use either.
USE="-nptl"
or
USE="nptl -nptlonly" and mv /lib/tls /lib/tls.disable
Note: recompiling glibc w/ different threading options may break other programs. I had to recompile openoffice, nautilus, etc... because the new glibc didn't have the proper header options. This may not have been caused by the nptl vs non-nptl, but by other things (ref the stage1 from stage3 tarball howto on these forums).
3. (WARNING) SATA: Several people reporting to xen-devel have been unable to get VIA and nVidia SATA to run on Xen. I have a VIA chipset on an MSI mb that I worked on for about 3 hours to with no success. The problem appears as a kernel lockup after SATA detection: drives are detected, but getting and handing off an interrupt fails (IRQ11 or 17 or something goes to IRQ0) and the kernel hangs. If anyone has success with Xen and SATA, particularly with these chipsets please write in.
4. For the first "test" domain download ttylinux from:
http://sourceforge.net/project/showfile ... _id=280428
or
http://www.minimalinux.org/ttylinux/showpage.php?pid=4
Xen Ebuilds in Portage
As root:
Download the necessary files
xen-2.0.6.ebuild
xend-conf
xend-init
xendomains-conf
xendomains-init
from
http://bugs.gentoo.org/show_bug.cgi?id=70161
There may be updates to these files or patches that need to go in the files directory, so move those into the proper locations as well. I'd like to be more helpful on exactly the files, but Xen is not in Portage yet. The above has worked for me as of 06/02/05.
Next, create the Xen ebuild in your portage overlay (look at make.conf for the portage overlay):
Code: Select all
BASEDIR="/usr/local/portage/app-emulation/xen"
FILEDIR="${BASEDIR}/files/2.0.6"
mkdir -p ${FILEDIR}
cp xen-2.0.6.ebuild ${BASEDIR}
cp xend-conf ${FILEDIR}
cp xend-init ${FILEDIR}
cp xendomains-conf ${FILEDIR}
cp xendomains-init ${FILEDIR}
find ${BASEDIR} -type f -exec chmod 644 {} \;
find ${BASEDIR} -type d -exec chmod 755 {} \;
ebuild ${BASEDIR}/xen-2.0.6.ebuild digest
Also put the xen0-sources ebuild in your portage overlay:
Code: Select all
BASEDIR="/usr/local/portage/sys-kernel/xen0-sources"
mkdir -p ${BASEDIR}
cp xen0-sources-2.6.11.10.ebuild ${BASEDIR}
find ${BASEDIR} -type f -exec chmod 644 {} \;
find ${BASEDIR} -type d -exec chmod 755 {} \;
ebuild ${BASEDIR}/xen0-sources-2.6.11.10.ebuild digest
emerge -av xen xen0-sources
----
Xen Kernel(s)
There are two types of Linux kernels in Xen - privileged and unprivileged.
The privileged kernel is used to boot the system initially into Xen and contains all the hardware drivers and modules of a typical kernel. After booting into the privileged kernel (Xen0) this running "domain" is called Domain-0.
Unprivileged kernels, on the other hand have no kernel modules and obtain most/all of their hardware information from Xen and the Xen0 domain. (Note that this isn't strictly true and I'm still figuring out the details).
To make matters even more confusing a Xen0 kernel may be used as a XenU kernel. This is not how I explain the installation. I copy the entire xen0 sources to a xenU directory and keep my xen0 and xenU builds entirely separate. Choose for yourself, but I had extra disk space and I wanted to keep both xen0 and xenU kernels for experimentation.
To separate the xen0 and xenU kernels:
Code: Select all
cd /usr/src
cp -Rp linux-2.6.11.10-xen0/ linux-2.6.11.10-xenUCode: Select all
cd linux-2.6.11.10-xen0
make mrproper
cp ../2.6.11.config .config
make oldconfig
make menuconfig
stated in the Xen User's Guide section 2.3.3.
Note: oldconfig doesn't work great sometimes and I have (over 4 installations) found that I have to manually say y/n/m over
the entire kernel. Still, it may save time.
Make sure the following options are used at a minimum for the Xen0 kernel
Code: Select all
*
* XEN
*
Privileged Guest (domain 0) (XEN_PRIVILEGED_GUEST) [N/y/?] (NEW) y
Physical device access (XEN_PHYSDEV_ACCESS) [Y/?] (NEW) y
Block-device backend driver (XEN_BLKDEV_BACKEND) [Y/n/?] (NEW)
Network-device backend driver (XEN_NETDEV_BACKEND) [Y/n/?] (NEW)
Block-device frontend driver (XEN_BLKDEV_FRONTEND) [Y/n/?] (NEW)
Network-device frontend driver (XEN_NETDEV_FRONTEND) [Y/n/?] (NEW)
Pipelined transmitter (DANGEROUS) (XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER) [N/y/?] (NEW)
Scrub memory before freeing it to Xen (XEN_SCRUB_PAGES) [Y/n/?] (NEW)Code: Select all
CONFIG_BLK_DEV_IDEDISK y
CONFIG_BLK_DEV_IDEGENERIC y
CONFIG_PARTITION_ADVANCED y
Does anyone have any RAID and LVM experiences?"VFS: Cannot open root device "306" or hda6
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on hda6"
Finally, compile in Ethernet Bridging as a module as Xen requires it for all networking.
Code: Select all
Device Drivers -> Networking support -> Networking options -> 802.1d Ethernet Bridgning as module
Compile and install the xen0 kernel:
Code: Select all
make && make modules && make modules_install
mount /boot
cp vmlinuz /boot/vmlinuz-2.6.11.10-xen0
cp System.map /boot/System.map-2.6.11.10-xen0
ln -sf /boot/System.map-2.6.11.10-xen0 /boot/System.map
To install kernel modules external to the kernel (nvidia, madwifi, etc...)title Xen 2.0 / XenLinux 2.6.11 primary domain
root (hd0,0)
kernel (hd0,0)/xen.gz dom0_mem=262144
module (hd0,0)/vmlinuz-2.6.11.10-xen0 root=/dev/hda3
Code: Select all
cd /usr/src
rm linux
ln -s linux-2.6.11.10-xen0 linux
Code: Select all
rm linux
ln -s linux-2.6.11-gentoo-r9 linux
XenU Kernel
XenU kernels are reusable and not used during the physical system boot process. They are used by Xen when launching a new domain from Domain-0. Consequently there is no need to put the xenU kernels in /boot. After little deliberation and much knee jerk reaction I put my xenU kernel in /usr/share (LSB compliant?).
Code: Select all
cd /usr/src/linux-linux-2.6.11.10-xenU
make mrproper
make xenU_defconfig
make
mkdir -p /usr/share/xen/xenU-kernels
cp vmlinuz /usr/share/xen/xenU-kernels/vmlinuz-2.6.11.10-xenU
Test Domain Preparation
move your ttylinux domain download (the test domain) to a known directory (/opt)
vi /etc/xen/ttylinux and add the following lines:
Code: Select all
kernel = "/usr/share/xen/xenU-kernels/vmlinuz-2.6.11.10-xenU"
memory = 64
name = "ttylinux"
nics = 1
ip = "192.168.0.11"
disk = ['file:/opt/ttylinux-xen,sda1,w']
root = "/dev/sda1 ro"
- kernel - is the xenU kernel
memory - is the memory Xen allocates to the VM in MB
name - is the VM name (the domain name that shows up under xm list)
nics - number of virtual network cards in domain
ip - for static networking (best if it's on the same network as you usually
use)
disc - the ttylinux install is on a file which will be presented to the VM
operating system as a writable sda1
root - the root for ttylinux is on /dev/sda1 and is read only
This isn't strictly necessary, but made things easier for me. I had some hardware problems with some pieces of the default runlevel (pcmcia card on my
laptop), and I wanted to experiment with Xen without X.
As root:
Code: Select all
mkdir /etc/runlevels/xen
for script in `ls /etc/runlevels/default/`
do
rc-update add ${script} xen
done
rc-update del xdm xen
rc-update del pcmcia xen
/etc/inittab:
Code: Select all
l4:4:wait:/sbin/rc xenCode: Select all
module (hd0,0)/vmlinuz-2.6.11.10-xen0 root=/dev/hda3 4Reboot into Xen0 Domain
Log in as root:
(Assuming vc/1)
If domains should crash or fail to launch in the first place, check these Xen logs for debugging:
/var/log/xend-debug.log
/var/log/xend.log
See what domains are running:
Code: Select all
xm listCreate New Test VM
Ctrl+Alt+F2 to get to vc/2
Log in as root:
Code: Select all
modprobe loop
xm create /etc/xen/ttylinux -cpassword : root
ls (and have fun in a very small linux)
Switch back to Domain-0 (Ctrl+Alt+F1)
xm list
you should see Domain0 and ttylinux domain
Simple Xen Commands
- xm list
xm create </path/to/domain/file>
xm shutdown <domain_name>
xm destroy <domain_name>
xm console <domain_name>
You should also have networking as Xen (tries) to automatically bring up a bridge from Domain-0 to other domains.
This worked for me in general. ttylinux doesn't have ssh, but does have a http daemon. Try hitting port 80.
TODO
Again, please forgive me for any inconsistencies, incompleteness etc... I figured having a start that people could try and
comment on was better than nothing. The fine people working in http://bugs.gentoo.org/show_bug.cgi?id=70161 have done a great job and it's time to get users on Xen and Xen in Portage.
As the I hope you find this helpful and welcome any feedback.
Here are a few things I will be working on:
1. Enumerating hardware problems and solutions (SATA, etc...)
2. Creating and using other domain images (I have worked with Red Hat ES 4 to good effect).
3. Troubleshooting domains and domain bootup.
4. Figuring out the true differences between Xen0 an XenU and how to get limited H/W drivers in XenU domains (iptables for example).
) ones together, but the short-time-memoryless idiot I am, I forgot to copy them on my iBook, so I could post them, when I find someplace with internet access