Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Xen Para virtualization HOWTO
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
trad511
n00b
n00b


Joined: 29 Nov 2003
Posts: 27

PostPosted: Thu Jun 02, 2005 11:11 pm    Post subject: Xen Para virtualization HOWTO Reply with quote

This HOWTO assumes that you have an inkling of what Xen does and just want to get it up and running to experiment. Or, you may be like me and need to get an entire LVS Linux cluster up and running for work, but don't have 6+ separate machines.

If you're interested in full details of Xen please visit the Xen website:

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/

Briefly, however, Xen is a virtualization technique which allows several (Linux and BSD at this time) Operating Systems to run on a single piece of hardware. Xen is actually a para virtualization technique (read the real documentation for the full description), but is most similar to VMWare ESX (the full Enterprise Edition) in operation. Xen is not an emulation technique like Wine which uses a software layer to allow (in the case of Wine), MS Windows DLL's to "talk" with Linux programs. Xen is a specialized version of the Linux kernel which "grabs" all the hardware from the system during bootup and then allocates abstracted access to that hardware for the virtual Operating Systems (domains). (No flame wares meant to be started here, I simply want to categorize where Xen fits in the grand scheme).

I highly recommend the FAQ and Architecture papers for an overview and the Users' Guide at

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html

for general installation instructions (non-Gentoo specific) and the full list of domain parameters and command/gui domain control
features. It's fascinating reading, but there is a lot to learn and I've tried to distill the essence of Xen and Gentoo-ify it here.

Happy Xen-ing and I hope this helps at least one person.

----

Prerequisites

1. Gentoo on x86 (or x86_64 installed 32 bit). The July '05 Xen release will be available for x86_64 full 64 bit and will clean and thin the Xen code. My understanding is that the current Xen code is off a 2.4 kernel or very early 2.6 kernel and that v3.0 will update much of
the code and thin it out.

2. (WARNING) glibc: Xen currently doesn't play well with NPTL (tls emulation is very slow), so use either.

USE="-nptl"
or
USE="nptl -nptlonly" and mv /lib/tls /lib/tls.disable

Note: recompiling glibc w/ different threading options may break other programs. I had to recompile openoffice, nautilus, etc... because the new glibc didn't have the proper header options. This may not have been caused by the nptl vs non-nptl, but by other things (ref the stage1 from stage3 tarball howto on these forums).

3. (WARNING) SATA: Several people reporting to xen-devel have been unable to get VIA and nVidia SATA to run on Xen. I have a VIA chipset on an MSI mb that I worked on for about 3 hours to with no success. The problem appears as a kernel lockup after SATA detection: drives are detected, but getting and handing off an interrupt fails (IRQ11 or 17 or something goes to IRQ0) and the kernel hangs. If anyone has success with Xen and SATA, particularly with these chipsets please write in.

4. For the first "test" domain download ttylinux from:

http://sourceforge.net/project/showfiles.php?group_id=86024&package_id=134747&release_id=280428

or

http://www.minimalinux.org/ttylinux/showpage.php?pid=4


Xen Ebuilds in Portage

As root:

Download the necessary files

xen-2.0.6.ebuild
xend-conf
xend-init
xendomains-conf
xendomains-init

from

https://bugs.gentoo.org/show_bug.cgi?id=70161

There may be updates to these files or patches that need to go in the files directory, so move those into the proper locations as well. I'd like to be more helpful on exactly the files, but Xen is not in Portage yet. The above has worked for me as of 06/02/05.

Next, create the Xen ebuild in your portage overlay (look at make.conf for the portage overlay):

Code:
BASEDIR="/usr/local/portage/app-emulation/xen"
FILEDIR="${BASEDIR}/files/2.0.6"

mkdir -p ${FILEDIR}

cp xen-2.0.6.ebuild ${BASEDIR}
cp xend-conf ${FILEDIR}
cp xend-init ${FILEDIR}
cp xendomains-conf ${FILEDIR}
cp xendomains-init ${FILEDIR}

find ${BASEDIR} -type f -exec chmod 644 {} \;
find ${BASEDIR} -type d -exec chmod 755 {} \;

ebuild ${BASEDIR}/xen-2.0.6.ebuild digest



Also put the xen0-sources ebuild in your portage overlay:

Code:
BASEDIR="/usr/local/portage/sys-kernel/xen0-sources"

mkdir -p ${BASEDIR}

cp xen0-sources-2.6.11.10.ebuild ${BASEDIR}

find ${BASEDIR} -type f -exec chmod 644 {} \;
find ${BASEDIR} -type d -exec chmod 755 {} \;

ebuild ${BASEDIR}/xen0-sources-2.6.11.10.ebuild digest

emerge -av xen xen0-sources


if you have a system w/o X: USE="-gtk -gtk2" b/c otherwise dependency for Python twisted will bring in X.

----

Xen Kernel(s)

There are two types of Linux kernels in Xen - privileged and unprivileged.

The privileged kernel is used to boot the system initially into Xen and contains all the hardware drivers and modules of a typical kernel. After booting into the privileged kernel (Xen0) this running "domain" is called Domain-0.

Unprivileged kernels, on the other hand have no kernel modules and obtain most/all of their hardware information from Xen and the Xen0 domain. (Note that this isn't strictly true and I'm still figuring out the details).

To make matters even more confusing a Xen0 kernel may be used as a XenU kernel. This is not how I explain the installation. I copy the entire xen0 sources to a xenU directory and keep my xen0 and xenU builds entirely separate. Choose for yourself, but I had extra disk space and I wanted to keep both xen0 and xenU kernels for experimentation.

To separate the xen0 and xenU kernels:

Code:
cd /usr/src

cp -Rp linux-2.6.11.10-xen0/ linux-2.6.11.10-xenU


Xen0 Kernel:

Code:
cd linux-2.6.11.10-xen0

make mrproper
cp ../2.6.11.config .config
make oldconfig
make menuconfig


where I usually store my current config (in this case 2.6.11.conf) in /usr/src. I'm using an existing kernel configuration file as
stated in the Xen User's Guide section 2.3.3.

Note: oldconfig doesn't work great sometimes and I have (over 4 installations) found that I have to manually say y/n/m over
the entire kernel. Still, it may save time.

Make sure the following options are used at a minimum for the Xen0 kernel

Code:
*
* XEN
*
Privileged Guest (domain 0) (XEN_PRIVILEGED_GUEST) [N/y/?] (NEW) y
Physical device access (XEN_PHYSDEV_ACCESS) [Y/?] (NEW) y
  Block-device backend driver (XEN_BLKDEV_BACKEND) [Y/n/?] (NEW)
  Network-device backend driver (XEN_NETDEV_BACKEND) [Y/n/?] (NEW)
Block-device frontend driver (XEN_BLKDEV_FRONTEND) [Y/n/?] (NEW)
Network-device frontend driver (XEN_NETDEV_FRONTEND) [Y/n/?] (NEW)
  Pipelined transmitter (DANGEROUS) (XEN_NETDEV_FRONTEND_PIPELINED_TRANSMITTER) [N/y/?] (NEW)
Scrub memory before freeing it to Xen (XEN_SCRUB_PAGES) [Y/n/?] (NEW)


Also, make sure to set

Code:
CONFIG_BLK_DEV_IDEDISK y
CONFIG_BLK_DEV_IDEGENERIC y
CONFIG_PARTITION_ADVANCED y


to prevent a non-bootable system failing with

Quote:
"VFS: Cannot open root device "306" or hda6
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on hda6"


Does anyone have any RAID and LVM experiences?

Finally, compile in Ethernet Bridging as a module as Xen requires it for all networking.

Code:
Device Drivers -> Networking support -> Networking options -> 802.1d Ethernet Bridgning as module



Compile and install the xen0 kernel:

Code:
make && make modules && make modules_install

mount /boot

cp vmlinuz /boot/vmlinuz-2.6.11.10-xen0
cp System.map /boot/System.map-2.6.11.10-xen0
ln -sf /boot/System.map-2.6.11.10-xen0 /boot/System.map


vi /boot/grub/grub.conf and add the following entry:

Quote:
title Xen 2.0 / XenLinux 2.6.11 primary domain
root (hd0,0)
kernel (hd0,0)/xen.gz dom0_mem=262144
module (hd0,0)/vmlinuz-2.6.11.10-xen0 root=/dev/hda3


To install kernel modules external to the kernel (nvidia, madwifi, etc...)

Code:
cd /usr/src
rm linux
ln -s linux-2.6.11.10-xen0 linux


Code:
rm linux
ln -s linux-2.6.11-gentoo-r9 linux


This allows the drivers to compile and install with the Xen kernel.

XenU Kernel

XenU kernels are reusable and not used during the physical system boot process. They are used by Xen when launching a new domain from Domain-0. Consequently there is no need to put the xenU kernels in /boot. After little deliberation and much knee jerk reaction I put my xenU kernel in /usr/share (LSB compliant?).

Code:
cd /usr/src/linux-linux-2.6.11.10-xenU
make mrproper
make xenU_defconfig
make

mkdir -p /usr/share/xen/xenU-kernels
cp vmlinuz /usr/share/xen/xenU-kernels/vmlinuz-2.6.11.10-xenU




Test Domain Preparation

move your ttylinux domain download (the test domain) to a known directory (/opt)

vi /etc/xen/ttylinux and add the following lines:

Code:

kernel = "/usr/share/xen/xenU-kernels/vmlinuz-2.6.11.10-xenU"
memory = 64
name = "ttylinux"
nics = 1
ip = "192.168.0.11"
disk = ['file:/opt/ttylinux-xen,sda1,w']
root = "/dev/sda1 ro"


where

    kernel - is the xenU kernel
    memory - is the memory Xen allocates to the VM in MB
    name - is the VM name (the domain name that shows up under xm list)
    nics - number of virtual network cards in domain
    ip - for static networking (best if it's on the same network as you usually
    use)
    disc - the ttylinux install is on a file which will be presented to the VM
    operating system as a writable sda1
    root - the root for ttylinux is on /dev/sda1 and is read only


Create Xen Runlevel

This isn't strictly necessary, but made things easier for me. I had some hardware problems with some pieces of the default runlevel (pcmcia card on my
laptop), and I wanted to experiment with Xen without X.

As root:

Code:
mkdir /etc/runlevels/xen

for script in `ls /etc/runlevels/default/`
do
    rc-update add ${script} xen
done

rc-update del xdm xen
rc-update del pcmcia xen


To use this runlevel I edited /etc/inittab and grub.conf and changed the following:

/etc/inittab:
Code:
l4:4:wait:/sbin/rc xen


/boot/grub/grub.conf:
Code:
module (hd0,0)/vmlinuz-2.6.11.10-xen0 root=/dev/hda3 4




Reboot into Xen0 Domain

Log in as root:

(Assuming vc/1)



If domains should crash or fail to launch in the first place, check these Xen logs for debugging:

/var/log/xend-debug.log
/var/log/xend.log

See what domains are running:

Code:
xm list



Create New Test VM

Ctrl+Alt+F2 to get to vc/2

Log in as root:

Code:
modprobe loop

xm create /etc/xen/ttylinux -c


username : root
password : root

ls (and have fun in a very small linux)


Switch back to Domain-0 (Ctrl+Alt+F1)

xm list

you should see Domain0 and ttylinux domain

Simple Xen Commands

    xm list
    xm create </path/to/domain/file>
    xm shutdown <domain_name>
    xm destroy <domain_name>
    xm console <domain_name>


You will have to destroy the ttylinux domain from Domain-0

You should also have networking as Xen (tries) to automatically bring up a bridge from Domain-0 to other domains.
This worked for me in general. ttylinux doesn't have ssh, but does have a http daemon. Try hitting port 80.

TODO

Again, please forgive me for any inconsistencies, incompleteness etc... I figured having a start that people could try and
comment on was better than nothing. The fine people working in https://bugs.gentoo.org/show_bug.cgi?id=70161 have done a great job and it's time to get users on Xen and Xen in Portage.

As the I hope you find this helpful and welcome any feedback.

Here are a few things I will be working on:

1. Enumerating hardware problems and solutions (SATA, etc...)
2. Creating and using other domain images (I have worked with Red Hat ES 4 to good effect).
3. Troubleshooting domains and domain bootup.
4. Figuring out the true differences between Xen0 an XenU and how to get limited H/W drivers in XenU domains (iptables for example).
_________________
Success is the ability to go from one failure to another with no loss of enthusiasm. - Winston Churchill


Last edited by trad511 on Wed Jun 22, 2005 9:59 pm; edited 3 times in total
Back to top
View user's profile Send private message
cprior
n00b
n00b


Joined: 03 Jun 2004
Posts: 57
Location: #gentoo ;)

PostPosted: Fri Jun 03, 2005 9:16 am    Post subject: Reply with quote

for the xen0 kernel, make sure to set
CONFIG_BLK_DEV_IDEDISK y
CONFIG_BLK_DEV_IDEGENERIC y
CONFIG_PARTITION_ADVANCED y
to prevent a non-bootable system failinf with
"VFS: Cannot open root device "306" or hda6
Please append a correct "root=" boot option
Kernel panic: VFS: Unable to mount root fs on hda6"
(description taken from http://kerneltrap.org/node/970 as I cannot reproduce my exact messages---any more *g*)
Back to top
View user's profile Send private message
Shienarier
Apprentice
Apprentice


Joined: 16 Jun 2003
Posts: 278

PostPosted: Mon Jun 13, 2005 12:02 pm    Post subject: Reply with quote

Quote:
Where am i supposed to get xen.gz?

I got it. I guess that you have to have /boot mounted in order to get it in /boot/xen.gz
Back to top
View user's profile Send private message
trad511
n00b
n00b


Joined: 29 Nov 2003
Posts: 27

PostPosted: Thu Jun 16, 2005 2:33 am    Post subject: Reply with quote

Shienarier wrote:
Quote:
Where am i supposed to get xen.gz?

I got it. I guess that you have to have /boot mounted in order to get it in /boot/xen.gz


Sorry if I misunderstand, but are you talking about:

a. the initial copying of xen.gz to (a mounted) /boot, or
b. grub handoff to the kernel (as in grub.conf: kernel /xen.gz

If a. then I may have things in the wrong order. If b. then I don't understand.

Just trying to figure out where the HOWTO is confusing. Updating/beautifying now and will post in a couple days.
_________________
Success is the ability to go from one failure to another with no loss of enthusiasm. - Winston Churchill
Back to top
View user's profile Send private message
groovin
Guru
Guru


Joined: 07 Feb 2004
Posts: 429
Location: California, USA

PostPosted: Thu Jun 16, 2005 2:57 am    Post subject: Reply with quote

this looks good. i tried out xen but not in such a clean fashion as illustrated here. my problem was i had to use one of xen's kernels for the host which didnt have support for any of my hardware.
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Thu Jun 16, 2005 2:17 pm    Post subject: Reply with quote

Domain 0 shouldn't need the frontend drivers in the kernel under most circumstances.
Back to top
View user's profile Send private message
trad511
n00b
n00b


Joined: 29 Nov 2003
Posts: 27

PostPosted: Wed Jun 22, 2005 8:27 pm    Post subject: Reply with quote

cgs wrote:
Domain 0 shouldn't need the frontend drivers in the kernel under most circumstances.


I was under the impression that the Domain 0 kernel had all the drivers and the other kernels were unprivileged - meaning that
they didn't have any special hardware drivers. I very well could be wrong, but went on the following from the Xen Users' Manual:

http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html#SECTION02230000000000000000

2.3.2

Quote:
...one with a `-xen0' extension which contains hardware device drivers and drivers for Xen's virtual devices, and one with a `-xenU' extension that just contains the virtual ones.


2.3.3

Quote:
If you wish to build a customized XenLinux kernel (e.g. to support additional devices or enable distribution-required features), you can use the standard Linux configuration mechanisms, specifying that the architecture being built for is xen, e.g:


# cd linux-2.6.11-xen0
# make ARCH=xen xconfig
# cd ..
# make

You can also copy an existing Linux configuration (.config) into linux-2.6.11-xen0 and execute:


# make ARCH=xen oldconfig

You may be prompted with some Xen-specific options; we advise accepting the defaults for these options.

Note that the only difference between the two types of Linux kernel that are built is the configuration file used for each. The "U" suffixed (unprivileged) versions don't contain any of the physical hardware device drivers, leading to a 30% reduction in size; hence you may prefer these for your non-privileged domains. The `0' suffixed privileged versions can be used to boot the system, as well as in driver domains and unprivileged domains.


My experience and the documentation led me to believe that if you download Xen and install via Section 2.3.1 you would not need
any special configurations for drivers (does it compile them all????). I never did try to compile Domain 0 with no drivers (or did and
can't remember). Let me know if the Domain 0 configuration works without any drivers.
_________________
Success is the ability to go from one failure to another with no loss of enthusiasm. - Winston Churchill
Back to top
View user's profile Send private message
trad511
n00b
n00b


Joined: 29 Nov 2003
Posts: 27

PostPosted: Wed Jun 22, 2005 8:52 pm    Post subject: Unprivileged Domains Reply with quote

Where I currently have an unresolved problem (the reason I've not update the the domain/image section of the howto) is with a
RHEL-4 (CentOS-4) install image.

The way I installed the image was to boot the system with the installation CD and create a single partition installation (in my
case this was /dev/hda10). Because I was using this outside of Xen I updated my Gentoo grub.conf and rebooted into the new
installation no problem. Remember here that this installation will do hardware probing during install and then create an initrd based
on kernel modules specific to your installation. Also, the kernel installed with the installation includes all modules needed to run
the system (including iptables modules which I'll come back to in a moment).

To add this domain to Xen in the most simple manner I could think of I copied my /etc/xen/ttylinux configuration file to /etc/xen/rheltest.
Editing this file I used the XenU domain in /usr/share/xen/xenU-kernels/vmlinuz-2.6.11.10-xenU. Note that this is the exact same
kernel as used with ttylinux and that I changed a few networking parameters to make the domain unique.

Starting up this new domain I got a hung kernel boot. A search on the web stated that the RHEL initrd had to be used during the
boot process. So, ever in a rush to get this going, I added the initrd to the rheltest config file. The final file looks like:

(I'll post this later as I'm not able to access the machine right now)

Note that now I am using the standard XenU kernel with the RHEL-4 initrd, which doesnt' seem quite right....
But, starting the domain this time works.

Here's the problem: All the hardware drivers and services relying on them from the original RH install fail. No wonder as there
are no USB kernel modules now. There is no hardware access to /dev/rtc. So, during startup, the Xen RHEL domain throws up a
bunch of non-fatal errors, runs Kudzu to reconfigure, etc... Ultimately when you go through and allow Kudzu to remove all these
devices from your configuration (don't do this with networking) the system will boot cleanly (minus the rtc which is solved elsewhere).

This domain is usable, but my lack of knowledge is in how to allow the domain virtual access to the hardware it wants. Haven't
researched and experimented enough here so if anyone has some tips and tricks I'd appreciate it.

The other problem with this domain is that there are absolutely no kernel modules and thus none of the iptables modules exist.
Even if the domain is virtual we still want access to all the networking functionality. My hypothesis is that you need to selectively
compile in modules ot the XenU kernel - like the iptables module - in order for domains to be fully functional. Haven't had a chance
to test this out yet.

If anyone has experience knowing how exactly to work with the unprivileged domains and their drivers/modules please post.
_________________
Success is the ability to go from one failure to another with no loss of enthusiasm. - Winston Churchill
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Wed Jun 22, 2005 9:11 pm    Post subject: Reply with quote

Here's how it works. Domain 0 has access to all physical devices since it's privileged, after all. Domain 0 would have configured both the physical device drivers as well as the frontend drivers so that it can export them as virtual disk and/or virtual network devices to the unprivileged domains.

Unprivileged domains typically have frontend drivers, but not backend drivers. The frontend drivers connect to Domain 0's backend drivers via shared mem and everyone's happy. Typically, the unprivileged domain _only_ needs these frontend drivers configured. Domain 0 needs the physical device drivers and the backend drivers in its kernel, not in the unprivileged kernel.

Unprivileged domains can be configured with backend drivers so that they may serve physical device access to other domains. The unprivileged domain must get access to the physical device via the pci= settings in the domain configuration. It must also have the physical device driver configured in the kernel.

A domain cannot:

export a netif (act as a xen network backend) and import a netif (act as a xen network frontend) at the same time

export a blkif (act as a xen block device backend) and import a blkif (act as a xen block device frontend) at the same time

See User Manual section 7.3.
Back to top
View user's profile Send private message
ElCondor
Guru
Guru


Joined: 10 Apr 2002
Posts: 520
Location: Vienna, Austria, Europe

PostPosted: Wed Jul 06, 2005 5:24 pm    Post subject: XenKernel keeps resetting Reply with quote

Could anyone with a running xen0 please post (or mail me) a working .config for the linux-2.6.11.10-xen0 kernel? After about 10 different kernels which all reset during boot (somewhere at PS/2 .. mice ... ) instantly without giving me the chance to actually read at which point the kernel resets, I'm a little .. fed up with configuring by blind guessing (and I don't want to continue till I'm desperate) :(

Thanks in Advance

* ElCondor pasa*
_________________
Here I am the victim of my own choices and I'm just starting!
Back to top
View user's profile Send private message
ElCondor
Guru
Guru


Joined: 10 Apr 2002
Posts: 520
Location: Vienna, Austria, Europe

PostPosted: Wed Jul 06, 2005 6:09 pm    Post subject: Reply with quote

Got it! appending a "noreboot" to the xen.gz kernel shows the kernel-problem :)

so, what I got is:
Quote:
VFS: Cannot open root device "hda5" or unknown-block(0,0)
Please append a corrent "root=" boot option
Kernel panic - not syncing: VFS: Unable to mount root fs on unkown-block(0,0)

:(
I also manually added (because those 2 parameters show up nowhere in the .config or menuconfig anywhere!)
Code:
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDEGENERIC=y

to the .config file. The parameters
Code:
CONFIG_PARTITION_ADVANCED=y
CONFIG_MSDOS_PARTITION=y

are already set by menuconfig. when I compile the kernel, I get the following message:
Quote:
.config:229: trying to assign nonexistent symbol BLK_DEV_IDEGENERIC

:8O: what did I miss / do wrong?? :?:
Actually I'm rather lost at this point :oops:

* ElCondor pasa *
_________________
Here I am the victim of my own choices and I'm just starting!
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Wed Jul 06, 2005 6:57 pm    Post subject: Reply with quote

ElCondor wrote:
I also manually added (because those 2 parameters show up nowhere in the .config or menuconfig anywhere!)
Code:
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDEGENERIC=y


If it's not in a Kconfig file, it's probably not valid.

The correct variables are CONFIG_IDE and CONFIG_BLK_DEV_IDE.

If you are manually editing your .config file and not following the documentation, don't expect too much help from anyone.
Back to top
View user's profile Send private message
ElCondor
Guru
Guru


Joined: 10 Apr 2002
Posts: 520
Location: Vienna, Austria, Europe

PostPosted: Wed Jul 06, 2005 9:17 pm    Post subject: Reply with quote

cgs wrote:
The correct variables are CONFIG_IDE and CONFIG_BLK_DEV_IDE.

Thanks, I'm going to try that tomorrow :)
cgs wrote:
If you are manually editing your .config file and not following the documentation, don't expect too much help from anyone.

It's difficult to follow the documentation when parameter names are mixed up completely ;)

* ElCondor pasa *
_________________
Here I am the victim of my own choices and I'm just starting!
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Wed Jul 06, 2005 9:25 pm    Post subject: Reply with quote

ElCondor wrote:

It's difficult to follow the documentation when parameter names are mixed up completely ;)


Which ones are?
Back to top
View user's profile Send private message
ElCondor
Guru
Guru


Joined: 10 Apr 2002
Posts: 520
Location: Vienna, Austria, Europe

PostPosted: Thu Jul 07, 2005 9:31 am    Post subject: Reply with quote

CONFIG_IDE and CONFIG_BLK_DEV_IDE vs CONFIG_BLK_DEV_IDEGENERIC was a little .. disturbing - since I tried to keep the kernel small and removed everything unnecessary (and a little more as we found out ;) )

Thanks again for the help, xen runs fine now :)

* ElCondor pasa *
_________________
Here I am the victim of my own choices and I'm just starting!
Back to top
View user's profile Send private message
braynyac
n00b
n00b


Joined: 11 Mar 2003
Posts: 10

PostPosted: Sat Jul 23, 2005 8:04 pm    Post subject: Xend service not starting Reply with quote

First of all, wanted to say nice howto =) Second, I'm having trouble getting Xend to start. I am booted into the xen kernel, as a 'uname -a' shows:
Code:
uname -a
Linux stalin 2.6.11.10-xen #1 Fri Jul 22 15:31:58 EDT 2005 i686 AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux

When I try to start xend, it fails, and when I check my logs, this is what /var/log/xend-debug.log shows:
Code:
Traceback (most recent call last):
  File "/usr/sbin/xend", line 121, in ?
    sys.exit(main())
  File "/usr/sbin/xend", line 107, in main
    return daemon.start()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 525, in start
    self.run()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 612, in run
    self.listenEvent(xroot)
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 629, in listenEvent
    return reactor.listenTCP(port, protocol, interface=interface)
  File "/usr/lib/python2.3/site-packages/twisted/internet/default.py", line 283, in listenTCP
    p.startListening()
  File "/usr/lib/python2.3/site-packages/twisted/internet/tcp.py", line 594, in startListening
    raise CannotListenError, (self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on localhost:8001: (98, 'Address already in use').


I installed the xen ebuild per the instructions in the post, and I'm not sure what's going wrong, as I'm not a programmer and don't know Python =). Any help would be appreciated, and thanks in advance,

~braynyac
_________________
I never really understood how there could be things that would drive you insane just because you knew them until I ran into Windows.
- Peter da Silva
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Sat Jul 23, 2005 10:00 pm    Post subject: Re: Xend service not starting Reply with quote

braynyac wrote:
First of all, wanted to say nice howto =) Second, I'm having trouble getting Xend to start. I am booted into the xen kernel, as a 'uname -a' shows:
Code:
uname -a
Linux stalin 2.6.11.10-xen #1 Fri Jul 22 15:31:58 EDT 2005 i686 AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux

When I try to start xend, it fails, and when I check my logs, this is what /var/log/xend-debug.log shows:
Code:
Traceback (most recent call last):
  File "/usr/sbin/xend", line 121, in ?
    sys.exit(main())
  File "/usr/sbin/xend", line 107, in main
    return daemon.start()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 525, in start
    self.run()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 612, in run
    self.listenEvent(xroot)
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 629, in listenEvent
    return reactor.listenTCP(port, protocol, interface=interface)
  File "/usr/lib/python2.3/site-packages/twisted/internet/default.py", line 283, in listenTCP
    p.startListening()
  File "/usr/lib/python2.3/site-packages/twisted/internet/tcp.py", line 594, in startListening
    raise CannotListenError, (self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on localhost:8001: (98, 'Address already in use').


I installed the xen ebuild per the instructions in the post, and I'm not sure what's going wrong, as I'm not a programmer and don't know Python =). Any help would be appreciated, and thanks in advance,

~braynyac


Edit /etc/xen/xend-config.sxp and change the port number from 8001 to something else, or stop whatever else is listening on that port (execute ``netstat -lnp | grep :8001'' as root). Maybe xend is already running?
Back to top
View user's profile Send private message
braynyac
n00b
n00b


Joined: 11 Mar 2003
Posts: 10

PostPosted: Sun Jul 24, 2005 5:19 am    Post subject: Reply with quote

cgs, thanks for the help =) I installed Twisted 2.0 and Twisted-web 0.5, and I now have xend and xendomains started. Now I'm getting some other problems...When I start the service I get this message:
Code:
 * Starting Xen control daemon ...
/usr/lib/python2.3/site-packages/xen/xend/server/SrvDir.py:3: DeprecationWarning: twisted.protocols.http has moved to twisted.web.http. See http://twistedmatrix.com/projects/web.
  from twisted.protocols import http

Everything starts OK, but it's annoying. The Twisted web site says that this may happen due to a fork in the project and that it's ok to ignore, but I'd still like to prevent it from showing up each time I start the xend service.

Also, when I run
Code:
xm list
as root, I get this:
Code:
Traceback (most recent call last):
  File "/usr/sbin/xm", line 9, in ?
    main.main(sys.argv)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 808, in main
    xm.main(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 106, in main
    self.main_call(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 124, in main_call
    p.main(args[1:])
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 343, in main
    doms.sort()
AttributeError: 'str' object has no attribute 'sort'


This happens after the xend and xendomains services are running.

If I try to create a new domain, I get this:
Code:
stalin ~ # xm create /etc/xen/ttylinux -c
Using config file "/etc/xen/ttylinux".
Traceback (most recent call last):
  File "/usr/sbin/xm", line 9, in ?
    main.main(sys.argv)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 808, in main
    xm.main(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 106, in main
    self.main_call(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 124, in main_call
    p.main(args[1:])
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 259, in main
    create.main(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/create.py", line 548, in main
    (dom, console) = make_domain(opts, config)
  File "/usr/lib/python2.3/site-packages/xen/xm/create.py", line 504, in make_domain
    dominfo = server.xend_domain_create(config)
  File "/usr/lib/python2.3/site-packages/xen/xend/XendClient.py", line 207, in xend_domain_create
    {'op'      : 'create',
  File "/usr/lib/python2.3/site-packages/xen/xend/XendClient.py", line 148, in xendPost
    return self.client.xendPost(url, data)
  File "/usr/lib/python2.3/site-packages/xen/xend/XendProtocol.py", line 79, in xendPost
    return self.xendRequest(url, "POST", args)
  File "/usr/lib/python2.3/site-packages/xen/xend/XendProtocol.py", line 143, in xendRequest
    resp = conn.getresponse()
  File "/usr/lib/python2.3/httplib.py", line 778, in getresponse
    response.begin()
  File "/usr/lib/python2.3/httplib.py", line 273, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.3/httplib.py", line 237, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine


Thanks in advance for your help!!!

~braynyac
_________________
I never really understood how there could be things that would drive you insane just because you knew them until I ran into Windows.
- Peter da Silva
Back to top
View user's profile Send private message
cgs
n00b
n00b


Joined: 16 Jun 2005
Posts: 6

PostPosted: Sun Jul 24, 2005 9:45 am    Post subject: Re: Xend service not starting Reply with quote

cgs wrote:
braynyac wrote:
First of all, wanted to say nice howto =) Second, I'm having trouble getting Xend to start. I am booted into the xen kernel, as a 'uname -a' shows:
Code:
uname -a
Linux stalin 2.6.11.10-xen #1 Fri Jul 22 15:31:58 EDT 2005 i686 AMD Athlon(tm) XP 2000+ AuthenticAMD GNU/Linux

When I try to start xend, it fails, and when I check my logs, this is what /var/log/xend-debug.log shows:
Code:
Traceback (most recent call last):
  File "/usr/sbin/xend", line 121, in ?
    sys.exit(main())
  File "/usr/sbin/xend", line 107, in main
    return daemon.start()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 525, in start
    self.run()
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 612, in run
    self.listenEvent(xroot)
  File "/usr/lib/python2.3/site-packages/xen/xend/server/SrvDaemon.py", line 629, in listenEvent
    return reactor.listenTCP(port, protocol, interface=interface)
  File "/usr/lib/python2.3/site-packages/twisted/internet/default.py", line 283, in listenTCP
    p.startListening()
  File "/usr/lib/python2.3/site-packages/twisted/internet/tcp.py", line 594, in startListening
    raise CannotListenError, (self.interface, self.port, le)
twisted.internet.error.CannotListenError: Couldn't listen on localhost:8001: (98, 'Address already in use').


I installed the xen ebuild per the instructions in the post, and I'm not sure what's going wrong, as I'm not a programmer and don't know Python =). Any help would be appreciated, and thanks in advance,

~braynyac


Edit /etc/xen/xend-config.sxp and change the port number from 8001 to something else, or stop whatever else is listening on that port (execute ``netstat -lnp | grep :8001'' as root). Maybe xend is already running?

Which version of twisted do have installed? Refer to i think,, um... 82 and 83 or 84 and 85 of the xen ebuild bug report in the Gentoo bug list and make sure you have the right version.
Back to top
View user's profile Send private message
braynyac
n00b
n00b


Joined: 11 Mar 2003
Posts: 10

PostPosted: Mon Jul 25, 2005 1:21 pm    Post subject: Reply with quote

cgs, I had Twisted 2.0.0 installed, as well as Twisted-web 0.5.0. With these I was getting the errors previously mentioned. I referred to the comments you mentioned, and unmerged both of those packages, then emerged Twisted 1.3.0. Now I get this when I start xendomains:
Code:

Traceback (most recent call last):
  File "/usr/sbin/xm", line 9, in ?
    main.main(sys.argv)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 808, in main
    xm.main(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 106, in main
    self.main_call(args)
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 124, in main_call
    p.main(args[1:])
  File "/usr/lib/python2.3/site-packages/xen/xm/main.py", line 343, in main
    doms.sort()
AttributeError: 'str' object has no attribute 'sort'


I also get the same errors when I try to run 'xm list'.

Again, I am now on Twisted 1.3.0, and I can get xend and xendomains started, but I can not create any new domains, or list any current domains. Thanks again!

Edit
I just stopped both xend and xendomains and then restarted. Both came up without any errors. However, when I try to get the status of the xendomains service or run 'xm list' I now get this:
Code:

stalin ~ # service xendomains status
 * status:  started
(111, 'Connection refused')
Error: Error connecting to xend, is xend running?


Or this:
Code:

stalin ~ # xm list
(111, 'Connection refused')
Error: Error connecting to xend, is xend running?


Here is some more info:
Code:

stalin ~ # ps aux |grep xend
root     17029  0.0  2.9   9936  7604 ?        S    09:46   0:00 python /usr/sbin/xend start

Code:

stalin ~ # service xend status
 * status:  started


Any ideas? I have changed the ports as cgs said above...they are now 8003,8004, and I checked to make sure nothing else was listening on those ports before I set them. So once again, I'm stumped =).

~braynyac
_________________
I never really understood how there could be things that would drive you insane just because you knew them until I ran into Windows.
- Peter da Silva
Back to top
View user's profile Send private message
braynyac
n00b
n00b


Joined: 11 Mar 2003
Posts: 10

PostPosted: Tue Jul 26, 2005 1:50 pm    Post subject: Reply with quote

Well, I think I solved the issue...

As mentioned earlier, I had something running on port 8000 (Shoutcast), which caused xend to not start. So I changed the ports in /etc/xen/xend-config.sxp to 8003 and 8004 (top two lines). This allowed xend and xendomains to start, but would not let me run 'xm list'. I shut off Shoutcast, changed the ports back to their defaults, stopped and started xend and xendomains, and 'xm list' now works!

Anyone know why changing the default ports would cause xend to break?

Thanks!

~braynyac
_________________
I never really understood how there could be things that would drive you insane just because you knew them until I ran into Windows.
- Peter da Silva
Back to top
View user's profile Send private message
Cenrim
Tux's lil' helper
Tux's lil' helper


Joined: 04 Dec 2004
Posts: 142
Location: Germany

PostPosted: Sun Sep 04, 2005 2:17 pm    Post subject: Reply with quote

did anybody wrote some gentoo-style init scripts for xen?
I put two (prolly quite ugly and buggy :roll: ) ones together, but the short-time-memoryless idiot I am, I forgot to copy them on my iBook, so I could post them, when I find someplace with internet access :oops:
Back to top
View user's profile Send private message
Cenrim
Tux's lil' helper
Tux's lil' helper


Joined: 04 Dec 2004
Posts: 142
Location: Germany

PostPosted: Tue Sep 06, 2005 1:40 pm    Post subject: Reply with quote

so, there are these little...

/etc/init.d/xend:
Code:

#!/sbin/runscript
#
# xend          Script to start and stop the Xen control daemon.
#                       gentoo-style hack ;)
#
# Author:       of the original bash script:
#               Keir Fraser <keir.fraser@cl.cam.ac.uk>
#               of this ugly 'hack':
#               Florian Ermisch <floh@eb104.tu-berlin.de>
#
# description: Starts and stops the Xen control daemon.

depend() {
   need localmount           ### so we can access root-filesystems whereever they are (i.e. /usr/share/xen/)
}

checkpriv() {
   if [ ! -e /proc/xen/privcmd ] ; then
      eerror "xend have to be started in a provileged Domain"
      return 1
   fi
}

start() {
   checkpriv || return 1
   ebegin "Starting xend"
   xend start
   eend $?
}

stop() {
   ebegin "Stopping xend"
   xend stop
   eend $?
}


and /etc/init.d/xendomains:
Code:

#!/sbin/runscript
#
# /etc/init.d/xendomains
# Start / stop domains automatically when domain 0 boots / shuts down.
#
# chkconfig: 345 99 00
# description: Start / stop Xen domains. ### I got no ****ing clue what chkconfig is ^^"
#
# The original script offers fairly basic functionality. It worked on LSB-compliant
# Linux Distributions. This one is way more ugly, but works on Gentoo. ;P
#
# Original based on the example in the "Designing High Quality Integrated Linux
# Applications HOWTO" by Avi Alkalay
# <http://www.tldp.org/HOWTO/HighQuality-Apps-HOWTO/>,
### This one on what I've seen in other scripts and and some trying ;)
###               Florian Ermisch <floh@eb104.tu-berlin.de>

opts="start stop restart status stopall"
INITD=/etc/init.d
AUTODIR=/etc/xen/auto
LOCKFILE=/var/lock/subsys/xendomains
TIMEOUT=60 #timeout for stopall

depend () {
   need xend sshd
   ### sshd just in case a domain hangs really badly & has to be destroyed remotely
}

checkpriv() {
   if ! [ -e /proc/xen/privcmd ]; then
   eerror "xendomains have to be startet in a privileged Domain"
   return 1
   fi
}
checklock() {
   if [ -e $LOCKFILE ]; then
   eerror "xendomains are locked, already started?"
   return 1
   fi
}

start() {
   checkpriv || return 1
   checklock || return 1
   
       einfo "Starting Xen domains:"

    # We expect config scripts for auto starting domains to be in
    # AUTODIR - they could just be symlinks to files elsewhere
    if [ -d $AUTODIR ] && [ $(ls $AUTODIR | wc -l) -gt 0 ]; then
   touch $LOCKFILE
       # Create all domains with config files in AUTODIR.
   for dom_auto in  $AUTODIR/*; do
      ebegin    "   Starting $dom_auto"
      xm create --quiet --defconfig $dom_auto
      eend $?
   done
    fi
}

stop()
{
    # NB. this shuts down ALL Xen domains (politely), not just the ones in
    # AUTODIR/*
    # This is because it's easier to do ;-) but arguably if this script is run
    # on system shutdown then it's also the right thing to do.
   ebegin "Shutting down all Xen domains"
   xm shutdown --all --wait --halt
   RETVAL=$?
   [ $RETVAL -eq 0 ] && rm -f $LOCKFILE
   eend $?
}
stopall() {
    ###            some of my domains didn't shut down cleanly, so this option
    ###            lists the still running ones after ${TIMEOUT} seconds
   status
   ebegin "Shutting down all remaining Xen domains"
   xm shutdown --all --halt
   COUNT=0
   echo -n "   "
   while [ "$COUNT" -lt "$TIMEOUT" ]; do
      echo -n "."
      sleep 2s
      if [ `xm list | wc -l` -eq 2 ]; then
         COUNT=$TIMEOUT
      fi
      COUNT=`expr ${COUNT} + 2`
   done   
   echo
   if ! [ `xm list | wc -l` -eq 2 ]; then
      eerror "After $TIMEOUT seconds still running domains:"
      for dom_running in `xm list | grep -v Name | grep -v Domain-0 | cut -d " " -f 1`; do
         eerror "   $dom_running"
      done
      EE=1
   fi
   eend $EE
}

restart()
{
   svc_stop
   svc_start
}

# same as restart for now - commented out to avoid confusion
#reload()
#{
#    restart
#}
status() {
   if [ -e $LOCKFILE ]; then
      for dom_running in `xm list | grep -v Name | grep -v Domain-0 | cut -d " " -f 1`; do
         einfo "$dom_running:   started"
      done
                            ### Lists the running domains
   else   
      eerror "Not by xendomains started domains:"
      if [ `xm list | wc -l` -eq 2 ]; then
         echo "   None"
      fi
      for dom_running in `xm list | grep -v Name | grep -v Domain-0 | cut -d " " -f 1`; do
         echo "   $dom_running"
      done
   fi
}


oh.. f***, I'm realizing, I did'nt comment this stuff... :oops:
'K, and I cut great blocks of out commented stuff from the original script...
and didn't post the header stuff... :roll:

so, edit will follow... ^^""

edit: inserted some stuff
sorry guys, I need a xen runnin machine, so I can understand my own stuff... :oops:
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum