Xen 2.6.25 dom0 kernel ebuild

[andylyon]

andylyon

Any problems with these kernels? I'm sorry if you mentioned this in the thread, its hard for me to read with the 2 kernel config dumps taking up > 80% of the thread.

I've not had any problems specific to these kernels, but I've had very little feedback so its hard to say if they are stable or not.

Each kernel is tested with 32 and 64 bit vm's, both hvm and pv, windows and linux, I run several vm's at once with lots of intensive processes running in each one.

Bear in mind that the Suse kernel I take the patches from is for 11.1 beta 3, I would not use these kernels on a production system, not yet.

I'd love to get more feedback!

Andy

[drescherjm]

I am using the 2.6.21 xen kernel that was in portage. And it works but I would like to upgrade since there are problems with the nic driver and I want to start using samba dfs with cifs. This was not working for me until 2.6.26 kernels.

I will probably try your kernels when I get back from vacation in the 3rd week of November. I do not want to implement this and have the system down and get calls to fix this from the beach.

[trikolon]

running the latest ebuild for >12h without any problems.

i have a simple question:

i habe 4 real ethenet devices eth0 - eth3. eth0 is bridged to xenbrI, eth1 is bridged to xenbrE, eth2 is bridged to xenbrW and eth3 is bridged to xenbrD.
i am running ipfire (something like ipcop) as a hvm domain with all 4 bridges without any problems. but i am not able to get in any other domU network up and running.
for example, i want a debian server paravirtualized. so i gave him 'bridged=xenbrI' and want to use dhcp to assign the ip adresses. but nothing happens. no adresses, no ping is possible..
is it because i already use the xenbrI bridge in my running ipcop or is something else missing?

edit: ----> solved! i should use an kernel with xen support, d'oh!!

[andylyon]

I have updated the ebuild to 2.6.27-r2 for genpatches 4.

I made a mistake yesterday and updated the xen patches without changing the filename, that is also updated in this version so you will need to download both files.

Andy

[forall]

Hi

I using the latest patches from http://code.google.com/p/gentoo-xen-ker ... loads/list, compiled kernel, but when my kernel is loading I have error:



Somebody have a same problem?

[trikolon]

after testing the second release, i can say, everything runs smooth.

does anybody know why there is no speedstepping for the dom0 available?
power management -> CPU frequency scaling is just empty...

[stof]

I'm not definitely sure, if this is related to the kernel or if i just missed a kernel feature. Your kernel seems to run fine in the dom0 and i also got HVM running. The strange thing is: If i use your kernel for a paravirtualized domU, it doesn't start up - except if there is already a hvm domU running or a domU with an older kernel. If your kernel is the first one i try to start, it crashes. This sounds strange but i verified this multiple times.

I've also tried older 2.6.18 and 2.6.21 domU kernels. They always run fine. But i have to confess that i tried to disable as much as possible from the 2.6.27 domU kernel. So maybe it was too much cleanup.

Here some output:

Code: Select all

[xen0 xen]# xm create web0.cfg -c
Using config file "./web0.cfg".
Started domain web0
                   [xen0 xen]#

Code: Select all

[xen0 xen]# xm dmesg
...
(XEN) mm.c:707:d10 Error getting mfn a7fd2 (pfn d5dd2) from L1 entry 00000000a7fd2061 for dom10
(XEN) traps.c:437:d10 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S (ff19d95e)
(XEN) Domain 10 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.3.0  x86_32p  debug=n  Not tainted ]----
(XEN) CPU:    1
(XEN) EIP:    e019:[<c0110b80>]
(XEN) EFLAGS: 00000282   EM: 1   CONTEXT: pv guest
(XEN) eax: ffffffea   ebx: f56fb000   ecx: a7fd2061   edx: 00000000
(XEN) esi: 00000006   edi: a7fd2061   ebp: 00000000   esp: c04bdf2c
(XEN) cr0: 8005003b   cr4: 000026f0   cr3: 001d0d20   cr2: 00000000
(XEN) ds: e021   es: e021   fs: e021   gs: 0000   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c04bdf2c:
(XEN)    c0110b80 0001e019 00010082 00000000 00000061 00000000 000a7fd2 c010e805
(XEN)    00104000 f56fb000 00000000 a7fd2000 00000063 000004ef c04cbddb 00000061
...

[andylyon]

Hi

I using the latest patches from http://code.google.com/p/gentoo-xen-ker ... loads/list, compiled kernel, but when my kernel is loading I have error:



Somebody have a same problem?

How did you compile the kernel? you cannot use a bzImage with xen, you must use make vmlinuz, like this:

cd /usr/src/linux-2.6.27-xen-r2/
make vmlinuz
cp arch/x86/boot/vmlinuz /boot/vmlinuz-2.6.27-xen-r2
cp System.map /boot/System.map-2.6.27-xen-r2

and then in grub.conf:

kernel /xen.gz
module /vmlinuz-2.6.27-xen-r2 root=/dev/<blah>


Andy

[andylyon]

I'm not definitely sure, if this is related to the kernel or if i just missed a kernel feature. Your kernel seems to run fine in the dom0 and i also got HVM running. The strange thing is: If i use your kernel for a paravirtualized domU, it doesn't start up - except if there is already a hvm domU running or a domU with an older kernel. If your kernel is the first one i try to start, it crashes. This sounds strange but i verified this multiple times.

I've also tried older 2.6.18 and 2.6.21 domU kernels. They always run fine. But i have to confess that i tried to disable as much as possible from the 2.6.27 domU kernel. So maybe it was too much cleanup.

Here some output:

Code: Select all

[xen0 xen]# xm create web0.cfg -c
Using config file "./web0.cfg".
Started domain web0
                   [xen0 xen]#

Code: Select all

[xen0 xen]# xm dmesg
...
(XEN) mm.c:707:d10 Error getting mfn a7fd2 (pfn d5dd2) from L1 entry 00000000a7fd2061 for dom10
(XEN) traps.c:437:d10 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S (ff19d95e)
(XEN) Domain 10 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.3.0  x86_32p  debug=n  Not tainted ]----
(XEN) CPU:    1
(XEN) EIP:    e019:[<c0110b80>]
(XEN) EFLAGS: 00000282   EM: 1   CONTEXT: pv guest
(XEN) eax: ffffffea   ebx: f56fb000   ecx: a7fd2061   edx: 00000000
(XEN) esi: 00000006   edi: a7fd2061   ebp: 00000000   esp: c04bdf2c
(XEN) cr0: 8005003b   cr4: 000026f0   cr3: 001d0d20   cr2: 00000000
(XEN) ds: e021   es: e021   fs: e021   gs: 0000   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c04bdf2c:
(XEN)    c0110b80 0001e019 00010082 00000000 00000061 00000000 000a7fd2 c010e805
(XEN)    00104000 f56fb000 00000000 a7fd2000 00000063 000004ef c04cbddb 00000061
...

That is very strange, I must admit I've not tested any of the kernels as the very first domU, as soon as my machine boots it starts up several windows hvms which I use for my work, sometimes I boot a pv domU but not often.

I will give it a try and let you know if I have the same problem, and if I can fix it , I have an idea what it might be....

Andy

[andylyon]

after testing the second release, i can say, everything runs smooth.

does anybody know why there is no speedstepping for the dom0 available?
power management -> CPU frequency scaling is just empty...

If you disable ACPI the CPU frequency scaling options reappear....... I will look into it.

Andy

[trikolon]

any news?

[uprooter]

Hello.

Im working with andy's 2.6.27 patches for a week .
It looks quite stable (I even put a fax server with direct serial access under DOMU )
however I encountered 2 problems

when starting a domU the xen console stops working after "Freeing unused kernel memory: ..."
I saw this post : http://wiki.xensource.com/xenwiki/XenFa ... f99d883097
but it doesn't seems to be the issue under gentoo since the files already exists under /dev/

second thing is that the kernel does not detect the RTC but i guess it's just my bad luck with that cheap hardware.


Any idea (on the first issue)?

[trikolon]

@ uprooter: i had similar problems trying to get an ubuntu domu up and running. it is working for my with a domu gentoo kernel

after almost 2 weeks of testing i can say, i dont have any problems! its stable, its fast, but:
- i would like to see cpu frequency und acpi working hand in hand
- and, i am not able to use virt-manager from any pc in the lan.. has anybody virt-manager up and running?

[galahad7]

Am hoping someone can give me a hand to get this ebuild installed.

I've downloaded the xen-sources-2.6.27-r2.ebuild and the xen-patches-2.6.27-4.tar.bz2 which appear earlier in this thread.
Then have
- created /usr/local/portage/sys-kernel/xen-sources and copied the files in
- added PORTDIR_OVERLDAY=/usr/local/portage

My understanding is next thing to do is to type

Code: Select all

ebuild xen-sources-2.6.27-r2.ebuild digest

Once I do this the system is looking for the patches at http://distfiles.gentoo.org/distfiles/x ... -4.tar.bz2 . How should the ebuild be set up so that it gets xen-patches-2.6.27-4.tar.bz2 from the local file system?

Thanks
Stephen

[stof]

Put the patches file into /usr/portage/distfiles.

[galahad7]

Ah, of course. Thanks.

[andylyon]

@ uprooter: i had similar problems trying to get an ubuntu domu up and running. it is working for my with a domu gentoo kernel

after almost 2 weeks of testing i can say, i dont have any problems! its stable, its fast, but:
- i would like to see cpu frequency und acpi working hand in hand
- and, i am not able to use virt-manager from any pc in the lan.. has anybody virt-manager up and running?

The CPU Frequency scaling kernel options are not available because dom0 does not manage the cpu frequency, Xen does!

From http://www.nabble.com/Power-Management- ... 35782.html

"Nothing is missing except a tool. /proc/cpuinfo or other linux
equivalent interfaces can't reveal cpufreq information, because
now freq scaling is controlled by Xen instead of dom0, and there's
no fixed mapping between dom0's vcpu and underlying real cpus.
In 3.3, xen does exports hypercall to query runtime cpufreq info
in libxc library, however no tool is using at the time. In post-3.3,
we actually developed a so-called 'xenpm' utility for such purpose.
We're back-porting it to xen-3.3 tree, and you should be able to
see it soon."

Regarding virt-manager, I have used it on my gentoo xen system but only briefly as I prefer CLI tools, I was able to start and stop vm's and the graphs shown in the GUI updated every few seconds, I didnt test any more than that.

andy

[drescherjm]

Back from vacation. Got it working. I had a few build problems related to the _dom0 method found in

http://en.gentoo-wiki.com/wiki/Xen

but eventually I worked them out.

One such problem was for some reason the vmlinuz file ended up in this folder

Code: Select all

/usr/src/linux/_dom0/arch/x86/boot

I initially thought that this would be a 32 bit kernel but indeed it is 64 bits.

Code: Select all

datastore1 ~ # uname -a
Linux datastore1 2.6.27-xen-r2 #1 SMP Mon Nov 17 13:48:35 EST 2008 x86_64 AMD Athlon(tm) 64 Processor 3200+ AuthenticAMD GNU/Linux

[andylyon]

@ uprooter: i had similar problems trying to get an ubuntu domu up and running. it is working for my with a domu gentoo kernel

after almost 2 weeks of testing i can say, i dont have any problems! its stable, its fast, but:
- i would like to see cpu frequency und acpi working hand in hand
- and, i am not able to use virt-manager from any pc in the lan.. has anybody virt-manager up and running?

The CPU Frequency scaling kernel options are not available because dom0 does not manage the cpu frequency, Xen does!

From http://www.nabble.com/Power-Management- ... 35782.html

"Nothing is missing except a tool. /proc/cpuinfo or other linux
equivalent interfaces can't reveal cpufreq information, because
now freq scaling is controlled by Xen instead of dom0, and there's
no fixed mapping between dom0's vcpu and underlying real cpus.
In 3.3, xen does exports hypercall to query runtime cpufreq info
in libxc library, however no tool is using at the time. In post-3.3,
we actually developed a so-called 'xenpm' utility for such purpose.
We're back-porting it to xen-3.3 tree, and you should be able to
see it soon."

Having given this some thought I realize it makes no sense at all to control cpu frequency or other pm features from dom0, dom0 is not supposed to be heavily loaded and indeed its load have no correlation with the vm load at all, for example every single physical cpu could be loaded to 100% and the dom0 could be idle.

[andylyon]

Hello.

Im working with andy's 2.6.27 patches for a week .
It looks quite stable (I even put a fax server with direct serial access under DOMU )
however I encountered 2 problems

when starting a domU the xen console stops working after "Freeing unused kernel memory: ..."
I saw this post : http://wiki.xensource.com/xenwiki/XenFa ... f99d883097
but it doesn't seems to be the issue under gentoo since the files already exists under /dev/

second thing is that the kernel does not detect the RTC but i guess it's just my bad luck with that cheap hardware.


Any idea (on the first issue)?

did you pass console=xvc0 to the domU kernel ? I had the exact same problem until I informed the kernel of the correct console to use.

Andy

[andylyon]

I'm not definitely sure, if this is related to the kernel or if i just missed a kernel feature. Your kernel seems to run fine in the dom0 and i also got HVM running. The strange thing is: If i use your kernel for a paravirtualized domU, it doesn't start up - except if there is already a hvm domU running or a domU with an older kernel. If your kernel is the first one i try to start, it crashes. This sounds strange but i verified this multiple times.

I've also tried older 2.6.18 and 2.6.21 domU kernels. They always run fine. But i have to confess that i tried to disable as much as possible from the 2.6.27 domU kernel. So maybe it was too much cleanup.

Here some output:

Code: Select all

[xen0 xen]# xm create web0.cfg -c
Using config file "./web0.cfg".
Started domain web0
                   [xen0 xen]#

Code: Select all

[xen0 xen]# xm dmesg
...
(XEN) mm.c:707:d10 Error getting mfn a7fd2 (pfn d5dd2) from L1 entry 00000000a7fd2061 for dom10
(XEN) traps.c:437:d10 Unhandled invalid opcode fault/trap [#6] on VCPU 0 [ec=0000]
(XEN) domain_crash_sync called from entry.S (ff19d95e)
(XEN) Domain 10 (vcpu#0) crashed on cpu#1:
(XEN) ----[ Xen-3.3.0  x86_32p  debug=n  Not tainted ]----
(XEN) CPU:    1
(XEN) EIP:    e019:[<c0110b80>]
(XEN) EFLAGS: 00000282   EM: 1   CONTEXT: pv guest
(XEN) eax: ffffffea   ebx: f56fb000   ecx: a7fd2061   edx: 00000000
(XEN) esi: 00000006   edi: a7fd2061   ebp: 00000000   esp: c04bdf2c
(XEN) cr0: 8005003b   cr4: 000026f0   cr3: 001d0d20   cr2: 00000000
(XEN) ds: e021   es: e021   fs: e021   gs: 0000   ss: e021   cs: e019
(XEN) Guest stack trace from esp=c04bdf2c:
(XEN)    c0110b80 0001e019 00010082 00000000 00000061 00000000 000a7fd2 c010e805
(XEN)    00104000 f56fb000 00000000 a7fd2000 00000063 000004ef c04cbddb 00000061
...

That is very strange, I must admit I've not tested any of the kernels as the very first domU, as soon as my machine boots it starts up several windows hvms which I use for my work, sometimes I boot a pv domU but not often.

I will give it a try and let you know if I have the same problem, and if I can fix it , I have an idea what it might be....

Andy

I've made a little progress on this problem, I may have a fix in the next few days, sorry its taking so long but the issue exists upstream (in opensuse b3) so I its not a simple "patching gone wrong" type of problem.

Andy

[drescherjm]

I am having problems building the domU kernel.

Code: Select all

datastore1 _domU # make
make -C /usr/src/linux-2.6.27-xen-r2 O=/usr/src/linux-2.6.27-xen-r2/_domU/.
  GEN     /usr/src/linux-2.6.27-xen-r2/_domU/Makefile
scripts/kconfig/conf -s arch/x86/Kconfig
  Using /usr/src/linux-2.6.27-xen-r2 as source for kernel
  GEN     /usr/src/linux-2.6.27-xen-r2/_domU/Makefile
  CHK     include/linux/version.h
  CHK     include/linux/utsrelease.h
  CALL    /usr/src/linux-2.6.27-xen-r2/scripts/checksyscalls.sh
  CHK     include/linux/compile.h
  CC      arch/x86/kernel/cpu/common_64.o
  LD      arch/x86/kernel/cpu/built-in.o
  LD      arch/x86/kernel/built-in.o
  CC      kernel/sched.o
  CC      kernel/panic.o
  CC      kernel/cpu.o
/usr/src/linux-2.6.27-xen-r2/kernel/cpu.c: In function 'disable_nonboot_cpus':
/usr/src/linux-2.6.27-xen-r2/kernel/cpu.c:410: error: implicit declaration of function '_cpu_down'
make[3]: *** [kernel/cpu.o] Error 1
make[2]: *** [kernel] Error 2
make[1]: *** [sub-make] Error 2
make: *** [all] Error 2

[trikolon]

andylyon hat Folgendes geschrieben:
trikolon hat Folgendes geschrieben:
@ uprooter: i had similar problems trying to get an ubuntu domu up and running. it is working for my with a domu gentoo kernel

after almost 2 weeks of testing i can say, i dont have any problems! its stable, its fast, but:
- i would like to see cpu frequency und acpi working hand in hand
- and, i am not able to use virt-manager from any pc in the lan.. has anybody virt-manager up and running?


The CPU Frequency scaling kernel options are not available because dom0 does not manage the cpu frequency, Xen does!

From http://www.nabble.com/Power-Management- ... 35782.html

"Nothing is missing except a tool. /proc/cpuinfo or other linux
equivalent interfaces can't reveal cpufreq information, because
now freq scaling is controlled by Xen instead of dom0, and there's
no fixed mapping between dom0's vcpu and underlying real cpus.
In 3.3, xen does exports hypercall to query runtime cpufreq info
in libxc library, however no tool is using at the time. In post-3.3,
we actually developed a so-called 'xenpm' utility for such purpose.
We're back-porting it to xen-3.3 tree, and you should be able to
see it soon."



Having given this some thought I realize it makes no sense at all to control cpu frequency or other pm features from dom0, dom0 is not supposed to be heavily loaded and indeed its load have no correlation with the vm load at all, for example every single physical cpu could be loaded to 100% and the dom0 could be idle.

ok, i understand. so, it is good, the way it is.

have u tried the opensuse 11.1 b5 kernel? maybe they fixed some things there?

Does anybody know, how to get a working pv-grub.gz? xen-tools did not install it. here are my use flags for xen-tools-3.3.0

Code: Select all

[ebuild   R   ] app-emulation/xen-tools-3.3.0  USE="api hvm pygrub screen -acm -custom-cflags -debug -doc -flask" 0 kB

[andylyon]

andylyon hat Folgendes geschrieben:
trikolon hat Folgendes geschrieben:
@ uprooter: i had similar problems trying to get an ubuntu domu up and running. it is working for my with a domu gentoo kernel

after almost 2 weeks of testing i can say, i dont have any problems! its stable, its fast, but:
- i would like to see cpu frequency und acpi working hand in hand
- and, i am not able to use virt-manager from any pc in the lan.. has anybody virt-manager up and running?


The CPU Frequency scaling kernel options are not available because dom0 does not manage the cpu frequency, Xen does!

From http://www.nabble.com/Power-Management- ... 35782.html

"Nothing is missing except a tool. /proc/cpuinfo or other linux
equivalent interfaces can't reveal cpufreq information, because
now freq scaling is controlled by Xen instead of dom0, and there's
no fixed mapping between dom0's vcpu and underlying real cpus.
In 3.3, xen does exports hypercall to query runtime cpufreq info
in libxc library, however no tool is using at the time. In post-3.3,
we actually developed a so-called 'xenpm' utility for such purpose.
We're back-porting it to xen-3.3 tree, and you should be able to
see it soon."



Having given this some thought I realize it makes no sense at all to control cpu frequency or other pm features from dom0, dom0 is not supposed to be heavily loaded and indeed its load have no correlation with the vm load at all, for example every single physical cpu could be loaded to 100% and the dom0 could be idle.

ok, i understand. so, it is good, the way it is.

have u tried the opensuse 11.1 b5 kernel? maybe they fixed some things there?

The patches are from kernel-source-2.6.27.4-10.2.src.rpm which is currently the latest kernel src rpm for 11.1 beta.

Does anybody know, how to get a working pv-grub.gz? xen-tools did not install it. here are my use flags for xen-tools-3.3.0

Code: Select all

[ebuild   R   ] app-emulation/xen-tools-3.3.0  USE="api hvm pygrub screen -acm -custom-cflags -debug -doc -flask" 0 kB

I had a similar problem with the stubdom tools not being built, I emailed Robert Buchholz who maintains the ebuild and he explained that the build script downloads files in the build phase and that causes problems, so stubdom and pv-grub support is disabled in the ebuild.

To get the stubdom tools I had to build from source.

Andy

[andylyon]

I am having problems building the domU kernel.

Code: Select all

datastore1 _domU # make
make -C /usr/src/linux-2.6.27-xen-r2 O=/usr/src/linux-2.6.27-xen-r2/_domU/.
  GEN     /usr/src/linux-2.6.27-xen-r2/_domU/Makefile
scripts/kconfig/conf -s arch/x86/Kconfig
  Using /usr/src/linux-2.6.27-xen-r2 as source for kernel
  GEN     /usr/src/linux-2.6.27-xen-r2/_domU/Makefile
  CHK     include/linux/version.h
  CHK     include/linux/utsrelease.h
  CALL    /usr/src/linux-2.6.27-xen-r2/scripts/checksyscalls.sh
  CHK     include/linux/compile.h
  CC      arch/x86/kernel/cpu/common_64.o
  LD      arch/x86/kernel/cpu/built-in.o
  LD      arch/x86/kernel/built-in.o
  CC      kernel/sched.o
  CC      kernel/panic.o
  CC      kernel/cpu.o
/usr/src/linux-2.6.27-xen-r2/kernel/cpu.c: In function 'disable_nonboot_cpus':
/usr/src/linux-2.6.27-xen-r2/kernel/cpu.c:410: error: implicit declaration of function '_cpu_down'
make[3]: *** [kernel/cpu.o] Error 1
make[2]: *** [kernel] Error 2
make[1]: *** [sub-make] Error 2
make: *** [all] Error 2

Can you put your .config somwhere I can get it? id rather you didnt post it into the forum as it takes up so much space, but i need the config to look into this problem.

Andy