KVM: gentoo guest doesn't start

Message

ats2 · Post by **ats2** » Thu Mar 31, 2011 8:53 pm

Hi,
I'm trying to create a gentoo guest from an ubuntu machine (but I have experience with installing gentoo on physical servers at work).
Here's the problem : at first reboot, the vm hangs on "Starting from hard disk" and waits for ages.

I've tried various drivers in the kernel. lspci from a livecd told me the correct one was PIIX3, so I compiled it, with no success.
I also tried starting with virtio driver (all virtio stuff compiled in the kernel). No luck.

So I was wondering if someone had met a similar issue and could help...
Thanks in advance.

ats2

PS : of course, KVM works well in this machine otherwise. I've checked and installed various distributions and other OS'es like BSD and Windows as guests. Only gentoo resists and I can't figure out why

Before anyone asks, here's a part of the machine conf (last try with virtio driver) :

Code: Select all

<domain type='kvm'>
  <name>Gentoo</name>
  <uuid>0eed5888-0eca-2857-48b3-911f95edf604</uuid>
  <memory>1024000</memory>
  <currentMemory>1024000</currentMemory>
  <vcpu>1</vcpu>
  <os>
    <type arch='i686' machine='pc-0.12'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/kvm</emulator>
    <disk type='file' device='disk'>
      <driver name='qemu' type='qcow2'/>
      <source file='/vmachines/gentoo.img'/>
      <target dev='vda' bus='virtio'/>
    </disk>
...

ats2 · Post by **ats2** » Fri Apr 01, 2011 8:48 am

Well, could someone could post a link with a /usr/src./linux/.config from a KVM gentoo guest, so that I could compare with mine?

It would help me a lot!

Thanks in advance

ats2

ghutzl · Post by **ghutzl** » Fri Apr 01, 2011 11:40 am

Hello!

This is my .config from a working gentoo guest under kvm:

[EDIT] The direct paste of the .config file was probybly too large, so I put it into pastebin:

http://pastebin.com/DkFhRRVW

ghutzl · Post by **ghutzl** » Fri Apr 01, 2011 12:05 pm

Hi!

I think I remember now what I did with this machine. I used genkernel which I normally don't use because I create the kernel on my own. So I am not sure if the config I have pasted is the one you are looking for. I should be similar to the kernel config of the gentoo boot disk.

Here is the output of lspci -k on that machine:

Code: Select all

00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
        Subsystem: Qumranet, Inc. Qemu virtual machine
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
        Subsystem: Qumranet, Inc. Qemu virtual machine
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
        Subsystem: Qumranet, Inc. Qemu virtual machine
        Kernel driver in use: PIIX_IDE
        Kernel modules: ata_piix
00:01.2 USB Controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
        Subsystem: Qumranet, Inc. Qemu virtual machine
        Kernel driver in use: uhci_hcd
        Kernel modules: uhci-hcd
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
        Subsystem: Qumranet, Inc. Qemu virtual machine
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
        Subsystem: Qumranet, Inc. Device 1100
00:03.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 20)
        Subsystem: Qumranet, Inc. Device 1100
        Kernel driver in use: 8139cp
        Kernel modules: 8139cp, 8139too
00:04.0 Multimedia audio controller: Intel Corporation 82801AA AC'97 Audio Controller (rev 01)
        Subsystem: Intel Corporation Device 0000
        Kernel driver in use: Intel ICH
        Kernel modules: snd-intel8x0
00:05.0 RAM memory: Qumranet, Inc. Virtio memory balloon
        Subsystem: Qumranet, Inc. Device 0005
        Kernel driver in use: virtio-pci
        Kernel modules: virtio_pci

You see it is actually using the ata_piix kernel module for the IDE interface it found. Maybe that is what you are missing?

This is the xml config file of my kvm virtual machine in case it helps:

Code: Select all

<domain type='kvm' id='1'>
  <name>virt-gentoo</name>
  <uuid>05ee83fc-358c-f8a7-0ad7-f0069afcc6d7</uuid>
  <memory>1048576</memory>
  <currentMemory>1048576</currentMemory>
  <vcpu current='1'>4</vcpu>
  <os>
    <type arch='x86_64' machine='pc-0.13'>hvm</type>
    <boot dev='hd'/>
    <boot dev='cdrom'/>
    <bootmenu enable='no'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>
  <cpu match='exact'>
    <model>phenom</model>
    <vendor>AMD</vendor>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-kvm</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw'/>
      <source dev='/dev/vg0/virt-gentoo'/>
      <target dev='hda' bus='ide'/>
      <alias name='ide0-0-0'/>
      <address type='drive' controller='0' bus='0' unit='0'/>
    </disk>
    <disk type='file' device='cdrom'>
      <driver name='qemu' type='raw'/>
      <source file='/home/hutzl/download/linux/gentoo/iso/install-x86-minimal-20110301.iso'/>
      <target dev='hdc' bus='ide'/>
      <readonly/>
      <alias name='ide0-1-0'/>
      <address type='drive' controller='0' bus='1' unit='0'/>
    </disk>
    <controller type='ide' index='0'>
      <alias name='ide0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    </controller>
    <interface type='network'>
      <mac address='52:54:00:92:91:67'/>
      <source network='default'/>
      <target dev='vnet0'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/16'/>
      <target port='0'/>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/16'>
      <source path='/dev/pts/16'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <input type='mouse' bus='ps2'/>
    <graphics type='vnc' port='5900' autoport='yes' keymap='de'/>
    <sound model='ac97'>
      <alias name='sound0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </sound>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <alias name='video0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <alias name='balloon0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>

ats2 · Post by **ats2** » Fri Apr 01, 2011 2:50 pm

Wow, thanks a lot!
I'll try that tonight and post the results... You may be right about the missing module.
Well, more tomorrow. Thx again!

ats2

ats2 · Post by **ats2** » Sat Apr 02, 2011 9:15 am

It didn't work. I've checked ata_piix is compiled in the kernel, and it is already. Actually, the only real difference I can see is that your machine is amd and mine intel, but I don't think it is relevant in that particular issue...
I have tried modifying the xml file, with no more success. Maybe it is related with the way I build the machine, although I just do it the same way as with other linux distributions.
I'll let it go for the time being.
Well, it was worth a try. Thanks for the idea and help anyway

Cheers.

ats2

jormartr · Post by **jormartr** » Sat Apr 02, 2011 10:34 am

Is the machine working with ide mode? I assume it is.

This is what i learnt on these url's: bugs.gentoo.org and
wiki.sihnon.net, also, I am using genkernel, with virtio block being a module.

add to /etc/genkernel.conf

Code: Select all

MODULES_KVM="virtio virtio_balloon virtio_ring virtio_pci virtio_blk virtio_net"

grub won't recognize vd* drives, so, when invoking it, to make it know about its existence:

Code: Select all

device (hd0) /dev/vda

add to the kernel boot parameters:

Code: Select all

doload=virtio_pci,virtio_blk

This is how I work with virtio. I am not completely sure if this is what you want to accomplish, but I post here, so you maybe could take profit.

ats2 · Post by **ats2** » Mon Apr 04, 2011 9:23 am

Thanks for your answer.

Actually, no, the machine doesn't even start in ide mode. It boots from cdrom and that's all. What's weird is that ata_piix is loaded (compiled in the kernel) and it works with a debian virtual machine, for example, but not with gentoo...
I'll try the virtio driver with your settings in a few days and give the results here.

Thanks for the help.

ats2

jormartr · Post by **jormartr** » Mon Apr 04, 2011 9:38 pm

Do not try to make it booting with virtio if it does not boot on pata.

With pata it just should work; once you get it working, then you may want to change to virtio, as it has better performance, but may not work on the first shoot (based on my experience).

ats2 · Post by **ats2** » Tue Apr 05, 2011 9:33 pm

Well, it just won't boot even though pata is enabled (I've checked three times and reinstalled the VM two times in a row).
I give up, it's consuming too much time.

Thanks everybody for your help anyway. Always a great forum

ats2

cach0rr0 · Post by **cach0rr0** » Wed Apr 06, 2011 8:18 am

ats2 wrote:It didn't work. I've checked ata_piix is compiled in the kernel, and it is already. Actually, the only real difference I can see is that your machine is amd and mine intel, but I don't think it is relevant in that particular issue...

with virtio, you shouldn't need any of those.

this is my config for 2.6.36-hardened-r9

http://pastebin.com/raw.php?i=NWYGdFWj

Have a look through it for the VIRTIO modules I have enabled

Specifically:

CONFIG_VIRTIO_BLK=y

If you have that set, and you start your guest appropriately, your drives should show up as /dev/vda, /dev/vdb, and so forth.
Which requires some trickery with grub (you have to do a custom device.map, and do --device-map=/path/to/custom/device.map run installing grub), but otherwise, should be treated the same as any other disk (e.g. within fstab, and within your grub.conf entry, e.g. root=/dev/vda3)

This is how I start the VM from the host OS (no .conf files or anything of the sort)

Code: Select all

qemu-kvm -drive file=/kvm/web/apache.img,if=virtio,boot=on -net nic,model=virtio,macaddr=00:1d:92:ab:3f:77 -net tap,ifname=tap0,script=no,downscript=no -m 1024 -vnc 127.0.0.1:5 -balloon virtio

notice the virtio for -drive, and -net
also notice the boot=on

Even though i have mine on hardened-sources, a similar configuration (minux the grsec and PaX stuff) should do the job. Virtio pretty well handles everything, including your NIC, and your hard drive.

When you're going through menuconfig on the guest, hit the /forward slash to search, search for 'VIRTIO', see if you have the same stuff enabled I do. If you do, you don't need to muck about with ATA_PIIX or any of that nonsense.

my info from the Guest OS, if you're interested:

lspci -n

Code: Select all

00:00.0 0600: 8086:1237 (rev 02)
00:01.0 0601: 8086:7000
00:01.1 0101: 8086:7010
00:01.3 0680: 8086:7113 (rev 03)
00:02.0 0300: 1013:00b8
00:03.0 0200: 1af4:1000
00:04.0 0500: 1af4:1002
00:05.0 0100: 1af4:1001

literally, that's it

grub.conf

Code: Select all

default 0
timeout 10
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title Gentoo Hardened (2.6.36-r9 virtio)
root (hd0,0)
kernel /boot/kernel-new root=/dev/vda3

fstab

Code: Select all


# <fs>                  <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
/dev/vda1               /boot           ext2            noauto,noatime  1 2
/dev/vda3               /               ext3            noatime         0 1
/dev/vda2               none            swap            sw              0 0
/dev/vda5               /tmp            reiserfs        nodev,nosuid,noexec     0 0
/dev/vda6               /opt            reiserfs        nodev,nosuid    0 0
/dev/vda7               /home           xfs             nodev,nosuid,logbufs=8  0 0
/dev/vda8               /usr            reiserfs        nodev           0 0
/dev/vda9               /var            ext3            defaults        0 0
/dev/vda10              /www            reiserfs        nodev,nosuid    0 0

shm                     /dev/shm        tmpfs           nodev,nosuid,noexec     0 0

hopefully you can sort out the screwy .conf file syntax between all of that and my kvm startup commands

Caprisun · Post by **Caprisun** » Mon Aug 13, 2012 4:46 am

ats2 wrote:Well, it just won't boot even though pata is enabled (I've checked three times and reinstalled the VM two times in a row).
I give up, it's consuming too much time.

Thanks everybody for your help anyway. Always a great forum

ats2

I'm not sure if you're still trying, but after having the same problem I got this working today.

Code: Select all


x86: 3.3.8 Gentoo-Sources Kernel
Virtualization:

CONFIG_PARAVIRT_GUEST=y
# CONFIG_PARAVIRT_TIME_ACCOUNTING is not set
CONFIG_PARAVIRT=y
# CONFIG_PARAVIRT_SPINLOCKS is not set
CONFIG_PARAVIRT_CLOCK=y
# CONFIG_PARAVIRT_DEBUG is not set
CONFIG_VIRT_TO_BUS=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_CONSOLE=y
# CONFIG_HW_RANDOM_VIRTIO is not set
# CONFIG_FB_VIRTUAL is not set
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRT_DRIVERS=y
# CONFIG_DEBUG_VIRTUAL is not set
CONFIG_VIRTUALIZATION=y

PCI:

CONFIG_PCI_QUIRKS=y
# CONFIG_ACPI_PCI_SLOT is not set
# Bus options (PCI etc.)
CONFIG_PCI=y
# CONFIG_PCI_GOBIOS is not set
# CONFIG_PCI_GOMMCONFIG is not set
# CONFIG_PCI_GODIRECT is not set
CONFIG_PCI_GOANY=y
CONFIG_PCI_BIOS=y
CONFIG_PCI_DIRECT=y
CONFIG_PCI_MMCONFIG=y
CONFIG_PCI_DOMAINS=y
# CONFIG_PCI_CNB20LE_QUIRK is not set
CONFIG_PCIEPORTBUS=y
# CONFIG_HOTPLUG_PCI_PCIE is not set
CONFIG_PCIEAER=y
# CONFIG_PCIE_ECRC is not set
# CONFIG_PCIEAER_INJECT is not set
CONFIG_PCIEASPM=y
# CONFIG_PCIEASPM_DEBUG is not set
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_PCI_STUB is not set
# CONFIG_PCI_IOV is not set
# CONFIG_PCI_PRI is not set
# CONFIG_PCI_PASID is not set
# CONFIG_PCI_IOAPIC is not set
CONFIG_PCI_LABEL=y
CONFIG_HOTPLUG_PCI=y
# CONFIG_HOTPLUG_PCI_FAKE is not set
# CONFIG_HOTPLUG_PCI_COMPAQ is not set
# CONFIG_HOTPLUG_PCI_IBM is not set
# CONFIG_HOTPLUG_PCI_ACPI is not set
# CONFIG_HOTPLUG_PCI_CPCI is not set
# CONFIG_HOTPLUG_PCI_SHPC is not set
# CONFIG_BLK_DEV_PCIESSD_MTIP32XX is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIAL_8250_PCI=y
# CONFIG_I2C_DESIGNWARE_PCI is not set
# CONFIG_I2C_PXA_PCI is not set
# PCI-based Watchdog Cards
# CONFIG_PCIPCWATCHDOG is not set
# CONFIG_WDTPCI is not set
CONFIG_VIRTIO_PCI=y
CONFIG_GENERIC_PCI_IOMAP=y

I tried being conservative after enabling what seemed to be all of the virtio drivers. But I also broke the kernel.

After recompiling clean 3 times, I stopped disabling PCI devices and left SATA drives alone while turning off LVM/RAID,USB.

I think PCI is being used as a backend for connections of virtio devices...though I haven't done enough reading to confirm that. Perhaps to make use of hotplugging? Anyway I've only been compiling this again for 48 hours myself.

After leaving the configuration alone and sticking to basic virtio stuff with NFS customized along with filesystems, the system booted with virtio.

I would suggest backing up your bzimage and .config, then widdling down the kernel config.

I was also able to boot Windows XP 32bit. Doing updates is a good idea before applying the drivers from Redhat for Virtio. It does shave time off of the boot, however.

I hope you keep trying!

Ant P. · Post by **Ant P.** » Tue Aug 14, 2012 6:06 pm

The PIIX3 emulated devices are only there so you can run dumb OSes (windows). A correctly configured KVM should have lspci output like this:

Code: Select all

00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
	Subsystem: Red Hat, Inc Qemu virtual machine
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
	Subsystem: Red Hat, Inc Qemu virtual machine
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
	Subsystem: Red Hat, Inc Qemu virtual machine
00:01.2 USB controller: Intel Corporation 82371SB PIIX3 USB [Natoma/Triton II] (rev 01)
	Subsystem: Red Hat, Inc Qemu virtual machine
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
	Subsystem: Red Hat, Inc Qemu virtual machine
00:02.0 VGA compatible controller: VMware SVGA II Adapter
	Subsystem: VMware SVGA II Adapter
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
	Subsystem: Red Hat, Inc Device 0001
	Kernel driver in use: virtio-pci
00:04.0 SCSI storage controller: Red Hat, Inc Virtio block device
	Subsystem: Red Hat, Inc Device 0002
	Kernel driver in use: virtio-pci
00:05.0 RAM memory: Red Hat, Inc Virtio memory balloon
	Subsystem: Red Hat, Inc Device 0005
	Kernel driver in use: virtio-pci
00:07.0 Communication controller: Red Hat, Inc Virtio console
	Subsystem: Red Hat, Inc Device 0003
	Kernel driver in use: virtio-pci
00:08.0 Ethernet controller: Red Hat, Inc Virtio network device
	Subsystem: Red Hat, Inc Device 0001
	Kernel driver in use: virtio-pci

The .config should contain these:

Code: Select all

CONFIG_PARAVIRT_GUEST=y
CONFIG_PARAVIRT=y
CONFIG_PARAVIRT_SPINLOCKS=y
CONFIG_PARAVIRT_CLOCK=y
CONFIG_VIRT_TO_BUS=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_CONSOLE=y
CONFIG_HW_RANDOM_VIRTIO=y
CONFIG_VIRTIO=y
CONFIG_VIRTIO_RING=y
CONFIG_VIRTIO_PCI=y
CONFIG_VIRTIO_BALLOON=y
CONFIG_VIRTIO_MMIO=y
# CONFIG_VIRT_DRIVERS is not set
# CONFIG_VIRTUALIZATION is not set