Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
howto install pvgrub for gentoo xen
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
newtonian
Guru
Guru


Joined: 19 Jan 2005
Posts: 465
Location: Hokkaido Japan

PostPosted: Sun Dec 26, 2010 8:06 am    Post subject: howto install pvgrub for gentoo xen Reply with quote

From what I've seen posted in the xen forums, gentoo doesn't add pvgrub to their xen distribution because it uses downloads software from other packages. There is pygrub useflag available for xen so you can use that. Although pvgrub is supposed more secure.

Peter Groben has created an overlay for pvgrub that worked very well for me. This howto goes over the steps it took me to get pvgrub installed in a gentoo dom0 and the configs needed for the domU running from a a gentoo-sources kernel compiled within the domU.

pvgrub really got me out of a tight spot for 2 reasons.
1. I couldn't re-compile my dom0 kernel because it is hosting several production domU's and I can't risk any downtime should the compile go wrong.
2. With pvgrub domU's can freely build their own kernel and modules without ever having to upload anything to the dom0.

dom0 pvgrub, setup the overlay and build the package

Create a new directory that will be the root of your own portage tree, e. g.
Code:
 mkdir /usr/local/portage

add the following to /etc/make.conf if you don't already have an overlay setup:
Code:
echo 'PORTDIR_OVERLAY=/usr/local/portage' >> /etc/make.conf

create a xen-pvgrub directory
Code:
mkdir -p /usr/local/portage/app-emulation/xen-pvgrub/

move to the xen-pvgrub folder and download the contents of the files directory and the ebuild you need ie.
Code:
cd /usr/local/portage/app-emulation/xen-pvgrub/
mkdir files

populate the files directory with patches:
Code:
cd /usr/local/portage/app-emulation/xen-pvgrub/files/
wget http://subversion.fem.tu-ilmenau.de/repository/fem-overlay/trunk/app-emulation/xen-pvgrub/files/xen-pvgrub-3.3.1-sandbox-fix.patch
wget http://subversion.fem.tu-ilmenau.de/repository/fem-overlay/trunk/app-emulation/xen-pvgrub/files/xen-pvgrub-3.3.2-32bit-ioemu-fix.patch
wget http://subversion.fem.tu-ilmenau.de/repository/fem-overlay/trunk/app-emulation/xen-pvgrub/files/xen-pvgrub-4.0.9999-sandbox-fix.patch

get the ebuild that you need
Code:
cd /usr/local/portage/app-emulation/xen-pvgrub/
wget http://subversion.fem.tu-ilmenau.de/repository/fem-overlay/trunk/app-emulation/xen-pvgrub/xen-pvgrub-3.3.2.ebuild
wget http://subversion.fem.tu-ilmenau.de/repository/fem-overlay/trunk/app-emulation/xen-pvgrub/xen-pvgrub-4.0.1.ebuild


create a digest for your ebuild ie.
Code:
ebuild xen-pvgrub-3.3.2.ebuild digest
ebuild xen-pvgrub-4.0.1.ebuild digest


emerge the package
Code:
echo 'app-emulation/xen-pvgrub' >> /etc/portage/package.keywords

Code:
emerge -av =xen-pvgrub-4.0.1


if all goes well, you should get the following results from equery:
Code:
equery files xen-pvgrub

Code:
/usr
/usr/lib
/usr/lib/xen
/usr/lib/xen/boot
/usr/lib/xen/boot/pv-grub-x86_32.gz
/usr/lib/xen/boot/pv-grub-x86_64.gz


updated settings for the domU:

Code:
cat /mnt/data/xen/configs/mydomU.conf

Code:
name   = "mydomU"

#pvgrub here
kernel = "/usr/lib/xen/boot/pv-grub-x86_64.gz"
extra='(hd2)/boot/grub/grub.conf'
#end pvgrub

memory = 1024
vcpus  = 4

disk = [ 'phy:virtual-servers/dev-root,xvda4,w', \
         'phy:virtual-servers/dev-swap,xvda2,w', \
    'phy:virtual-servers/dev-tmp,xvda3,w' ]
vif = [ "rate=750KB/s, mac=00:11:50:00:00:62, bridge=xenbr0" ]


domU settings with standard gentoo sources
Now we're done with the settings on the dom0 side.

From within domU, create a /boot/grub/grub.conf. There is no need to emerge grub. pvgrub only reads the grub.conf text file.
Code:
mkdir -p /boot/grub/
vim /boot/grub/grub.conf


Sample grub.conf when using pvgrub and gentoo-sources in domU
Code:
default 0
timeout 5

title linux-2.6.36-gentoo-r5
root (hd2)
kernel /boot/linux-2.6.36-gentoo-r5 root=/dev/xvda4 ro


Code:
emerge gentoo-sources


create a new symlink
Code:
cd /usr/src/ rm linux; ln -s linux-2.6.36-gentoo-r5 linux


setup the kernel
Code:
cd /usr/src/linux; make menuconfig

Quote:
Processor type and features ---> Paravirtualized guest support ---> [*] Xen guest support
Device Drivers ---> Block Devices ---> [*] Xen virtual block device support
Device Drivers ---> Network device support ---> [*] Xen network device frontend driver
Device Drivers ---> [*] Xen memory balloon driver
[*] Scrub pages before returning them to system
[*] Xen /dev/xen/evtchn device
[*] Xen filesystem
[*] Create xen entries under /sys/hypervisor

build the kernel, also were using pvgrub so it's OK to add modules to the kernel!

Code:
make -j5 && make modules_install


Code:
cp vmlinux /boot/linux-2.6.36-gentoo-r5


update fstab, change block device name like sda to xvda
Code:
sed -i 's/sda/xvda/' /etc/fstab


sample /etc/fstab:
Code:

/dev/xvda2               none            swap            sw                             0 0
/dev/xvda3               /tmp            tmpfs           nodev,nosuid,noexec   0 0
/dev/xvda4               /                  ext3            noatime                      0 1

shm                     /dev/shm        tmpfs           nodev,nosuid,noexec     0 0

In /etc/inittab change tty1 to hvc0 and add hvc0 to /etc/securetty
Code:
sed -i 's/tty1/hvc0/' /etc/inittab
echo 'hvc0' >> /etc/securetty


shutdown the domU and create the domU from the dom0 side. You should get the
Code:
my domUprompt # shutdown -h now


from the dom0 side create the new domU and have it boot into pvgrub:

Code:
xm create -c /mnt/data/xen/configs/mydomUconfig



Code:
   GNU GRUB  version 0.97  (1048576K lower / 0K upper memory)

 +-------------------------------------------------------------------------+
 | linux-2.6.36-gentoo-r5                                                  | 
 | Xen Linux dev-built-linux-2.6.34-xen-r4                                 |
 |                                                                         |
 |                                                                         |
 |                                                                         |
 |                                                                         | 
 +-------------------------------------------------------------------------+
    Use the ^ and v keys to select which entry is highlighted.
    Press enter to boot the selected OS, 'e' to edit the
    commands before booting, or 'c' for a command-line.



You should now be free to create and install new modules and kernels all within the domU filesystem.

Cheers,

Links:


Last edited by newtonian on Tue Dec 28, 2010 1:00 pm; edited 1 time in total
Back to top
View user's profile Send private message
aTan
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2007
Posts: 134
Location: Czech Republic (Ukraine)

PostPosted: Sun Dec 26, 2010 11:03 pm    Post subject: Reply with quote

Hi. Thank you for pvgrub howto.

When I'm trying to emerge xen-pvgrub I'm getting this error:

Code:
gcc -DCONFIG_GRUB   -fno-strict-overflow -O1 -fno-omit-frame-pointer -fno-optimize-sibling-calls  -m32 -march=i686 -m32 -march=i686 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wno-unused-value -Wdeclaration-after-statement  -fno-stack-protector -fno-exceptions -fno-builtin -Wall -Werror -Wredundant-decls -Wno-format -Wno-redundant-decls -fno-stack-protector -fgnu89-inline -Wstrict-prototypes -Wnested-externs -Wpointer-arith -Winline -g -DGNT_DEBUG -DGNTMAP_DEBUG -D__INSIDE_MINIOS__ -m32 -march=i686 -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../extras/mini-os/include -D__MINIOS__ -DHAVE_LIBC -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../extras/mini-os/include/posix -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../tools/xenstore  -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../extras/mini-os/include/x86 -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../extras/mini-os/include/x86/x86_32 -U __linux__ -U __FreeBSD__ -U __sun__ -nostdinc -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/../extras/mini-os/include/posix -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/cross-root-i686/i686-xen-elf/include -isystem /usr/lib/gcc/x86_64-pc-linux-gnu/4.5.1/include -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/lwip-x86_32/src/include -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/lwip-x86_32/src/include/ipv4 -I/var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/include -I../xen/include -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os/../../extras/mini-os/include -D__MINIOS__ -DHAVE_LIBC -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os/../../extras/mini-os/include/posix -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os/../../tools/xenstore -D__XEN_INTERFACE_VERSION__=0x00030205  -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os/../../extras/mini-os/include/x86 -isystem /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os/../../extras/mini-os/include/x86/x86_32 -c sched.c -o /var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/mini-os-x86_32-grub/sched.o
cc1: warnings being treated as errors
netfront.c:41:8216: error: variably modified ‘tx_freelist’ at file scope
netfront.c:44:8218: error: variably modified ‘rx_buffers’ at file scope
netfront.c:45:8218: error: variably modified ‘tx_buffers’ at file scope
make[1]: *** [/var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom/mini-os-x86_32-grub/netfront.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/extras/mini-os'
make: *** [pv-grub] Error 2
make: Leaving directory `/var/tmp/portage/portage/app-emulation/xen-pvgrub-4.0.1/work/xen-4.0.1/stubdom'
emake failed
 * ERROR: app-emulation/xen-pvgrub-4.0.1 failed:
 *   compile pv-grub_x86_32 failed
 *
 * Call stack:
 *     ebuild.sh, line  56:  Called src_compile
 *   environment, line 2688:  Called die
 * The specific snippet of code:
 *       emake XEN_TARGET_ARCH="x86_32" -C stubdom pv-grub || die "compile pv-grub_x86_32 failed";


Code:
Portage 2.2.0_alpha10 (default/linux/amd64/10.0/desktop, gcc-4.5.1, glibc-2.12.1-r3, 2.6.36-zen1-bfs x86_64)
=================================================================
System uname: Linux-2.6.36-zen1-bfs-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_3800+-with-gentoo-2.0.1
Timestamp of tree: Sun, 26 Dec 2010 01:45:01 +0000
distcc 3.1 x86_64-pc-linux-gnu [disabled]
ccache version 3.1.3 [enabled]
app-shells/bash:     4.1_p9
dev-java/java-config: 2.1.11-r3
dev-lang/python:     2.6.6-r1, 2.7.1, 3.1.3
dev-util/ccache:     3.1.3
dev-util/cmake:      2.8.3-r1
sys-apps/baselayout: 2.0.1-r1
sys-apps/openrc:     0.6.8
sys-apps/sandbox:    2.4
sys-devel/autoconf:  2.13, 2.68
sys-devel/automake:  1.6.3-r1, 1.8.5-r4, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils:  2.21
sys-devel/gcc:       4.4.5, 4.5.1-r1
sys-devel/gcc-config: 1.4.1
sys-devel/libtool:   2.4-r1
sys-devel/make:      3.82
virtual/os-headers:  2.6.34 (sys-kernel/linux-headers)
Repositories: gentoo gnome zugaina rion kde-sunset kde sunrise local
ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -march=native -pipe -fomit-frame-pointer -fno-strict-overflow -fno-delete-null-pointer-checks -fno-tree-vrp -mno-align-stringops -minline-stringops-dynamically -mno-push-args -fno-ident"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config /usr/share/openvpn/easy-rsa"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo"
CXXFLAGS="-O2 -march=native -pipe -fomit-frame-pointer -fno-strict-overflow -fno-delete-null-pointer-checks -fno-tree-vrp -mno-align-stringops -minline-stringops-dynamically -mno-push-args -fno-ident"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y --keep-going"
FEATURES="assume-digests binpkg-logs buildpkg ccache distlocks fixlafiles fixpackages metadata-transfer news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
GENTOO_MIRRORS="ftp://ftp.linux.cz/pub/linux/gentoo"
LANG="en_US.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en cs ru"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp/portage"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/var/lib/layman/gnome /var/lib/layman/zugaina /var/lib/layman/rion /var/lib/layman/kde-sunset /var/lib/layman/kde /var/lib/layman/sunrise /system/portage-local"
SYNC="rsync://rsync.de.gentoo.org/gentoo-portage"
USE="3dnow 3dnowext 64bit 7zip X a52 aac acl acpi alsa amd64 avi bash-completion bitmap-fonts bluetooth branding bzip2 cairo cddb cdr cli cpudetection cracklib crypt css cups cxx dbus devmap dlloader dri dts dv dvd dvdr dvdread emboss emul-linux-x86 encode exif extensions fam fbcon ffmpeg firefox flac fortran ftp fuse gdbm gdu gif glut gnutls gpm gstreamer gtk gtk2 htmlhandbook iconv ieee1394 ipv6 jpeg kde lame lcms libg++ libnotify lm_sensors logitech-mouse mad mikmod mmx mng modules mp3 mp4 mpeg mplayer mudflap multilib musepack musicbrainz ncurses nls nptl nptlonly nvidia ogg opengl openmp oss pam pango pcre pdf pdflib perl png policykit ppds pppd python qt3 qt3support qt4 quicktime readline reflection sdl semantic-desktop session spell spl sse sse-filters sse2 sse3 ssl startup-notification svg symlink sysfs tcpd threads tiff tk truetype truetype-fonts type1-fonts udev unicode usb utf8 vcd vorbis x264 xcb xcomposite xml xorg xscreensaver xulrunner xv xvid zlib" ALSA_CARDS="intel8x0" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic auth_digest authn_anon authn_dbd authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock dbd deflate dir disk_cache env expires ext_filter file_cache filter headers ident imagemap include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_ajp proxy_balancer proxy_connect proxy_http rewrite setenvif so speling status unique_id userdir usertrack vhost_alias" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev mouse keyboard" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en cs ru" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Back to top
View user's profile Send private message
aTan
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2007
Posts: 134
Location: Czech Republic (Ukraine)

PostPosted: Sun Dec 26, 2010 11:16 pm    Post subject: Reply with quote

Switching back to gcc-4.4 fixed the error.
Back to top
View user's profile Send private message
aTan
Tux's lil' helper
Tux's lil' helper


Joined: 06 Jan 2007
Posts: 134
Location: Czech Republic (Ukraine)

PostPosted: Tue Dec 28, 2010 10:40 pm    Post subject: Reply with quote

Is it possible to use pygrub with nfsroot?
Back to top
View user's profile Send private message
newtonian
Guru
Guru


Joined: 19 Jan 2005
Posts: 465
Location: Hokkaido Japan

PostPosted: Wed Dec 29, 2010 2:04 am    Post subject: Booting from nfsroot Reply with quote

aTan wrote:
Is it possible to use pygrub with nfsroot?


The guy in this post seems to have done it in pygrub, I'd imagine pvgrub would be just about the same setup.
http://forums.citrix.com/thread.jspa?threadID=240826&tstart=45

HTH,
Back to top
View user's profile Send private message
newtonian
Guru
Guru


Joined: 19 Jan 2005
Posts: 465
Location: Hokkaido Japan

PostPosted: Wed Dec 29, 2010 2:11 am    Post subject: Reply with quote

aTan wrote:
Is it possible to use pygrub with nfsroot?


Also in pvgrub you be changing the domU config in /mnt/data/xen/configs/ on dom0

from something like this:

Code:
kernel = "/usr/lib/xen/boot/pv-grub-x86_32.gz"
extra = "(hd0,0)/grub/menu.lst"


to something like this:

Code:
kernel = "/usr/lib/xen/boot/pv-grub-x86_32.gz"
extra = "(nd)/grub/menu.lst"

You should setup the DHCP server and TFTP server correctly. Create ${tftproot}/grub/menu.lst; copy necessary files (vmlinuz, initrd, etc.).

From: http://wiki.xensource.com/xenwiki/PvGrub

I haven't tested a pvgrub nfsroot boot but I'd assume the following:
/usr/lib/xen/boot/pv-grub-x86_32.gz would be coming from the dom0 file system and the location of grub/menu.lst would be dependent on
the ip address given by the dhcp server and the /etc/exports ip to file system mapping.

Hope this helps,
Back to top
View user's profile Send private message
alexxy
Developer
Developer


Joined: 28 Jun 2005
Posts: 50
Location: Gatchina, St. Petersburg, Russia

PostPosted: Thu Apr 07, 2011 10:29 am    Post subject: Reply with quote

Thanks for pvgrub ebuilds. I made fixes for then and going to add them to main tree
_________________
Gentoo Team Russia
Working on Gentoo for iPAQ hx4700
Back to top
View user's profile Send private message
newtonian
Guru
Guru


Joined: 19 Jan 2005
Posts: 465
Location: Hokkaido Japan

PostPosted: Fri Apr 08, 2011 2:47 am    Post subject: Thanks!! Reply with quote

alexxy wrote:
Thanks for pvgrub ebuilds. I made fixes for then and going to add them to main tree


That was Peter Groben's excellent work on the ebuilds, so I can't take credit for them but great to hear they'll be added to the main tree.

Thanks!!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum