Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
raid5, cryptsetup and lvm (for booting a dom0 xen kernel)
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
nemster
n00b
n00b


Joined: 11 Jun 2004
Posts: 27

PostPosted: Sun Nov 04, 2007 12:33 pm    Post subject: raid5, cryptsetup and lvm (for booting a dom0 xen kernel) Reply with quote

ok heres a short guide how i sucessfully brought up a xen dom0 ontop of an encrypted lvm.
details which should be clear have been left out as this is clearly for people who already setup gentoo boxes.
please be familiar with the raid, lvm2, xen and cryptsetup howtos from the wiki!

i have the following partitions for my raid5: sda1 sdb1 and sdc1.
sdd1 is the usb stick which hosts /boot!

for the filesystem part:
make a raid5:
Code:
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=3 /dev/sda1 /dev/sdb1 /dev/sdc1

encrypt it:
Code:
cryptsetup --cipher twofish-lrw-benbi --key-size 384 -h wp512 luksFormat /dev/md0 /path/to/keyfile

(for twofish-lrw-benbi you need cryptsetup --version to be 1.0.5. if your on a livecd you need to download a static version or so...)
open up:
Code:
cryptsetup luksOpen --key-file /path/to/keyfile /dev/md0 md0e

make lvm setup:
Code:
pvcreate /dev/mapper/md0e
vgcreate vg0 /dev/mapper/md0e
lvcreate -L10g -ndom0 vg0
lvcreate -L2g -nswap0 vg0
mkfs.ext3 /dev/vg0/dom0
mkswap /dev/vg0/swap0


mount and install your system.
install mdadm and cryptsetup and busybox static!

my /etc/portage/package.keywords
Code:
sys-kernel/xen-sources ~amd64
app-emulation/xen ~amd64
app-emulation/xen-tools ~amd64
sys-devel/dev86 ~amd64
sys-fs/cryptsetup-luks ~amd64

build your kernel with make menuconfig.

dont forget to build raid, lvm and the appropriate encrypting algos into it!
copy vmlinuz to your boot
(i put the kernel and the initramfs onto a usb stick.)

ok now comes the tricky part. creating an initramfs.
first i did one with genkernel and then modified it. it works. however its faster and clearer to make one yourself.
its also fun and not so hard to do.
i took http://de.gentoo-wiki.com/Cryptsetup-luks_initramfs as a base.

so create the layout
Code:

mkdir /root/initramfs
cd /root/initramfs
mkdir {bin,dev,mnt,new-root,proc,sys}
mknod --mode=0660 dev/null c 1 3
mknod --mode=0600 dev/console c 5 1
cp /bin/busybox.static bin/busybox
cp /sbin/lvm bin/
cp /sbin/mdadm bin/
cp /bin/cryptsetup bin/
cd bin
ln -s busybox sh
ln -s busybox mount
ln -s lvm pvscan
ln -s lvm vgscan
ln -s lvm vgchange
ln -s busybox umount

check all bins in bin with ldd, they must obviously be static!

ok, we need an init script. so heres mine:

Code:

#!/bin/sh
export PATH=/bin
umask 0077
mount -t proc proc /proc
mount -t sysfs sysfs /sys
mount -t tmpfs tmpfs /dev
busybox --install -s
echo "mdev"
mdev -s
echo /bin/mdev > /proc/sys/kernel/hotplug

echo "waiting for usbstick"
while ! mount -n -o ro /dev/sdd1 /mnt ; do
sleep 4
#do mdev -s because i had to wait till my usb stick was recognized!
mdev -s
done
echo "ok"

# tty fix
rm /dev/tty
ln -s /dev/console /dev/tty

#put your raid together.
mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1
echo "mount encrypted raid"
cryptsetup luksOpen --key-file /mnt/keyfile /dev/md0 md0e

#do lvm stuff
/bin/vgscan --ignorelockingfailure --mknodes 2>/dev/null
/bin/vgchange -ay --ignorelockingfailure 2>/dev/null

#next time we should do this as kernel parameter ;)
mount /dev/vg0/dom0 /new-root

echo > /proc/sys/kernel/hotplug
umount -l /mnt /proc /sys /dev
exec switch_root /new-root /sbin/init



ok. set it executable:
Code:
chmod 755 /root/initramfs/init


pack the initramfs and put it into /boot:
Code:
mount /dev/sdd1 /boot
cd /root/initramfs
find . | cpio --quiet -o -H newc | gzip > /boot/initramfs-crypt


if you think your system is ready, pray a bit and then reboot ;)
have fun!

if you think there is an easier or better way to do this please post.
my idea with the key on the usb stick is, that i delet the slot after boot and make a new key which is only copied to /boot when i need a reboot!
i found it difficult because the standard genkernel initramfs makes lvscan before the cryptsetup part.

and heres my dom0 kernel config for xen-sources:
http://nopaste.nasreddine.com/69f2ab41c5.html

cheers
Back to top
View user's profile Send private message
kezyxes
n00b
n00b


Joined: 29 Apr 2005
Posts: 11
Location: Plymouth, MN

PostPosted: Sun Nov 04, 2007 7:15 pm    Post subject: Reply with quote

Amazing!

Thanks!
Back to top
View user's profile Send private message
nemster
n00b
n00b


Joined: 11 Jun 2004
Posts: 27

PostPosted: Mon Nov 05, 2007 6:42 am    Post subject: Reply with quote

next goal is to have a network on the initramfs which rsyncs the key and if its not present open up a connection for remote maintainance.
Back to top
View user's profile Send private message
likewhoa
l33t
l33t


Joined: 04 Oct 2006
Posts: 731
Location: Brooklyn, New York

PostPosted: Mon Nov 05, 2007 6:48 am    Post subject: Reply with quote

now that's keeping busybox busy.
Back to top
View user's profile Send private message
nemster
n00b
n00b


Joined: 11 Jun 2004
Posts: 27

PostPosted: Mon Nov 05, 2007 8:35 am    Post subject: Reply with quote

btw:

Code:
dd if=/dev/vg1/domU1 of=/dev/vg1/test

1124649+0 records in
1124649+0 records out
575820288 bytes (576 MB) copied, 42.9553 s, 13.4 MB/s


Code:
cat /proc/cpuinfo
processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 67
model name      : AMD Athlon(tm) 64 X2 Dual Core Processor 6000+
stepping        : 3
cpu MHz         : 3153.618
cache size      : 1024 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mtrr mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni cx16 lahf_lm cmp_legacy svm cr8_legacy
bogomips        : 6311.56
TLB size        : 1024 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc

processor       : 1
vendor_id       : AuthenticAMD
---snip, same again---



Code:
name         : twofish
driver       : twofish-x86_64


Code:
Linux alchemist 2.6.20-xen-r6 #8 SMP Sun Nov 4 17:11:55 CET 2007 x86_64 AMD Athlon(tm) 64 X2 Dual Core Processor 6000+ AuthenticAMD GNU/Linux




funny, reading seems to make no difference:

Code:

dd if=/dev/vg1/domU1 of=/dev/null
2306833+0 records in
2306832+0 records out
1181097984 bytes (1.2 GB) copied, 9.53293 s, 124 MB/s

alchemist ~ # dd if=/dev/md1 of=/dev/null
2451755+0 records in
2451755+0 records out
1255298560 bytes (1.3 GB) copied, 9.93347 s, 126 MB/s

dd if=/dev/sda2 of=/dev/null
1331937+0 records in
1331936+0 records out
681951232 bytes (682 MB) copied, 9.26358 s, 73.6 MB/s

so raid 5 gives about 50% read speed here :)
o/c cpu usage of cryptd is quite high
ill make some bonnie++ tests when i find time
Back to top
View user's profile Send private message
nemster
n00b
n00b


Joined: 11 Jun 2004
Posts: 27

PostPosted: Sun Sep 07, 2008 4:40 pm    Post subject: Reply with quote

some time has passed now.

after some research i have not found another tutorial on this.

is it now possible to do this directly with genkernel?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum