View previous topic :: View next topic |
Author |
Message |
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Tue Jan 17, 2012 1:44 pm Post subject: btrfs, raid1, rootpartition, dracut, fstab, grub and more |
|
|
Hello there..
I'm running into some troubles with using btrfs as root partition.
I did the following:
1.) Created partitions on my hdds:
Code: |
Device Boot Start End Blocks Id System
/dev/sda1 * 63 385559 192748+ 83 Linux
/dev/sda2 385560 2345489 979965 82 Linux swap / Solaris
/dev/sda3 2345490 100004624 48829567+ 83 Linux
/dev/sda4 100004625 625142447 262568911+ 83 Linux
|
Code: |
/dev/sdb1 * 63 385559 192748+ 83 Linux
/dev/sdb2 385560 2345489 979965 82 Linux swap / Solaris
/dev/sdb3 2345490 100004624 48829567+ 83 Linux
/dev/sdb4 100004625 586114703 243055039+ 83 Linux
|
sd[ab]1: boot; supposed to be a raid but I'm using only sda1 at the moment. Format: ext2
sd[ab]2: swap
sd[ab]3: root; formated with btrfs as raid1
sd[ab]4: nothing yet, want to use it as /home
2.) Formated the btrfs partitions and created raid:
Code: |
mkfs.btrfs -L navi -m raid1 -d raid1 /dev/sda3 /dev/sdb3
|
Code: |
Gentoo-2012 ~ # btrfs-show
Label: navi uuid: 264a5ae5-bf14-44df-a800-f6936991b852
Total devices 2 FS bytes used 2.42GB
devid 1 size 46.57GB used 6.78GB path /dev/sda3
devid 2 size 46.57GB used 6.76GB path /dev/sdb3
Btrfs Btrfs v0.19
|
3.) Created subvolumes:
Code: |
mount -t btrfs /dev/sda3 /mnt/btrfs
btrfsctl -S system /mnt/btrfs
btrfsctl -S var /mnt/btrfs
umount /mnt/btrfs
|
Code: |
Gentoo-2012 ~ # btrfs subvolume list /mnt/btrfs
ID 256 top level 5 path system
ID 257 top level 5 path var
|
4.) Mounted the volumes:
Code: |
mount /dev/sda3 -o subvol=system /mnt/gentoo
mkdir -p /mnt/gentoo/var
mount /dev/sda3 -o subvol=var /mnt/gentoo/var
mkdir -p /mnt/gentoo/boot
mount /dev/sda1 /mnt/gentoo/boot
|
mount output:
Code: |
Gentoo-2012 ~ # mount
[...]
/dev/sda3 on /mnt/gentoo type btrfs (rw,subvol=system)
/dev/sda1 on /mnt/gentoo/boot type ext2 (rw)
/dev/sda3 on /mnt/gentoo/var type btrfs (rw,subvol=var)
|
5.) Installed Gentoo, compiled kernel with btrfs support built in (not as module)
6.) Installed Grub on /dev/sda1
Code: |
Gentoo-2012 grub # cat grub.conf
# This is a sample grub.conf for use with Genkernel, per the Gentoo handbook
# http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=1&chap=10#doc_chap2
# If you are not using Genkernel and you need help creating this file, you
# should consult the handbook. Alternatively, consult the grub.conf.sample that
# is included with the Grub documentation.
default 0
timeout 30
#splashimage=(hd0,0)/boot/grub/splash.xpm.gz
title Navi
root (hd0,0)
kernel /boot/vmlinuz-3.2.1-gentoo root=UUID=264a5ae5-bf14-44df-a800-f6936991b852 rootflags=subvol=system video=1280x800-24@60 quiet
initrd /boot/initramfs-3.2.1-gentoo.img
# vim:ft=conf:
|
7.) Configured fstab:
Code: |
UUID=264a5ae5-bf14-44df-a800-f6936991b852 / btrfs defaults,device=/dev/sda3,device=/dev/sdb3 0 1
UUID=264a5ae5-bf14-44df-a800-f6936991b852 /var btrfs subvol=var,compression 0 0
/dev/sda1 /boot ext2 noauto,noatime 1 2
/dev/sda2 none swap sw,pri=1 0 0
/dev/sdb2 none swap sw,pri=1 0 0
|
8.) Installed dracut with btrfs useflag:
Code: |
Gentoo-2012 etc # cat dracut.conf
# Sample dracut config file
logfile=/var/log/dracut.log
fileloglvl=6
# Exact list of dracut modules to use. Modules not listed here are not going
# to be included. If you only want to add some optional modules use
# add_dracutmodules option instead.
#dracutmodules+=""
# Dracut modules to omit
#omit_dracutmodules+=""
# Dracut modules to add to the default
add_dracutmodules+="btrfs"
# additional kernel modules to the default
#add_drivers+=""
# list of kernel filesystem modules to be included in the generic initramfs
#filesystems+=""
# build initrd only to boot current hardware
#hostonly="yes"
#
# install local /etc/mdadm.conf
mdadmconf="no"
# install local /etc/lvm/lvm.conf
lvmconf="no"
# A list of fsck tools to install. If it's not specified, module's hardcoded
# default is used, currently: "umount mount /sbin/fsck* xfs_db xfs_check
# xfs_repair e2fsck jfs_fsck reiserfsck btrfsck". The installation is
# opportunistic, so non-existing tools are just ignored.
#fscks=""
# inhibit installation of any fsck tools
#nofscks="yes"
|
Code: |
dracut "" 3.2.1-gentoo
|
Code: |
Gentoo-2012 boot # ls
System.map-3.2.1-gentoo boot config-3.2.1-gentoo grub initramfs-3.2.1-gentoo.img lost+found vmlinuz-3.2.1-gentoo
|
9.) Rebooted
The reboot went flawless, the root fs was mounted as btrfs and my new install was accessable in read and write mode.
I did a second reboot and then it happened. Initramfs complained about the filesystem, that it can't be mounted because of wrong filesystem or a damaged superblock, that I should use btrfsck to check the volume and found myself in the rescue shell.
I've rebooted into LiveDVD, checked the volume and everything seemed to be fine. I was able to mount it, to alter the data on it, btrfsck didn't come up with an error msg. Everything was cute and snuggly, no data were missing and the partition worked just fine.
I've rebooted again in the hope that everything would work again, but no. Nothing at all worked. Initramfs came up with the same error messages as above.
And again reboot into LiveDVD. I've used rsync to backup my new gentoo to an usb drive, killed all partitions and redid all the steps above beside the gentoo install, I've just rsynced it back.
Everything worked again after rebooting into my system. Initramfs mounted the btrfs volumes without complainig and I had read and write access to my gentoo again.
But.. after a second reboot.. yuck.. initramfs came up with the same errors about wrong fs, superblock, not able to mount, rescue shell and all that stuff.
The partitions are again clean and accessable after all. I have them mounted here in the LiveDVD at the moment.
Now the big question. What could be the problem? I'm really out of clues right now and could need some suggestions. ^^
bb Lia |
|
Back to top |
|
|
massimo Veteran
Joined: 22 Jun 2003 Posts: 1226
|
Posted: Wed Jan 18, 2012 9:50 am Post subject: |
|
|
I can only see one tiny thing which will probably not solve you issue but since you use an initramfs you should not have to give each device in the raid in the corresponding fstab entry, hence this should also work:
Code: |
UUID=264a5ae5-bf14-44df-a800-f6936991b852 / btrfs defaults 0 1
|
_________________ Hello 911? How are you? |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Wed Jan 18, 2012 10:03 am Post subject: |
|
|
Hmm.. does that mean that I don't need an initramfs at all when I add the devices into fstab? |
|
Back to top |
|
|
massimo Veteran
Joined: 22 Jun 2003 Posts: 1226
|
Posted: Wed Jan 18, 2012 10:16 am Post subject: |
|
|
That worked for but I don't use btrfs for /. _________________ Hello 911? How are you? |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Wed Jan 18, 2012 10:30 am Post subject: |
|
|
I've just tested it and the kernel came up with this error:
Code: |
Root-NFS: no NFS server address
VFS: unable to mount root fs via NFS, trying floppy.
Kernel Panic - not syncing: VFS unable to mount root fs on unknown-block(2,0)
|
Why does my kernel try to mount root via NFS? Huh? o.O |
|
Back to top |
|
|
massimo Veteran
Joined: 22 Jun 2003 Posts: 1226
|
Posted: Wed Jan 18, 2012 11:48 am Post subject: |
|
|
Is btrfs compiled into your kernel or as module? _________________ Hello 911? How are you? |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Wed Jan 18, 2012 12:10 pm Post subject: |
|
|
It is compiled into it and not as module.
Code: |
Gentoo-2012 linux # cat .config | grep BTRFS
CONFIG_BTRFS_FS=y
# CONFIG_BTRFS_FS_POSIX_ACL is not set
|
|
|
Back to top |
|
|
massimo Veteran
Joined: 22 Jun 2003 Posts: 1226
|
Posted: Wed Jan 18, 2012 6:34 pm Post subject: |
|
|
What if you do not us UUID for identifying the partition in both places grub.conf and fstab. _________________ Hello 911? How are you? |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Wed Jan 18, 2012 6:37 pm Post subject: |
|
|
I've tested that already. I actually came up with the UUIDs because I thought addressing the device with /dev/sda3 would be the issue. |
|
Back to top |
|
|
gimpel Advocate
Joined: 15 Oct 2004 Posts: 2720 Location: Munich, Bavaria
|
Posted: Sat Jan 21, 2012 11:07 pm Post subject: |
|
|
Seems you installed into a subvolume, but your fstab entry for / doesn't have a subvol=system option set.
Not sure if it matters actually. Passing the correct rootflags to grub should be sufficent. _________________ http://proaudio.tuxfamily.org/wiki - pro-audio software overlay
Last edited by gimpel on Sat Jan 21, 2012 11:21 pm; edited 1 time in total |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Sat Jan 21, 2012 11:19 pm Post subject: |
|
|
Well no, subvol isn't needed in fstab for root. The subvol for root is managed by boot parameter in grub.
But I guess I know the issue. The subvol for /var is busy during shutdown and not clearly unmounted.
I'll test tomorrow if it will work when i have /var on the system subvol. |
|
Back to top |
|
|
massimo Veteran
Joined: 22 Jun 2003 Posts: 1226
|
Posted: Sun Jan 22, 2012 5:52 pm Post subject: |
|
|
I've got /var on another subvol without having any issues. _________________ Hello 911? How are you? |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Tue Jan 24, 2012 2:07 pm Post subject: |
|
|
Okies so /var is working as subvol. I've killed the raid1 for now and installed everything on sda only. Maybe there's some hardware issue with sdb, dunno.
I'm going to do some hardware tests after everything is up and running on sda.
Reboots are working without raid though. |
|
Back to top |
|
|
renkinjutsu n00b
Joined: 28 Jul 2010 Posts: 28
|
Posted: Wed Jan 25, 2012 8:06 pm Post subject: |
|
|
Based on some Fedora bugs, dracut doesn't seem to handle multi-device btrfs volumes correctly. The thing is that if you have a multi-device btrfs volume, you have to make the system aware of it before mounting to boot.
This is done by scanning for btrfs volumes
Code: | btrfs device scan <device> |
You won't need to specify the device to scan. You can simply say btrfs device scan without arguments and it'll scan every node in /dev, but this will take longer.
Here's instructions how to make a rule in dracut to handle this (written for fedora though)
http://happy.penguins.tw/?p=142 |
|
Back to top |
|
|
lialee n00b
Joined: 17 Jan 2012 Posts: 8 Location: Germany
|
Posted: Thu Jan 26, 2012 10:25 am Post subject: |
|
|
Hmm.. btrfs comes with a similar file already.
/usr/lib/dracut/modules.d/90btrfs/btrfs_timeout.sh:
Code: |
#!/bin/bash
# -*- mode: shell-script; indent-tabs-mode: nil; sh-basic-offset: 4; -*-
# ex: ts=8 sw=4 sts=4 et filetype=sh
type getarg >/dev/null 2>&1 || . /lib/dracut-lib.sh
info "Scanning for all btrfs devices"
/sbin/btrfs device scan 2>&1 | vinfo
exit 0
|
That was also executed on boot as far as i recall.
I guess the name of the file doesn't matter, does it? o.O |
|
Back to top |
|
|
drhirsch n00b
Joined: 08 May 2004 Posts: 66 Location: Germany
|
Posted: Sat Feb 04, 2012 7:03 pm Post subject: |
|
|
Since my new backup hard drive arrived, I thougt I'd give btrfs and its builtin raid abilities a shot.
Setup was similar to yours, I didn't use subvolumes and instead some more partitions. /root should live on a raid1 btrfs. For additional spice I decided to use the new GPT partitions to boot as well, which lead to some minor fighting with grub2. Basically I went to the same trouble - mounting, copy, working fine, some additional trouble with grub claiming to not find itself, but after a successful boot I got "kernel panic - unable to mount root".
I seems it is a known bug that a btrfs raid can't always be mounted giving only the UUID or one of the device names. Further testing revealed that I could for example not use /dev/sdc2 for a successful mount, where as /dev/sdd2 worked - for the next partition sdd3 worked but sdc3 did not. So it seems a successful mount to depend on somewhat random conditions (The partitions were created in strictly the same order, and the mkfs.btrfs always used the same device order too).
Googling told me that a scan is required, exactly as in the dracut btrfs module. So I installed dracut using this thread and it worked for me
So I can't tell you why your kernel still refuses to mount, but at least for me your ideas were helpful |
|
Back to top |
|
|
kingfame_147 Apprentice
Joined: 11 Oct 2008 Posts: 171
|
Posted: Mon Feb 06, 2012 6:19 pm Post subject: |
|
|
renkinjutsu wrote: | Based on some Fedora bugs, dracut doesn't seem to handle multi-device btrfs volumes correctly. The thing is that if you have a multi-device btrfs volume, you have to make the system aware of it before mounting to boot.
This is done by scanning for btrfs volumes
Code: | btrfs device scan <device> |
You won't need to specify the device to scan. You can simply say btrfs device scan without arguments and it'll scan every node in /dev, but this will take longer.
Here's instructions how to make a rule in dracut to handle this (written for fedora though)
http://happy.penguins.tw/?p=142 |
Like lialee allready pointed out dracut has a btrfs module for this. I'm using btrfs for / and have subvolumes for /etc, /var and /root. My problem is that /etc needs to be mounted before the initram starts the normal boot. So I have to use the fstab.sys from dracut to mount my /etc subvolume. This only works if I mount it "ro", because otherwise it complains that /sysroot/etc is busy. But then gentoo doesn't mount /etc as rw during the "real" boot. I haven't found a solution for that other then writing a dracut module like "mount -o rw,remount /etc, /var, /root" which is an ugly workaround. If anyone has a solution for that please let me know
B2T:
That is strange. Have you tried to mount the device by hand in the rescue shell? Does it work? If not: Have you tried to do a "btrfs device scan" before, just to be sure. Have you tried to mount it with "-o degraded" in case dracut has a problem with one of your devices? I did quite a bit of research 2 weeks ago about dracut, btrfs and gentoo. I'm happy to help if you have any questions to clear things up. Please try my hints above and let me know if one of them helped.
@drhirsch: I've never had any problems mounting btrfs raid devices with only one device given. I can use UUID, label, or one of the /dev devices and it works for years. You should def. send this bug to the mailing list with some more information about that |
|
Back to top |
|
|
drhirsch n00b
Joined: 08 May 2004 Posts: 66 Location: Germany
|
Posted: Tue Feb 07, 2012 12:26 am Post subject: |
|
|
Mounting works for you because your system already knows of the multi volume btrfs partitions. Possibly your system has done a `btrfs device scan` or similar before.
Try to boot a multivolume btrfs at / without an initramfs, just with specifying a "root=/dev/sdxx" or "root=UUID=..." at the kernel command line.
A successful mount may depend on the relative humidity or the moon phase, and afaik this bug is known for at least a year.
And before you ask - of course the btrfs module is builtin in the kernel |
|
Back to top |
|
|
kingfame_147 Apprentice
Joined: 11 Oct 2008 Posts: 171
|
Posted: Tue Feb 07, 2012 8:29 am Post subject: |
|
|
drhirsch wrote: | Mounting works for you because your system already knows of the multi volume btrfs partitions. Possibly your system has done a `btrfs device scan` or similar before.
Try to boot a multivolume btrfs at / without an initramfs, just with specifying a "root=/dev/sdxx" or "root=UUID=..." at the kernel command line.
A successful mount may depend on the relative humidity or the moon phase, and afaik this bug is known for at least a year.
And before you ask - of course the btrfs module is builtin in the kernel :-) |
I'm well aware of the "btrfs device scan" command: I'm using btrfs for 2 years with own created initramfs, dracut etc.
Could you please point me to this bug? I'm subscribed to the btrfs mailing list and can't remember a single user complaining about any problems with that, except for users complaining about this "bug" because they forgot to "btrfs device scan". |
|
Back to top |
|
|
drhirsch n00b
Joined: 08 May 2004 Posts: 66 Location: Germany
|
Posted: Tue Feb 07, 2012 10:19 pm Post subject: |
|
|
As I wrote - if you boot without a ramdisk, you have not "btrfs device scan" available, the kernel has to mount the root device given in the command line. This fails. I still consider it a bug. |
|
Back to top |
|
|
kingfame_147 Apprentice
Joined: 11 Oct 2008 Posts: 171
|
Posted: Tue Feb 07, 2012 10:27 pm Post subject: |
|
|
No, thats not a bug at all :) That's a simple design decision.
You can still put all devices of the mult. disc btrfs into the fstab and then you don't need the "btrfs device scan". How should your system now which devices belong to a mult. disc btrfs without running such a command? Sure, you could do something like "Scan all devices automatically while mounting", but I like the manual approach more.
This "issue" is far far away from being a bug. You can criticize this approach and discuss about it. I mean if you start your X but haven't loaded e.g. the radeon module your X won't start. That's no bug, that's just the way it works :)
If btrfs will have a fsck and more distributions will use it I'm pretty sure that all distributions will do the scan command for you. |
|
Back to top |
|
|
drhirsch n00b
Joined: 08 May 2004 Posts: 66 Location: Germany
|
Posted: Wed Feb 08, 2012 8:30 am Post subject: |
|
|
It consider it a bug, because it is not possible to mount a multi device root btrfs with the kernel alone. At this time you don't have neither /sbin/btrfs nor /etc/fstab - yet every other fs and his sister can be mounted, but not btrfs multidevice.
For instance a md0 raid mounts without any hassle - the kernel simply scans all possible md devices and even tries permutations of them, in the case they have been swapped.
So for every file system out there the design decision was to make it able to mount it with the kernel alone - only for btrfs multi device it is different. So let me guess - it isn't a feature - it's a bug
PS: On the btrfs wiki at https://btrfs.wiki.kernel.org/articles/u/s/i/Using_Btrfs_with_Multiple_Devices_9854.html it is said "Once you create a multi-device filesystem, you can use any device in the FS for the mount command" - this is misleading. Some lines later the "btrfs device scan" is mentioned, but depending on the size of the screen of the mobile you used to read the wiki, you may not get that far
Last edited by drhirsch on Wed Feb 08, 2012 9:01 am; edited 1 time in total |
|
Back to top |
|
|
frostschutz Advocate
Joined: 22 Feb 2005 Posts: 2977 Location: Germany
|
Posted: Wed Feb 08, 2012 8:40 am Post subject: |
|
|
drhirsch wrote: | For instance a md0 raid mounts without any hassle - the kernel simply scans all possible md devices and even tries permutations of them |
Actually that's one piece of functionality the kernel devs want to get rid of.
It only works for old md metadata 0.90 anyway, for 1.x you must use initramfs to assemble the raid.
The general rule is that for multi device and other complex setups (LVM, encryption, etc) you just have to have an initramfs figuring things out for you. |
|
Back to top |
|
|
kingfame_147 Apprentice
Joined: 11 Oct 2008 Posts: 171
|
Posted: Wed Feb 08, 2012 8:42 am Post subject: |
|
|
Sure you can mount any multi device btrfs if you tell the mount cmd all devices belonging to the filesystem. Something like
Code: | mount -o device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde /dev/sdb /mnt |
This works at any time when you have the mount command available. E.g. you can put that into your grub rootflags like
Code: | rootflags=subvol=@root,device=/dev/sdb,device=/dev/sdc,device=/dev/sdd,device=/dev/sde |
No bug at all, like I said before :) Ofc this isn't as handy as just mount /dev/sdb, but then you need the "btrfs device scan". |
|
Back to top |
|
|
drhirsch n00b
Joined: 08 May 2004 Posts: 66 Location: Germany
|
Posted: Wed Feb 08, 2012 9:15 am Post subject: |
|
|
I wasn't aware of the rootflags option, my fault. So it is possible to mount something without a ramdisk - OK, is't not a bug, it's a errr design decision |
|
Back to top |
|
|
|