Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
ZFS as root filesystem crash [Solved]
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Mon Jul 08, 2013 10:37 am    Post subject: ZFS as root filesystem crash [Solved] Reply with quote

Hello everyone,

I am having trouble getting ZFS to work as a root filesystem, using a pool of four SSD drives. Specifically, I get a "unable to mount rootfs" type crash during boot. I hope someone can help. :)

There are a few resources I have mostly used:

https://github.com/ryao/zfs-overlay/blob/master/zfs-install
http://wiki.gentoo.org/wiki/ZFS

I used the wiki's instructions to build spl (0.6.1) and zfs (0.6.1) statically into my kernel (gentoo-sources-3.9.9).

Code:
# zpool status

Code:
  pool: rpool
 state: ONLINE
  scan: none requested
config:

        NAME                                             STATE     READ WRITE CKSUM
        rpool                                            ONLINE       0     0     0
          ata-OCZ-REVODRIVE3_MI_X2_OCZ-3ZJNZG8X9FTZRFDA  ONLINE       0     0     0
          ata-OCZ-REVODRIVE3_MI_X2_OCZ-5FRTV727B8V2KTUT  ONLINE       0     0     0
          ata-OCZ-REVODRIVE3_MI_X2_OCZ-1WF7Q3HFF49PUKIQ  ONLINE       0     0     0
          ata-OCZ-REVODRIVE3_MI_X2_OCZ-1TEZK3R3T5MY77U8  ONLINE       0     0     0

errors: No known data errors


Code:
# zfs list

Code:
NAME                     USED  AVAIL  REFER  MOUNTPOINT
rpool                   5.57G   213G   136K  none
rpool/GENTOO            3.81G   213G   136K  none
rpool/GENTOO/build-dir   136K   213G   136K  /mnt/gentoo//var/tmp/portage
rpool/GENTOO/distfiles  2.91G   213G  2.91G  /mnt/gentoo//usr/portage/distfiles
rpool/GENTOO/portage     931M   213G   931M  /mnt/gentoo//usr/portage
rpool/HOME               420K   213G   144K  /mnt/gentoo//home
rpool/HOME/root          140K   213G   140K  /mnt/gentoo//root
rpool/ROOT              1.76G   213G   136K  none
rpool/ROOT/gentoo       1.76G   213G  1.76G  /mnt/gentoo/


Code:
/boot/grub2/grub.cfg excerpt:

Code:
### BEGIN /etc/grub.d/10_linux ###
menuentry 'Gentoo GNU/Linux' --class gentoo --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-58CE-D843' {
        load_video
        insmod gzio
        insmod part_gpt
        insmod zfs
        set root='hd0,gpt1'
        if [ x$feature_platform_search_hint = xy ]; then
          search --no-floppy --fs-uuid --set=root --hint-bios=hd0,gpt1 --hint-efi=hd0,gpt1 --hint-baremetal=ahci0,gpt1  58CE-D843
        else
          search --no-floppy --fs-uuid --set=root 58CE-D843
        fi
        echo    'Loading Linux x86_64-3.9.9-gentoo ...'
        linux   /kernel-3.9.9-gentoo root=rpool/ROOT/gentoo ro
}


This is a slight modification of ryao's working grub.cfg, and I have tried many different root= options, such as root=ZFS=rpool/ROOT/gentoo.

I boot the same kernel with a non-ZFS root system just fine, and importing the zpool then works as well, as you can see from the output above. I'm just not able to boot with the ZFS pool as root fs.

Thank you for reading and I would appreciate help towards a solution. :)

--
_________________
Dive into Gentoo Linux and emerge into a new world


Last edited by Aonoa on Thu Jul 11, 2013 7:04 pm; edited 1 time in total
Back to top
View user's profile Send private message
peje
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2003
Posts: 87

PostPosted: Mon Jul 08, 2013 1:11 pm    Post subject: Reply with quote

Have you tried to force the export of rpool before rebooting? And may have a look at the mountpoints there should be an / entry.
export like this: zpool export -f rpool
cu Peje
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Mon Jul 08, 2013 1:51 pm    Post subject: Reply with quote

No, I have not tried to force the export. I will give that a try, thank you.

About the mountpoints, I know I created it with this command:
Code:
zfs create -o mountpoint=/ rpool/ROOT/gentoo


I also tried:
Code:
zfs set mountpoint=/ rpool/ROOT/gentoo


It is still listed as /mnt/gentoo, but I don't know if that is a problem? I assumed it was because of the import command I issued:
Code:
zpool import -R /mnt/gentoo rpool

_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
ryao
Developer
Developer


Joined: 27 Feb 2012
Posts: 118

PostPosted: Mon Jul 08, 2013 4:23 pm    Post subject: Re: ZFS as root filesystem crash Reply with quote

Building ZFS into your kernel does not currently permit you to boot with a ZFS rootfs because there is no way to import the pool to permit the rootfs mount to work. For now, you will need to use an initramfs. I am developing a patch that should enable the rootfs mount to work, but it is not ready yet.

With that said, root=ZFS=rpool/ROOT/gentoo is a flag for the initramfs. When the patch is available, your kernel commandline would contain something like "root=/dev/sda1 rootfstype=zfs zfs.zfs_bootfs=rpool/ROOT/gentoo". My plan is to make setting zfs.zfs_bootfs optional, although I do not have that working yet. The patch itself is also not working yet because the kernel will panic during its initialization procedure. Hopefully, I will be able to solve that problem soon.
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Mon Jul 08, 2013 4:40 pm    Post subject: Reply with quote

Aha, I was actually intending to try a initramfs soon if I hadn't solved things. Do you have any estimate on a timeframe for that patch? As in, days/weeks/months?

You have saved me a lot of trouble, ryao, thank you very much. :)
_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
peje
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2003
Posts: 87

PostPosted: Mon Jul 08, 2013 5:01 pm    Post subject: Reply with quote

@thanks ryaou for your work. I followed more or less your howto - builded spl and zfs as modules.
Now on 3 of my maschines zfs is working very well.
an zfs list looks like this:
VDR ~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 14,4G 22,3G 136K none
rpool/GENTOO 1,11G 22,3G 136K none
rpool/GENTOO/distfiles 230M 22,3G 230M /usr/portage/distfiles
rpool/GENTOO/portage 902M 22,3G 902M /usr/portage
rpool/HOME 436K 22,3G 136K /home
rpool/HOME/root 300K 22,3G 300K /root
rpool/ROOT 8,10G 22,3G 136K none
rpool/ROOT/gentoo 8,10G 22,3G 5,54G /
rpool/swap 5,16G 27,4G 9,23M -

Thats the way it should be? I had troubles that the mountpoints were like in Aonoas install /mnt/gentoo...
cu Peje
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Mon Jul 08, 2013 6:01 pm    Post subject: Reply with quote

I have made a initramfs now, and have some progress, but it fails with the following and I do not know what else to put in there. I tried entering just rpool/ROOT/gentoo, but no luck.

Code:

>> Loading modules
     :: Scanning for zfs..spl, zunicode, zavl, znvpair, zcommon, zfs loaded.
>> ZFS pool rpool already imported.
>> Determining root device...
cannot open 'rpool/ROOT/gentoo' : dataset does not exist
!! rpool/ROOT/gentoo is not a filesystem
!! Could not find the root block device in ZFS=rpool/ROOT/gentoo.


I used the following to generate the initramfs:
Code:
genkernel initramfs --no-mountboot  --zfs --bootloader=grub2

_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
peje
Tux's lil' helper
Tux's lil' helper


Joined: 11 Jan 2003
Posts: 87

PostPosted: Mon Jul 08, 2013 6:35 pm    Post subject: Reply with quote

Aona do you really have an mountpoint / in your rpool?
cu Peje
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Mon Jul 08, 2013 6:43 pm    Post subject: Reply with quote

My rpool is not at / when I've imported it to /mnt/gentoo, no.

I'm getting this error during initramfs creation, though:
Code:
cp: cannot stat ‘/etc/zfs/zdev.conf’: No such file or directory
* Could not copy file /etc/zfs/zdev.conf for ZFS


EDIT (modifying the post according to new results)
The above zdev.conf error was fixed by emerging a newer (~amd64) genkernel (3.4.47).

It is kernel panicking after saying ZFS pool already imported and rpool/ROOT/gentoo dataset doesn't exist.
_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
ryao
Developer
Developer


Joined: 27 Feb 2012
Posts: 118

PostPosted: Tue Jul 09, 2013 8:35 pm    Post subject: Reply with quote

peje wrote:
@thanks ryaou for your work. I followed more or less your howto - builded spl and zfs as modules.
Now on 3 of my maschines zfs is working very well.
an zfs list looks like this:
VDR ~ # zfs list
NAME USED AVAIL REFER MOUNTPOINT
rpool 14,4G 22,3G 136K none
rpool/GENTOO 1,11G 22,3G 136K none
rpool/GENTOO/distfiles 230M 22,3G 230M /usr/portage/distfiles
rpool/GENTOO/portage 902M 22,3G 902M /usr/portage
rpool/HOME 436K 22,3G 136K /home
rpool/HOME/root 300K 22,3G 300K /root
rpool/ROOT 8,10G 22,3G 136K none
rpool/ROOT/gentoo 8,10G 22,3G 5,54G /
rpool/swap 5,16G 27,4G 9,23M -

Thats the way it should be? I had troubles that the mountpoints were like in Aonoas install /mnt/gentoo...
cu Peje


That is fine. During the install, the pool's datasets are mounted with the prefix /mnt/gentoo. That is what -R /mnt/gentoo does.

Aonoa wrote:
I have made a initramfs now, and have some progress, but it fails with the following and I do not know what else to put in there. I tried entering just rpool/ROOT/gentoo, but no luck.

Code:

>> Loading modules
     :: Scanning for zfs..spl, zunicode, zavl, znvpair, zcommon, zfs loaded.
>> ZFS pool rpool already imported.
>> Determining root device...
cannot open 'rpool/ROOT/gentoo' : dataset does not exist
!! rpool/ROOT/gentoo is not a filesystem
!! Could not find the root block device in ZFS=rpool/ROOT/gentoo.


I used the following to generate the initramfs:
Code:
genkernel initramfs --no-mountboot  --zfs --bootloader=grub2


It sounds like the pool import was not done correctly. Type "shell" to get a debug shell. Then use the zpool command to fix things. That probably would involve zpool status, zpool export rpool and zpool import -f -N rpool. Then type control-d to continue the boot process. You might want to regenerate your initramfs afterward.
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Wed Jul 10, 2013 6:39 am    Post subject: Reply with quote

I will try your suggestions when I get home later today, ryao. I am also wondering if my problems could be related to me using disk by-id aliases like this:

Code:
 ata-OCZ-REVODRIVE3_MI_X2_OCZ-3ZJNZG8X9FTZRFDA


In my rpool, instead of /dev/sdX, could those id's be unavailable that early on in the boot process?

Thank you. :)


EDIT:

It seems my thoughts regarding by-id were correct, and I'll try genkernel-next as in this thread: https://forums.gentoo.org/viewtopic-t-962598-start-0-postdays-0-postorder-asc-highlight-zfs+initramfs.html

_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
Aonoa
Guru
Guru


Joined: 23 May 2002
Posts: 574
Location: Oslo, Norway

PostPosted: Wed Jul 10, 2013 5:48 pm    Post subject: Reply with quote

Installing genkernel-next-18 did not help, unfortunately. Besides that, the system is just giving kernel panics, and hence I don't get any shell when it's unable to properly import the pool. I also tried to re-create the zpool.cache file and initramfs again, still just kernel panic.


----
Finally
After adding --udev and --disklabel to my genkernel line (I think udev was the necessary one), the resulting initramfs gave me a successful boot. It feels really good to have made it work after all this tinkering and tons of reboots. :D
_________________
Dive into Gentoo Linux and emerge into a new world
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum