Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Native ZFS root tutorials
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
EatMeerkats
Apprentice
Apprentice


Joined: 15 Mar 2006
Posts: 234

PostPosted: Wed May 23, 2012 11:41 pm    Post subject: Reply with quote

There's a good guide for ZFS root at https://github.com/ryao/zfs-overlay/blob/master/zfs-install
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Thu May 24, 2012 7:05 am    Post subject: Reply with quote

kernelOfTruth wrote:
disi wrote:
Hi Pendor,

thanks for the guide :)

I tried to follow your guide yesterday, but got stuck with grub2-mkconfig, which refused to run the drive probes, because it couldn't load the zfs module (insmod fails with the kernel on the cd).
I guess the reason was, that I used zfs-fuse of a systemrescuecd. Do you know any boot media with zfsonlinux on it or which one do you use to install?

thanks


afaik the Gentoo live-DVD (12.1) has zfsonlinux included


Thanks you two, it's funny that the last thing I thought of was to use a Gentoo Live CD to install Gentoo :lol:
Pendor provides a repository to build yourself an Ubuntu Live CD with zfsonlinux as well:
https://github.com/dajhorn/pkg-zfs/wiki/HOWTO-install-Ubuntu-to-a-Native-ZFS-Root-Filesystem

Setting up the chroot environment is actual straight forward, I still struggle with grub a little.
Also had a lot of compile problems -.- like:
zfs-9999 fails because spl-9999 doesn't build against kernel-3.4.0, grub-9999 fails to build because autopoint fails, which means rebuild gettext, this only builds on the livecd I used with the 'path=' trick etc. etc.

Anyway, this is a good experience :)
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 04, 2012 6:18 am    Post subject: Reply with quote

kernelOfTruth wrote:
afaik the Gentoo live-DVD (12.1) has zfsonlinux included


That version is old has issues that have since been fixed in the upstream GIT, but it is sufficient for doing an install. :)

EatMeerkats wrote:
There's a good guide for ZFS root at https://github.com/ryao/zfs-overlay/blob/master/zfs-install


You found my notes on how to do a ZFS install. I plan to turn those into official documentation at some point, but for now, I am waiting until I solve some lingering issues.

Note that overlay is where I develop things that is meant to go into portage in the future, which is why it is not in the official overlay list. You can find bits and pieces of the FreeBSD bootloader port that I am doing. Until the port of FreeBSD's bootloader is in the main tree, people should not touch it. My overlay's sys-fs/zfs and sys-kernel/spl ebuilds are relatively safe. Those ebuilds pull from my GIT forks of the upstream code

disi wrote:
zfs-9999 fails because spl-9999 doesn't build against kernel-3.4.0, grub-9999 fails to build because autopoint fails, which means rebuild gettext, this only builds on the livecd I used with the 'path=' trick etc. etc.


There is a patch floating around that can enable it to build, but it doesn't meet upstream's coding standards. I plan to fix this, but I can't promise it within any particular time frame.
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Sat Jun 23, 2012 11:40 pm    Post subject: Reply with quote

I just followrd the guide step by step and got stuck at
Code:
(chroot) Gentoo-2012 ~ # grub2-install /dev/sda
Path `/boot/grub2' is not readable by GRUB on boot. Installation is impossible. Aborting.

So I try again with a small (|100MB) /boot partition...
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Sun Jun 24, 2012 2:58 pm    Post subject: Reply with quote

disi wrote:
I just followrd the guide step by step and got stuck at
Code:
(chroot) Gentoo-2012 ~ # grub2-install /dev/sda
Path `/boot/grub2' is not readable by GRUB on boot. Installation is impossible. Aborting.

So I try again with a small (|100MB) /boot partition...


GRUB2 is somewhat temperamental, which is one of the reasons why I want to move away from depending on it. Make sure that GRUB2 installed its files in /boot/grub2.
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Sun Jun 24, 2012 5:17 pm    Post subject: Reply with quote

I give up for now.

It worked on a virtual machine in VirtualBox using your guide but a 100MB extra partition for /boot.

On the actual hardware, I cannot get grub to mount the root filesystem (testing the latest not git and -9999, the same with genkernel)

Here are the options I played around with:
Code:
[insmod zfs]
[boot=zfs] [dozfs] root=ZFS[=rpool/ROOT/gentoo] [real_root=ZFS[=rpool/ROOT/gentoo]] [ro]


What I just remember, I used a different kernel config than in the vm... are there any kernel configs it needs?

//edit: all right, do not try to have /etc as an extra volume :x and there must be some kernel option missing in my custom config, genkernel boots now...
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 25, 2012 4:56 am    Post subject: Reply with quote

disi wrote:
I give up for now.

It worked on a virtual machine in VirtualBox using your guide but a 100MB extra partition for /boot.

On the actual hardware, I cannot get grub to mount the root filesystem (testing the latest not git and -9999, the same with genkernel)

Here are the options I played around with:
Code:
[insmod zfs]
[boot=zfs] [dozfs] root=ZFS[=rpool/ROOT/gentoo] [real_root=ZFS[=rpool/ROOT/gentoo]] [ro]


What I just remember, I used a different kernel config than in the vm... are there any kernel configs it needs?

//edit: all right, do not try to have /etc as an extra volume :x and there must be some kernel option missing in my custom config, genkernel boots now...


If you find a missing kernel option, please let me know. I try to be as thorough as possible with the config checks.
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Mon Jun 25, 2012 6:54 am    Post subject: Reply with quote

I suggest to give a working grub.cfg as example. For me, with an extra 100MB boot partition, it looks like this (I assume it needs an insmod or something if you have /boot in the pool?):
Code:
linux   /kernel root=ZFS=rpool/ROOT/gentoo


In my kernel config was no support for initramfs :roll: So the error was always unknown block and cannot mount root fs, set a proper 'root=' kernel option, this was kind of misleading.

Then it was loading everthing, but came up with no more services left in this runlevel and asking for the Runlevel. This was because I had /etc/ on an extra volume, this could be worked around if we do not set the pool to legacy mountpoint. Anyway I moved the files back onto the root volume and works fine now.

One more thing I noticed, if you do not install anything from upstream before running genkernel for the first time, you are missing the tools to compress the initramfs. They are pulled in when you install e.g. zfs-9999. I went manually into the genkernel.conf and told it to use gzip as method.
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Jun 25, 2012 9:25 pm    Post subject: Reply with quote

disi wrote:
I suggest to give a working grub.cfg as example. For me, with an extra 100MB boot partition, it looks like this (I assume it needs an insmod or something if you have /boot in the pool?):
Code:
linux   /kernel root=ZFS=rpool/ROOT/gentoo


That file is a bit big to reproduce here. I have posted it to github:

https://gist.github.com/2991337

disi wrote:
In my kernel config was no support for initramfs :roll: So the error was always unknown block and cannot mount root fs, set a proper 'root=' kernel option, this was kind of misleading.


I have committed a check for this. It should be on the mirrors (along with sys-fs/zfs-0.6.0_rc9-r3) soon.

disi wrote:
Then it was loading everthing, but came up with no more services left in this runlevel and asking for the Runlevel. This was because I had /etc/ on an extra volume, this could be worked around if we do not set the pool to legacy mountpoint. Anyway I moved the files back onto the root volume and works fine now.


The system should not be able to boot with /etc on a different mount point because /etc/fstab contains the information needed to mount /etc. ZFS datasets are an exception to this, provided that you do not use mountpoint=legacy.

disi wrote:
IOne more thing I noticed, if you do not install anything from upstream before running genkernel for the first time, you are missing the tools to compress the initramfs. They are pulled in when you install e.g. zfs-9999. I went manually into the genkernel.conf and told it to use gzip as method.


Which tools? Is this cpio?
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Tue Jun 26, 2012 8:26 am    Post subject: Reply with quote

ryao wrote:
Which tools? Is this cpio?


Good question, default is compression method 'best' for genkernel, I think. Whatever is used there and the error is kind of:
'' command not found, please install '' (original with those empty quotes)

if I set it to no compression or gzip it works.

I don't think its cpio, because compression comes afterwards?

So it doesn't tell you what is missing, if you emerge something from upstream there must be some dependency which makes genkernel compression to work. This happens using the latest stage3 tarball.

//edit:
Technically genkernel-3.4.36 enables all compression methods for initramfs per default in the kernel config and doesn't check if the program is installed (e.g. on a stage3)?
Code:
*--compress-initramfs-type*=<arg>::
   Compression type for initramfs (best, xz, lzma, bzip2, gzip, lzop).
   "Best" causes selection of the best available algorithm that is selected
   in your kernel configuration.


Or it sets a blank compression method, but wants to compress. This would explain the empty quotes in the error message...
Code:
[[ -z "${!cmd_variable_name}" ]] && gen_die "Compression '${compression}' is not available. Please install package '${!pkg_variable_name}'."


So we get something, if COMPRESS_INITRD_TYPE is set to anything useful:
Code:
         local compression
         case ${COMPRESS_INITRD_TYPE} in
            xz|lzma|bzip2|gzip|lzop) compression=${COMPRESS_INITRD_TYPE} ;;
            lzo) compression=lzop ;;

Otherwise it checks for 'best' or 'fastest', checks what is enabled in the kernel, starts with xz and breaks the loop if set to best.
So xz it is as best (default) and lzop is fastest... but here must be something going wrong because compression seems empty if xz-utils is not part of stage3 tarballs (not installed)?

Code:
               for tuple in \
                     'CONFIG_RD_XZ    cmd_xz    xz' \
                     'CONFIG_RD_LZMA  cmd_lzma  lzma' \
                     'CONFIG_RD_BZIP2 cmd_bzip2 bzip' \
                     'CONFIG_RD_GZIP  cmd_gzip  gzip' \
                     'CONFIG_RD_LZO   cmd_lzop  lzop'; do
                  set -- ${tuple}
                  kernel_option=$1
                  cmd_variable_name=$2
                  if grep -sq "^${kernel_option}=y" ${KERNEL_DIR}/.config && test -n "${!cmd_variable_name}" ; then
                     compression=$3
                     [[ ${COMPRESS_INITRD_TYPE} == best ]] && break
                  fi
               done


This
Code:
test -n "${!cmd_variable_name}"
is only true, if I have e.g. '/bin/xz' in the history of bash? So it loops through and compression stays empty...?

Or this is all stupid :) And I missed something...
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Fri Jun 29, 2012 5:16 am    Post subject: Reply with quote

disi wrote:
ryao wrote:
Which tools? Is this cpio?


Good question, default is compression method 'best' for genkernel, I think. Whatever is used there and the error is kind of:
'' command not found, please install '' (original with those empty quotes)

if I set it to no compression or gzip it works.

I don't think its cpio, because compression comes afterwards?

So it doesn't tell you what is missing, if you emerge something from upstream there must be some dependency which makes genkernel compression to work. This happens using the latest stage3 tarball.

//edit:
Technically genkernel-3.4.36 enables all compression methods for initramfs per default in the kernel config and doesn't check if the program is installed (e.g. on a stage3)?
Code:
*--compress-initramfs-type*=<arg>::
   Compression type for initramfs (best, xz, lzma, bzip2, gzip, lzop).
   "Best" causes selection of the best available algorithm that is selected
   in your kernel configuration.


Or it sets a blank compression method, but wants to compress. This would explain the empty quotes in the error message...
Code:
[[ -z "${!cmd_variable_name}" ]] && gen_die "Compression '${compression}' is not available. Please install package '${!pkg_variable_name}'."


So we get something, if COMPRESS_INITRD_TYPE is set to anything useful:
Code:
         local compression
         case ${COMPRESS_INITRD_TYPE} in
            xz|lzma|bzip2|gzip|lzop) compression=${COMPRESS_INITRD_TYPE} ;;
            lzo) compression=lzop ;;

Otherwise it checks for 'best' or 'fastest', checks what is enabled in the kernel, starts with xz and breaks the loop if set to best.
So xz it is as best (default) and lzop is fastest... but here must be something going wrong because compression seems empty if xz-utils is not part of stage3 tarballs (not installed)?

Code:
               for tuple in \
                     'CONFIG_RD_XZ    cmd_xz    xz' \
                     'CONFIG_RD_LZMA  cmd_lzma  lzma' \
                     'CONFIG_RD_BZIP2 cmd_bzip2 bzip' \
                     'CONFIG_RD_GZIP  cmd_gzip  gzip' \
                     'CONFIG_RD_LZO   cmd_lzop  lzop'; do
                  set -- ${tuple}
                  kernel_option=$1
                  cmd_variable_name=$2
                  if grep -sq "^${kernel_option}=y" ${KERNEL_DIR}/.config && test -n "${!cmd_variable_name}" ; then
                     compression=$3
                     [[ ${COMPRESS_INITRD_TYPE} == best ]] && break
                  fi
               done


This
Code:
test -n "${!cmd_variable_name}"
is only true, if I have e.g. '/bin/xz' in the history of bash? So it loops through and compression stays empty...?

Or this is all stupid :) And I missed something...


This is mostly correct. The only error is your interpretation of ${!cmd_variable_name}. It says to look for a variable that is named the value of cmd_variable_name and take the value of it. As an example, `var=value name=var;echo "${!name}"` will output `value`.

This is a minor issue and I am somewhat inundated with more pressing concerns right now. Would you file a bug report in the Gentoo bug tracker?
Back to top
View user's profile Send private message
disi
Veteran
Veteran


Joined: 28 Nov 2003
Posts: 1354
Location: Out There ...

PostPosted: Fri Jun 29, 2012 8:04 am    Post subject: Reply with quote

ryao wrote:

This is a minor issue and I am somewhat inundated with more pressing concerns right now. Would you file a bug report in the Gentoo bug tracker?


Already done... did you look into the other thread? I was too drunk yesterday (Germany is out of the cup :( ) and forgot to check for reloading the module, if possible at all.
_________________
Gentoo on Uptime Project - Larry is a cow
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Fri Jun 29, 2012 6:24 pm    Post subject: Reply with quote

disi wrote:
ryao wrote:

This is a minor issue and I am somewhat inundated with more pressing concerns right now. Would you file a bug report in the Gentoo bug tracker?


Already done... did you look into the other thread? I was too drunk yesterday (Germany is out of the cup :( ) and forgot to check for reloading the module, if possible at all.


The short answer is that it is not possible to reload the module when using a ZFS rootfs. The long answer is that it is possible to hack something together to use a tmpfs and switch_root/pivot_root to do it. It would be similar to the zfs-shutdown and linuxrc scripts in portage's ./sys-fs/zfs/files directory. I designed those to unmount the rootfs and export its pool at shutdown. In theory, they could be adapted to do what you want, but doing that would be time consuming and many would find it to be challenging.

I think it would be better to focus on implementing the ability to set these options in genkernel. If I were to dedicate an afternoon for it, I would definitely get it working. The only issue is making time. There are literally a few dozen different things on my to do list at the moment, of which this is one of them.

Quite frankly, the best thing would be to modify the kernel code to permit this setting to be changed without having to reload the module. I do not think I will have time to do it this year and I do not know of anyone who does.
Back to top
View user's profile Send private message
yngwin
Retired Dev
Retired Dev


Joined: 19 Dec 2002
Posts: 4572
Location: Suzhou, China

PostPosted: Sun Jul 01, 2012 5:14 am    Post subject: Reply with quote

As I just said on the other thread:

On the Gentoo Wiki there is a request for a Gentoo on ZFS article. That would be a good place to get all the relevant documentation in one place. If you guys could get that started, that would be grand... I am available for help on editing / wikifying (among other places in the #gentoo-wiki channel).
_________________
"Those who deny freedom to others deserve it not for themselves." - Abraham Lincoln
Free Culture | Defective by Design | EFF
Back to top
View user's profile Send private message
Tyler_Durden
Apprentice
Apprentice


Joined: 27 Jul 2004
Posts: 189
Location: Germany

PostPosted: Sat Aug 04, 2012 8:06 pm    Post subject: Reply with quote

ryao wrote:

EatMeerkats wrote:
There's a good guide for ZFS root at https://github.com/ryao/zfs-overlay/blob/master/zfs-install


You found my notes on how to do a ZFS install. I plan to turn those into official documentation at some point, but for now, I am waiting until I solve some lingering issues.



Really a nice guide to play with zfs. Thank you for this!

BUT, i had to find out the very hard way, that grep 2.13 can be really an unpleasant showstopper half the way building up a new system on zfs 8O

And all my new zpools had always failed to boot the first time with some dataset error. A new import/export of the pool in busybox was the solution.
_________________
Gentoo 17.1 x86_64
Intel Core i9-9900K
Asus MAXIMUS XI HERO
AMD Radeon 6800XT
64GB DDR4
Samsung SSD 970 EVO Plus 1TB
8x Seagate Archive (SATA-RAID 64TB)
Digital Devices Cine S2 V6.5 DVB Adapter
Back to top
View user's profile Send private message
tuxhat
n00b
n00b


Joined: 31 Aug 2012
Posts: 1

PostPosted: Fri Aug 31, 2012 9:52 am    Post subject: Reply with quote

EatMeerkats wrote:
There's a good guide for ZFS root at https://github.com/ryao/zfs-overlay/blob/master/zfs-install

Yes this guide is very nice, but one problem is that when I do a genkernel it says "warning no /boot" and i think it fails in the end.

I like this guide, but it sort of confuses me. I want to put this on a empty SATA 500gb, I want to use ZFS ... I want to use the unstable packages, i want to also use grub legacy instead of grub2 and also it would be nice to have a straight forward guide, though i really like this i get sort of a error when like trying to rc-update add zfs boot. but honestly i will give this a try again, is it possible if i have my gentoo system on another hdd which i do, can i use that system to do this zfs installation on my SATA HDD drive which is sdb and my ide 250gb gentoo system is sda .. can i put a zfs installation using my gentoo system . ? its using stable.. can i turn that system to a unstable sysem.. yes i'm not the professional at gentoo since it will be my second time on host.. but this time i want to do it perfect and right and i want to use zfs filesystem really bad.

If you can , give me some tips on what i should do, and how i should do this all correctly, NOTE i do have acknowledgement on setting up gentoo and everything using their official guide i setup gentoo 4 times 3 times in Virtualbox and one time on a acutal host... i'm just sort of stuck at the /boot with genkernel and the rc-update add zfs boot.. other than that its pretty straight forward.

is it possible i can make a /boot before i jump to the genkernel and just mount it ?

all the help would be cool eh,

Thanks.
Tux Hat
Back to top
View user's profile Send private message
Raffi
l33t
l33t


Joined: 17 Mar 2003
Posts: 731
Location: Moscow, Id.

PostPosted: Fri Mar 15, 2013 4:36 pm    Post subject: Reply with quote

I've converted a couple of machines over to running full zfs systems booting via grub2. While the guides pointed to in this thread were helpful, I did do things a bit different. While I'm pretty sure what I did is OK, I thought I'd run it by other's who have been working with this stuff.

The first difference in how I have done things is in using my own custom initrd. I did that since I had been using my own custom lvm script for a while and it was easier for me to modify it than it was to learn a new initrd system. The second difference is in not creating another gpt boot partition. From what I read, the partition 9 that zfs creates when you give it the entire disk is/was probably used by Solaris for booting. Since it was already there and of sufficient size, I simply changed the partition type so that grub would use it. So far, this seems to be working well and I'm hoping that I won't run into trouble at some later time.
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Mar 18, 2013 7:10 pm    Post subject: Reply with quote

tuxhat wrote:
EatMeerkats wrote:
There's a good guide for ZFS root at https://github.com/ryao/zfs-overlay/blob/master/zfs-install

Yes this guide is very nice, but one problem is that when I do a genkernel it says "warning no /boot" and i think it fails in the end.

I like this guide, but it sort of confuses me. I want to put this on a empty SATA 500gb, I want to use ZFS ... I want to use the unstable packages, i want to also use grub legacy instead of grub2 and also it would be nice to have a straight forward guide, though i really like this i get sort of a error when like trying to rc-update add zfs boot. but honestly i will give this a try again, is it possible if i have my gentoo system on another hdd which i do, can i use that system to do this zfs installation on my SATA HDD drive which is sdb and my ide 250gb gentoo system is sda .. can i put a zfs installation using my gentoo system . ? its using stable.. can i turn that system to a unstable sysem.. yes i'm not the professional at gentoo since it will be my second time on host.. but this time i want to do it perfect and right and i want to use zfs filesystem really bad.

If you can , give me some tips on what i should do, and how i should do this all correctly, NOTE i do have acknowledgement on setting up gentoo and everything using their official guide i setup gentoo 4 times 3 times in Virtualbox and one time on a acutal host... i'm just sort of stuck at the /boot with genkernel and the rc-update add zfs boot.. other than that its pretty straight forward.

is it possible i can make a /boot before i jump to the genkernel and just mount it ?

all the help would be cool eh,

Thanks.
Tux Hat


I am late to reply here, but I plan to update that soon. Anyway having a separate /boot dataset on ZFS is fine. I am thinking of changing zfs-install to create it.

Raffi wrote:
I've converted a couple of machines over to running full zfs systems booting via grub2. While the guides pointed to in this thread were helpful, I did do things a bit different. While I'm pretty sure what I did is OK, I thought I'd run it by other's who have been working with this stuff.

The first difference in how I have done things is in using my own custom initrd. I did that since I had been using my own custom lvm script for a while and it was easier for me to modify it than it was to learn a new initrd system. The second difference is in not creating another gpt boot partition. From what I read, the partition 9 that zfs creates when you give it the entire disk is/was probably used by Solaris for booting. Since it was already there and of sufficient size, I simply changed the partition type so that grub would use it. So far, this seems to be working well and I'm hoping that I won't run into trouble at some later time.


That will likely be fine. The only potential issue is what will happen if you ever replace the disk with a new one and use autoexpand.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum