Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Server Boot on RAID1 + BTRFS
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Feb 25, 2018 6:19 pm    Post subject: Server Boot on RAID1 + BTRFS Reply with quote

Bonjour à toute la communauté,

Je veux monter un serveur avec 6 disques.

Les deux premiers disques sont en RAID 1 avec mdadm. C'est sur ces deux premiers disques que le système boote et que toutes les partitions sont montées sauf /home.

Sur les quatre autres disques, il y a un RAID 5 avec le système de fichier BTRFS monté ensuite sous /home.

Mon problème est le suivant, je n'arrive pas à booter sur mon RAID 1.

Voici un topo de tout ce que j'ai fait :

---------------------------------------------------------------------------

Hello to all the community

I want to make a server with 6 disks.

The two first disks are in RAID1 with mdadm. It's on these two first disks that the system boots up, and that all the partitions are mounted, but /home.

On the 4 other disks, there is a RAID5 done with BTRFS file system, that is mounted on /home.

My problem is : I don't manage to boot on my RAID1 array!

Here the script I have done:

Code:
nic=eno16777736
ip=192.168.52.137
netmask=255.255.255.0
gateway=192.168.52.2
DEVBOOT1=/dev/sda
DEVBOOT2=/dev/sdb
DEVR1=/dev/sdc
DEVR2=/dev/sdd
DEVR3=/dev/sde
DEVR4=/dev/sdf
sizeOfBoot=128MB
sizeOfSw=2GB
sizeOfSbin=2GB
sizeOfUsr=10GB
sizeOfVar=3GB
sizeOfVarTmp=10GB
sizeOfVarLog=2GB
sizeOfVarLogAudit=1GB
sizeOfTmp=1GB
numOfCoresPlusOne=2

# chiffres pour une petite VM

### PART 1

loadkeys fr

ifconfig $nic $ip netmask $netmask
route add default gw $gateway
echo "nameserver 208.67.222.222" > /etc/resolv.conf
echo "nameserver 208.67.220.220" >> /etc/resolv.conf
ping –c 3 www.gentoo.org

echo "
n         # /boot
P
1

+"$sizeOfBoot"
t
fd
n         # sw
p
2

+"$sizeOfSw"
t

82
n         # /sbin
p
3

+"$sizeOfSbin"
t

fd
n         #
e


n         # /usr

+"$sizeOfUsr"
t

fd
n         # /var

+"$sizeOfVar"
t

fd
n         # /var/tmp

+"$sizeOfVarTmp"
t

fd
n         # /var/log

+"$sizeOfVarLog"
t

fd
n         # /var/log/audit

+"$sizeOfVarLogAudit"
t

fd
n         # /tmp

+"$sizeOfTmp"
t

fd
n         # /


t

fd
a
1
w
" | fdisk $DEVBOOT1 || exit 1

sfdisk –d $DEVBOOT1 | sfdisk $DEVBOOT2
mdadm --create /dev/md/0 --level=1 --metadata=0.90 --raid-devices=2 "$DEVBOOT1"1 "$DEVBOOT2"1
mdadm --create /dev/md/1 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"3 "$DEVBOOT2"3
mdadm --create /dev/md/2 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"5 "$DEVBOOT2"5
mdadm --create /dev/md/3 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"6 "$DEVBOOT2"6
mdadm --create /dev/md/4 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"7 "$DEVBOOT2"7
mdadm --create /dev/md/5 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"8 "$DEVBOOT2"8
mdadm --create /dev/md/6 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"9 "$DEVBOOT2"9
mdadm --create /dev/md/7 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"10 "$DEVBOOT2"10
mdadm --create /dev/md/8 --level=1 --metadata=1.2 --raid-devices=2 "$DEVBOOT1"11 "$DEVBOOT2"11

mkfs.vfat /dev/md/0      # /boot
mkswap /dev/sda2 && swapon /dev/sda2
mkfs.btrfs /dev/md/1   # /sbin
mkfs.btrfs /dev/md/2   # /usr
mkfs.btrfs /dev/md/3   # /var
mkfs.btrfs /dev/md/4   # /var/tmp
mkfs.btrfs /dev/md/5   # /var/log
mkfs.btrfs /dev/md/6   # /var/log/audit
mkfs.btrfs /dev/md/7   # /tmp
mkfs.btrfs /dev/md/8   # /
mkfs.btrfs -m raid5 –d raid5 $DEVR1 $DEVR2 $DEVR3 $DEVR4

### MANUAL

# Verifier /etc/resolv.conf

ntpd –g –q

### PART 2

mount /dev/md/8 /mnt/gentoo

mkdir -p /mnt/gentoo/boot /mnt/gentoo/sbin /mnt/gentoo/usr /mnt/gentoo/var/tmp /mnt/gentoo/var/log/audit /mnt/gentoo/tmp /mnt/gentoo/home

chmod 1777 /mnt/gentoo/tmp /mnt/gentoo/var/tmp

cd /mnt/gentoo

wget http://distfiles.gentoo.org/releases/amd64/autobuilds/20180206T214502Z/stage3-amd64-20180206T214502Z.tar.xz

### MANUAL

mount /dev/md/0 /mnt/gentoo/boot
mount /dev/md/1 /mnt/gentoo/sbin
mount /dev/md/2 /mnt/gentoo/usr
mount /dev/md/3 /mnt/gentoo/var
mount /dev/md/4 /mnt/gentoo/var/tmp
mkdir /mnt/gentoo/var/tmp
mount /dev/md/4 /mnt/gentoo/var/tmp
mount /dev/md/5 /mnt/gentoo/var/log
mkdir /mnt/gentoo/var/log
mount /dev/md/5 /mnt/gentoo/var/log
mount /dev/md/6 /mnt/gentoo/var/log/audit
mkdir /mnt/gentoo/var/log/audit
mount /dev/md/6 /mnt/gentoo/var/log/audit
mount /dev/md/7 /mnt/gentoo/tmp
mount $DEVR1 /mnt/gentoo/home

tar xpf stage3-* --xattrs-include='*.*' --numeric-owner

### PART 3

nano -w /mnt/gentoo/etc/portage/make.conf
   # CFLAGS="-march=native -O2 -pipe"
   # CXXFLAGS="${CFLAGS}"
   # PORTDIR="/usr/portage"
   # DISTDIR="/usr/portage/distfiles"
   # PKGDIR="/usr/portage/packages"
   # LC_MESSAGES=C
   # MAKEOPTS="-j"$numOfCoresPlusOne""

mirrorselect –i –o >> /mnt/gentoo/etc/portage/make.conf
   # Sélectionner quelques mirroirs en France, Allemagne…

mkdir –p /mnt/gentoo/etc/portage/repos.conf

cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf

cp --dereference /etc/resolv.conf /mnt/gentoo/etc/

mount --types proc /proc /mnt/gentoo/proc
mount --rbind /sys /mnt/gentoo/sys
mount --make-rslave /mnt/gentoo/sys
mount --rbind /dev /mnt/gentoo/dev
mount --make-rslave /mnt/gentoo/dev

### MANUAL

chroot /mnt/gentoo /bin/bash

### PART 4 CHROOTED (LONG)

source /etc/profile
export PS1="(chroot) ${PS1}"

emerge-webrsync && emerge --sync

emerge --oneshot portage

#eselect profile list
eselect profile set 40
   # hardened/linux/amd64 (stable)

emerge --ask --update --deep --newuse @world
   # time 30 m

### PART 5 CHROOTED

echo "Europe/Paris" > /etc/timezone
emerge --config sys-libs/timezone-data
nano –w /etc/locale.gen
   # Choisir un anglais UTF8
locale-gen
eselect locale list
   # Choisir le numero Num
Num=3
eselect locale set $Num

env-update && source /etc/profile && export PS1="(chroot) $PS1"

emerge --ask sys-kernel/hardened-sources

etc-update
   # -3

emerge --ask sys-kernel/hardened-sources
emerge -na =sys-apps/gradm-3.1*

emerge --ask sys-kernel/genkernel

emerge mdadm
rc-update add mdraid boot

emerge --ask sys-fs/btrfs-progs

### MANUAL (LONG) - KERNEL

# make config
# time make
# time make_modules
# make install

# genkernel --btrfs --mdadm --install initramfs all
# genkernel --btrfs --mdadm --install initramfs

### PART 6

nano –w /etc/fstab

nano -w /etc/conf.d/hostname

nano -w /etc/conf.d/net

ln -s net.lo net.$nic

rc-update add net.$nic default

passwd

nano -w /etc/conf.d/keymaps

emerge --ask app-admin/syslogd

rc-update add syslogd default

emerge --ask sys-process/cronie

rc-update add cronie default

rc-update add sshd default

emerge --ask net-misc/dhcpcd

emerge --ask sys-boot/syslinux

### PART 7 (BOOTLOADER)

# CEST LA QUE JAI  BESOIN DE VOTRE AIDE
# THERE I NEED HELP PLEASE



Donc pour résumer, voici ce que j'ai fait.

Dans la partie 7 (bootloader) j'ai essayé d'utiliser grub2.

- Grub : il me dit qu'il n'arrive pas à monter les grappes raid (pourtant mon fichier /etc/mdadm.conf est correctement rempli et j'ai bien rajouté l'option GRUB_CMDLINE_LINUX_DEFAULT="domdadm").

----------------------------------------------

I tried to use Grub:2 but it keeps saying me that it can't mount my arrays. Besides, my /etc/mdadm.conf is correctly written, and I've added the option domdadm to /etc/default/grub...

Guys in my case often do a simple mdadm --detail --scan >> /etc/mdadm.conf" but I've already done that.


-------------------------

Les gens dans mon cas à ce que j'en ai compris font un bête et méchant
Code:
mdadm --detail --scan >> /etc/mdadm.conf"


et l'erreur disparaît, mais pas chez moi ... :(

Mon fstab ressemble à cela : Here is my fstab :

Code:

/dev/md/0   /boot             vfat   noauto,noatime   1 2
/dev/md/8   /                   btrfs  noatime              0 0
/dev/sda2    none             swap  sw                     0 0
/dev/md/2   /usr                btrfs   defaults             0 0
/dev/md/3   /var                btrfs   defaults             0 0
/dev/md/4   /var/tmp         btrfs   defaults             0 0
/dev/md/5   /var/log          btrfs   defaults             0 0
/dev/md/6   /var/log/audit btrfs   defaults             0 0
/dev/md/7   /tmp btrfs       btrfs   defaults             0 0



Please help me, après des heures de recherche et une petite expérience de Gentoo je n'y arrive pas.

Mon kernel a bien les bons paramètres au niveau RAID, et concernant le reste je démarre sans problème si je ne cherche pas à faire de RAID sur le disque de boot.


-----------

My kernel has all it is ok to work including RAID1 support option and BTRFS of course...

After hours and hours of research I don't manage to make it boot.


-----------


Quelqu'un a-t-il une idée ? :o / Can someone give me a hint?

Bien à vous et bravo à cette formidable distribution. / greeetings to all of you

Output de la commande df -h : / Here is the ouptut of df -h command:

Code:

Filesystem     Size     Used     Avail     Use%     Mounted on
/dev/md8       22G    272M      20G        2%     /
/dev/md0     122M      33M     90M       27%    /boot
/dev/md1      1.9G      31M     1.7G        2%    /sbin
/dev/md2      9.4G     3.2G     5.9G       35%   /usr
/dev/md3      2.8G      45M     2.5G         2%   /var
/dev/md4      9.4G     826M    7.8G        10%  /var/tmp
/dev/md5      1.9G       17M    1.7G          1%  /var/log
/dev/md6      954M      17M    842M         2%  /var/log/audit
/dev/md7      954M      17M    841M         2% /tmp
cgroup_root     10M         0       10M         0% /sys/fs/cgroup
udev                10M     4.0K      10M         1% /dev
shm               369M         0      369M        0% /dev/shm


Franck M.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Feb 25, 2018 6:44 pm    Post subject: Reply with quote

Francky,

Welcome to Gentoo.

There is no need to make bilingual posts. I read through the French, very slowly, as my French is not good and almost moved your post to the French forum.

Why do you have vfat on /boot?
Are you using EFI or not.

Your md0 is raid1 with metadata 0.90, so other than the vfat filesystem, it should just work.
vfat is a problem as it does not support symbolic links.

Where does it go wrong?

What is the error message?

You say you tried to use Grub2. What boot loader are you using now?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Feb 25, 2018 6:53 pm    Post subject: Reply with quote

NeddySeagoon wrote:
Francky,

Welcome to Gentoo.

There is no need to make bilingual posts. I read through the French, very slowly, as my French is not good and almost moved your post to the French forum.

Why do you have vfat on /boot?
Are you using EFI or not.

Your md0 is raid1 with metadata 0.90, so other than the vfat filesystem, it should just work.
vfat is a problem as it does not support symbolic links.

Where does it go wrong?

What is the error message?


I have vfat on /boot I don't know why. Should I change from vfat to something else or is it ok with fat?
I am not willing to use UEFI.

So I should change my md0 partitioning? Please give me info.

It goes wrong when I reboot.

It simply boot my kernel then after a few seconds I got a message :

Code:

mdadm: No arrays found in config file or automatically
>> Determining root device . . .
!! Could not find the root block device in UUID=75184c4-b9da-43a1-b100-e7cc61dbd8a2.
!! Please specify another value or:
!! - press Enter fo the same
!! - type "shell" for a shell
!! - type "q" to skip. . .

I want to use EXTLINUX or GRUB2, if possible LILO is my preferred but I prefer you tell me which one to choose is easier.
root block device(UUID=751484c4-b9da-43a1-b108-e7cc61dbd8a2) ::
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Feb 25, 2018 7:14 pm    Post subject: Reply with quote

Francky,

Your kernel loads, so vfat is working for you.

Make friends with wgetpaste.
Use it to put your kernel .config file onto a pastebin site and post the link.
It too big to fit into a post.

Also post the output of lspci.

What bootloader do you use?
Puts its configuration file onto a pastebin.

With your system set up to chroot, what does
Code:
blkid
show.

At the moment it looks like the initrd is missing or not being mounted.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Feb 25, 2018 7:59 pm    Post subject: A Reply with quote

[quote="NeddySeagoon"]Francky,

Your kernel loads, so vfat is working for you.

Make friends with wgetpaste.
Use it to put your kernel .config file onto a pastebin site and post the link.
It too big to fit into a post.

Also post the output of lspci.

What bootloader do you use?
Puts its configuration file onto a pastebin.

With your system set up to chroot, what does
Code:
blkid
show.

At the moment it looks like the initrd is missing or not being mounted.

My .config : https://pastebin.com/JFnKDg2W

lspci output:
Code:

00:00.0 Host Bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX Host Bridge (rev 01)
00:01.0 PCI bridge: Intel Corporation 440BX/ZX/DX - 82443BX/ZX/DX AGP Bridge (rev 01)
00:07.0 ISA bridge: Intel Corporation 82371AB/EB/MB PIIX4 ISA (rev 08)
00:07.1 IDE interface: Intel Corporation 82371AB/EB/MB PIIX4 IDE (rev 01)
00:07.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 08)
00:07.7 System peripheral: VMware Virtual Machine Communication Interface (rev 10)
00:0f.0 VGA compatible controller: VMware SVGA II Adapter
00:10.0 SCSI storage controller: LSI Logic / Symbios Logic 52c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI (rev 01)
00:11.0 PCI bridge: VMware PCI bridge (rev 02)
00:15.0 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.1 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.2 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.4 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.5 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.6 PCI bridge: VMware PCI Express Root Port (re v01)
00:15.7 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.0 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.1 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.2 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.3 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.4 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.5 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.6 PCI bridge: VMware PCI Express Root Port (re v01)
00:16.7 PCI bridge: VMware PCI Express Root Port (re v01)
00:17.O PCI bridge: VMware PCI Express Root Port (re v01)
00:17.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:17.7 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.0 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.1 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.2 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.3 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.4 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.5 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.6 PCI bridge: VMware PCI Express Root Port (rev 01)
00:18.7 PCI bridge: VMware PCI Express Root Port (rev 01)
02:00.0 USB Controller: VMware USB1.1 UHCI Controller
02:01.0 Ethernet controller: Intel Corporation 82545EM Gigabit Ethernet Controller (Copper) (rev 01)
02:02.0 USB controller: VMware USB2 EHCI Controller


I don't know what bootloader I should use, but I'd like to turn to EXTLINUX or GRUB:2...

Output of blkid:
https://pastebin.com/hJzwgtUd
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Feb 25, 2018 8:28 pm    Post subject: Reply with quote

Francky,

All the boot loaders should work.
Whichever one you are using now is doing something as the kernel loads.

Please post or pastebin its config file.

You have
Code:
CONFIG_MD_AUTODETECT=y
in your kernel. That works only of raid metadata 0.90, that's your /boot.
The partitions must be type 0xfd.
However, it won't matter for booting. Boot loaders have to make their own arrangements to load the kernel and initrd.
That will not assemble the rest of your raid sets.

grub legacy ignores the raid, that's why it has to be raid metadata 0.90
grub2 can read raid properly
I suspect that lilo ignores the raid at boot, as it loads a block list.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Feb 25, 2018 8:52 pm    Post subject: Reply with quote

NeddySeagoon wrote:
Francky,

All the boot loaders should work.
Whichever one you are using now is doing something as the kernel loads.

Please post or pastebin its config file.

You have
Code:
CONFIG_MD_AUTODETECT=y
in your kernel. That works only of raid metadata 0.90, that's your /boot.
The partitions must be type 0xfd.
However, it won't matter for booting. Boot loaders have to make their own arrangements to load the kernel and initrd.
That will not assemble the rest of your raid sets.

grub legacy ignores the raid, that's why it has to be raid metadata 0.90
grub2 can read raid properly
I suspect that lilo ignores the raid at boot, as it loads a block list.


OK

I am using Grub:2

I have done

grub-install /dev/sda (first disk of the array)
grub-install /dev/sdb (second disk of the array)
then grub-mkconfig -o /boot/grub/grub.cfg

The grub.cfg can be read here :
https://pastebin.com/KYVVmTx7

Then I've added the correct "domdadm" option but no result.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Feb 25, 2018 9:19 pm    Post subject: Reply with quote

Francky,

Your grub.cfg says
Code:
linux   /vmlinuz-4.8.17-hardened-r2-FHFl1 root=UUID=751484c4-b9da-43a1-b100-e7cc61dbd8a2 ro  domdadm

but from blkid
Code:
/dev/md8: UUID="751484c4-b9da-43a1-b100-e7cc61dbd8a2"
so you are passing the UUID of the raid set, not the UUID of the root filesystem contained on that raid set..
Code:
/dev/md8: UUID="751484c4-b9da-43a1-b100-e7cc61dbd8a2" UUID_SUB="4561ee17-ee4b-422c-a5f7-319e3b63e29c" TYPE="btrfs"


I don't know how you pass a btrfs sub volume. From the Wiki its just root=UUID=4561ee17-ee4b-422c-a5f7-319e3b63e29 for you.

That is unlikely to work if mdadm is really not assembling the raid sets.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Mon Feb 26, 2018 6:48 am    Post subject: Reply with quote

NeddySeagoon wrote:
so you are passing the UUID of the raid set, not the UUID of the root filesystem contained on that raid set.. .


My / is a RAID 1 array named /dev/md8, so I passed the UUID of this raid set as my root.

How to get the UUID of the root filesystem contained on that raid set as you said ? Could it be the problem ?

What I don't understand is why the raid arrays are not automatically mounted at boot time...

Thanks
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Mon Feb 26, 2018 1:29 pm    Post subject: Reply with quote

Francky,

Lots of different objects have UUIDs
blkid shows most of them.

Partitions have a UUID

Code:
/dev/sda1: UUID="50e9fc0f-e7ed-8f75-cb20-1669f728008a" TYPE="linux_raid_member" PARTUUID="36452d6c-01"
/dev/sdb1: UUID="50e9fc0f-e7ed-8f75-cb20-1669f728008a" TYPE="linux_raid_member" PARTUUID="36452d6c-01"

That's a bit unusual. The PARTUUIDs should be unique ... that tells that you partitioned one drive and copied it to the other.
Here the partition UUIID is PARTUUID="36452d6c-01

Raid sets have a UUID.
Code:
mdadm -E /dev/sda1

That the UUID you can feed to mdadm -A -uuid=..
Here its UUID="50e9fc0f-e7ed-8f75-cb20-1669f728008a"
mdadm will find all the members of the raid set with that uuid.

blkid does not show the filesystem UUID there.
Knowing that md0 is composed of sda1 and sdb1, we can see
Code:
/dev/md0: UUID="c9df4e13-ec58-4795-9a62-7bd0cd3e2c40" TYPE="ext2"
which is the filesystem UUID.

Simarly for md8, which I think in your root.
Code:
/dev/md8: UUID="751484c4-b9da-43a1-b100-e7cc61dbd8a2" UUID_SUB="4561ee17-ee4b-422c-a5f7-319e3b63e29c" TYPE="btrfs"

I think you need to pass 4561ee17-ee4b-422c-a5f7-319e3b63e29c to the kernel in root=UUID=

For a trial, edit the grub.cfg. If root is not on md8, that UUID above will be incorrect. Its the UUID from the UUID_SUB that you need to tell the kernel for its root.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Mon Feb 26, 2018 2:21 pm    Post subject: Reply with quote

NeddySeagoon wrote:

Simarly for md8, which I think in your root.
Code:
/dev/md8: UUID="751484c4-b9da-43a1-b100-e7cc61dbd8a2" UUID_SUB="4561ee17-ee4b-422c-a5f7-319e3b63e29c" TYPE="btrfs"

I think you need to pass 4561ee17-ee4b-422c-a5f7-319e3b63e29c to the kernel in root=UUID=

For a trial, edit the grub.cfg. If root is not on md8, that UUID above will be incorrect. Its the UUID from the UUID_SUB that you need to tell the kernel for its root.


Hello Neddy

My root is yes on md8, so the UUID you told was correct, but grub ended up telling me it could not find root block device in that UUID...

I think the problem is not here because each time I boot, I have a mdadm error prior to the root block device error: mdadm: No arrays found in config file or automatically.

I don't understand why I have this message because I have done everything right, my /etc/mdadm.conf contains the arrays, I've done mdadm --detail --scan >> /etc/mdadm.conf.

An idea why mdadm does not assemble my raid arrays ?

--------------


What about using btrfs with mkfs.btrfs -m raid1 -d raid1 /dev/sdaX /dev/sdbX for all X ? so, no more mdadm, only BTRFS Raid? could it be a workaround?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Mon Feb 26, 2018 2:52 pm    Post subject: Reply with quote

Francky,

The boot process involves breaking lots of circular dependencies.
You say
Francky wrote:
/etc/mdadm.conf contains the arrays
.

That file is on the root filesystem, so it can't be read until the raid sets are assembled and root is mounted.
That's too late to assemble the raid sets to mount root. This one of those circular dependencies.

grub loads your kernel and initrd. The initrd is a small root filesystem in a file.
When the kernel starts, it mounts the initrd as a temporary root filesystem and executes the init script there.
Somehow, that init script must assemble your raid sets. If /etc/mdadm.conf is to be used, it must be here.

Your initrd, as made by genkernel, is a cpio archive, possibly containing other cpio archives.
Use the cpio command to unpack it. Find the init script, it will either be /init or /sbin/init and put it onto a pastebin.

When booting fails, you should be dropped into a busybox shell.
Can you see your hard drives and partitions in /dev ?
If not, the kernel is not seeing your drives. That's a problem.
What modules are loaded?
You need fusion_spi (that may not be 100% correct).

If your drives and partitions are there, what about your raid sets?
/dev/md*
If they are missing can you assemble them by hand using the tools in the initrd?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Mon Feb 26, 2018 3:23 pm    Post subject: Reply with quote

NeddySeagoon wrote:
Francky,

The boot process involves breaking lots of circular dependencies.
You say
Francky wrote:
/etc/mdadm.conf contains the arrays
.

That file is on the root filesystem, so it can't be read until the raid sets are assembled and root is mounted.
That's too late to assemble the raid sets to mount root. This one of those circular dependencies.

grub loads your kernel and initrd. The initrd is a small root filesystem in a file.
When the kernel starts, it mounts the initrd as a temporary root filesystem and executes the init script there.
Somehow, that init script must assemble your raid sets. If /etc/mdadm.conf is to be used, it must be here.

Your initrd, as made by genkernel, is a cpio archive, possibly containing other cpio archives.
Use the cpio command to unpack it. Find the init script, it will either be /init or /sbin/init and put it onto a pastebin.

When booting fails, you should be dropped into a busybox shell.
Can you see your hard drives and partitions in /dev ?
If not, the kernel is not seeing your drives. That's a problem.
What modules are loaded?
You need fusion_spi (that may not be 100% correct).

If your drives and partitions are there, what about your raid sets?
/dev/md*
If they are missing can you assemble them by hand using the tools in the initrd?


Yes I am dropped to the busybox shell, but I don't see any drives or partitions in /dev, so that means my kernel does not see my drives.

You are talking about fusion_spi. I have taken a look to the .config of my kernel and it has :

CONFIG_FUSION=y
CONFIG_FUSION_SPI=m
CONFIG_FUSION_FC=m
CONFIG_FUSION_SAS=m
etc as m.

I would really appreciate your help to uncompress that initrd image as I did not manage to using simple cpio try out.

How do I use CPIO to unpack my initramfs ? Maybe I should add some commands that assemble the arrays then repack the initrd?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Mon Feb 26, 2018 3:45 pm    Post subject: Reply with quote

Francky,

Code:
CONFIG_FUSION_SPI=m
says that the driver for your emulated SCSI card is made as a module.
Unless it is loaded, your hard drives will not be visible.
No hard drives, no partitions so mdadm has nothing to work with.

Unfortunately, the kernel help does not give the module name
Reading the kernel source for the module drivers/message/fusion/mptspi.c, close to the top of the file is
Code:
#define MYNAM           "mptspi"


So, boot into the rescue shell and check that your drives are not in /dev.
Code:
modprobe mptspi

and check again.
If your drives have appeared, we know that the initrd is not loading mptspi.

With your drives in /dev, what happens if you run the init script?
There will be a few errors but your system might boot too.

If the above works, rebuild your kernel with CONFIG_FUSION_SPI built in, not a module.
Then the initrd does not need to load it. It will be easier that fixing the init script in the initrd.

Code:
cpio --extract < /path/to/input/file > /path/to/output/dir

cpio works with stdin and sdout < redirects stdin. > redirects stdout.
It makes a real mess of the console if you leave stdout as the console.
Close your eyes and type reset
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Thu Mar 01, 2018 5:47 pm    Post subject: Reply with quote

modprobe command did not work, apparently like if modprobe use was not possible in that tiny rescue shell? (ash : no modprobe or something like this ... )

All those difficulties make me ask myself if I should not turn to *true* material raid for the boot / root and let the BTRFS Raid for all the rest.

What do you think of the idea in itself?

I have found that link and that's why I'm thinking of material card for the RAID. http://www.openwebit.com/c/list-of-real-sata-raid-cards-for-linux/
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Thu Mar 01, 2018 7:56 pm    Post subject: Reply with quote

Francky,

Hardware raid1 for boot is going to be very expensive and most of the time it won't be doing anything.
I don't use btrfs. I'll wait a few years for it to mature.

I find it difficult to believe that busybox modprobe is not in the initrd.
Its needed by the initrd to load modules.

Maybe the symlink is missing in the initrd?

Try busybox modprobe <module_name>
In the initrd
Code:
/bin/busybox --install -s
will generate all the symbolic links for you.
If you do that as root on your actual root filesystem lots of things will be overwritten and will no longer work.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Fri Mar 02, 2018 9:59 pm    Post subject: Reply with quote

OK so I figure that my problem is with my initrd.

For info, the real info on the busybox shell is "modprobe: can't open 'modules.dep': No such file or directory"

Could the way be to create from scratch an initramfs without using genkernel ?

Would that have a better chance of working, and how to do that if it is interesting ?

Seen nothing on Gentoo wiki apart from genkernel and dracut.

Suggestion ?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Fri Mar 02, 2018 11:08 pm    Post subject: Reply with quote

Francky,

Use genkernel but configure everything you need to boon as <*>.
Then there are no modules that need to be loaded.

That way you can can make your initrd by hand only with the user space tools you need ta assemble your raid sets.
It becomes like firmware. As its self contained and not tied to any kernel, why change it.
One way is covered under Custom Initramfs

Don't embed the initrd into the kernel, at least until it works. Even then its not a good idea.
The embedded initrd is only required by users using an efi stub kernel, with no boot loader.

There is another way described in Early Userspace Mounting
This is what i use now, I think its simpler..
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sat Mar 03, 2018 8:53 am    Post subject: Reply with quote

Hello !

I have decided to create my own initramfs.

So I followed the tuto of Gentoo you showed to me.

I used ldd to find all the dependencies to mount and mdadm and add them to my /lib64 of the initramfs.

Then I've written this little tiny script for init :

Code:

#!/bin/busybox

rescue_shell () {
   echo "Something went wrong. Dropping to a shell."
   exec sh
}

mount -t proc none /proc
mount -t sysfs none /sys

mdadm --create /dev/md/8 --level=1 --metadata=1.2 --raid-devices=2 /dev/sda11 /dev/sdb11

mount -o ro /dev/md/8 /mnt/root || rescue_shell

umount /proc
umount /sys

exec switch_root /mnt/root /sbin/init


I packed it...

What is fun is the following :

When I boot my custom initramfs, it reports the following error:

mdadm: cannot open /dev/sda11: No such device or address
and the other error following are normal as there are no /dev/md/8 root device to mount / on ...

When using the busybox shell and ls -la /dev/, I see everything, including /dev/sda11 !!! So why does mdadm says to me it cannot open /dev/sda11, if it is here?

ls /dev returns:

Code:

console
null
sda1
sda10
sda11
sda2
sda3
sda4
sda5
sda6
sda7
sda8
sda9
sdb1
sdb10
sdb11
sdb2
sdb3
sdb4
sdb5
sdb6
sdb7
sdb8
sdb9
tty
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sat Mar 03, 2018 10:42 am    Post subject: Reply with quote

Francky,

I hope that this command failed!
Code:
mdadm --create /dev/md/8 --level=1 --metadata=1.2 --raid-devices=2 /dev/sda11 /dev/sdb11

--create makes a new raid set by overwriting the raid metadata. Your own data will still be there but unless the metadata is *identical*, your data will not be accessible.
Think of it like making a new empty partition table on a HDD.

The command you need is --assemble. See
Code:
man mdadm


You have a static /dev in your initrd. That's fine, it works here. Unlike a dynamic /dev, an entry does not mean that there is actually any hardware under the /dev node.
Is your mptspi module loaded or built into the kernel?

Your initrd script does not load it, so if it not built in, its not available.
My initrd init script may help.
It full of old code commented out and debug statements (echo and sleep), so I could debug it.

My initramfs_list is there for reference too.
That lets me use the Early Userspace Mounting method
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Mar 04, 2018 6:18 pm    Post subject: Reply with quote

Hello

I have made progresses, I recompiled the kernel and modified a bit my init :

Code:

#!/bin/busybox sh

rescue_shell () {
   echo "Something went wrong. Dropping to a shell."
   exec sh
}

mount -t proc none /proc
mount -t sysfs none /sys
mount -t devtmpfs none /dev

mdadm --assemble /dev/md/8 /dev/sda11 /dev/sdb11

raidautorun /dev/md8

mount -o ro /dev/md/8 /mnt/root || rescue_shell

umount /proc
umount /sys
umount /dev

echo "IT IS OK TO LAUNCH SWITCH_ROOT!"

exec switch_root /mnt/root /sbin/init"


So now when I reboot my computer, what happens is :

Code:
[...]
BTRFS info (device md8): disk space caching is enabled
BTRFS info (device md8): has skinny extents
IT IS OK TO LAUNCH SWITCH_ROOT


So I am very happy cause that means that mdadm --assemble worked and mount of root is done!

but... but...

Just after I have the following:

Code:
switch_root: can't execute '/sbin/init': No such file or directory
Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100



Haaaaaaaaaa :o :evil: :!: :?: :idea:

Help ? :P :oops:
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Mar 04, 2018 6:35 pm    Post subject: Reply with quote

Francky,

Yes, your raid appears to be assembled.

I suspect that your root is not mounted correctly.
Is root actually a subvolume of the BTRFS filesystem on the raid set?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 54096
Location: 56N 3W

PostPosted: Sun Mar 04, 2018 6:38 pm    Post subject: Reply with quote

Francky,

Yes, your raid appears to be assembled.

I suspect that your root is not mounted correctly.
Is root actually a subvolume of the BTRFS filesystem on the raid set?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Mar 04, 2018 6:39 pm    Post subject: Reply with quote

To help us, I have tried to read some lines before thanks to adding a /bin/sleep:

Code:

md: could not open unknown-block(0,11)
md: Scanned 18 and added 2 devices.
md: autorun ...
md: considering sda1 ...
md:  adding sda1 ...
md:  adding sdb1 ...
md: created md0
md: bind<sdb1>
md: bind<sda1>
md: running: <sda1><sdb1>
md/raid1:md0: active with 2 out of 2 mirrors
md0: detected capacity change from 0 to 127860736
md: ... autorun DONE.
UDF-fs: warning (device md8): udf_fill_super: No partition found (2)
(mount,1584,0):ocfs2_fill_super:1024 ERROR: superblock probe failed!
(mount,1584,0):ocfs2_fill_super:1218 ERROR: status = -22
gfs2: gfs2 mount does not exist
BTRFS: device fsid 751484c4-b9da-43a1-b100-e7cc61db8a2 devid 1 transid 99 /dev/md/8
Back to top
View user's profile Send private message
Francky
n00b
n00b


Joined: 25 Feb 2018
Posts: 14
Location: France

PostPosted: Sun Mar 04, 2018 6:43 pm    Post subject: Reply with quote

The number
Code:
BTRFS: device fsid 751484c4-b9da-43a1-b100-e7cc61db8a2 devid 1 transid 99 /dev/md/8
that appears is not the UUID_SUB of the /dev/md8 (root) but the UUID...

Any idea to pass the UUID_SUB ? May I use mdadm --assemble with UUIDs instead of disks ?

Many thanks :-)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum