Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Separate /usr on Linux requires initramfs
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Thu Oct 10, 2013 10:00 am    Post subject: Separate /usr on Linux requires initramfs Reply with quote

Hi gentooers!

I have some questions here.

Quote:
Linux systems which have / and /usr on separate file systems but do not
use an initramfs will not be supported starting on 01-Nov-2013.

If you have / and /usr on separate file systems and you are not
currently using an initramfs, you must set one up before this date.
Otherwise, at some point on or after this date, upgrading packages
will make your system unbootable.

For more information on setting up an initramfs, see this URL:

https://wiki.gentoo.org/wiki/Initramfs/HOWTO

Due to many upstream changes, properly supporting Linux systems that
have /usr missing at boot time has become increasingly difficult.
Despite all our efforts, it already breaks in some exotic
configurations, and this trend is likely to grow worse.

For more information on the upstream changes and why using an initramfs
is the cleanest route forward, see the following URLs:

http://freedesktop.org/wiki/Software/systemd/separate-usr-is-broken
https://blog.flameeyes.eu/2013/01/the-boot-process


I am running gentoo on a server with minimal server stuff, No X, no nothing. I use GPT, ext4 and softwareraid (MD) autodetect from kernel WITHOUT any sort of initrd

Code:

/dev/md1        /boot                   ext4            noatime,noauto                          1       2
/dev/md2        none                    swap            loop=/dev/loop6,encryption=AES256,sw    0       0
/dev/md3        /                       ext4            noatime                                 0       1
/dev/md5        /home                   ext4            noatime,nodev,nosuid                    0       2
/dev/md6        /tmp                    ext4            noatime,nodev,noexec,nosuid             0       2
/dev/md7        /usr                    ext4            noatime,nodev                           0       2
/dev/md8        /var                    ext4            noatime,nodev,nosuid                    0       2
/dev/md9        /server                 ext4            noatime,nodev,noexec,nosuid             0       2



As it seams, a Separate /usr partition will not work in the future.

1. Does this mean that it may work or does it mean 100% unbootabe system?

2. I have a static kernel with no modules support with Separate /boot, / and /usr
This have nothing todo with a Separate /boot and / right?

3. It seams to be 2 options here. Move /usr content to / or use an initrd, i would move it but then i must resize and stuff.

4. If i want a initrd as mimial as possible (Just be able to boot, NO modules support in kernel) , is it possible to use an initrd without module support in the kernel and what exactly kernel config options is needed?

5. Should i use dracut, or make my own generic initrd that should work with all kernels?


Any tips, whats your solution?
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
ulenrich
Veteran
Veteran


Joined: 10 Oct 2010
Posts: 1480

PostPosted: Thu Oct 10, 2013 10:24 am    Post subject: Reply with quote

Code:
etc # zcat /proc/config.gz |grep -i initrd ; emerge -p -1 dracut
CONFIG_BLK_DEV_INITRD=y
# CONFIG_ACPI_INITRD_TABLE_OVERRIDE is not set

These are the packages that would be merged, in order:

Calculating dependencies  .... ....... done!
[ebuild   R    ] sys-kernel/dracut-034  USE="-debug device-mapper -net (-selinux)" DRACUT_MODULES="-biosdevname -bootchart -btrfs caps -cifs crypt -crypt-gpg -crypt-loop dash -dmraid -dmsquash-live -gensplash -iscsi -livenet -lvm -mdraid -multipath -nbd -nfs -plymouth -ssh-client -syslog systemd" 0 kB

I guess initrd is working with forbidden modules load configured.

Many Redhat-conspiracy devotees think dracut is bloated, but for me
- it runs in 9 seconds: "dracut -H initrd-3.11.4-1.bfs 3.11.4-1.bfs"
- it results a 6 MByte initrd file

I don't trust Gentoo maintainers of genkernel hacking around to get a slim initrd. One Gentoo developer in another thread just admitted Udev is needed in an initrd when using LVM2. The Redhat-haters in this forum made a self-full-filling prophecy out of it ...

Instead I split out device-mapper from Lvm2 to get a dmsetup only out of it (As it is supposed to do by upstream and done by Debian), because I need it for LUKS cryptsetup my /home partition:
https://bugs.gentoo.org/show_bug.cgi?id=479950


Last edited by ulenrich on Thu Oct 10, 2013 10:47 am; edited 1 time in total
Back to top
View user's profile Send private message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Thu Oct 10, 2013 10:47 am    Post subject: Reply with quote

OK, so. For a working initrd you only need:

CONFIG_BLK_DEV_INITRD=y

In the kernel. Why the hell is dracut masked and if its bloated what tool to use instead? Or is manual the only way?

If i choose the initrd solution for this issue what do i need to do more than 1 Kernel option, create an initrd file and put it in lilo.conf

Code:

 image=/boot/vmlinuz
  label="Linux"
  root="/dev/md3"
  initrd=/boot/initrd.img


I still wondering if this initrd file is needed on a server, what will break?. Is there anyone here that have som issues without initrd and a separate /usr part on a server system?

MD autodetect is in the kernel and all drivers is static. If you look att application such as PHP, Mysql, Nginx, syslog-ng, they should work fine. Or are they going to move commands like mount, bash to /usr too? I am confused ;)
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Thu Oct 10, 2013 11:53 am    Post subject: Reply with quote

A lot of things may be moving to /usr in the future. For that reason I do not use a separate /usr anymore. If you already have a partition for everything, your / will be a mostly empty filesystem anyway, so no reason not to put those couple of files onto the /usr partition as well, making /usr your new /. This transition can be handled easily from a rescue system, and you could keep using your Initramfs-free setup as long as the deprecated RAID autodetect keeps working.

Not that there's anything wrong with a nice and custom made Initramfs. ( http://gentoo-en.vfose.ru/wiki/Initramfs ) Plus it enables you to use newer md raid metadata, gpt partitions, etc. (things that aren't supported by the md autodetection).

Mine currently looks like this:
Code:

#!/bin/busybox sh

rescue_shell() {
    echo "Something went wrong. Dropping you to a shell."
    busybox --install -s
    exec /bin/sh
}

# Prepare
mount -t devtmpfs none /dev
mount -t proc none /proc
mount -t sysfs none /sys
echo 0 > /proc/sys/kernel/printk

# Assemble RAID:
( sleep 2 # disk not ready?
  mdadm --assemble --scan
  sleep 2
) &

# Unlock Key
cryptsetup luksOpen --header /root/key.luks /root/key KEY

wait # for mdadm

# Unlock SSD
( cryptsetup luksOpen --allow-discards --key-file=/dev/mapper/KEY --keyfile-offset=0 --keyfile-size=512 `findfs UUID="ae797aa3-83af-4f5d-9f87-9461044d7fd9"` luksSSD1
  # lvm vgscan
  lvm lvchange -a y SSD/root
  sleep 2
) &

# Unlock HDD
for i in 1 2 3 4 5 6 7 8
do
    cryptsetup luksOpen --key-file=/dev/mapper/KEY --keyfile-offset=$(($i*512)) --keyfile-size=512 /dev/md"$i" luksHDD"$i" &
done

wait # for cryptsetup / LVM

# Mount Root
mount -o ro `findfs UUID="fa15678f-7e7e-4a47-8ed2-7cea7a5d037d"` /mnt/root || rescue_shell

# Clean up
cryptsetup luksClose KEY
echo 1 > /proc/sys/kernel/printk
umount /dev /proc /sys

# Switcheroo
exec switch_root /mnt/root /sbin/init
Back to top
View user's profile Send private message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Thu Oct 10, 2013 12:16 pm    Post subject: Reply with quote

Nice one frostschutz!

Quote:

Embedding into the Kernel
If you want the initramfs to be embedded into the kernel image, edit your kernel config and set Initramfs source file(s) to the root of your initramfs, (e.g /usr/src/initramfs):


This seams to be my best option, because its completely transparent

What binarys do you have in your Initramfs?
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
frostschutz
Advocate
Advocate


Joined: 22 Feb 2005
Posts: 2977
Location: Germany

PostPosted: Thu Oct 10, 2013 2:35 pm    Post subject: Reply with quote

busybox, cryptsetup, mdadm, lvm, all of them built statically by Gentoo. If Gentoo removes those static use flags, I'll have to start including libraries as well...
Back to top
View user's profile Send private message
hydrapolic
Tux's lil' helper
Tux's lil' helper


Joined: 07 Feb 2008
Posts: 126

PostPosted: Wed Oct 16, 2013 7:28 am    Post subject: Reply with quote

My computer is partitioned like the following:

sigle disk /dev/sda

/dev/sda1: boot
/dev/sda2: root
/dev/sda3: lvm

The partitions usr,var,tmp,opt,home are on lvm.

This is an amd64~ machine, and messages like the following started to come up:

* setting up tmpfiles.d entries for /dev ...
/lib64/rc/sh/tmpfiles.sh: line 237: uniq: command not found

With dracut-034 USE="device-mapper" and DRACUT_MODULES="lvm" and with "rd.auto rd.lvm=1" kernel options, it seems like /usr is still not mounted and the error appears. Anyone got it working with dracut?
With genkernel-3.4.47 (genkernel --lvm initramfs) and with dolvm kernel option it works just fine and the error disappeared.
Back to top
View user's profile Send private message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Wed Oct 16, 2013 8:37 am    Post subject: Reply with quote

Sorry, I dont use genkernel or modules at all. I want my system as static as possible. I have decided to wait until some problem occur on my client first ;)

If anyone having issues with a plain separate / and /usr (Not related to LVM, btrfs or other layers) only ext4 like me, post your issues here ;)
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
lepgalle
n00b
n00b


Joined: 03 Apr 2011
Posts: 19

PostPosted: Sun Dec 08, 2013 9:12 pm    Post subject: Reply with quote

jenkler wrote:
If anyone having issues with a plain separate / and /usr (Not related to LVM, btrfs or other layers) only ext4 like me, post your issues here ;)


Yes, since yesterday. Nothing serious I guess however after update tmpfiles.d appeared in sysinit boot level which requires uniq to run. Sure enough it is in /usr/bin This is the first time an error is generated during boot because of the separate /usr partition.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21490

PostPosted: Sun Dec 08, 2013 11:18 pm    Post subject: Reply with quote

The failure due to uniq being on /usr is funny, since the line that failed for you is probably tmpfiles_basenames="`printf "${tmpfiles_basenames}
" | sort | uniq`"
, printf is a shell command, sort is in /bin, and sort understands -u to render the sorted output unique. Therefore, the first failure is entirely because someone wrote a suboptimal shell pipeline.
Back to top
View user's profile Send private message
lepgalle
n00b
n00b


Joined: 03 Apr 2011
Posts: 19

PostPosted: Mon Dec 09, 2013 12:13 am    Post subject: Reply with quote

Hu wrote:
The failure due to uniq being on /usr is funny, since the line that failed for you is probably tmpfiles_basenames="`printf "${tmpfiles_basenames}
" | sort | uniq`"
, printf is a shell command, sort is in /bin, and sort understands -u to render the sorted output unique. Therefore, the first failure is entirely because someone wrote a suboptimal shell pipeline.


Indeed, you are right. It complains about line 237 which is exactly the one you picked. So, just for fun I edited the corresponding line in /lib64/rc/sh/tmpfiles.sh to tmpfiles_basenames="`printf "${tmpfiles_basenames}
" | sort -u`"
and watch what is happening during next boot (I understand that this edit will not be permanent).

Thanks
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21490

PostPosted: Mon Dec 09, 2013 12:28 am    Post subject: Reply with quote

If it works correctly, it would be worth filing a bug to have that change incorporated upstream. Fewer processes in a pipeline is almost always a good thing.
Back to top
View user's profile Send private message
hydrapolic
Tux's lil' helper
Tux's lil' helper


Joined: 07 Feb 2008
Posts: 126

PostPosted: Mon Dec 09, 2013 9:16 am    Post subject: Reply with quote

Thank you for the tip Hu, a bug report is opened: https://bugs.gentoo.org/show_bug.cgi?id=493736
Back to top
View user's profile Send private message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Mon Dec 09, 2013 1:21 pm    Post subject: Reply with quote

I Solved it by one big partition (ext4) 3 TB and a swap file. No more issues yey ;)
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
hydrapolic
Tux's lil' helper
Tux's lil' helper


Joined: 07 Feb 2008
Posts: 126

PostPosted: Wed Dec 11, 2013 7:51 pm    Post subject: Reply with quote

@Hu,
your fix will appear in the next release of OpenRC, thanks for it

@jenkler,
I use a single root partition for xen dom0, but on other servers it makes sense to split up to increase security (mount options) or to prevent the logs to fill up your whole disk or maybe to use different filesystems for different tasks (ext4 is fine, but I had problems with it on vmware, others like jfs/xfs are also worth considering - check out the kernel changelog, you'll find commits fixing data corruption/memory leaks for ext4)
Back to top
View user's profile Send private message
jenkler
Apprentice
Apprentice


Joined: 28 Apr 2003
Posts: 222
Location: Sweden - Stockholm

PostPosted: Thu Dec 12, 2013 12:07 pm    Post subject: Reply with quote

I use monit to monitor disk and no users have shell access on my systems ;) I use 5% of my disk now so the log fillup is no problem for me. And you can allways use logrotate.

Its nice to have only one part (easy to backup and so on)
_________________
Hello from me: Jenkler IT AB (swedish) (use google translate). Check out my Linux manpages for web in english.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum