/usr merge what?

Message

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 9:03 pm

Following from the previous locked post.

I'm using a separate /usr, and I never built an initramfs.
How do I avoid to get /bin merged into /usr?
Is it ok if I remove "baselayout" from the @system set?

Which are the reasons for merging /bin into /usr?
Which are the reasons to not do such a thing?

I hope to not be inopportune,
Leonardo

Post by **asturm** » Fri Mar 26, 2021 9:08 pm

You don't have to do anything. The only thing that was posted on the dev ML was a new version of a script to convert one's system to test a usrmerge setup.

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 9:19 pm

Ok, this makes more sense. Thanks for the quick answer.
I tought it was going to be deployed on every system.
I didn't interpreted the message correctly.

Leonardo

Post by **NeddySeagoon** » Fri Mar 26, 2021 9:34 pm

Leonardo.b,

The /usr merge has been talked about for a long time, maybe 10 years.
There have been some steps along the way to make it easy to implement.
Its not happening right now.

Post by **Ionen** » Fri Mar 26, 2021 9:42 pm

gentoo-dev ML shouldn't be considered a source of news either way

"if" (big if) this was to go forward there would be both a news item and most likely new profiles intended to work with it similarly to what happened with lib32, and then "if" deprecation+removal of old profiles happen it would likely take a few years (17.0 is even still there despite deprecated), no reason to think about this right now.

But the usrmerge setup exists right now for those that want it (along with a masked USE), and so do scripts for migration (at your own risks). The new script posted on gentoo-dev ML is also currently unsafe and needs work (it's just there for review, not actually being used on a production system, I've had a few bad things happen but it was a throwaway test chroot).

Edit: that aside, regardless of usrmerge, be warned that separate /usr without an initramfs is still not really supported by gentoo, maintainers "may" still fix issues but aren't required to leaving this a bit up in the air. So if want to keep separate /usr may still want to consider an initramfs.

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 10:09 pm

Neddy Seagoon,
Now I see. I found some of these 10 years I was missing around in the net.
I checked the Ubuntu usb I used during the install, and in fact /bin is a symlink.

I read some arguments defending or accusing this decision.
For a newcomer without background knowledge, it looks pointless but harmless.

Post by **pietinger** » Fri Mar 26, 2021 10:21 pm

Maybe a silly question (I dont have a seperate /usr but I hate initramfs):

Why do you must have initramfs when using a seperate /usr. IMHO it should be possible to keep a small /bin and /sbin for booting and later you use UnionFS for mounting the external /usr. Doesnt this work ?

Tony0945 · Post by **Tony0945** » Fri Mar 26, 2021 10:30 pm

Leonardo.b wrote:How do I avoid to get /bin merged into /usr?
Is it ok if I remove "baselayout" from the @system set?

Code: Select all

~ $ grep baselayout /etc/portage/package.mask
sys-apps/baselayout::gentoo       # changes PATH + ?

My real baselayout is in overlay

I doubt if removing baselayout would work. Just freeze whatever version you are running and put it in local overlay.
From time to time you will have to do maintenance on your ebuild.

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 10:31 pm

Ionen, it's fine, I can work around the need for an initramfs.
Even while using udevd, I think some inittab magic could provide /usr before devfs is mounted.
But I didn't tested this, so I can't say.

Naib · Post by **Naib** » Fri Mar 26, 2021 10:33 pm

pietinger wrote:Maybe a silly question (I dont have a seperate /usr but I hate initramfs):

Why do you must have initramfs when using a seperate /usr. IMHO it should be possible to keep a small /bin and /sbin for booting and later you use UnionFS for mounting the external /usr. Doesnt this work ?

The problem is some init system require information (bins, libs or data) within the /usr directory structure.
Without such information you are unable to reach the mount stage of init and thus are unable to proceed.

THUS a new root must be created storing the minimum needed to boot to at least the mount stage and then pivot /root to your real root that now has /usr mounted and full access to the required data

Post by **Ionen** » Fri Mar 26, 2021 10:39 pm

At initramfs is essentially a minimal rootfs as-is, except you don't have to worry about things not working together given it's all contained. No worries about disabling USE because a feature of /bin/need-before-mounting-usr needs /usr/lib/libmassive-with-many-deps-that-even-reads-from-usr-share.so

mike155 · Post by **mike155** » Fri Mar 26, 2021 11:04 pm

Why do you must have initramfs when using a seperate /usr.

That's simply not true. I have machines with separated /usr and with unified /usr - and none of them has an initramfs.

Post by **asturm** » Fri Mar 26, 2021 11:12 pm

That it is possible for many systems does not mean it can (or will) be guaranteed for all.

But this is not really ontopic anymore for usrmerge.

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 11:26 pm

pietinger,
you people type too fast, and my English is slow.

To mount /usr, you need the dev nodes.
You can't mount /dev/sda without /dev/sda being present already.

The device nodes are created by the kernel in the devfs, if devfs is built in the kernel.
Otherwise, you have a NeddySeagoon's Old Fashion Gentoo, where /dev is created only once, and it stays on the root filesystem.
There is nothing against a separate /usr in this setup. Here /usr just works.

Since the devfs provides kernel nodes only, with default (very strict) permissions, usually people use some daemons to fix the permissions on these nodes.
Avaiable daemons are mdevd, s6-devd, eudevd and udevd - as far as I know.
mdevd and s6-devd read a configuration file, and they fix the permissions on demand.
They are installed in /sbin, and they just work without trouble.

udevd and evdev are similar to these, with the addition of automagic loading of kernel modules.
Again, this is not a problem for a separate /usr, unless udev requires bits installed in /usr.
If I didn't understand wrong, it used to be the case. Not sure if has been fixed, or if it's true it has ever been broken.

I can think to a couple of workaround for that.
static nodes under /dev, and than devfs mounted in top of them for example.
Amen, I am all static so I don't care.

Tony0945,
I gave a look to pkg/sys-apps/baselayout/CONTENTS. I didn't think to it before.
It installs /usr/share/something, /etc/profile, /etc/inputrc, basic stuff.
I tought it was the one to set the base layout of the system, but probably I was wrong.
I am already rolling my own configuration on these files, so I freezed it too.
Until I have a backup to test its removal.

Leonardo

Leonardo.b · Post by **Leonardo.b** » Fri Mar 26, 2021 11:34 pm

Ionen, as a fallback I keep busybox and a bunch of dev nodes installed in /recovery.
If my system would read into /usr/share to boot - I would consider it already broken.

astrum, right.
Moderators, feel free to split move or whatever. I can change the subject into "/usr merge what? + initramfs discussion" too.

Post by **NeddySeagoon** » Fri Mar 26, 2021 11:36 pm

There was a council decision in 2017 that devs need not support a separate /usr without an initrd.

The immediate problem then was that udev needed files on /usr to be able to start. [profile=146493]steveL[/profile] produced the patches [topic=901206]HOWTO: udev, sep /usr, no initramfs. eudev:3.1.5 rc:0.23.2[/topic] and instructions to delay udev starting until after localmount had mounted /usr.

After some fettling, a bug to add the patch set was submitted (which I can't find) and subsequently rejected.

The status quo is still that devs need not support a separate /usr without an initrd, which is not the same as mandating an initrd.
It just means that if it stops working one day, sympathy will be minimal.

Tony0945 · Post by **Tony0945** » Sat Mar 27, 2021 12:44 am

Leonardo.b,

I'm very poor at shell and unfamiliar with most eclasses. Looking at the actual baselayout ebuilds they seem to create the multilib stucture and install some other files.
Presumably, a future baselayout would creat a different structure i.e, the /bin yo /usr/bin symlink.

Also there are basic files involved:

Code: Select all

 $ equery f baselayout
 * Searching for baselayout ...
 * Contents of sys-apps/baselayout-2.6-r1:
/etc
/etc/env.d
/etc/env.d/50baselayout
/etc/filesystems
/etc/gentoo-release
/etc/hosts
/etc/inputrc
/etc/issue
/etc/issue.logo
/etc/modprobe.d
/etc/modprobe.d/aliases.conf
/etc/modprobe.d/i386.conf
/etc/networks
/etc/os-release
/etc/profile
/etc/protocols
/etc/services
/etc/shells
/etc/sysctl.conf
/usr
/usr/share
/usr/share/baselayout
/usr/share/baselayout/fstab
/usr/share/baselayout/group
/usr/share/baselayout/issue.devfix
/usr/share/baselayout/passwd
/usr/share/baselayout/shadow
/usr/share/doc
/usr/share/doc/baselayout-2.6-r1
/usr/share/doc/baselayout-2.6-r1/ChangeLog.bz2

I could not find dependencies on baselayout but then I have moved other base ebuilds to overlay also, notably openrc and eudev.

At a minimum, if you are concerned, instead of making baselayout::gentoo, mask out greater than your current version. Then you will have a heads up from portage before it installs the new baselayout, possibly making changes you haven't reviewed and may not desire.
Why? that is a dangerous question to ask on these forums. Content yourself with "How?" until that becomes verboten also.

GDH-gentoo · Post by **GDH-gentoo** » Sat Mar 27, 2021 1:53 am

Yes, it is possible to have a separate /usr and no initramfs, but it depends on how many userspace tools are needed to get a particular system up and running, and how they are packaged. /usr-merge will likely make it impossible, though, since with such a setup the root filesystem can no longer contain programs and libraries. Or kernel modules. Or firmware.

Current stable eudev does not require having /usr pre-mounted, and neither does current stable OpenRC (except for one warning that can be fixed by a symlink to /bin/busybox). It seems that people conflate udev, which is a mechanism, with udev rules, which are input to that mechanism, and can be installed by any package. Udev rules can instruct the udev daemon to execute a program, and of course it is possible to write a rule that mentions a program installed in /usr —which I suppose that can also happen in e.g. /etc/mdev.conf—. That will fail if the rule is triggered before /usr is mounted. But that's not the daemon's fault. It is just doing what it is told to. It's the rule's fault, and the the fault of the package that installs such a rule.

Another related aspect is that having a slim initramfs requires making it manually.

Post by Hu » Sat Mar 27, 2021 4:17 am

For the sake of clarity, I want to point out that there are two different /usr related issues. What Neddy (and others) described, refers to the situation where the filesystem containing /usr/* is not the filesystem that contains the directory usr in the root directory. That is, you would see this in mount:

Code: Select all

/dev/sda1 on / ...
/dev/sda2 on /usr ...

This is the situation that is officially unsupported without an initramfs, because some tools assume that once / is mounted, it is legal to access /usr/bin too. An initramfs that mounts /usr before starting the init system satisfies this assumption. A system where /usr/* are on the root filesystem satisfies this assumption without an initramfs. As others noted above, for some systems, this assumption can be broken and nothing bad will happen. For some, breaking it will break a program.

usrmerge refers to the not yet implemented on Gentoo, though advocated on some distributions, plan to do, in effect:

Code: Select all

mv /bin/* /usr/bin
mv /sbin/* /usr/sbin
rmdir /bin /sbin
ln -s usr/bin /bin
ln -s usr/sbin /sbin

That is, replace /bin, /sbin, and possibly a few other directories, with symlinks pointing to the like-named directories in /usr.

[Edit: per later posts, IUSE=split-usr is part of usrmerge, not part of storing /usr separately. I removed some phrasing that states the contrary. -Hu]

GDH-gentoo · Post by **GDH-gentoo** » Sat Mar 27, 2021 2:22 pm

Hu wrote:That is, replace /bin, /sbin, and possibly a few other directories, with symlinks pointing to the like-named directories in /usr.

And also /lib*.

Tony0945 · Post by **Tony0945** » Sat Mar 27, 2021 2:36 pm

Thank you, Hu, for that explanation. I had assumed that split-usr meant that /usr/bin was split from /bin et cetera, what I consider the "normal" situation. I have 24 packages built with split-usr turned on.
I do not use a separate /usr. Should I recompile those packages with the flag off? Should I make sure all are in local overlay for the forthcoming collapse of the filesystem into everything in /usr except pseudo-systems?

Post by **Ionen** » Sat Mar 27, 2021 2:57 pm

Tony0945 wrote:Should I recompile those packages with the flag off? Should I make sure all are in local overlay for the forthcoming collapse of the filesystem into everything in /usr except pseudo-systems?

Should do nothing, the change isn't even planned yet. As I mentioned earlier "if" it happens there'll be a news item and a profile migration like with lib32 which could drag over years, not something to worry about right now.

The split-usr flag is currently forced-on and force-disabling it is at your own risks, it requires migration (and proper testing) and is not something to just flip. Like if no /bin symlink programs that try to access some commands with full path (like /bin/cp) will now fail given it's only in /usr/bin with -split-usr

Post by **asturm** » Sat Mar 27, 2021 3:03 pm

Tony0945 wrote:Should I recompile those packages with the flag off? Should I make sure all are in local overlay for the forthcoming collapse of the filesystem into everything in /usr except pseudo-systems?

Sigh.

When in doubt, consult portage:

Code: Select all

$ euse -i split-usr
global use flags (searching: split-usr)
************************************************************
[+  D F ] split-usr - Enable behavior to support maintaining /bin, /lib*, /sbin and /usr/sbin  separately from /usr/bin and /usr/lib*

Tony0945 · Post by **Tony0945** » Sat Mar 27, 2021 4:04 pm

Ionen, asturm, that is at odds with what Hu said above. I have a single root partition. it even contains /boot. The only outside partition mounted is /boot/efi and that's only for updates.

Post by **asturm** » Sat Mar 27, 2021 4:17 pm

Yes and no. If you want to know what IUSE=split-usr does, the only possible valid reference is portage. It is completely detached from how "split-usr" may be used in the forums to describe having /usr as a separate partition.

/usr merge what?

/usr merge what?

Re: /usr merge what?