Update causes mayhem--sys-apps/systemd-utils to blame?

[alamahant]

CAUTION!
Hi Guys today's portage output was a bit strange

Code: Select all

emerge -uDUav --keep-going --with-bdeps=y --exclude=gcc @world

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild     U  ] net-misc/rsync-3.2.4-r1::gentoo [3.2.4::gentoo] USE="acl iconv ipv6 ssl xattr -examples -lz4 -stunnel -system-zlib -verify-sig -xxhash -zstd" PYTHON_SINGLE_TARGET="python3_9%* -python3_8% -python3_10%" 0 KiB
[ebuild     U  ] sys-libs/binutils-libs-2.38-r2:0/2.38::gentoo [2.38-r1:0/2.38::gentoo] USE="nls -64-bit-bfd (-cet) -multitarget -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild     U  ] sys-devel/binutils-2.38-r2:2.38::gentoo [2.38-r1:2.38::gentoo] USE="gold nls plugins (-cet) (-default-gold) -doc -multitarget -pgo -static-libs -test -vanilla" 0 KiB
[ebuild     U  ] media-libs/freetype-2.12.0-r1:2::gentoo [2.12.0:2::gentoo] USE="X adobe-cff bzip2 cleartype-hinting harfbuzz png svg utils -brotli -debug -doc -fontforge (-infinality) -static-libs" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild     U  ] app-text/ghostscript-gpl-9.56.1:0/9.56::gentoo [9.55.0-r1:0/9.55::gentoo] USE="X cups dbus gtk (unicode) -static-libs" L10N="-de (-ja) -ko -zh-CN -zh-TW" 61,126 KiB
[ebuild     U  ] xfce-base/xfce4-panel-4.17.0::gentoo [4.16.3::gentoo] USE="dbusmenu -introspection -vala" 0 KiB
[ebuild     U  ] virtual/libudev-232-r6:0/1::gentoo [232-r5:0/1::gentoo] USE="-systemd" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] sys-apps/systemd-utils-250.4::gentoo  USE="(split-usr) tmpfiles udev -boot (-selinux) -sysusers -test" ABI_X86="(64) -32 (-x32)" 0 KiB
[uninstall     ] sys-apps/systemd-tmpfiles-249.9::gentoo  USE="(-selinux) -test" 
[blocks b      ] <sys-apps/systemd-tmpfiles-250 ("<sys-apps/systemd-tmpfiles-250" is soft blocking sys-apps/systemd-utils-250.4)
[uninstall     ] sys-fs/udev-249.9::gentoo  USE="acl kmod (split-usr) (-selinux) -test" ABI_X86="(64) -32 (-x32)" 
[blocks b      ] <sys-fs/udev-250 ("<sys-fs/udev-250" is soft blocking sys-apps/systemd-utils-250.4)
[ebuild     U  ] virtual/udev-217-r4::gentoo [217-r3::gentoo] 0 KiB
[ebuild     U  ] sys-apps/portage-3.0.30-r4::gentoo [3.0.30-r2::gentoo] USE="(ipc) native-extensions rsync-verify xattr -apidoc -build -doc -gentoo-dev (-selinux) -test" PYTHON_TARGETS="python3_9 -pypy3 -python3_8 -python3_10" 0 KiB
[ebuild     U  ] virtual/tmpfiles-0-r2::gentoo [0-r1::gentoo] 0 KiB

Total: 11 packages (10 upgrades, 1 new, 2 uninstalls), Size of downloads: 61,126 KiB
Conflict: 2 blocks (all satisfied)

Many soft blocks,probably caused by
sys-apps/systemd-utils
ending up with a system with broken DE(just blinking cursor--maybe because of xfce-base/xfce4-panel-4.17.0?)
and a failure to initialize net(in my case br0).
WEIRD
NOTE:I tried to locate who is the culprit pulling
systemd-utils
and I think it might be
virtual/tmpfiles-0-r2
which includes in the ebuild

Code: Select all

RDEPEND="
	!prefix-guest? (
		|| (
			sys-apps/systemd-utils[tmpfiles]
			sys-apps/systemd-tmpfiles
			sys-apps/systemd
		)
	)"

BUT I DO have
sys-apps/systemd-tmpfiles
already installed.
Also in systemd-tempfiles i see

Code: Select all

RDEPEND="sys-apps/systemd-utils[tmpfiles]"

Something seems SERIOUSLY wrong.

PS:After reverting to pre-update backup ALL is fine again.

[Ionen]

systemd-utils combines sys-fs/udev, systemd-boot, and systemd-tmpfiles making them obsolete (thus the removal, there should be a news item soon'ish to make things clearer).

That aside, I can't really think of many reasons why this would cause problems because it installs the same things (and USE="tmpfiles udev" is enabled here).

Do you have an INSTALL_MASK with /lib/systemd or similar with custom package.env rules by any chances? This could cause you to lose the udev daemon and explain things.

[alamahant]

Do you have an INSTALL_MASK with /lib/systemd or similar with custom package.env rules by any chances? This could cause you to lose the udev daemon and explain things.

Not that i am aware of.
I dont think so.
Should i open a bug report for systemd-utils
?

[Ionen]

Unsure if was the cause of your issues but did find one thing that was wrong

May or may not help:
https://gitweb.gentoo.org/repo/gentoo.g ... 7992bb0902

Wait until --sync gives -r1 (systemd-utils-250.4-r1)

Edit: also we have a potentially similar sounding bug at #838940

[proteusx]

Fortunately I saw this before it caused me any grief.
Whenever I see *systemd* in the emerge list I scrutinize it.
My solution:

Code: Select all

echo ">=sys-fs/eudev-3.2.11-r2" >> /etc/portage/package.mask/systemd-crap.mask

Also, it may be a good idea to mask these too

Code: Select all

>=virtual/udev-217-r4
>=virtual/libudev-232-r6 

[Naib]

systemd-utils combines sys-fs/udev, systemd-boot, and systemd-tmpfiles making them obsolete (thus the removal, there should be a news item soon'ish to make things clearer).

That aside, I can't really think of many reasons why this would cause problems because it installs the same things (and USE="tmpfiles udev" is enabled here).

Do you have an INSTALL_MASK with /lib/systemd or similar with custom package.env rules by any chances? This could cause you to lose the udev daemon and explain things.

It doesn't seem as simple as that...
I use eudev but this new package is blocking

Code: Select all

[blocks B      ] sys-apps/systemd-utils[udev] ("sys-apps/systemd-utils[udev]" is soft blocking sys-fs/eudev-3.2.11-r2)
[blocks B      ] sys-fs/eudev ("sys-fs/eudev" is soft blocking sys-apps/systemd-utils-250.4-r1)

[Naib]

Fortunately I saw this before it caused me any grief.
Whenever I see *systemd* in the emerge list I scrutinize it.
My solution:

Code: Select all

echo ">=sys-fs/eudev-3.2.11-r2" >> /etc/portage/package.mask/systemd-crap.mask

Also, it may be a good idea to mask these too

Code: Select all

>=virtual/udev-217-r4
>=virtual/libudev-232-r6 

I don't blame you, but this package is being rude and assuming it will be on your system even though ZERO reason for it to be there ..

more packages for my mask file

[alamahant]

Ok
I can confirm that

Code: Select all

xfce-base/xfce4-panel-4.17.0

was not the cause of DE crash.
I hat it updated in a systemd system and DE works fine.
So it must be the
systemd-utils
thingie.

[Ionen]

I use eudev but this new package is blocking

Proper fix is to disable USE=udev on systemd-utils if you use eudev (systemd-utils[udev] == sys-fs/udev, so can't be together, but systemd-utils[tmpfiles,-udev] is fine), there'll be a news item soon'ish to explain that given portage's output is confusing.

[proteusx]

... but this package is being rude and assuming it will be on your system even though ZERO reason for it to be there ..

virtual/tmpfiles is the root of all evil but we do not really need it.

Code: Select all

echo "virtual/tmpfiles-0-r2" >> /etc/portage/profile/package.provided

[Heavenly]

EDIT:
On the development side, the current ebuild (250.4-r1) of "systemd-utils" has "kmod" as a hard dependency where I assume it should be under the ?udev USE section.
My setup doesn't use kmod so I had to edit the ebuild but thought it should probably be changed upstream. If I'm missing the reason for hard depends then I would like to know the reason.

[Ionen]

Wait until --sync gives -r1 (systemd-utils-250.4-r1)

That was most likely the issue according to reports, so do upgrade to -r1 for the fix.

[Hu]

... but this package is being rude and assuming it will be on your system even though ZERO reason for it to be there ..

virtual/tmpfiles is the root of all evil but we do not really need it.

Code: Select all

echo "virtual/tmpfiles-0-r2" >> /etc/portage/profile/package.provided

How should readers determine whether or not they need it on a given system? My understanding is that some packages malfunction if tmpfiles processing is skipped during early boot-up. If no packages needed tmpfiles support, why would the Gentoo maintainers go to the trouble of packaging it?

[proteusx]

Hu.

I have /tmp and /run mounted on tmpfs and I am using openrc-0.17
So there is no need for tmpfiles.
And with vitrual/tmpfiles in "package.provided" there are no side effects whatsoever for nearly 5 years.

The maintainers can package whatever "systemd-style" nonsense they like.
Equally I can choose to ignore them and modify my Gentoo as I see fit and share my experience in the forum.

[Hu]

Most people do not use openrc-0.17 anymore, and that is a key prerequisite for prohibiting installation of virtual/tmpfiles. If you want to encourage people not to install a freestanding tmpfiles package, you need to also encourage them to use an openrc that does not need such a package. Otherwise, you are setting them up for trouble that someone else will have to diagnose and clean up.

[Naib]

Most people do not use openrc-0.17 anymore, and that is a key prerequisite for prohibiting installation of virtual/tmpfiles. If you want to encourage people not to install a freestanding tmpfiles package, you need to also encourage them to use an openrc that does not need such a package. Otherwise, you are setting them up for trouble that someone else will have to diagnose and clean up.

you mean like the dev's pushing this out half baked and people coming here for help? that sort of thing. ZERO news item and it bricks a boot ...

[crowbert]

I'm not clear on why kmod is a dependency here. I make a point of compiling a kernel with all required modules built-in, sometimes not even including module support in the kernel (to reduce the attack surface on a server). I understand that udev may want to load modules when a new device is found, but that's certainly not required for a functional system, and before this change, nothing was pulling it in.

[Heavenly]

I'm not clear on why kmod is a dependency here. I make a point of compiling a kernel with all required modules built-in, sometimes not even including module support in the kernel (to reduce the attack surface on a server). I understand that udev may want to load modules when a new device is found, but that's certainly not required for a functional system, and before this change, nothing was pulling it in.

Yeah, I was confused by this as well. My kernel is built-in with no module support so the pulling of "kmod" took me by surprise when I updated. I'm hoping they change this upstream so I don't have to fork the ebuild each update. I just moved the "kmod" dependency to the ?udev section in the ebuild since I saw mention of "add kmod for udev" in the package changelog, though I'm not entirely convinced it's needed at all?

[sam_]

I'm not clear on why kmod is a dependency here. I make a point of compiling a kernel with all required modules built-in, sometimes not even including module support in the kernel (to reduce the attack surface on a server). I understand that udev may want to load modules when a new device is found, but that's certainly not required for a functional system, and before this change, nothing was pulling it in.

Yeah, I was confused by this as well. My kernel is built-in with no module support so the pulling of "kmod" took me by surprise when I updated. I'm hoping they change this upstream so I don't have to fork the ebuild each update. I just moved the "kmod" dependency to the ?udev section in the ebuild since I saw mention of "add kmod for udev" in the package changelog, though I'm not entirely convinced it's needed at all?

Fixed in 1 and 2.

As for a news item, it's in the works.

[codefossa]

I'm not sure if it's related, but I had very similar problems to OP, though not using eudev. While I now have no major issues, I do get clock skew warnings during boot that did not exist before this upgrade. Is this anything I should worry about, or is there a way for me to fix it?

Code: Select all

 * /proc is already mounted
 * /run/openrc: creating directory
 * /run/lock: correcting mode
 * /run/lock: correcting owner
 * Caching service dependencies ...
 [ ok ]
 * Caching service dependencies ...
 [ ok ]
 * Clock skew detected with `(null)'
 * Adjusting mtime of `/run/openrc/deptree' to Mon Apr 18 10:29:09 2022
 * WARNING: clock skew detected!
 * Mounting security filesystem ...
 [ ok ]
 * Mounting debug filesystem ...
 [ ok ]
 * Mounting config filesystem ...
 [ ok ]
 * Mounting persistent storage (pstore) filesystem ...
 [ ok ]
 * Mounting efivarfs filesystem ...
 [ ok ]
 * Mounting cgroup filesystem ...
 [ ok ]
 * Remounting devtmpfs on /dev ...
 [ ok ]
 * Mounting /dev/mqueue ...
 [ ok ]
 * Creating list of required static device nodes for the current kernel ...
 [ ok ]
 * Create Static Devices Nodes in /dev ...
 [ ok ]
 * Starting udev ...
 [ ok ]
 * Generating a rule to create a /dev/root symlink ...
 [ ok ]
 * Populating /dev with existing devices through uevents ...
 [ ok ]

rc sysinit logging stopped at Mon Apr 18 06:35:50 2022


rc boot logging started at Mon Apr 18 06:35:50 2022

 * WARNING: clock skew detected!
 * Setting system clock using the hardware clock [Local Time] ...
 [ ok ]
 * The binfmt-misc module needs to be loaded by the modules service or built in.
 * Mounting misc binary format filesystem ...
 [ ok ]
 * Loading custom binary format handlers ...
 [ ok ]
 * Checking local filesystems  ...
fsck.fat 4.2 (2021-01-31)
/dev/nvme0n1p5: 14 files, 30757/130812 clusters
 [ ok ]
 * Remounting filesystems ...
 [ ok ]
 * Updating /etc/mtab ...
 * Creating mtab symbolic link
 [ ok ]
 * Activating swap devices ...
 [ ok ]
 * Mounting local filesystems ...
 [ ok ]
 * Configuring kernel parameters ...
 [ ok ]
 * Creating user login records ...
 [ ok ]
 * Wiping /tmp directory ...
 [ ok ]
 * Setting hostname to gentoo  ...
 [ ok ]
 * Setting terminal encoding [UTF-8] ...
 [ ok ]
 * Setting keyboard mode [UTF-8] ...
 [ ok ]
 * Loading key mappings [us] ...
 [ ok ]
 * Bringing up network interface lo ...
 [ ok ]
 * Saving key mapping ...
 [ ok ]
 * Saving terminal encoding ...
 [ ok ]
 * Create Volatile Files and Directories ...
 [ ok ]
 * Initializing random number generator ...
 [ ok ]

rc boot logging stopped at Mon Apr 18 10:35:52 2022


rc default logging started at Mon Apr 18 10:35:52 2022

 * WARNING: clock skew detected!
 * Starting dbus ...
 [ ok ]
 * Starting NetworkManager ...
 [ ok ]
Connecting.......            1sConnecting........           1sConnecting.........          1sConnecting..........         1sConnecting...........        1sConnecting............       1sConnecting.............      1sConnecting..............     1sConnecting...............    1sConnecting...............    0s [offline]
 * Marking NetworkManager as inactive. It will automatically be marked
 * as started after a network connection has been established.
 * WARNING: NetworkManager has started, but is inactive
 * Starting chronyd ...
 [ ok ]
 * WARNING: netmount will start when NetworkManager has started
 * Setting up sddm ...
 [ ok ]
 * Starting local ...
 [ ok ]

rc default logging stopped at Mon Apr 18 10:35:54 2022

And here is what I saw yesterday. I am up to date with testing as of now though, so this may be slightly outdated, but includes what seems to have caused the issue.

Code: Select all

These are the packages that would be merged, in order:

Calculating dependencies  ... ... done!
[ebuild     U  ] dev-util/boost-build-1.79.0::gentoo [1.78.0-r1::gentoo] USE="-examples" 110,798 KiB
[ebuild     U  ] net-misc/rsync-3.2.4-r1::gentoo [3.2.3-r5::gentoo] USE="acl iconv ipv6 ssl xattr -examples -lz4 -stunnel -system-zlib -verify-sig -xxhash -zstd" PYTHON_SINGLE_TARGET="python3_9%* -python3_8% -python3_10%" 1,089 KiB
[ebuild     U  ] dev-libs/libpcre2-10.40:0/3::gentoo [10.39-r1:0/3::gentoo] USE="bzip2 jit pcre16 readline (split-usr) unicode zlib -libedit -pcre32 -static-libs -verify-sig%" ABI_X86="(64) -32 (-x32)" 1,726 KiB
[ebuild     U  ] dev-lang/perl-5.34.1-r1:0/5.34::gentoo [5.34.1:0/5.34::gentoo] USE="gdbm -berkdb -debug -doc -ithreads -minimal -quadmath" 177 KiB
[ebuild     U  ] sys-apps/coreutils-9.1::gentoo [9.0-r2::gentoo] USE="acl nls (split-usr) xattr -caps -gmp -hostname -kill -multicall (-selinux) -static -test -vanilla -verify-sig" 5,585 KiB
[ebuild     U  ] virtual/perl-Compress-Raw-Zlib-2.103.0::gentoo [2.101.0::gentoo] 0 KiB
[ebuild     U  ] virtual/perl-Compress-Raw-Bzip2-2.103.0::gentoo [2.101.0::gentoo] 0 KiB
[ebuild     U  ] virtual/perl-IO-Compress-2.103.0::gentoo [2.102.0::gentoo] 0 KiB
[ebuild     U  ] sys-libs/binutils-libs-2.38-r2:0/2.38::gentoo [2.38-r1:0/2.38::gentoo] USE="nls -64-bit-bfd (-cet) -multitarget -static-libs" ABI_X86="(64) -32 (-x32)" 190 KiB
[ebuild     U  ] media-libs/freetype-2.12.0-r1:2::gentoo [2.12.0:2::gentoo] USE="X adobe-cff bzip2 cleartype-hinting png svg -brotli -debug -doc -fontforge -harfbuzz (-infinality) -static-libs -utils" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild     U  ] sys-devel/binutils-2.38-r2:2.38::gentoo [2.38-r1:2.38::gentoo] USE="gold nls plugins (-cet) (-default-gold) -doc -multitarget -pgo -static-libs -test -vanilla" 0 KiB
[ebuild   R    ] dev-libs/libgcrypt-1.10.1:0/20::gentoo  USE="asm -doc -static-libs -verify-sig (-custom-cflags%)" ABI_X86="(64) -32 (-x32)" CPU_FLAGS_X86="-aes -avx -avx2 -padlock -sha -sse4_1" 0 KiB
[ebuild     U  ] dev-util/glib-utils-2.72.1::gentoo [2.72.0::gentoo] PYTHON_SINGLE_TARGET="python3_9 -python3_8 -python3_10" 4,777 KiB
[ebuild     U  ] dev-util/gdbus-codegen-2.72.1::gentoo [2.72.0::gentoo] PYTHON_SINGLE_TARGET="python3_9 -python3_8 -python3_10" 0 KiB
[ebuild     U  ] media-fonts/fontawesome-6.1.1:0/6::gentoo [5.15.3:0/5::gentoo] USE="X otf -ttf" 13,015 KiB
[ebuild     U  ] sys-devel/clang-14.0.1-r1:14::gentoo [14.0.1:14::gentoo] USE="(pie) static-analyzer xml -debug -default-compiler-rt -default-libcxx -default-lld -doc -llvm-libunwind -test" ABI_X86="(64) -32 (-x32)" LLVM_TARGETS="(AArch64) (AMDGPU) (ARM) (AVR) (BPF) (Hexagon) (Lanai) (MSP430) (Mips) (NVPTX) (PowerPC) (RISCV) (Sparc) (SystemZ) (VE) (WebAssembly) (X86) (XCore) (-ARC) (-CSKY) (-M68k)" PYTHON_SINGLE_TARGET="python3_9 -python3_8 -python3_10" 6 KiB
[ebuild     U  ] app-text/ghostscript-gpl-9.56.1:0/9.56::gentoo [9.55.0-r1:0/9.55::gentoo] USE="X cups dbus gtk (unicode) -static-libs" L10N="-de (-ja) -ko -zh-CN -zh-TW" 61,126 KiB
[ebuild     U  ] dev-libs/glib-2.72.1:2::gentoo [2.72.0:2::gentoo] USE="dbus elf (mime) xattr -debug -fam -gtk-doc (-selinux) -static-libs -sysprof -systemtap -test -utils" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild     U  ] x11-libs/pango-1.50.7::gentoo [1.50.6::gentoo] USE="X introspection -debug -sysprof" ABI_X86="(64) -32 (-x32)" 4,195 KiB
[ebuild     U  ] xfce-base/xfce4-panel-4.17.0::gentoo [4.16.3::gentoo] USE="dbusmenu -introspection -vala" 1,358 KiB
[ebuild     U  ] www-client/google-chrome-100.0.4896.127::gentoo [100.0.4896.88::gentoo] USE="(-selinux)" L10N="am ar bg bn ca cs da de el en-GB es es-419 et fa fi fil fr gu he hi hr hu id it ja kn ko lt lv ml mr ms nb nl pl pt-BR pt-PT ro ru sk sl sr sv sw ta te th tr uk vi zh-CN zh-TW" 85,024 KiB
[ebuild     U  ] virtual/libudev-232-r7:0/1::gentoo [232-r5:0/1::gentoo] USE="-systemd" ABI_X86="(64) -32 (-x32)" 0 KiB
[ebuild  N     ] sys-apps/systemd-utils-250.4-r1::gentoo  USE="(split-usr) tmpfiles udev -boot (-selinux) -sysusers -test" ABI_X86="(64) -32 (-x32)" 10,872 KiB
[uninstall     ] sys-fs/udev-249.9::gentoo  USE="acl kmod (split-usr) (-selinux) -test" ABI_X86="(64) -32 (-x32)"
[blocks b      ] <sys-fs/udev-250 ("<sys-fs/udev-250" is soft blocking sys-apps/systemd-utils-250.4-r1)
[ebuild     U  ] virtual/udev-217-r5::gentoo [217-r3::gentoo] 0 KiB
[ebuild     U  ] sys-apps/systemd-tmpfiles-250::gentoo [249.9::gentoo] USE="(-selinux%) (-test%)" 0 KiB
[blocks b      ] <sys-apps/systemd-tmpfiles-250 ("<sys-apps/systemd-tmpfiles-250" is soft blocking sys-apps/systemd-utils-250.4-r1)
[ebuild     U  ] virtual/tmpfiles-0-r3::gentoo [0-r1::gentoo] 0 KiB

Total: 26 packages (24 upgrades, 1 new, 1 reinstall, 1 uninstall), Size of downloads: 299,932 KiB
Conflict: 2 blocks (all satisfied)

[susancragin]

You may also want to check this old post.
viewtopic-p-8700665.html

[Naib]

do you get clock skew warning every boot?
it might be todo with daylight saving and your bios. Is your BIOS set to UTC

[Naib]

You may also want to check this old post.
viewtopic-p-8700665.html

old post? thats this thread and its from yesterday

[susancragin]

Sorry my mistake. Old post is below.
viewtopic-p-7608362.html

[codefossa]

do you get clock skew warning every boot?
it might be todo with daylight saving and your bios. Is your BIOS set to UTC

BIOS is set to local time, and yes I get the warning on every boot. Just to note. Without changing BIOS, I could revert the upgrade and I would not get the clock skew warning. Tested that a few times as the upgrade originally broke my system.