Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Avoiding opentmpfiles minimising silent breakage
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
jonathan183
Guru
Guru


Joined: 13 Dec 2011
Posts: 318

PostPosted: Wed Sep 26, 2018 10:53 pm    Post subject: Avoiding opentmpfiles minimising silent breakage Reply with quote

Ed: updated title - it became apparent that opentmpfiles is what I wanted to avoid and I have tmpfiles with openrc-0.13.11 already
Ed2: the thread title should probably have been how to deal with tmpfiles while retaining openrc-0.13.11

I have been running Gentoo for some time and have managed to avoid things like systemd by having in /etc/portage/package.mask

Code:
### prevent any systemd install
sys-apps/systemd
>sys-apps/usbutils-008
sys-auth/consolekit
sys-auth/polkit
sys-fs/udev


I currently have openrc in a local overlay
Code:
[ebuild   R    ] sys-apps/openrc-0.13.11::Jonathan_overlay  USE="ncurses netifrc newnet unicode -debug -pam (-prefix) (-selinux) -static-libs -tools" 0 KiB


Since sudo now requires opentmpfiles I currently have it in a local overlay with
Code:
inherit eutils pam multilib libtool
#inherit eutils pam multilib libtool tmpfiles


I want to avoid things being silently broken so have up until now avoided adding anything associated with tmpfiles in package.provided.
I have noticed in this thread a package.provided entry has been suggested.
Is the approach I am currently taking really any better at revealing issues and avoiding package breakage than adding an entry in package.provided ?
Is there a better way of avoiding silent breakage ?

I did suggest a USE flag in sudo but since any openrc currently in the tree would install tmpfiles this is unlikely to be supported ...


Last edited by jonathan183 on Fri Sep 28, 2018 1:52 pm; edited 3 times in total
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Wed Sep 26, 2018 11:13 pm    Post subject: Reply with quote

Code:
virtual/tmpfiles-0


add to /etc/portage/profile/package.provided and use the regular sudo ebuild, no need for it in the local overlay (one less file to take care of)

opentmpfiles is being pulled in by tmpfiles.eclass by way of the virtual/tmpfiles, the above will fix that since the old openrc works just fine as is.
The above will work with anything that uses the tmpfiles.eclass.

Edit to add: it works because I run openrc-0.13.11 myself.
I also update eix lately (also uses tmpfiles.eclass) and it's fine too.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
Tony0945
Watchman
Watchman


Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA

PostPosted: Wed Sep 26, 2018 11:53 pm    Post subject: Reply with quote

The only file I see installed by -r2 that is different from -r1 (no tmpfiles) is /usr/lib/tmpfiles.d/sudo.conf
Code:
MSI sudo # cat /usr/lib/tmpfiles.d/sudo.conf
# Create an empty sudo time stamp directory on OSes using systemd.
# Sudo will create the directory itself but this can cause problems
# on systems that have SELinux enabled since the directories will be
# created with the user's security context.
d /run/sudo 0711 root root
D /run/sudo/ts 0700 root root
Draw your own conclusions. Note that I have not done an exhaustive analysis of the package to see if tmpfiles are referenced anywhere else.
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 12:02 am    Post subject: Reply with quote

I see these ebuild using tmpfiles.eclass

app-admin/sudo
app-admin/puppetserver
app-emulation/virtualbox
app-metrics/collectd
app-portage/eix
dev-db/mysql-init-scripts
mail-filter/policyd-weight
net-ftp/proftpd
net-im/coturn
net-misc/connman
net-misc/quagga
net-misc/gerbera
net-p2p/resilio-sync
sys-apps/logwatch
www-servers/apache
x11-wm/xpra

over time there will probably be more added.
Lots of ebuilds place files in /usr/lib/tmpfiles.d manually right now, which is probably why they went with tmpfiles.eclass


edit to add: I can't say I agree with the whole concept of trying to make openrc look like systemd though, which is what this is an attempt at.
But it's either go with package.provided or start modifying lots more ebuilds as they transition.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21490

PostPosted: Thu Sep 27, 2018 1:32 am    Post subject: Reply with quote

Removing the inherit of the tmpfiles eclass, if that is all you do, is probably the wrong solution. It may work sometimes, but it will not define the functions normally provided by tmpfiles.eclass, so you will get errors in the ebuild (possibly non-fatal) about the absence of those functions. Faking the presence of virtual/tmpfiles is less trouble and safer. If allows the ebuild to install the tmpfiles configuration files (but you could delete them via INSTALL_MASK if you wish), so if you do later need to use the tmpfiles capability, the configuration files will already be there. They are only data files, so if you do not run a tmpfiles manager, they are inactive.

Personally, I'd find a way to provide working tmpfiles service on the system and then let the packages use tmpfiles as normal. If your chosen version of openrc does not natively handle tmpfiles, you will need some other processor for them. Standard processors may not accept your old openrc, so you might need a local patch to get the processor to work.

Regardless of the path you choose, keep in mind that your ultimate goal should be for the computer to do what you need. If you spend substantial time fighting the package system, you aren't spending that time doing something you like. Pick a solution that minimizes your ongoing system maintenance burden.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6920

PostPosted: Thu Sep 27, 2018 1:54 am    Post subject: Re: Avoiding tmpfiles minimising silent breakage Reply with quote

jonathan183 wrote:
I want to avoid things being silently broken so have up until now avoided adding anything associated with tmpfiles in package.provided.

What do you do to avoid having a broken /run structure? Manually add mkdir entries to /etc/local.d?
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Thu Sep 27, 2018 4:59 am    Post subject: Reply with quote

Hu wrote:
Personally, I'd find a way to provide working tmpfiles service on the system and then let the packages use tmpfiles as normal.

++
Unavoidable for:
jonathan183 wrote:
I want to avoid things being silently broken

tmpfiles is by packages not only used for a correctly set up /run or /tmp, but tmpfiles.eclass is also used for a correct setup of files in some /var/* directories which on some systems might be temporary.

The reason why sys-apps/opentmpfiles blocks <sys-apps/openrc-0.23 is that openrc provided the tmpfiles service on its own in versions 0.xx-0.22*.
So "actually" the blocker should be on a certain range of openrc versions, but ranges are not supported by portage/pms. (For users without an ancient openrc in their local overlay, the given blocker is a correct workaround in the absence of ranges.)
Therefore, the non-breaking solution is to put (and update from time to time) sys-apps/opentmpfiles into your local overlay, removing the blocker.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7470

PostPosted: Thu Sep 27, 2018 7:02 am    Post subject: Reply with quote

it is avoidable if the package itself make no use of tmpfiles.d
but in your case
Code:
      --enable-tmpfiles.d="${EPREFIX}"/usr/lib/tmpfiles.d

so your problem is not only avoiding tmpfiles.d eclass, but also make sudo work properly without tmpfiles.d presence, which doesn't look sane with that default parameter set at build time.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Sep 27, 2018 9:05 am    Post subject: Reply with quote

mv wrote:
tmpfiles is by packages not only used for a correctly set up /run or /tmp, but tmpfiles.eclass is also used for a correct setup of files in some /var/* directories which on some systems might be temporary.

mv ... that is a misuse of the word(s) "correct[ly]", but we've been over this previously, so, again, when, and by what mysterious process, did these unnamed files in /var/* become "temporary"? ... that's a rhetorical question btw.

@jonathan183 ... that version of openrc includes opentmpfiles, and if you have 'tmpfiles.dev' and 'tmpfiles.setup' in the runlevel, then nothing unexpected should happen were you to package.provided sys-apps/opentmpfiles. If those are not in the runlevel then you would need to check the contents of /usr/lib/tmpfiles.d and (if needed) modify the initscript so that necessary files/directories are created, or in the case of sudo, eix, etc, then ignore and/or do whats needed via /etc/local.d ... all the while praising the gods of system improvement for making your system more robust :)

best ... khay
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 10:03 am    Post subject: Reply with quote

Yes the older versions had tmpfiles that worked, /lib64/rc/sh/tmpfiles.sh

It got removed then added back by way of sys-apps/opentmpfiles, all in the name of making openrc compatible with systemd. *sigh*

Either solution works,

add to package.provided
Code:
virtual/tmpfiles-0


OR

modify virtual/tmpfiles by adding something like
Code:
<sys-apps/openrc-0.23.0

to the selection criteria (I think this will work, but it's off the top of my head and not tried)

In either case, the result is opentimfiles is not used, not loaded and the original openrc tmpfiles code will work in its place.


edit to add: This is the offending portion of code
Code:
grep virtual /usr/portage/eclass/tmpfiles.eclass
RDEPEND="virtual/tmpfiles"


Modifying opentmpfiles is not the answer.
This is what needs to change if one goes the direction of modifying ebuilds.
Code:
cat /usr/portage/virtual/tmpfiles/tmpfiles-0.ebuild
...
RDEPEND="
   || (
      sys-apps/opentmpfiles
      sys-apps/systemd
   )"


edit to add 2: This is what is in openfilestmp tarball
Code:
./tmpfiles
./openrc
./openrc/opentmpfiles-setup.initd
./openrc/opentmpfiles-setup.confd
./openrc/opentmpfiles-dev.initd
./openrc/opentmpfiles-dev.confd
./license
./README.md
./Makefile

The init.d/conf.d stuff was originall called tmpfiles.dev and tmpfiles.setup in old openrc
and tmpfiles was called tmpfiles.sh in the old openrc.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
jonathan183
Guru
Guru


Joined: 13 Dec 2011
Posts: 318

PostPosted: Thu Sep 27, 2018 11:11 am    Post subject: Reply with quote

Thanks everyone for your help ...
virtual/tmpfiles-0 in package.provided it is 8)

Ed: It is opentmpfiles which I wanted to avoid, this became clear as a result of the earlier posts in this thread.
I already had tmpfiles as part of openrc-0.13.11.
Adding virtual/tmpfiles-0 to package.provided is the right solution for me.
I have removed sudo from my local overlay and reverted to the main tree version.


Last edited by jonathan183 on Fri Sep 28, 2018 12:10 pm; edited 1 time in total
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 12:18 pm    Post subject: Reply with quote

From the top of the tmpfiles shell script from opentmpfiles
Quote:
# This is a reimplementation of the systemd tmpfiles.d code
# Control creation, deletion, and cleaning of volatile and temporary files
#
# Copyright (c) 2012 Gentoo Foundation
# Released under the 2-clause BSD license.
#
# This instance is a pure-POSIX sh version, written by Robin H Johnson
# <robbat2@gentoo.org>, based on the Arch Linux version as of 2012/01/01:
# http://projects.archlinux.org/initscripts.git/tree/arch-tmpfiles
#
# See the tmpfiles.d manpage as well:
# https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html
# This script should match the old manpage
# http://0pointer.de/public/systemd-man/tmpfiles.d.html
# as of 2012/03/12 and also implements some more recent features


Why? Why does openrc have to emulate systemd or why does the lead dev (Hubbs) think that it has to.
That's so glaringly stupid its not funny.
If systemd does something stupid and the computer crashes and burns, does openrc have to follow suit?
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Sep 27, 2018 2:44 pm    Post subject: Reply with quote

Anon-E-moose wrote:
tmpfiles shell script from opentmpfiles:
# This is a reimplementation of the systemd tmpfiles.d code

Why? Why does openrc have to emulate systemd or why does the lead dev (Hubbs) think that it has to.

Anon-E-moose ... that's an easy one:

Filesystem_Hierarchy_Standard#FHS_compliance wrote:
Modern Linux distributions include a /run directory as a temporary filesystem (tmpfs) which stores volatile runtime data, following the FHS version 3.0. According to the FHS version 2.3, such data were stored in /var/run but this was a problem in some cases because this directory is not always available at early boot.

You see, it's a problem, your /var/run isn't availabe for volitle runtime data at early boot, and now we have /run (for "early boot") lets move all kinds of stuff there (like lockfiles, or what-have-you), and declare other stuffs under /var volitile, because it's obvious that because ... bwahhhahhhaaa. Having done that we now need a method of making sure that the necessary directory structure is now created, with the correct permissions, etc, ... see, much better, what could go wrong.

Anon-E-moose wrote:
If systemd does something stupid and the computer crashes and burns, does openrc have to follow suit?

Absolutely, the reasoning being that "gentoo follows upstream", anyone who doesn't see the sense in that is poo-pooing our fine work and can be dismissed as a "vocal minority".

best ... khay
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 3:07 pm    Post subject: Reply with quote

Upstream is not forcing us to do this, it's Hubbs insane desire to make openrc look like systemd.
Since there are holdouts not wanting a systemd only distro, he's decided to slowly destroy it by making it look like the warmed over coprophilic pile called systemd.
That's all on Hubbs and should be a reason for his dismissal from power in gentoo.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Sep 27, 2018 3:17 pm    Post subject: Reply with quote

Anon-E-moose wrote:
Upstream is not forcing us to do this, it's Hubbs insane desire to make openrc look like systemd.

Anon-E-moose ... I'm just mirroring the reasoning generally provided, "gentoo follows upstream" isn't my invention, it's used to justify such changes.

best ... khay
Back to top
View user's profile Send private message
Tony0945
Watchman
Watchman


Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA

PostPosted: Thu Sep 27, 2018 3:25 pm    Post subject: Reply with quote

My overlay ebuild http://dpaste.com/27WCY34
I cut off at 0.17 because that's where I decided to stop. Higher may work as well. Can anyone verify that the change was made at 0.23? In any case other stuff has been added along the way like automatically loading all modules. I'd really like to study all the openrc tarballs to see what really changed but i can't find them all.

Package.provided was a good idea but I think that it is planned to drop package.provided from portage in the near future.
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 3:28 pm    Post subject: Reply with quote

khayyam I know. ;)

I just wonder if freedesktop.org said to cut off your pinky finger, if Hubbs would run to the kitchen and grab a knife and start sawing.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Thu Sep 27, 2018 3:41 pm    Post subject: Reply with quote

Tony, https://github.com/openrc/openrc/ should have all of them going back to version 18, I have 13 version, because it's what I use.

Edit to add: quick glance shows version 22 with tmpfiles for init.d/conf.d and tmpfiles.sh, version 23 doesn't show them

ETA2: this may get you all the old tarballs https://gitweb.gentoo.org/proj/openrc.git/refs/tags
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Thu Sep 27, 2018 4:06 pm    Post subject: Reply with quote

Anon-E-moose wrote:
khayyam I know. ;) I just wonder if freedesktop.org said to cut off your pinky finger, if Hubbs would run to the kitchen and grab a knife and start sawing.

Anon-E-moose ... if the rational given was that he would be 20% less likely to hit one of those digits with a hammer then, possibly, as that's about where upstream is at in terms of its thinking.

best ... khay
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Thu Sep 27, 2018 9:11 pm    Post subject: Reply with quote

khayyam wrote:
that is a misuse of the word(s) "correct[ly"

No, it's not. "correctly" means that you will not suffer from silent breakage which is what you will get if the ebuilds and init scripts do not create the directories in the way the application expects it; this breakage will happen without tmpfiles. One can program manual fixes afterwards, once one sees what gets broken and why, but the OP asked how to avoid silent breakage. Putting virtual/tmpfiles into package.provided is a receipt for silent breakage.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21490

PostPosted: Fri Sep 28, 2018 1:34 am    Post subject: Reply with quote

Regardless of whether systemd's design was a good one, the reality is that notable programs have adopted its tmpfiles design and now depend on certain directories being available without being created by the application. Those directories are required to exist for proper functionality. To ensure their existence, your choices are:
  • Ensure the directories are stored on a filesystem that persists across reboots. Given the wide variance in Gentoo systems, ensuring this is hard. It becomes even worse when you consider that the system administrator may transplant a functioning system by way of bulk file copy, so it is not sufficient to ensure that, at package install time, the filesystem is persistent. The administrator may later rearrange to have some filesystem become non-persistent and not reinstall the package, so any hook in the ebuild would not be given a chance to reject the change.
  • Patch the application to generate the directories when needed. This would need to be done to every application that otherwise requires tmpfiles. It would need to be maintained across version upgrades. It may be difficult to do correctly in cases where upstream wants a root:root mode 1777 directory for unprivileged users to write into. Since the users are unprivileged, you cannot simply create the directory on first run. It needs to be pre-created by a privileged process, which upstream might not provide.
  • Create the directories at boot. This is the current tmpfiles approach. If you are going to go this path, it is far easier to have an implementation that can read systemd's tmpfiles format (ugly though it is), so that you can reuse the tmpfiles configuration files created and maintained by the upstream authors. If you invent your own configuration format, then you must maintain all the per-package configuration files yourself, including updating them as requirements change.
Given those choices, supporting tmpfiles is the path of least resistance.
Back to top
View user's profile Send private message
Tony0945
Watchman
Watchman


Joined: 25 Jul 2006
Posts: 5127
Location: Illinois, USA

PostPosted: Fri Sep 28, 2018 2:23 am    Post subject: Reply with quote

Hu wrote:
Given those choices, supporting tmpfiles is the path of least resistance.

Actually, the path of least resistance is to abandon openrc, runit,and s6 and just adopt systemd.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 21490

PostPosted: Fri Sep 28, 2018 4:04 am    Post subject: Reply with quote

I took it as a given that systemd is not a valid choice due to its many misfeatures and the general attitude of its developers across a variety of issues. With that choice ruled out, supporting tmpfiles is, in my opinion, the least trouble.
Back to top
View user's profile Send private message
khayyam
Watchman
Watchman


Joined: 07 Jun 2012
Posts: 6227
Location: Room 101

PostPosted: Fri Sep 28, 2018 7:48 am    Post subject: Reply with quote

mv wrote:
tmpfiles is by packages not only used for a correctly set up /run or /tmp, but tmpfiles.eclass is also used for a correct setup of files in some /var/* directories which on some systems might be temporary.

khayyam wrote:
[...] that is a misuse of the word(s) "correct[ly]", but we've been over this previously, so, again, when, and by what mysterious process, did these unnamed files in /var/* become "temporary"? ... that's a rhetorical question btw.

mv wrote:
No, it's not. "correctly" means that you will not suffer from silent breakage which is what you will get if the ebuilds and init scripts do not create the directories in the way the application expects it; this breakage will happen without tmpfiles. One can program manual fixes afterwards, once one sees what gets broken and why, but the OP asked how to avoid silent breakage. Putting virtual/tmpfiles into package.provided is a receipt for silent breakage.

mv ... here we are again with your selective quoting (corrected above) ... is it not perfectly clear that by harking back to that previous discussion I'm speaking to the question of technical correctness, and not how one now copes with these ill conceived changes? Prior to the introduction of this unnecessary volitility there was no silent breakage, or issue in need of a solution, and so "what you will get" is premised by "what was done" not "what you must do now".

best ... khay
Back to top
View user's profile Send private message
Anon-E-moose
Watchman
Watchman


Joined: 23 May 2008
Posts: 6095
Location: Dallas area

PostPosted: Fri Sep 28, 2018 9:43 am    Post subject: Reply with quote

Using package provided when one is using an older openrc that supports tmpfiles does not do away with tmpfile support it does away with the ebuild conflict, it will not cause silent breakage, at least as far as tmpfile support.

For those who don't understand please read the above several times till you do understand.

Edit to add: and if someone says "in the future they may change opentmpfiles and that would break older stuff"
yes, they might do that, in that case a patch could be made either against the already installed pieces /etc/init.d/tmpfiles* /etc/conf.d/tmpfiles* /lib/sh/rc/tmpfiles.sh or put into a patch file for the ebuild and re-emerge the older openrc.
6 of one half-dozen of the other.

ETA2: I did look at the difference between what's in openrc-0.13 and opentmpfile-0.1.3 and the init.d files differ in where the tmpfiles shell script is called from, but the options passed are exactly the same, other than location of shell script.
Opentmpfiles puts the tmpfiles shell script in /bin instead of /lib/rc/sh/ which is stupid, AFAIC, as it's not something that should be used indiscriminately, which it might be if left in /bin.
The tmpfiles shell script has been changed, mostly to add new options, for dry-run, etc. but overall it does the same things as tmpfiles.sh from <openrc-0.23.
_________________
PRIME x570-pro, 3700x, 6.1 zen kernel
gcc 13, profile 17.0 (custom bare multilib), openrc, wayland
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum