| View previous topic :: View next topic |
| Author |
Message |
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
 Posted: Tue May 28, 2019 6:14 pm Post subject: Portage feature bloat or well needed: namespaces |
 |
|
Curious as to what people feel about the "features" added into portage as of late - specifically things that require kernel support, namely, namespace (PID, network, more?) and better sandboxing.
These features I suspect are to help against protect your system against rogue ebuilds that download scripts to run and install.
What is your opinion: feature bloat or necessary feature?
I just get caught up with it on machines with "minimal kernels" that have as many features removed to streamline them. Now that gentoo-sources have that patch to enable portage features, it helps against forgetting these things, but still kind of annoying to enable more kernel bloat...
Comments?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
 |
mike155
Advocate
Joined: 17 Sep 2010
Posts: 4438
Location: Frankfurt, Germany
|
| Posted: Tue May 28, 2019 7:03 pm Post subject: |
|
|
I would like to know more about these 'new' features. Is there a 'design document'? Or at least any documentation?
I'm not an expert, but I think that kernel namespaces - if applied correctly - can make the sandbox and the build process more secure. |
|
| Back to top |
|
|
szatox
Advocate
Joined: 27 Aug 2013
Posts: 3131
|
| Posted: Tue May 28, 2019 7:53 pm Post subject: |
|
|
If you wanted to make a rogue ebuild, why not inject your rogue code into the application itself?
We must trust developers and maintainers, there is no way around it. Is the new version any better than the regular sandbox when it comes to guarding our systems from silly mistakes? |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Tue May 28, 2019 8:24 pm Post subject: |
|
|
Network sandbox is the most important, it holds ebuilds to a minimum standard so they don't decay into lazy wrappers around security nightmares like Go, npm, or worse, curl-pipe-sh commands.
PID sandbox is there so when you Ctrl+C or compilation is finished it actually terminates all child processes. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Tue May 28, 2019 9:01 pm Post subject: |
|
|
Not sure how everyone felt before these changes, especially ones that need kernel options changed in order to allow for them to work. Again, it's just that more options are needed in the kernel where they weren't necessary in the past, and each "Y" adds memory requirements...
I didn't consider host/target situation and using it to help prevent build mistakes if the ebuild could possibly affect the host. Most of the times the ebuilds don't do anything that could harm the host while building for a target. If the ebuilds just don't do anything dangerous so that something like this doesn't happen, I wouldn't consider this to be a problem, so it seemed like feature bloat. But ebuilds that depend on upstream running scripts on your machine is potentially an issue.
I've never worried about process aborting, it had generally worked fine in the past and don't really mind having to do manual process cleanup when it doesn't work.
Again I'm not claiming these changes are completely superfluous, they do have their uses, but maintaining machines that don't have resources, these add up...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Tue May 28, 2019 10:00 pm Post subject: |
|
|
| eccerr0r wrote: | | Again I'm not claiming these changes are completely superfluous, they do have their uses, but maintaining machines that don't have resources, these add up... |
What do they add up to? I turned on all the namespace options on my ten-year-old netbook and didn't see a significant slowdown or loss of memory. The biggest resource hog in portage code remains the dependency resolver and ebuild parsing. |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
| Posted: Tue May 28, 2019 10:13 pm Post subject: |
|
|
| eccerr0r wrote: | Not sure how everyone felt before these changes, especially ones that need kernel options changed in order to allow for them to work. Again, it's just that more options are needed in the kernel where they weren't necessary in the past, and each "Y" adds memory requirements...
I didn't consider host/target situation and using it to help prevent build mistakes if the ebuild could possibly affect the host. Most of the times the ebuilds don't do anything that could harm the host while building for a target. If the ebuilds just don't do anything dangerous so that something like this doesn't happen, I wouldn't consider this to be a problem, so it seemed like feature bloat. But ebuilds that depend on upstream running scripts on your machine is potentially an issue.
I've never worried about process aborting, it had generally worked fine in the past and don't really mind having to do manual process cleanup when it doesn't work.
Again I'm not claiming these changes are completely superfluous, they do have their uses, but maintaining machines that don't have resources, these add up... |
At least it will still install and they aren't forcing you to make the kernel changes... "right now"
| Code: | >>> Installing (1 of 1) sys-apps/portage-2.3.67::gentoo
* Setting make.globals default DISTDIR="/usr/portage/distfiles" for backward compatibility
* Setting repos.conf default location = /usr/portage for backward compatibility
* Messages for package sys-apps/portage-2.3.67:
* CONFIG_IPC_NS: is not set when it should be.
* CONFIG_PID_NS: is not set when it should be.
* CONFIG_NET_NS: is not set when it should be.
* Please check to make sure these options are set correctly.
* Failure to do so may cause unexpected problems.
>>> Auto-cleaning packages...
>>> No outdated packages were found on your system.
* GNU info directory index is up-to-date.
|
You can still disable them though make.conf $FEATURES to avoid any future surprises?
I get where you are coming from wanting a extra-lean system. I think the choice to not use the features is still an option. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Tue May 28, 2019 10:48 pm Post subject: Re: Portage feature bloat or well needed: namespaces |
|
|
| eccerr0r wrote: | | These features I suspect are to help against protect your system against rogue ebuilds that download scripts to run and install. |
I kinda agree with eccerr0r there, i don't see the point of these features.
But i disagree with him about the possible cause, seriously if an ebuild is made to download some script from somewhere and run it, these features won't protect anyone anyway, while only sandbox could help prevent hell from them (a feature we already have).
But maybe i'm not aware if it's common practice, for me the only case i know are nvidia drivers, but while it download the .run file i don't think it actually run it.
Can we get a practical case of what these features cover? |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Wed May 29, 2019 12:23 am Post subject: |
|
|
This is all speculation as I haven't thought of all possibilities yet.
One specific problem is that devices files in a chroot are still pointing to the host. New kernel namespaces and possibly even containers will protect against this, and isn't a feature in base Unix. But I thought the devs were supposed to look for this :) *j/k* yes if the ebuild is forced to download a script and run, this can't be protected against.
Likewise the network namespace. Having a ebuild needing to go download a file outside of portage...evil! And this does protect against it likely, but true if it's in a filesystem sandbox, it shouldn't propagate out of the sandbox...
I just was a bit alarmed about the two minor annoyances with the pid namespace (spews warnings everywhere) and then the network namespace (causes distcc to chunk badly by making seemingly good IP addresses unreachable).
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed May 29, 2019 12:56 am Post subject: |
|
|
| eccerr0r wrote: |
Likewise the network namespace. Having a ebuild needing to go download a file outside of portage...evil! And this does protect against it likely, but true if it's in a filesystem sandbox, it shouldn't propagate out of the sandbox... |
well, downloading and running are two different things, gentoo-sources do that for its patches files. You mean not only it bug distcc, but also for kernel patches and the like?
Nor i see a value in protecting nfs (if user use portage on nfs), which again should be handle by sandbox imo.
I don't understand them correctly, but i think it's kinda a protection for specific cases, which raise the question: why enable that per default then?
Still with the "i don't understand them correctly" emphasis.
I was more expecting answer from skellr who claims "Necessary feature"... |
|
| Back to top |
|
|
berferd
Tux's lil' helper
Joined: 13 May 2004
Posts: 117
|
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed May 29, 2019 1:05 am Post subject: |
|
|
ok you help me a lot to decide
| Quote: | It should be noted that all of the features follow the systemd idea of
supporting Linux only and require fancy kernel features. |
and i have stop reading there, not for me  |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
| Posted: Wed May 29, 2019 1:14 am Post subject: |
|
|
| krinn wrote: | | I was more expecting answer from skellr who claims "Necessary feature"... |
It was one of the two answers the OP requested in the opening post. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21595
|
| Posted: Wed May 29, 2019 1:54 am Post subject: Re: Portage feature bloat or well needed: namespaces |
|
|
| eccerr0r wrote: | | Curious as to what people feel about the "features" added into portage as of late - specifically things that require kernel support, namely, namespace (PID, network, more?) and better sandboxing. |
I approve. I experimented with doing similar things with wrappers around emerge, but some of these only really work well if integrated directly into Portage. Network sandbox in particular is very painful to kludge in as a wrapper, since the fetch phase should have access, and later phases should be confined. To support that in a wrapper, you either run every emerge twice (once with --fetchonly outside the sandbox, then again with regular options inside) or you provide a proxy and configure FETCHCOMMAND to use it. With the feature integrated in Portage, Portage can know whether to move particular steps into the sandbox because it knows what phase is being processed.
| eccerr0r wrote: | | These features I suspect are to help against protect your system against rogue ebuilds that download scripts to run and install. |
| szatox wrote: | | If you wanted to make a rogue ebuild, why not inject your rogue code into the application itself? |
I think it is important in this discussion to distinguish between rogue code that intends to harm the system and rogue code that, while not malicious, would never pass a code review administered by an experienced maintainer. The sandbox is a great alternative to trusting that every ebuild and the associated upstream code was properly reviewed for good practice. It may help against some forms of malicious code as a nice side effect, but users should not expect the sandbox to provide perfect protection against an actively malicious adversary. Even if it keeps the ebuild confined, there is still the problem, as szatox noted, that this only confines the build process. Suppose a malicious upstream wrote a build script that is perfectly innocuous, but included malware in the installed program. The sandbox will keep the innocuous build on good behavior, then on completion, Portage will copy the malicious program to your filesystem, where it can lurk until someone runs it - outside the sandbox.
| szatox wrote: | | Is the new version any better than the regular sandbox when it comes to guarding our systems from silly mistakes? |
Yes. The classic sandbox can be defeated by a program that wishes to subvert it, because it is implemented as an in-process interceptor that tries to filter out disallowed actions. Additionally, it has the usual problem of an inline filter. It attempts to determine how the kernel will interpret the request, then analyzes whether that interpretation would violate policy. If it misinterprets how the kernel will handle the request, or if it fails to intercept a call, then a prohibited action will not be blocked. The namespace based approach instructs the kernel to enforce the restrictions, so it can only be bypassed if there is a bug in the kernel. Additionally, since kernel namespaces are available on all distributions, and are sometimes used to enforce security boundaries, it is reasonable to expect that the kernel namespace logic will receive more attention than the userspace sandbox library.
| eccerr0r wrote: | | I just was a bit alarmed about the two minor annoyances with the pid namespace (spews warnings everywhere) and then the network namespace (causes distcc to chunk badly by making seemingly good IP addresses unreachable). |
Better error messages may be appropriate here. Portage supposedly makes distcc do the right thing, according to the manual. If it isn't doing that for you, that is worth investigating. See man make.conf for the feature network-sandbox-proxy. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Wed May 29, 2019 5:48 am Post subject: |
|
|
So the ultimate reasoning for these things is just to be able for senior developers to portage-allow random lower level devs write ebuilds without having to go through them (let portage be the gatekeeper instead of manual review)?
So no real enduser benefit other than possible faster turnaround for ebuild writing?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Wed May 29, 2019 4:32 pm Post subject: |
|
|
The benefit is not for the end user. The benefit is for support groups who have to diagnose how some of the worse Help Vampires broke their system after the fact. Removing ways to screw things up horribly, when those people are already downloading random badly-written ebuilds off the web and running them without a glimmer of understanding, is only going to save hours in the long run.
Until Gentoo manages to fully shed this "ricer distro" reputation some of them assign to it, we're going to have to put up with them. May as well build them a padded cell in the meantime. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Wed May 29, 2019 5:14 pm Post subject: |
|
|
Once again more bloat for something that doesn't help the end user...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
asturm
Developer
Joined: 05 Apr 2007
Posts: 8935
|
| Posted: Wed May 29, 2019 5:55 pm Post subject: |
|
|
| Did you actually care to measure that 'bloat'? |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Wed May 29, 2019 6:22 pm Post subject: |
|
|
A little here won't hurt, a little there won't hurt. Just a bit more here...
... and we get the monster that we have to deal with today.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
| Posted: Wed May 29, 2019 7:05 pm Post subject: |
|
|
| eccerr0r wrote: | A little here won't hurt, a little there won't hurt. Just a bit more here...
... and we get the monster that we have to deal with today. |
I can agree with that idea. Some things can just creep in and you don't know how far you are into the rabbit hole until it's too late.
I don't see it here, yet. It's easy to opt-out of if you don't want it and it hasn't caused alot of issues since it was implemented. Who remembers the last bug report or forum post relevant to the feature?
Yet it should be disabled for everyone because of your corner-case situation when you can't be bothered to disable it in make.conf? OK, that's practical. |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Wed May 29, 2019 7:29 pm Post subject: |
|
|
Might I propose that if you're that worried about security causing slowdowns, throw your Intel CPU away.
We have numbers for those: they're at least 16% and growing. |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
| Posted: Wed May 29, 2019 7:38 pm Post subject: |
|
|
| Ant P. wrote: | Might I propose that if you're that worried about security causing slowdowns, throw your Intel CPU away.
We have numbers for those: they're at least 16% and growing. |
Yeah, it's always a seesaw situation. Someone always taking turns poking their head up into line of fire when they become popular: security through obscurity. |
|
| Back to top |
|
|
eccerr0r
Watchman
Joined: 01 Jul 2004
Posts: 9677
Location: almost Mile High in the USA
|
| Posted: Wed May 29, 2019 7:51 pm Post subject: |
|
|
But we're not talking about security here since we confirmed there is no end user benefit.
Yes these things can be disabled, alas sooner or later, like all feature creep, the disabling will break and thus become mandatory...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching? |
|
| Back to top |
|
|
skellr
l33t
Joined: 18 Jun 2005
Posts: 975
Location: The Village, Portmeirion
|
| Posted: Wed May 29, 2019 7:56 pm Post subject: |
|
|
| eccerr0r wrote: | But we're not talking about security here since we confirmed there is no end user benefit.
Yes these things can be disabled, alas sooner or later, like all feature creep, the disabling will break and thus become mandatory... |
I don't agree with your specific issue, but I do agree with your overall concern. It's something to keep an eye on.
You can just ignore me the "Layman", time will tell. |
|
| Back to top |
|
|
|
Gentoo Chat
