Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage feature bloat or well needed: namespaces
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
skellr
l33t
l33t


Joined: 18 Jun 2005
Posts: 901
Location: The Village, Portmeirion

PostPosted: Wed May 29, 2019 8:05 pm    Post subject: Reply with quote

Now , I understand why people want to punt the forums...
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Wed May 29, 2019 11:11 pm    Post subject: Reply with quote

skellr wrote:
Now , I understand why people want to punt the forums...

Yes indeed, this is because there are people who don't want to see anyone disagreeing with them and posts backhanded responses indicating their own opinion is correct and not want further discussion.

Except this is not world/national politics, but something that we as Gentoo users have to deal. Hence this is "Gentoo Chat" versus "OTW". The problem is that I know a lot of us do want lean, customized installations with as little enabled as possible to save memory or whatnot, one reason why there's such a systemd fiasco.

The whole point of this thread is to get some opinions from people on both sides, and glad some people added in ways to workaround the problem, though these opt outs will stop working at some point. Those who simply say "this is the way it is and deal with it" without giving good reason how it improves enduser experience is also part of the problem.

Again "necessary feature" why? We've had portage for years without it and it worked, why now? I'm just hoping it's not a "solution looking for a problem" kind of deal which is often attributed to one piece of software. As some people alluded to, perhaps indeed it's to deal with bad build scripts that do funny things with your computer and these do block them, but it's a good point if these are actually in the application themselves and these features no longer protect you after they're installed. Or perhaps is it specifically because of evil things like pip, cargo, cmake?, and other user mode installation scripts that weren't meant to be installed as root - if that's the case, then I do approve of it as I loathe pip/cargo when they're forced to be integrated with portage.

If it's just a developer vs developer war which is sort of the pip/cargo deal, then someone please say so, that's also a reasonable answer, just that end users end up cleaning the mess...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6014

PostPosted: Wed May 29, 2019 11:23 pm    Post subject: Reply with quote

Before you go off on one calling everyone else ignorant and arrogant:

You could've looked up /sandbox in make.conf(5) and disabled it within 2 minutes, but you decided this thread was a better use of everyone's time. You've produced literally nothing to demonstrate these things even cause a slowdown, yet you expect anyone to care.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Wed May 29, 2019 11:51 pm    Post subject: Reply with quote

Again, the problem is feature creep, and guarantee the workaround/defeature will break at some point. If it's valuable, I'd like to know exactly where the value is, who truly values it.

Else systemd is the best piece of software in the world because it has so many features!
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
skellr
l33t
l33t


Joined: 18 Jun 2005
Posts: 901
Location: The Village, Portmeirion

PostPosted: Thu May 30, 2019 12:47 am    Post subject: Reply with quote

eccerr0r wrote:
skellr wrote:
Now , I understand why people want to punt the forums...

Yes indeed, this is because there are people who don't want to see anyone disagreeing with them and posts backhanded responses indicating their own opinion is correct and not want further discussion.

Except this is not world/national politics, but something that we as Gentoo users have to deal. Hence this is "Gentoo Chat" versus "OTW". The problem is that I know a lot of us do want lean, customized installations with as little enabled as possible to save memory or whatnot, one reason why there's such a systemd fiasco.

My responses have been a bit back-handed. Not towards you, but your idea. It's a feature(s) that has been implemented for a few years now without a problem and now its an issue because of some corner-case situation? I get you don't want to fall into a rabbit hole and find out whats really going on until it's too late. This isn't Ubuntu yet. It's good to question things, but I really don't see the issue for something that can easily be disabled.

I included the "Forums getting punted" response because it is the same person that was questioning the forums was the same person that implemented the features you were having an issue with. It was just a tin hat/jab at the subsequent coincidence of your OP. maybe there is truth in sarcasm? Probably not. But we can go there because this is the "Forum" anyway? It has it's purpose. I'm sure they don't want to hear this debate in Bugzilla or gentoo-dev right? :) ohh, wait not everyone can participate to gentoo-dev ATM. And the Forum users tend to be separatists? Sorry, just Me going off on a tangent...

eccerr0r wrote:

The whole point of this thread is to get some opinions from people on both sides, and glad some people added in ways to workaround the problem, though these opt outs will stop working at some point. Those who simply say "this is the way it is and deal with it" without giving good reason how it improves enduser experience is also part of the problem.

Again "necessary feature" why? We've had portage for years without it and it worked, why now? I'm just hoping it's not a "solution looking for a problem" kind of deal which is often attributed to one piece of software. As some people alluded to, perhaps indeed it's to deal with bad build scripts that do funny things with your computer and these do block them, but it's a good point if these are actually in the application themselves and these features no longer protect you after they're installed. Or perhaps is it specifically because of evil things like pip, cargo, cmake?, and other user mode installation scripts that weren't meant to be installed as root - if that's the case, then I do approve of it as I loathe pip/cargo when they're forced to be integrated with portage.

If it's just a developer vs developer war which is sort of the pip/cargo deal, then someone please say so, that's also a reasonable answer, just that end users end up cleaning the mess...


I voted "necessary feature" because that was the criteria of your question. you only left one of two options. yes or no. I am very literal with those types of things. (dropped on the head as a baby). You want to know more than ask a broader question. IE: What else can be done.
Why did you ask the question in that manor? (Yes or no) troll?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14371

PostPosted: Thu May 30, 2019 2:32 am    Post subject: Reply with quote

eccerr0r wrote:
So the ultimate reasoning for these things is just to be able for senior developers to portage-allow random lower level devs write ebuilds without having to go through them (let portage be the gatekeeper instead of manual review)?
As Ant P. alluded, this isn't solely or perhaps at all about whether low level developers are granted access to the main Gentoo Portage tree. These sandboxes confine all ebuilds, regardless of the tree from which you get them, and confine the upstream build processes triggered by those ebuilds. This means that users who add overlays written by unskilled developers benefit from confining those ebuilds. It also means that various obnoxious practices in the build system (mainly, downloading and installing any automagic dependencies not present in the host system) are a hard failure for everyone with the sandbox enabled, instead of only failing for those rare individuals who mimic the sandbox through virtualization / firewall tricks. If the sandbox is purely opt-in, packages that ought to be broken by the sandbox may go unnoticed because nobody who opted in uses that package in the way that the sandbox traps. (For the case of the network sandbox and the automagic download, if the people running with the sandbox enabled have the dependency installed, the download will not be attempted, so there is nothing to block and no one will notice that the problem exists.) Thus, to get good coverage, the feature needs to be default-enabled. With the exception of a couple of threads where distcc broke down weirdly (which, as above, seems to be a case that Portage is supposed to handle, so the breakage appears to be a bug), I can't recall any cases where a request for help was traced to the namespace-based sandbox.

Personally, as long as the sandboxes only break things that already violated best practice, I like them. I definitely like the namespace-based sandbox more than the userland preloaded filter sandbox.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Thu May 30, 2019 7:48 am    Post subject: Reply with quote

Thanks Hu, that was a much better explanation why it was put there. Again I agree that if ebuilds go start using pip or cmake or cargo or wget to download more stuff behind portage's back, then this is a good thing to ensure it doesn't happen.

It was just a bit strange as that ideally once packages are in the tree, this type of trouble has been worked out and the end user does not need to see this - it should be a developer side only thing to ensure QA. Granted since we do sometimes see regular/real filesystem sandbox violations, there's no reason why it could happen for network access "escapes". Thus agreed it's a legitimate reason why these features should exist.

Again I was just caught off guard with the most confusing symptoms for a feature that defaulted on. Normally errors should give hints as to why a particular problem occurred. Both symptoms of the namespace (PID and network) caused very strange portage behaviors that aren't obvious to the underlying problem. Then when finding the true reason behind these errors - seemingly feature creep instead of a perhaps installation error, and the fact it sure seemed more like a developer QA tool (much like sandbox itself) that required kernel options - this was why the feature seemed unnecessary for endusers depending on ebuilds automagically doing the right thing.

If only distcc ended up causing "network sandbox" violations instead of simply giving up on the hosts for no apparent reason... or warning "distcc disabled due to network sandboxing" (prior to the use of automatic proxy) that would have gone a long way to figuring out what was going on...
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7198

PostPosted: Thu May 30, 2019 1:28 pm    Post subject: Reply with quote

Hu wrote:
As Ant P. alluded, this isn't solely or perhaps at all about whether low level developers are granted access to the main Gentoo Portage tree. These sandboxes confine all ebuilds, regardless of the tree from which you get them, and confine the upstream build processes triggered by those ebuilds. This means that users who add overlays written by unskilled developers benefit from confining those ebuilds.

Which again could be limited to custom overlays users (even when these users should assume using an overlay from M. RandomGuy)

Hu wrote:
It also means that various obnoxious practices in the build system (mainly, downloading and installing any automagic dependencies not present in the host system) are a hard failure for everyone with the sandbox enabled, instead of only failing for those rare individuals who mimic the sandbox through virtualization / firewall tricks.

Something quiet surprising for me Hu.
If the build system sucks, but the gentoo dev has let it do its shit, it's not portage fault ; you are just saying "oh the build system is crappy, portage will saved you". But at end: 1/ (wihtout) the build system do shit 2/ (with) the build system fail : in the two cases the gentoo dev has done shit.
They could put a QA tool to check that, or a QA process to validate ebuilds from portage tree (but this won't help custom repo users i agree)

Hu wrote:
If the sandbox is purely opt-in, packages that ought to be broken by the sandbox may go unnoticed because nobody who opted in uses that package in the way that the sandbox traps. (For the case of the network sandbox and the automagic download, if the people running with the sandbox enabled have the dependency installed, the download will not be attempted, so there is nothing to block and no one will notice that the problem exists.) Thus, to get good coverage, the feature needs to be default-enabled. With the exception of a couple of threads where distcc broke down weirdly (which, as above, seems to be a case that Portage is supposed to handle, so the breakage appears to be a bug), I can't recall any cases where a request for help was traced to the namespace-based sandbox.

Again, weird logic: in order to see the gentoo dev has done shit allowing an unwanted download, we must opt-in the option!

Hu wrote:
Personally, as long as the sandboxes only break things that already violated best practice, I like them. I definitely like the namespace-based sandbox more than the userland preloaded filter sandbox.

Personaly i would prefer portage to stay focus on what it should do, lot of functions has been add over time to portage, for no apparents benefits to have them "inside" portage rather than an external tool.
I would prefer portage code stay as tiny as it could, without considering saving memory... it would help everyone that wish look at portage understand it faster, than digging useless lines of code of extra features.
emerge --depclean : this could simply be done by an external tool (to cite an example)
portage is a pretty stable tool ; and while most of the problematic of portage speed has comes from growing ebuilds number in the tree, still, portage could really benefits from loosing fat to ditch any seconds of speed it could.

I'm not welcoming features for "special case" that could be handle by an external tool added inside portage code, even if the features are good ones.
They should create a QA tool to check ebuild, these features should be opt-in for ebuilds devs, or some build system tester, or some stablereq testing... not on every users, and certainly not inside portage.

What's next? portage will backup user / to protect from anything that might happend?
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Thu May 30, 2019 10:09 pm    Post subject: Reply with quote

<sarcasm>Hooray restore points!</sarcasm>
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14371

PostPosted: Fri May 31, 2019 2:36 am    Post subject: Reply with quote

eccerr0r wrote:
It was just a bit strange as that ideally once packages are in the tree, this type of trouble has been worked out and the end user does not need to see this - it should be a developer side only thing to ensure QA. Granted since we do sometimes see regular/real filesystem sandbox violations, there's no reason why it could happen for network access "escapes".
Ideally, these issues should be caught before the ebuild is added, yes. However, in practice, some maintainers seem to overestimate the quality of work produced by the upstream project and do not review all of upstream's changes before adding a new version to the tree. Thus, even if version X was fine, upstream may have done something that constitutes bad practice in version X+1 and, due to the excessive faith afforded them, it isn't caught before the ebuild is added to the tree. The namespace sandboxes can still promote those to hard errors, which hopefully will be caught by the maintainer before the ebuild is keyworded for general use. You could also look at it as a form of automated QA. It's much much cheaper for the maintainer to run the new version in a sandbox and look for breakage than to review the new version by hand looking for practices that the sandbox can break automatically. Then, since we've already gone to the trouble of having the sandbox exist, and we want as many maintainers as possible to enable it, there's an argument to have it default-enabled.
eccerr0r wrote:
Again I was just caught off guard with the most confusing symptoms for a feature that defaulted on. Normally errors should give hints as to why a particular problem occurred.
Yes, this is an area that could use improvement. Portage should more clearly explain what is wrong with your system configuration when you have the FEATURE enabled and the kernel support disabled. Ideally, it should explain or link to an explanation that both covers how to enable the required kernel feature and how to disable the Portage FEATURE, so that the user can make and easily execute an informed decision about how to proceed.
eccerr0r wrote:
If only distcc ended up causing "network sandbox" violations instead of simply giving up on the hosts for no apparent reason... or warning "distcc disabled due to network sandboxing" (prior to the use of automatic proxy) that would have gone a long way to figuring out what was going on...
Yes. In my opinion, this warrants a bug report. I doubt it would be easy to report a network sandbox error, since Portage isn't involved in the network flow. However, it should be practical both to force-disable distcc when the proxy is unavailable and to report that this was done.
krinn wrote:
Which again could be limited to custom overlays users (even when these users should assume using an overlay from M. RandomGuy)
It could, but as I write earlier in this post, some Gentoo maintainers seem to take the position that if upstream followed good practice in one release, then it is safe to assume upstream will follow good practice in every subsequent release, until demonstrated otherwise. In my opinion, this assumption does not hold for all projects. It is easier to confine all ebuilds, from all trees and all projects, than to try to identify which upstreams can be trusted to follow good practice in each and every release, forever. Using the sandbox consistently also means that users will be forced to either make it work, or affirmatively disable it, soon after they upgrade to a Portage version that understands it. If instead it were enabled only for some overlays, users could unwittingly run their main tree with a broken configuration for a long time, then be very puzzled why Portage suddenly starts throwing sandbox errors when they try to use an overlay. Also, as I write above, I think the user experience interacting with it when the kernel is not ready could be improved. When the kernel is ready, the experience is transparent. Portage uses the features and the user is not required to do anything.
krinn wrote:
If the build system sucks, but the gentoo dev has let it do its shit, it's not portage fault ; you are just saying "oh the build system is crappy, portage will saved you". But at end: 1/ (wihtout) the build system do shit 2/ (with) the build system fail : in the two cases the gentoo dev has done shit.
If the sandbox forces the build to fail, that serves as a flag to anyone watching that the build system needs to be reviewed and patched. If the build system is allowed to violate best practice, and everything works in the common flow, the bad practice may persist for far longer.
krinn wrote:
They could put a QA tool to check that, or a QA process to validate ebuilds from portage tree (but this won't help custom repo users i agree)
Yes, it could be done as QA only. However, as I noted in a prior post, if all your QA testers happen to have a configuration that skips the bad practice, and only QA testers run the sandbox, then the bad practice makes it into the main tree and lurks until somebody happens to build with a configuration that both provokes the bad practice and draws human attention to that practice. With the sandbox default-enabled, the first time that bad practice path is executed, the build will fail. A build failure is very effective at drawing human attention to problems. In the case of the network sandbox, suppose the bad practice was to download a source archive from Github and build it. As long as everyone who takes that path happens to have Github access, and almost any machine with an unfiltered Internet connection will, the bad practice will be harmless and go unreported. The network sandbox ensures that the first person whose system tries that will get a failure, which then turns into a help request and ultimately a bug report.
krinn wrote:
Hu wrote:
If the sandbox is purely opt-in, packages that ought to be broken by the sandbox may go unnoticed because nobody who opted in uses that package in the way that the sandbox traps.
Again, weird logic: in order to see the gentoo dev has done shit allowing an unwanted download, we must opt-in the option!
I don't understand your point here. Anyone with the sandbox enabled will catch the problem. Users probably won't opt in without an education campaign touting the new feature and its benefits, especially if there is any fear, uncertainty, or doubt about whether opting in will cause more problems than it avoids. Making the feature default-enabled increases the pool of testers, and builds confidence that the feature works because if it breaks, many people will notice it. If there were a risk to the user, or a reason to opt out other than a desire for a slim kernel, I might question the choice to default-enable it.
krinn wrote:
Personaly i would prefer portage to stay focus on what it should do, lot of functions has been add over time to portage, for no apparents benefits to have them "inside" portage rather than an external tool.
If you can propose a clean way to implement equivalent checks outside the core Portage program, I'd like to see a specification. Since different parts of the build need to have different levels of restriction, I think it is awkward to do this without at least hooks into ebuild, if not in emerge itself.
krinn wrote:
I would prefer portage code stay as tiny as it could, without considering saving memory... it would help everyone that wish look at portage understand it faster, than digging useless lines of code of extra features.
emerge --depclean : this could simply be done by an external tool (to cite an example)
portage is a pretty stable tool ; and while most of the problematic of portage speed has comes from growing ebuilds number in the tree, still, portage could really benefits from loosing fat to ditch any seconds of speed it could.
This feature has negligible speed impact. Almost all the work is handled by the kernel, and most of that is done through tricks associated with the implementation of namespaces.
krinn wrote:
They should create a QA tool to check ebuild, these features should be opt-in for ebuilds devs, or some build system tester, or some stablereq testing... not on every users, and certainly not inside portage.
This is not about only the ebuild, but also the upstream build. As above, as a rule, maintainers do not exhaustively review every new version looking for new ways that upstream is violating best practice. I don't believe it's possible to create a tool that can statically detect all the bad practices that the namespace sandbox can break. Also, as above, running this only on developer systems may miss problems that can be caught by running it everywhere. If this feature had a measurable cost to the users whose kernels already support it, I would be more receptive to the criticisms. As is, I think the benefits outweigh the minimal execution burden. (Also, note that I never condoned that it works poorly for users whose kernels aren't configured to support it.)
krinn wrote:
What's next? portage will backup user / to protect from anything that might happend?
Support for VFS snapshots to undo package installations would be nice, now that you mention it. The kernel support isn't sufficient for that yet, but it is sufficient for namespaces.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 18157

PostPosted: Fri May 31, 2019 3:33 am    Post subject: Reply with quote

@Hu: Thanks for your comments. I'm neutral on the features, and anything that can help reduce QA burden and possibly have other benefits seems like a net gain.

Hu wrote:
Making the feature default-enabled increases the pool of testers, and builds confidence that the feature works because if it breaks, many people will notice it.
That sounds an awful lot like conscripting users as beta testers. That practice is often looked upon poorly by the conscripts.

I wouldn't mind too much, except for a seeming tendency to not provide news announcements for major / significant changes (or even blockers depending on the impact).
_________________
Those who know what's best for us must rise and save us from ourselves.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Fri May 31, 2019 5:16 am    Post subject: Reply with quote

It's not only having a FEATURE enabled, the FEATURE was stealth enabled without enduser knowledge. It's one thing if an enduser manually enables it and breaks things, it's another when the system stealthily enables it and breaks things. If it had been implemented as a etc-update/dispatch-conf where the edit adds in these FEATURES, that would be acceptable and tip off endusers that something happened and worth at least a web search why the FEATURE is now there.

Agree with the enduser becoming unwitting QA frustrates endusers. Probably beating a dead horse but QA should have been done by the devs/testers, and endusers should not be forced to instrument their systems to become QA by default. Again this is in an ideal world - I do agree that sandbox is a good thing that catches build/install scripts stealthily adding files randomly in your filesystem or download a random file not part of portage, but it's something that in an ideal world, caught by the developers and thus not necessary.

On my "faster"/"high" memory machines I really don't mind doing these QA features, the slower/low memory/customized ones I do. On embedded machines I view CGROUPS as an unnecessary feature and tend to disable all of them to save both code and kernel data that keeps track of these things, and want the minimal, fastest route to distcc to help these machines get their updates built. Perhaps it's only a few kilobytes total, but there are a lot of these little things here and there, both in kernel and in user space.

I will have to say, what's done has been done. My slow machines hopefully will have a continual path to opt out of these QA checks and hope that my faster machines or other people will catch them first... I just hope the path does not evaporate some day.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7209
Location: Austria

PostPosted: Fri May 31, 2019 6:12 am    Post subject: Reply with quote

pjp wrote:
That sounds an awful lot like conscripting users as beta testers. That practice is often looked upon poorly by the conscripts.

Yet that is exactly what ~arch is for.
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7198

PostPosted: Fri May 31, 2019 12:37 pm    Post subject: Reply with quote

asturm wrote:
Yet that is exactly what ~arch is for.

Since when???
For ages, ~ has been unstable, which doesn't mean you'll be beta testers for devs, just that the package didn't met stabilization level yet expect by the maintainer.

Just like each time a doctor put you on a table to operate you, you have risks, in no way it mean the doctor will do experiment with you, that's not what you sign for.
Many ~ users accept the risk to use unstable packages, in no way it mean their system is a playground for gentoo devs.

You have arch testers guys for that or you can call for beta testers in forum (it has been made more than once), but don't assume ~ users are yours.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 18157

PostPosted: Fri May 31, 2019 10:58 pm    Post subject: Reply with quote

asturm wrote:
pjp wrote:
That sounds an awful lot like conscripting users as beta testers. That practice is often looked upon poorly by the conscripts.

Yet that is exactly what ~arch is for.
With a rolling release that does not provide the "safety net" of hard break version releases to add new features, the switch from testing to stable is still significant. That's why I qualified it with a reference to news announcements and potentially (ab?)using blockers.
_________________
Those who know what's best for us must rise and save us from ourselves.
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14371

PostPosted: Sat Jun 01, 2019 1:27 am    Post subject: Reply with quote

pjp wrote:
Hu wrote:
Making the feature default-enabled increases the pool of testers, and builds confidence that the feature works because if it breaks, many people will notice it.
That sounds an awful lot like conscripting users as beta testers. That practice is often looked upon poorly by the conscripts.
I would like to think that the namespace feature itself received adequate testing before it was default-enabled that this was safe, but I can't say for certain that it did. I can't dispute that this conscripts users. I concur that users who are conscripted, and subsequently suffer for having been conscripted, will be unhappy about it. Conscripts who never notice problems may not realize they were ever conscripted at all.
pjp wrote:
I wouldn't mind too much, except for a seeming tendency to not provide news announcements for major / significant changes (or even blockers depending on the impact).
Agreed, more direct user notification would be nice. I think I discovered the existence of the sandbox features by accident reading a manual page. My kernel already had all the right support, for unrelated reasons, before I updated to a version of Portage that expected to use the namespace features.
eccerr0r wrote:
It's not only having a FEATURE enabled, the FEATURE was stealth enabled without enduser knowledge. It's one thing if an enduser manually enables it and breaks things, it's another when the system stealthily enables it and breaks things. If it had been implemented as a etc-update/dispatch-conf where the edit adds in these FEATURES, that would be acceptable and tip off endusers that something happened and worth at least a web search why the FEATURE is now there.
Yes, that would have been a good way to roll it out. Alternatively, a news item that encourages users to enable it would have worked, and would have been a good venue to tout how thoroughly it had been tested by developers, to allay user fears that this was an experimental or unreliable feature.
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 7209
Location: Austria

PostPosted: Sat Jun 01, 2019 10:53 am    Post subject: Reply with quote

krinn wrote:
asturm wrote:
Yet that is exactly what ~arch is for.

Since when???
For ages, ~ has been unstable, which doesn't mean you'll be beta testers for devs, just that the package didn't met stabilization level yet expect by the maintainer.

Just because this thread has hyperbolicly declared it beta, does not mean it is. ~arch is where new features are hitting users first, plain simple.
_________________
backend.cpp:92:2: warning: #warning TODO - this error message is about as useful as a cooling unit in the arctic
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6014

PostPosted: Sat Jun 01, 2019 4:06 pm    Post subject: Reply with quote

In complete absence of any factual evidence to the contrary, I assert that this feature *increases* performance.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Sat Jun 01, 2019 4:40 pm    Post subject: Reply with quote

Ant P. wrote:
In complete absence of any factual evidence to the contrary, I assert that this feature *increases* performance.

While these sandboxing features do provide useful service, again, these things should have been checked by the developer not the enduser. By definition, adding instrumentation to detect problems is overhead. So I'd like you to refute the claim that sandboxing:

1. Adds additional instructions for the CPU to execute to check whether the operations leave the sandbox.
2. Adds additional memory requirements to ensure that each process is within a namespace
3. Because of the memory requirements, it will quicker push anonymous memory out to swap due to memory pressure.

Add the fact a recompile may be needed.
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14371

PostPosted: Sat Jun 01, 2019 8:58 pm    Post subject: Reply with quote

If your kernel is namespace-enabled, you're paying most of the performance cost whether or not you use the Portage feature at all. In a namespace-aware kernel, all processes are always in a namespace. Pid 1 starts in the initial namespace of each type, and its children are placed there unless moved elsewhere. If you never create extra namespaces, you can pretend you aren't on a namespace aware kernel. There isn't really a check for the process "leaving the namespace" because that's not how this works. Rather, the namespace is a subset of the full system, and anything not visible in the subset simply does not exist, as far as the sandboxed process can see. There's obviously overhead preparing the namespace, but once it is up, the normal rules apply. It's possible that the process in the namespace will be slightly more efficient if the namespace is minimal enough, since there is less work for the kernel to do due to less data with which to work, but that's probably down in the noise for most use cases.
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 7272
Location: almost Mile High in the USA

PostPosted: Sat Jun 01, 2019 10:33 pm    Post subject: Reply with quote

Exactly. Hence the recompile needed to enable namespace/cgroups which clearly adds more text to the kernel and associated data that needs to be tracked if it wasn't enabled in the first place.

I do give benefit of the doubt that these things are done O(1) in the kernel using some hash or using some LUT and not linearly searching, but it's still adding code/data to the kernel that needs to be run/checked, and overall best case would be a wash if it is more efficient, but one had to add text, increasing kernel memory usage.

---

BTW, how does one get distccmon working with the namespaces/cgroups, seems to report nothing now... (perhaps this query should be in its own thread...)
_________________
Intel Core i7 2700K@ 4.1GHz/HD3000 graphics/8GB DDR3/180GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum