Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Behavior of package sets regarding "world"
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2  
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  

should package sets be added to "world" by default?
yes, always
21%
 21%  [ 30 ]
no, never
8%
 8%  [ 12 ]
yes, but with (persistent) opt-out
55%
 55%  [ 77 ]
no, but with (persistent) opt-in
14%
 14%  [ 20 ]
Total Votes : 139

Author Message
skogs
n00b
n00b


Joined: 03 Mar 2006
Posts: 28

PostPosted: Tue Mar 18, 2008 1:38 am    Post subject: ... Reply with quote

It matters not to me. I'm just excited by Genone's shadow ship.
Back to top
View user's profile Send private message
bluesea
n00b
n00b


Joined: 10 May 2004
Posts: 27

PostPosted: Tue Mar 18, 2008 7:47 pm    Post subject: set intersections Reply with quote

A question: What happens when two sets contain common atoms? Will deleting one set "break" the second set? Will atoms need "referenced by" counts to avoid this?

Just thoughts. I'm not sure about how to vote right now, but whatever the result, I hope we can customize the behavior to our needs.
_________________
erutangiS
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Tue Mar 18, 2008 8:05 pm    Post subject: Re: set intersections Reply with quote

bluesea wrote:
A question: What happens when two sets contain common atoms? Will deleting one set "break" the second set? Will atoms need "referenced by" counts to avoid this?

With "deleting", do you mean removing the file containing the atoms in the set, or do you mean unmerging the set?
In the first case the set simply doesn't exist anymore, it doesn't affect other sets (unless of course it is referenced directly, but that should be obvious)
For teh second case there currently is no check if the atom is referenced by other sets, but that's something I wanted to add for some time (thanks for reminding me).
Back to top
View user's profile Send private message
bluesea
n00b
n00b


Joined: 10 May 2004
Posts: 27

PostPosted: Tue Mar 18, 2008 8:41 pm    Post subject: Re: set intersections Reply with quote

Genone wrote:
bluesea wrote:
A question: What happens when two sets contain common atoms? Will deleting one set "break" the second set? Will atoms need "referenced by" counts to avoid this?

With "deleting", do you mean removing the file containing the atoms in the set, or do you mean unmerging the set?
In the first case the set simply doesn't exist anymore, it doesn't affect other sets (unless of course it is referenced directly, but that should be obvious)
For teh second case there currently is no check if the atom is referenced by other sets, but that's something I wanted to add for some time (thanks for reminding me).


I meant the second. Sorry I wasn't too clear about that. If all elements in set 1 are (blindly) unmerged, then we will lose those elements shared with set 2. I assume re-emerging set 2 will fix this, but you'd have to know to do that (and not wait for a mission-critical function to fail). Also, unmerging an atom common to many sets could involve a lot of work.

I would assume that using a tagging method might work. Each atom has a set of tags associated with it: one tag per set to which it belongs. Unmerging a set removes tags from the atoms - if no tag is left, the atom is actually unmerged. Is this how you are implementing it?
_________________
erutangiS
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Tue Mar 18, 2008 9:39 pm    Post subject: Re: set intersections Reply with quote

bluesea wrote:
I would assume that using a tagging method might work. Each atom has a set of tags associated with it: one tag per set to which it belongs. Unmerging a set removes tags from the atoms - if no tag is left, the atom is actually unmerged. Is this how you are implementing it?

Nah, we're just going to iterate over all sets and see if it contains atoms matching the package being unmerged (with some necessary exceptions)
Back to top
View user's profile Send private message
Joe_Sextus
n00b
n00b


Joined: 28 Feb 2006
Posts: 45

PostPosted: Wed Mar 19, 2008 7:18 pm    Post subject: Reply with quote

I think it should be setup so that system + usersets + air = world. This way emerge -u air updates everything not in a set, but emerge -u world still updates everything.
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2592
Location: The Peanut Gallery

PostPosted: Thu Mar 20, 2008 8:28 am    Post subject: Reply with quote

tuam wrote:
I vote for "always add", because I like the current meaning of "world". For scenarios like "upgrade without @system" I suggest that the user maintain a package set like @myworld. The world file could then consist of the two lines @myworld and @system :wink:

I think that's becoming too tricky; easier if we can just do emerge @world -@system
Back to top
View user's profile Send private message
dol-sen
Developer
Developer


Joined: 30 Jun 2002
Posts: 2549
Location: Richmond, BC, Canada

PostPosted: Sun Mar 23, 2008 7:21 am    Post subject: Reply with quote

I have voted no, with persistent opt-in.

Sorry, It took so long for me to find this thread. Thank you GMN.

In porthole, (I know the new release is long, long overdue) It's upgrades list separation and sets support (regardless of the installed portage's feature availability) separates packages by the following code excerpt:
Code:
self.cat_order = ["System"] + sets_list + ["World", "Dependencies"]


but will be later changed to use

Code:
PRIORITIES = {"System": 0, "Sets":1, "World":2, "Dependencies":3}

So that as a package is found upgradeable it checks all package lists for the first occurrence of membership and then moves on to the next upgradeable package. So far it checks any user Sets lists in sorted order but I had intended to eventually make them user configurable for membership priority. Also the above PRIORITIES constant was started (but not yet implemented) to make it easily moved to porthole's config for easy editing/configuration.


So if anyone wants to see how it works, even if the portage version you have installed does not yet support user sets, try out porthole's cvs version, create some sets lists in /etc/portage/sets/ , pick some world packages eg: gnome-*/* to a file /etc/portage/sets/gnome and you'll have just made a gnome sets list. In using/developing porthole's sets support I have found that I prefer the separation. It makes it a lot easier to pick your upgrade choices according to your priorities.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...
Back to top
View user's profile Send private message
red-wolf76
l33t
l33t


Joined: 13 Apr 2005
Posts: 714
Location: Rhein-Main Area

PostPosted: Thu Apr 03, 2008 5:04 pm    Post subject: Reply with quote

I voted "yes with persistent opt-out", but I see a number of problems that make me want to change my vote on second thought. Not being a programmer, I don't have any clever ideas about solving them, but maybe someone else does.

Consider the following situation:
1.) world contains packageA (because you emerged it at some point) as well as @setA, which also contains packageA.
2.) if portage just blindly iterates through the world file whenever emerge -uDN world is called, this would mean that we would emerge packageA twice. Once for the entry in world proper, once for the @setA entry in world because it is contained in @setA's list.

It gets worse, when you get subsets containing redundancies or multiple sets contain the same package, as was mentioned somewhere further up.

It would probably be best to keep world in a clean format, while still being able to use sets. Either you involve an auto-cleaning mechanism that eliminates "top-level entries" for packages contained in one (or more) set(s) from the world file, or portage will need to generate some sort of "build list" everytime it is called and eliminate duplicates or "illegal removes" from that list on the fly.

The latter would have the charm of keeping sets that contain packageA intact when you remove another set that also contains packageA. Also, emerge -C packageA would need to have precedence over set dependencies, even if the next emerge -uD world would restore the package, since some packages don't like updating while installed for some reason.

Does any of this make sense?
_________________
0mFg, G3nt00 r0X0r$ T3h B1g!1111 ;)

Use sane CFLAGS! If for no other reason, do it for the lulz!
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Thu Apr 03, 2008 5:08 pm    Post subject: Reply with quote

red-wolf76 wrote:
Consider the following situation:
1.) world contains packageA (because you emerged it at some point) as well as @setA, which also contains packageA.
2.) if portage just blindly iterates through the world file whenever emerge -uDN world is called, this would mean that we would emerge packageA twice. Once for the entry in world proper, once for the @setA entry in world because it is contained in @setA's list.

It gets worse, when you get subsets containing redundancies or multiple sets contain the same package, as was mentioned somewhere further up.

Not a problem, that stuff is sorted out in the resolver.
Back to top
View user's profile Send private message
red-wolf76
l33t
l33t


Joined: 13 Apr 2005
Posts: 714
Location: Rhein-Main Area

PostPosted: Thu Apr 03, 2008 5:11 pm    Post subject: Reply with quote

Ah, then it already does some of the stuff that is suggested?

Actually, aren't sets and metapackages somewhat related? I mean, it's obvious they're not the same thing, but don't they serve sort of the same purpose?
_________________
0mFg, G3nt00 r0X0r$ T3h B1g!1111 ;)

Use sane CFLAGS! If for no other reason, do it for the lulz!
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Thu Apr 03, 2008 6:40 pm    Post subject: Reply with quote

red-wolf76 wrote:
Actually, aren't sets and metapackages somewhat related? I mean, it's obvious they're not the same thing, but don't they serve sort of the same purpose?

They overlap to some degree, yes, but both have certain unique properties at this point.
Back to top
View user's profile Send private message
lefsha
Veteran
Veteran


Joined: 30 Aug 2004
Posts: 1012
Location: Sankt-Petersburg, Russia

PostPosted: Sun Apr 13, 2008 1:22 am    Post subject: Re: Behavior of package sets regarding "world" Reply with quote

Genone wrote:
Some background information:
Now a question that has come up recently is if installing a package set should also add it to the "world" set by default. For example assuming you have a user-defined package set S containing packages A, B and C (which are not in "world"), should emerge @S insert S into the "world" set, so that a subsequent emerge -uD @world would also update A, B and C whenever possible, or would you prefer it to be not included, so for updating those packages you'd have to run a separate emerge -u @S.



If you ask yourself or someone else about it I would say you have NO understanding what you are doing and what the portage is all about. Please, leave developers group and make vacant place free for other guys, which would be smarter then you.

Gentoo getting worse and worse from one day to another, because such guys who have to learn at school trying to develop something without any understanding what they are doing.


Lesson 1.
World - is a keyword for all packages installed in the system.

Any group of packages must be a part of world - there is no other options.
As an example take a look on the system group.

It's surely fine to be able to create user defined groups, but the packages they consist from go in to the world list by default.

That is the only way how it should and must work.

One can always create many groups which will be not part of each other and can be managed separately.

P.S. I will qualify it as most dumbed poll I've ever seen on gentoo forum.
_________________
Lefsha
Back to top
View user's profile Send private message
ferringb
Developer
Developer


Joined: 03 Apr 2003
Posts: 355
Location: USA

PostPosted: Sun Apr 13, 2008 2:34 am    Post subject: Re: Behavior of package sets regarding "world" Reply with quote

Shifting the posting around a bit...

lefsha wrote:
Lesson 1.
World - is a keyword for all packages installed in the system.
Like hell it is.
World is two things- a file that holds the atoms specified manually on the commandline- /var/lib/portage/world, and the system pkgset (computed from profiles). The definition of 'world' varies across managers- pkgcore treats it as just the fileset (no implicit system set). Thing is, that *still* is not all pkgs on the system (there is a reason I added an installed pkgset to pkgcore, and why portage 2.2 has a similar pkgset). It's just the system specified nodes, and the root nodes for resolution.

If in doubt, check the results of emerge -Dpe world w/ /var/lib/portage/world in place, and w/out it.

lefsha wrote:
Any group of packages must be a part of world - there is no other options.
As an example take a look on the system group.

Again, world is just a file.... and as stated, it is not the pkgset that holds everything.

lefsha wrote:
It's surely fine to be able to create user defined groups, but the packages they consist from go in to the world list by default.

A revdep-rebuild pkgset that results in all repaired nodes being inserted into world is pretty worthless- it pins the deps meaning the resolver isn't able to make choices of it's own, instead is locked into past (undesired) choices.

Personally I won't be emulating that behaviour in pkgcore since it's (sorry genone) a misfeature. Automatic inclusion of pkgsets just because it was specified on the command as a merge target will wind up with a completely useless (imo) world file, requiring repeated cleansing of it.

lefsha wrote:
That is the only way how it should and must work.

Well... that's your opinion. Considering world has *never* meant what you think, I disagree a bit.

lefsha wrote:
P.S. I will qualify it as most dumbed poll I've ever seen on gentoo forum.
lefsha wrote:
If you ask yourself or someone else about it I would say you have NO understanding what you are doing and what the portage is all about. Please, leave developers group and make vacant place free for other guys, which would be smarter then you.

Bearing in mind the problems with your posting, the extreme arrogance/insulting nature, a person *really* should respond in similar kind to you. Quite frankly, I'm pretty god damn tempted to do so also considering your chosen target is a portage developer of 5+ years asking a pretty subtle question- feel free to thank his ass the next time he adds yet another feature folks take for granted.

Problem is, you didn't understand what you were discussing let alone what certain things actually *are* and instead of approaching things civilly, you fired off. I strongly suggest the next time you feel the urge to tell some random person on the web that they're an idiot and really should just go away, you choose the civil route. At the very least it has that lovely benefit of folks cutting you slack when you screwup.

lefsha wrote:
Gentoo getting worse and worse from one day to another, because such guys who have to learn at school trying to develop something without any understanding what they are doing.
Frankly behaviour of this sort is the basis of the "gentoo getting worse and worse".

Kindly avoid repeating it.
_________________
I don't want to be buried in a pet cemetery. ~Ramones
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Sun Apr 13, 2008 6:33 am    Post subject: Re: Behavior of package sets regarding "world" Reply with quote

lefsha wrote:
If you ask yourself or someone else about it I would say you have NO understanding what you are doing and what the portage is all about. Please, leave developers group and make vacant place free for other guys, which would be smarter then you.

Gentoo getting worse and worse from one day to another, because such guys who have to learn at school trying to develop something without any understanding what they are doing

Let me make that a formal warning. Next time you address any forum user in such a manner you will be banned.
The reason you aren't now, is that you've already made yourself look totally ridiculous by telling one of the developers of portage he doesn't know what he's talking about.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
red-wolf76
l33t
l33t


Joined: 13 Apr 2005
Posts: 714
Location: Rhein-Main Area

PostPosted: Sun Apr 13, 2008 12:44 pm    Post subject: Reply with quote

8O Whoa, some serious calming down may be required here, I think.

First of all, I'm not a developer. I'm not sure I'm all that knowledgeable as a user either. I do have an opinion of what I want portage to do for me, even if I am completely happy to leave the designers to actually make the decisions about whether this is at all sensible or how to go about it, if it is.

In an ideal world (pun intended to some degree), everyone would be writing ebuilds in a way that would allow portage to start calculating what to emerge from a minimum of "required" packages set down (by the distro, the profile, the user). In an ideal world, dependencies wouldn't show up anywhere, save in those cases where unmasking is required and stuff has to go in /etc/portage/package.keywords or even (if you're feeling lucky!) /etc/portage.unmask and would always be generated on the fly according to the requirements of your USE-flags and the few packages that you positively want to have installed.

As portage adds stuff to world after emerging it (after all, you told it that you wanted it installed. Period.) it seems likely that user error will cause world to become cluttered, if foo is already part of an installation because it is a dependency of bar and now becomes a top-level candidate as well because the user forgot to put --oneshot into the command. Sure, one may think of checking the dependency tree before letting portage add foo to world, but the question of how much thinking one wishes to free the user? Certainly, trying it the other way around (with --oneshot being the standard setting) should produce most hilarious results for packages that are not dependencies of others, should a user decide to try this amazing emerge --decplean he's been reading so much about in the fora.

The problem with sets - as I understood them - lies in the question of priorities when merging and unmerging them, or their individual packages. The set "foo" may contain package "bar", but what happens if "bar" is also part of world next to foo, or what would happen to bar if a second set containing a different non-slotted version gets installed...

But I'll gladly leave that to the actual boffins in charge of development. I'm actually quite happy with portage and the power it gives the user. Sure, there are a few caveats, but that holds for almost every tool out there, I guess.
_________________
0mFg, G3nt00 r0X0r$ T3h B1g!1111 ;)

Use sane CFLAGS! If for no other reason, do it for the lulz!
Back to top
View user's profile Send private message
Akkara
Administrator
Administrator


Joined: 28 Mar 2006
Posts: 5214
Location: &akkara

PostPosted: Mon Apr 14, 2008 8:01 am    Post subject: Reply with quote

I noticed that many of the reasons given in support of the various choices can be summarized as wishing for a particular set-operator over the default and user-defined sets.

Which suggests an idea: perhaps implementing set-operators directly.

So the person wanting to update everything except for their webapps, could perhaps emerge -u "world & !webapps" (to pick a C-like syntax for discussion; the actual syntax would most likely be different).

In fact, set definition itself could be done with such operators.

Then the "world" set can continue to be the set of all things explicitly emerged by the user (including sets), since exclusions would easily be handled by the operators.

Sets themselves would probably have to be dynamically evaluated. E.g., edefine nokernel = "world & !gentoo-sources" should expand the definition each time, not take the snapshot of world as it exists at the time this command is entered.

And there could even be some officially-published and updated sets that come with emerge --sync. For example, a set of all security updates, so that emerge -u security would update all security-related issues (and their deps), and nothing else.


But this is all probably well beyond the scope of the implementation that's in plan behind the scenes. Just ignore it if so; I don't intend to make impossible suggestions, only noticing the similarities between this and operators.

PS: I'd also like to take this opportunity for a quick thanks of appreciation for all who work on the various aspects of Gentoo!
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Mon Apr 14, 2008 11:17 am    Post subject: Reply with quote

Akkara wrote:
And there could even be some officially-published and updated sets that come with emerge --sync. For example, a set of all security updates, so that emerge -u security would update all security-related issues (and their deps), and nothing else.

Already exists (just that it's `emerge @security`).
Back to top
View user's profile Send private message
dagurasu
n00b
n00b


Joined: 29 Apr 2005
Posts: 74

PostPosted: Thu Apr 17, 2008 4:15 pm    Post subject: Reply with quote

steveL wrote:
I voted No. If sets are stuff that gets added to world (yes I know about -1) they might as well just be added at the command-line and then tracked as part of world.


I voted yes with opt-out. The above statement is not right, tracking them as part of world is not the same. Ex: all the tons of kde packages now. Yes they would be in world and will updated with emerge -u world as they should be, but one could also make a set so that if you only want to update kde you can just do that with one un-hackish command. Yes there are meta-packages, problem is they include alot of stuff I don't want. I want a customized kde set without having to manually update each one or write some silly script. I've done it in the past by editing the meta-builds and editing the version numbers as needed, but that's a big headache and I stopped doing it and just resorted to updating the whole world now usually.

One nice thing about world now (or before system was aparently removed) was that it could be used to duplicate a system or resotre a broken one (I've done this). If this is still possible by just copying in a whole directory of files that's ok too, but if it's more complicated, that's bad. There should be a simple file or set of files clearly defining everything, unless someone really asks otherwise, without having to dig through the package data base system or equery or whatever. If that isn't world, then what is it? I tend to agree with the comment that there could be a pseudo-world if people want a world that isn't really world, but then it shouldn't be called world. Why not have that? Call it the "whatever" set or something, then world can include it (or explicitly everything in it).

By the way, I do understand (unlike lefsha) that world was never meant to hold everything, but in principle it holds everything you WANT, and is used to figure out everything you thus need and I think this should remain true.
Back to top
View user's profile Send private message
steveL
Advocate
Advocate


Joined: 13 Sep 2006
Posts: 2592
Location: The Peanut Gallery

PostPosted: Fri Apr 18, 2008 8:26 pm    Post subject: Reply with quote

dagurasu wrote:
steveL wrote:
I voted No. If sets are stuff that gets added to world (yes I know about -1) they might as well just be added at the command-line and then tracked as part of world.


I voted yes with opt-out. The above statement is not right, tracking them as part of world is not the same.

That statement was written when I thought the individual packages would be added to world, as opposed to a reference to the set.
Quote:
I do understand that world was never meant to hold everything, but in principle it holds everything you WANT, and is used to figure out everything you thus need and I think this should remain true.

Agreed.
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9013
Location: beyond the rim

PostPosted: Mon May 12, 2008 4:54 pm    Post subject: Reply with quote

Ok, in 2.2_pre7 there will be a new option 'world-candidate' in sets.conf, and most default sets will have it set to 'False' (except for 'system' and file-based sets in /etc/portage/sets/), meaning that they will not be added to the 'world' set. Users can override it in /etc/portage/sets.conf, e.g. if you don't want 'system' to be recorded in 'world' you'd add the following:
Code:
[system]
world-candidate=False

Also, as of 2.2_pre6, sets added to the 'world' set are no longer recorded in the same physical file as normal packages (just in case anyone wonders why the set and the file differ).
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Goto page Previous  1, 2
Page 2 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum