| View previous topic :: View next topic |
| Author |
Message |
teika
Apprentice
Joined: 19 Feb 2011
Posts: 155
Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
 Posted: Mon Nov 28, 2011 12:48 am Post subject: Proposal: new emerge option --older-than |
 |
|
Hi. I think a new emerge option "--older-than" would facilitate things, at least for people like me: I rebuild (almost) all _only_ once a year. I never use "emerge -e world" which pulls in too many at once. I like piecemeal way; linux-headers first, then glibc, etc. But it's cumbersome.
Here comes --older-than. $ emerge -e --older-than sys-kernel/linux-headers sys-apps/util-linux should be read as "emerge util-linux, rebuilding dependent packages older-than linux-headers."; it will upgrade/rebuild all that util-linux needs, recursively solving dependency, except linux-headers and newer. Obviously --older-than is only useful when it's used with -e (=--emptytree) or -D (= --deep).
This is extremely useful. For example you can ensure that all Xorg related packages are rebuilt (or emacs dependency, or whichever) but without touching gcc, glibc, and so on.
Maybe three flavors would be good: --older-than-package / -file / -date, but I'm not sure.
Technically, it's already possible using "--excludes" option with find (1), but I'm speaking of convenience, not the possibility. ;)
# There's a subtlety. Suppose you run emerge -e --older-than foo new-pkg. If foo depends on bar, and bar is older than foo, then should it rebuild bar? Dunno, not an expert.
# Subtlety of the name: If new-pkg depends on baz, and baz is not installed, then baz should be pulled in, even though it's not "older-than" foo. But the accurate name --except-newer-and is daunting.
What do you think?
Last edited by teika on Thu Dec 01, 2011 8:55 am; edited 1 time in total |
|
| Back to top |
|
 |
dol-sen
Retired Dev
Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada
|
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10587
Location: Somewhere over Atlanta, Georgia
|
| Posted: Mon Nov 28, 2011 3:40 pm Post subject: |
|
|
I'm not saying that it would be beautiful, but I would like to see a syntax whereby these types of sets could be ephemerally created on the command line. It couldn't possibly be uglier than, say, the C++ pointer to a function returning a reference to an array syntax.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
teika
Apprentice
Joined: 19 Feb 2011
Posts: 155
Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
| Posted: Wed Nov 30, 2011 1:48 am Post subject: |
|
|
@dol-sen
I see, but it seems the opposite from what I want; --older-than is to *cut down* the rebuild/upgrade dependency which e.g. emacs requires. AgeSet is to choose *all* old packages, right? Or is it possible to choose both in AgeSet and dependency, i.e. a set intersection? (Seems no from a glance of the latest portage-2.2.)
Well, you may think it's not necessary, but I never do emerge [opts] world, so it'd be nice for me. Of course AgeSet benefits me, too. (I upgrade all once a year, breaking things into pieces.)
@John R. Graham
You said in the thread cited by dol-sen that -e doesn't update but simply rebuild. But it does seem to upgrade to the latest version. Right? (-e pretends nothing is installed, according to man emerge. If it doesn't upgrade, then the manual needs fix.)
I tried emerge -pe system. For example sys-apps/kbd is in system, and emerge says it'll be upgraded. |
|
| Back to top |
|
|
dol-sen
Retired Dev
Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada
|
| Posted: Wed Nov 30, 2011 2:23 am Post subject: |
|
|
No, -e does not specifically cause it to update. But since the tree is constantly changing, ebuilds, pkgs come and go and deps are changed. The process of running the -e can require some updates, which may trigger more updates. The main update stimulus is the -u and -D options.
No the AgeSet does not do set intersection like you want. A more common use of AgeSet is to rebuild all packages built with an older gcc/glibc toolchain for instance.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch... |
|
| Back to top |
|
|
teika
Apprentice
Joined: 19 Feb 2011
Posts: 155
Location: YYYY-MM-DD, period. Have you ever used the Internet?
|
| Posted: Wed Nov 30, 2011 1:06 pm Post subject: |
|
|
Hmm, perhaps sometimes portage developers define:
* rebuild: emerge again.
* upgrade: emerge if there's a new version, otherwise do nothing.
If so:
I understand what JRG meant: -e is not upgrade, only -u does. But this wording is different from ordinary users'. I understand the sentence "-e does not upgrade but rebuild" as "rebuild the same version".
For developers there must be reasons for this distinction, and it's good that there's nomenclature that allows quick and precise communication. But they're internal jargons, and you've got be careful.
My name is Yoda. This familyname is not rare in Japan. - 依田某
_________________
Hack of easy Shift / Ctrl / AltGr etc; save your pinkies, type without drudgery: topic 865313
XPAT - Xi, Putin, Abe and Trump - are security holes of their own nations.
Last edited by teika on Thu Dec 01, 2011 8:17 am; edited 2 times in total |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed Nov 30, 2011 2:25 pm Post subject: |
|
|
| teika wrote: |
I tried emerge -pe system. For example sys-apps/kbd is in system, and emerge says it'll be upgraded. |
Because it's rebuild kbd, and not rebuild kbd-0.9, and if you have then kbd-0.9 install, but the kbd-0.9 ebuild isn't in portage anymore, then rebuild kdb will rebuild (upgrade) to a newer version.
If you push in your world, not kbd, but kbd-0.9 instead, you'll get -e to try to rebuild kbd-0.9 and get a message that this ebuild doesn't exist in the tree anymore. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Mon Aug 27, 2012 6:10 pm Post subject: |
|
|
Sorry to reopen that old thread again, but I just recently saw the ageset, and unfortunately it seems it cannot do what you claim:
It can only update something older than a specified number of days, but neither something older than a specified date (unless you manually change the set definition every day) nor something older than a particular installation date.
After major gcc upgrades e.g. it would really be nice to see what was not yet recompiled with the new gcc.
I would not mind to specify the desired date once manually, but changing it every day makes no sense.
Of course, there is a solution (e.g. the eix-installed-after script with appropriate options, and of course the output of that script can be cast into a set), but a portage-only solution would be nice.
Perhaps it exists already and I just cannot see the obvious? |
|
| Back to top |
|
|
dol-sen
Retired Dev
Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada
|
| Posted: Tue Aug 28, 2012 1:53 am Post subject: |
|
|
Yeah, unfortunately I had nothing to do with the AgeSet's creation. I believe it was genone, who did the original set programming in portage.
I also agree with you that it would be good to have a native portage capability to perform age related rebuilds which can understand dates/times, num-of-days, etc.. Not have to edit the config each time. Now that my emaint rewrite has been merged into portage. It could be possible to create a new emaint module that is capable of taking the date/time cli options and modify the AgeSet config parameters within portage's set handling code to do this. I don't really know the best way to go about it until I've been poking around that code more.
Can you open an enhancement bug for this so it doesn't get forgotten about. I have several other things I need to work on before adding this to my chore list 
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch... |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9523
Location: beyond the rim
|
| Posted: Tue Aug 28, 2012 2:46 pm Post subject: |
|
|
While AgeSet takes "number of days" as filter argument it would be almost trivial to extend it to accept a date or also packagename as reference (or create a new subclass to to simplify argument handling).
For reference, this is the whole definition of the AgeSet class. All you'd have to do is parse a date string (or packagename) in the singleBuilder() method, pass that into the constructor and in _filter() compare it to the path mtime.
| Code: | 327 class AgeSet(EverythingSet):
328 _operations = ["merge", "unmerge"]
329
330 def __init__(self, vardb, mode="older", age=7):
331 super(AgeSet, self).__init__(vardb)
332 self._mode = mode
333 self._age = age
334
335 def _filter(self, atom):
336
337 cpv = self._db.match(atom)[0]
338 path = self._db.getpath(cpv, filename="COUNTER")
339 age = (time.time() - os.stat(path).st_mtime) / (3600 * 24)
340 if ((self._mode == "older" and age <= self._age) \
341 or (self._mode == "newer" and age >= self._age)):
342 return False
343 else:
344 return True
345
346 def singleBuilder(cls, options, settings, trees):
347 mode = options.get("mode", "older")
348 if str(mode).lower() not in ["newer", "older"]:
349 raise SetConfigError(_("invalid 'mode' value %s (use either 'newer' or 'older')") % mode)
350 try:
351 age = int(options.get("age", "7"))
352 except ValueError as e:
353 raise SetConfigError(_("value of option 'age' is not an integer"))
354 return AgeSet(vardb=trees["vartree"].dbapi, mode=mode, age=age)
355
356 singleBuilder = classmethod(singleBuilder)
|
As for the idea of creating sets on the commandline, I had actually implemented that long ago together with string based set operators. The latter were later removed in favor of atom-based operators (no idea if those are implemented by now), the former still exists. To use the above "rebuild all packages that were not rebuilt since last gcc update" example the syntax would be like this (untested):
| Code: | | emerge -p '@gcc-rebuild{class=dbapi.AgeSet,mode=older,refpackage=sys-devel/gcc}' |
|
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Sat Sep 01, 2012 6:49 pm Post subject: |
|
|
| Genone wrote: | | (or create a new subclass to to simplify argument handling). |
I have written a corresponding (separate) DateSet class in which the date can be specified in several ways: - Explicitly (in a format which can also be specified).
- Explicitly in seconds since epoch.
- Implicitly as the installation date of a package.
- Implicitly as a timestamp of a file.
The latter appears most useful to me, since e.g. you can touch some specified auxiliary file once you make a major gcc ugrade (and then minor gcc upgrades will not change this date).
Once bugzilla works again, I will open an enhancement bug with the patch. (Not sure whether the new class should be merged with the AgeSet class - as you remarked, it would be trivial to do so).
BTW: Is it really the proper way to check the date of a package by considering the timestamp of COUNTER? After all, portage supports BUILD_TIME since quite a while, but packages built with ancient portage versions not necessarily have it; isn't there a cleaner interface to get the build time?
Edit: Bugzilla is up again: Enhancement bug submitted. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9523
Location: beyond the rim
|
| Posted: Mon Sep 03, 2012 7:45 am Post subject: |
|
|
| mv wrote: | | BTW: Is it really the proper way to check the date of a package by considering the timestamp of COUNTER? After all, portage supports BUILD_TIME since quite a while, but packages built with ancient portage versions not necessarily have it; isn't there a cleaner interface to get the build time? |
Well, it didn't exist back when I wrote this stuff so COUNTER was the best option then, and I haven't really followed portage development over the last few years. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Mon Sep 03, 2012 7:38 pm Post subject: |
|
|
| Genone wrote: | | Well, it didn't exist back when I wrote this stuff so COUNTER was the best option then, and I haven't really followed portage development over the last few years. |
Thanks. Some investigation showed that the "portable" way is probably to use the BUILD_TIME key with aux_get - and if it doesn't exist, it is probably reasonable to assume that the build time is older than any chosen date/age. If you do not protest, I will file a corresponding bug (with patch) for the AgeSet and DateSet classes. |
|
| Back to top |
|
|
Genone
Retired Dev
Joined: 14 Mar 2003
Posts: 9523
Location: beyond the rim
|
| Posted: Tue Sep 04, 2012 6:40 am Post subject: |
|
|
| Feel free to update anything you think can be improved. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
