| View previous topic :: View next topic |
| Author |
Message |
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
 Posted: Sun May 07, 2017 4:37 pm Post subject: Let’s fund pkgcore work |
 |
|
pkgcore is the fastest tool to install a program from the portage tree. On my box | Code: | | time pmerge -p pkgcore |
finishes within 1.3s, compared to one full minute with emerge.
It’s search is so fast, that | Code: | | pquery --raw -nv --attr=keywords |
can replace eix without needing a separate database update step.
And it provides a drop-in replacement for emerge, including commandline options. Simply replace emerge with pmerge to get an instant speed-bump.
pkgcore achieves this by doing just the work that’s actually needed for the task. Taken together, that’s conceptual and technical excellence, deeply optimized to the requirements, and I pull my hat to its developerss.
However the PMS addition of subslots moved the target for that optimization. The dependency-space exploded, which pushes pkgcore into a costly path of its optimization when doing a world update. To fix this, it needs a restructuring of its dependency resolution, but that’s a large task which is hard to do in pure hobby time.
Therefore, after a year of prodding from me, radhermit set up a patreon account where we can provide funding. The initial goal is just 25$ per month: To show that there is actual interest in this work. I pledged $3. If just 8 of you join me, we’ll already have reached that goal.
If you want to help fund pkgcore, please join up on https://www.patreon.com/pkgcore
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )
pkgcore: So fast that it feels unreal - by doing only what is needed. |
|
| Back to top |
|
 |
geki
Advocate
Joined: 13 May 2004
Posts: 2387
Location: Germania
|
| Posted: Sun May 07, 2017 7:58 pm Post subject: |
|
|
Hmm, looks like a worthwhile project to me, since I am too short on time, unlike before. 
Though, somehow I cannot calculate anymore or I miss something.
The project amount before and after my pledge did not add up right,
although the fees were added on top of my pledge on the bill. 
_________________
hear hear |
|
| Back to top |
|
|
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
| Posted: Sun May 07, 2017 9:16 pm Post subject: |
|
|
| geki wrote: | Hmm, looks like a worthwhile project to me, since I am too short on time, unlike before.
Though, somehow I cannot calculate anymore or I miss something.
The project amount before and after my pledge did not add up right,
although the fees were added on top of my pledge on the bill. |
Could that be because I increased my pledge from $3 to $4?
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )
pkgcore: So fast that it feels unreal - by doing only what is needed. |
|
| Back to top |
|
|
geki
Advocate
Joined: 13 May 2004
Posts: 2387
Location: Germania
|
| Posted: Mon May 08, 2017 11:04 am Post subject: |
|
|
Well, I do not know. It was at 2 patreons and $6, I pledge $5, now it is at 3 patreons and $10.
_________________
hear hear |
|
| Back to top |
|
|
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
| Posted: Mon May 08, 2017 8:01 pm Post subject: |
|
|
| geki wrote: | | Well, I do not know. It was at 2 patreons and $6, I pledge $5, now it is at 3 patreons and $10. |
It is now at 4 patrons and $11.
I’m paying $4, you pay $5, so we’re together at $9. The other two might be pledging 1$ each.
@other patrons: If you are here, could you give your pledge amount?
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )
pkgcore: So fast that it feels unreal - by doing only what is needed. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Mon May 08, 2017 8:10 pm Post subject: |
|
|
Not saying how I know, but someone is mistaken. 
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
geki
Advocate
Joined: 13 May 2004
Posts: 2387
Location: Germania
|
| Posted: Mon May 08, 2017 9:04 pm Post subject: |
|
|
Well, as long as it is just alright as is, that's fine. I just wonder.
_________________
hear hear |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Mon May 08, 2017 9:16 pm Post subject: |
|
|
going faster, but doing less, you don't need any good algorithm to achieve this 
are you sure adding subslots and missing features would still make the dependencies resolution faster? Could be why author drops dev of pkgcore |
|
| Back to top |
|
|
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
| Posted: Fri May 12, 2017 5:57 pm Post subject: |
|
|
| krinn wrote: | going faster, but doing less, you don't need any good algorithm to achieve this
are you sure adding subslots and missing features would still make the dependencies resolution faster? Could be why author drops dev of pkgcore |
Yes, I am sure: subslots are already supported.
pkgcore was already much faster than portage when there were no subslots. It is still much faster, and it supports most of the latest features (i.e. EAPI 6: development is still ongoing, but in small steps due to limited free time of the devs), but a world update (pmerge -uDN world) requires too much memory.
To fix that high memory consumption on a world update, the core datastructures need a redesign. And that’s the task for which the core dev needs a larger slab of time.
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )
pkgcore: So fast that it feels unreal - by doing only what is needed. |
|
| Back to top |
|
|
NerdIII
n00b
Joined: 10 Oct 2010
Posts: 12
|
| Posted: Sun May 14, 2017 11:57 am Post subject: |
|
|
time emerge -p pkgcore: 5.0s
time pmerge -p pkgcore: 0.5s
time emerge -puDN @world: 111s
time pmerge -puDN @world: 6.4s
Yes, that's awesome. It just does not find any changed USE flags or downgraded packages at this time.  |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun May 14, 2017 2:59 pm Post subject: |
|
|
| Quiet impressive, this speak better NerdIII, i might gives it a try. |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Sun May 14, 2017 5:13 pm Post subject: |
|
|
| Maybe I'll finally give this a try. I could tolerate emerge's slowness but only because I'd been conditioned to accepting it was fast after using paludis for far too long (not gonna miss those half-hour dependency resolving times...) |
|
| Back to top |
|
|
NerdIII
n00b
Joined: 10 Oct 2010
Posts: 12
|
| Posted: Mon May 15, 2017 9:51 am Post subject: |
|
|
I said it wasn't coming up with the correct package list to merge. I was using the live ebuild and the merge list was just empty, missing some packages with changed use flags and some downgrades. So all I know is that it is fast at doing nothing at the moment. 
It could be my config files. They have grown messy over the years with lots of entries in package.accept_keywords, package.use and package.env.
EDIT:
I tried the live ebuild again today and it found a package to upgrade with | Code: | | pmerge -uDNav @world |
:
sys-boot/refind-0.10.7-r1::gentoo [0.10.7::gentoo]
This time I had no changed USE flags, but the output is missing downgrading packages again (feature or bug?). The times for my actual system upgrade emerge command look less fishy now:
emerge -uDNav @world: 143s
pmerge -uDNav @world: 40s |
|
| Back to top |
|
|
tranquilcool
Veteran
Joined: 25 Mar 2005
Posts: 1179
|
| Posted: Mon May 15, 2017 4:31 pm Post subject: |
|
|
it never worked for me.
pmerge pkgcore
ERROR: sys-apps/pkgcore-0.9.4 failed (unpack phase):
* S was null- , path=/usr/lib/pkgcore/ebd/helpers/5:/usr/lib/pkgcore/ebd/helpers/4:/usr/lib/pkgcore/ebd/helpers/common:/sbin:/bin:/usr/sbin:/usr/bin
_________________
this is a strange strange world. |
|
| Back to top |
|
|
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
|
| Back to top |
|
|
ArneBab
Guru
Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany
|
| Posted: Sun May 21, 2017 1:29 pm Post subject: |
|
|
| tranquilcool wrote: | it never worked for me.
pmerge pkgcore
ERROR: sys-apps/pkgcore-0.9.4 failed (unpack phase):
* S was null- , path=/usr/lib/pkgcore/ebd/helpers/5:/usr/lib/pkgcore/ebd/helpers/4:/usr/lib/pkgcore/ebd/helpers/common:/sbin:/bin:/usr/sbin:/usr/bin |
You’ll want to use the | Code: | | =sys-apps/pkgcore-9999 |
at the moment.
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )
pkgcore: So fast that it feels unreal - by doing only what is needed. |
|
| Back to top |
|
|
tranquilcool
Veteran
Joined: 25 Mar 2005
Posts: 1179
|
| Posted: Mon May 22, 2017 4:13 am Post subject: |
|
|
| ArneBab wrote: | | tranquilcool wrote: | it never worked for me.
pmerge pkgcore
ERROR: sys-apps/pkgcore-0.9.4 failed (unpack phase):
* S was null- , path=/usr/lib/pkgcore/ebd/helpers/5:/usr/lib/pkgcore/ebd/helpers/4:/usr/lib/pkgcore/ebd/helpers/common:/sbin:/bin:/usr/sbin:/usr/bin |
You’ll want to use the | Code: | | =sys-apps/pkgcore-9999 |
at the moment. |
yes, did use pkgcore-9999 and it worked. thanks.
_________________
this is a strange strange world. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Wed Nov 08, 2017 1:15 pm Post subject: |
|
|
Been funding through Patreon now for several months. As of yet, though, the required funding goal for the developer to continue work hasn't been reached.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Fri Dec 08, 2017 11:45 am Post subject: |
|
|
| pgkcore is user-friendly and impressive. It is great: we can avoid resorting to the plethora of "complementary" tools related at Portage! |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Fri Dec 08, 2017 6:01 pm Post subject: |
|
|
| If that's all you care about, you can use paludis instead. |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10589
Location: Somewhere over Atlanta, Georgia
|
| Posted: Fri Dec 08, 2017 7:26 pm Post subject: |
|
|
Also, isn't pkgcore one of the, "... plethora of "complementary" tools related at Portage?"
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
Mr. T.
Guru
Joined: 26 Dec 2016
Posts: 477
|
| Posted: Sun Dec 10, 2017 8:15 am Post subject: |
|
|
Should we modify the system to only integrate pkgcore? I do not know. pkgcore is qualified as a framework manager, it has its own configuration but uses the ebuilds repository.
I want to be able to study the package manager to answer some questions and solve problems. The design of Portage is undocumented (at least for the user), the technical
documentation is partial (scattered, not up to date) and the code may be of poor quality (obscure parts, lines of code not readable, complex algorithms...).
At the point where I am, I do not exclude to try several solutions. |
|
| Back to top |
|
|
The_Document
Apprentice
Joined: 03 Feb 2018
Posts: 275
|
| Posted: Sun Feb 25, 2018 6:42 am Post subject: |
|
|
| helecho wrote: | Should we modify the system to only integrate pkgcore? I do not know. pkgcore is qualified as a framework manager, it has its own configuration but uses the ebuilds repository.
I want to be able to study the package manager to answer some questions and solve problems. The design of Portage is undocumented (at least for the user), the technical
documentation is partial (scattered, not up to date) and the code may be of poor quality (obscure parts, lines of code not readable, complex algorithms...).
At the point where I am, I do not exclude to try several solutions. |
This means portage is not well coded, to say the least it had plenty of time to increase its performance and it hardly did. I tried pkgcore and its incredibly fast compared to portage however it doesn't honor useflag changes when doing updates, I would totally use if I found out how to do a portage equivalent of --jobs which I am sure pkgcore doesn't support. |
|
| Back to top |
|
|
|
Other Things Gentoo
