Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Emerge is very slow
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
vladimir_ttt
n00b
n00b


Joined: 22 Feb 2014
Posts: 2

PostPosted: Sat Feb 22, 2014 8:46 am    Post subject: Emerge is very slow Reply with quote

Hi,

I've started to use gentoo around 11 years ago.
I had Celeron 266MHz processor with 64Mb memory.
Any "emerge" command has run on ot in at most 20-40 sec up to result (without compilation phase).
Now I have computer 40 times faster with 8Gb memory and...

Code:
time emerge -pv world


First of all about times:
Code:

real    6m26.305s
user    5m22.163s
sys     0m1.728s


And second if it is a system that hasn't be updated an year I recieve a lot of output like this

Code:

dev-qt/qtcore:4

  (dev-qt/qtcore-4.8.5-r1::gentoo, ebuild scheduled for merge) pulled in by
    >=dev-qt/qtcore-4.8.5:4 required by (app-office/akonadi-server-1.10.3::gentoo, ebuild scheduled for merge)
    ~dev-qt/qtcore-4.8.5[aqua=,debug=] required by (dev-qt/qtsvg-4.8.5::gentoo, ebuild scheduled for merge)
    (and 12 more with the same problems)

  (dev-qt/qtcore-4.8.4-r4::gentoo, installed) pulled in by
    ~dev-qt/qtcore-4.8.4[-aqua,-debug,qt3support] required by (dev-qt/qt3support-4.8.4::gentoo, installed)
    (and 11 more with the same problem)

dev-qt/qtdbus:4

  (dev-qt/qtdbus-4.8.4::gentoo, installed) pulled in by
    ~dev-qt/qtdb
......


Who broke my perfect gentoo?
Why should I wait more than 6 min to receive no result?
I am almost on the edge to remove all my gentoos and pass to ubuntu!!!
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 10024
Location: Córdoba (Spain)

PostPosted: Sat Feb 22, 2014 9:01 am    Post subject: Reply with quote

I guess the answer for both questions is that portage does not a lot more than it used to do ten years ago, specially when it comes to resolving dependencies and blockers.

Times are much shorter, and updates easier to handle, if you keep your system up to date. Otherwise, you have to deal with heavy changes that will no doubt require human intervention to be ironed out.

If you manage to refrain your Ubuntu rush and want to update your machines, I advice you to update sys-apps/portage the first. After that, emerge -auDvN world and paste here the blockers or whatever errors you are getting.

If you still get qt blockers, most of them can be solved by just forcing the update without taking deps into consideration. Then go from there... Something in this line might get you started:

Code:
$ qlist -I dev-qt --nocolor | while read package; do emerge --oneshot --nodeps $package; done

_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
TomWij
Developer
Developer


Joined: 04 Jul 2012
Posts: 1551

PostPosted: Sun Feb 23, 2014 1:39 am    Post subject: Reply with quote

Too much packages, too much overlays, too much unnecessary USE flags, ...; there are various reasons, maybe you are just in a situation that involves a lot of backtracking. Passing --backtrack=0 is as fast as it used to be (30 seconds to me); however, if you do that you need to have quite a skill to resolve things manually (as backtracking resolves things for you). Maybe you could try to slowly lower the backtrack number to see what fits for you? Another heavy consumer of time are the sub slot rebuilds; while you could disable that, it brings you back to having to run `revdep-rebuild` (which, as you might remember, takes up even more time). On a side note, work on Portage has revived; and there are some commits coming up that shave off a bit of time, so, I expect to see the time it costs drop towards the future...
Back to top
View user's profile Send private message
Josef.95
Advocate
Advocate


Joined: 03 Sep 2007
Posts: 2288
Location: Germany

PostPosted: Sun Feb 23, 2014 2:15 am    Post subject: Reply with quote

You should dissolve the Qt libs conflict. (The output of the error message, however, is too short to be able to help)

.........................................................................
On my seven year old computer portage works fine.
(AMD Opteron Processor 185 2x2,6 GHz)
Code:
portageq match / '*/*' | wc -l
1118

time emerge -pv world
....
real    0m56.484s
user    0m55.487s
sys     0m0.913s
:)

(sorry for my bad english)
Back to top
View user's profile Send private message
666threesixes666
Veteran
Veteran


Joined: 31 May 2011
Posts: 1241
Location: 42.68n 85.41w

PostPosted: Sun Feb 23, 2014 4:36 am    Post subject: Reply with quote

patience is a virtue :twisted:
_________________
cat /etc/*-release
Funtoo Linux - baselayout 2.2.0
consider this warning no. 1
https://wiki.gentoo.org/index.php?title=Special:Contributions/666threesixes666&offset=&limit=500&target=666threesixes666
Back to top
View user's profile Send private message
vladimir_ttt
n00b
n00b


Joined: 22 Feb 2014
Posts: 2

PostPosted: Tue Feb 25, 2014 12:43 pm    Post subject: Reply with quote

I've solved the conflicts by
Code:
 emerge -e world
with removing some packages to process.
But the main problem is performance.
I try it at work on system updated up to and and with 4 times faster CPU (i5-4670) and 16Gb
Code:

real    2m38.606s
user    0m29.731s
sys     0m0.726s


It is almost 3 minutes for answer that everything up to date!!!

I remember that the degradation has happened at one around 2-3 years ago.
It had worked perfectly before that.

when it has happened I asked my brother what could be reason for this?
He answer may be they are checking whole emerge tree checksums before giving the answer :)
( last part was a joke )
Back to top
View user's profile Send private message
zsitvaij
n00b
n00b


Joined: 11 Jan 2006
Posts: 63

PostPosted: Sat Mar 01, 2014 1:07 pm    Post subject: Reply with quote

The main performance killer is subslot tracking. It's an anti-feature that's on by default and offers little over preserved-libs and revdep-rebuild. The only things that come to mind are rebuilding maxima after an sbcl update and qt-core after an icu update, and those give very obvious signs of being broken when they are.

For comparison, on an i3-550@3.2 GHz:
Code:
# bsdtar -cf /dev/null /usr/portage /var/db/pkg

(this makes sure all needed information is cached and not read from disk, so it doesn't factor into performance. bsdtar, because gnu tar knows that tarring to /dev/null is pointless and just reads the file descriptors and not the contents.)
Code:
# time emerge -pvuDN1 @world
These are the packages that would be merged, in order:
Calculating dependencies... done!
Total: 0 packages, Size of downloads: 0 kB

emerge -pvuDN1 @world  727.34s user 6.94s system 99% cpu 12:16.08 total

(now let's add '--ignore-built-slot-operator-deps y' to EMERGE_DEFAULT_OPTS in make.conf)
Code:
# time emerge -pvuDN1 @world
These are the packages that would be merged, in order:
Calculating dependencies... done!
Total: 0 packages, Size of downloads: 0 kB

emerge -pvuDN1 @world  88.10s user 1.37s system 99% cpu 1:29.82 total


This is with 709 packages in world.
Back to top
View user's profile Send private message
daddio
Tux's lil' helper
Tux's lil' helper


Joined: 11 Oct 2003
Posts: 135
Location: Richland WA

PostPosted: Fri May 02, 2014 4:26 am    Post subject: horribly slow Reply with quote

Performance truly has become horrific.

It seemed to become much worse about 2 years ago, and has steadily worsened since.

I hope we can get this fixed. Its deeply embarrassing to find dependency generation taking longer in many cases that the compiling.

zsitvaij, thanks for the heads up on subslot tracking... I don't know whether to look at the code, or just turn it off.
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 7835
Location: Somewhere over Atlanta, Georgia

PostPosted: Fri May 02, 2014 1:21 pm    Post subject: Reply with quote

If it automatically resolves more blocks than it used to (and it does), I'm willing to put up with the slowdown within reason. Execution time has gone up & down over the years as features have been added and then subsequently optimized. Nobody makes me sit around and watch the spinner until it's done. All that said, the chances of me switching to Ubuntu (or anything else) because Portage is slow (but reliable) are less than the chance that the proverbial tissue paper dog has of catching the asbestos cat on a romp through Hell. ;)

- John
_________________
This space intentionally left blank.
Back to top
View user's profile Send private message
Anon-E-moose
Advocate
Advocate


Joined: 23 May 2008
Posts: 2368
Location: Dallas area

PostPosted: Fri May 02, 2014 1:45 pm    Post subject: Reply with quote

time emerge -pvuD --with-bdeps y @world

These are the packages that would be merged, in order:

Calculating dependencies... done!

Total: 0 packages, Size of downloads: 0 kB

real 0m32.966s
user 0m32.296s
sys 0m0.340s

----
time emerge -pve --with-bdeps y @world

Total: 929 packages (929 reinstalls), Size of downloads: 38,588 kB

real 0m31.972s
user 0m31.247s
sys 0m0.378s


I've not noticed any large slowdown, perhaps something else is going on on your system.

Note: It will be a little slower after a fresh boot as nothing will be in memory cache.
_________________
Asus m5a99fx, FX 8320 - amd64-multilib, 3.15.9-zen, glibc-2.17, gcc-4.7.3-r1, eudev
xorg-server-1.16, openbox w/lxpanel, nouveau, oss4
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 2403
Location: UK

PostPosted: Sat May 03, 2014 6:36 pm    Post subject: Reply with quote

Consider trying a different package manager if Portage is failing you. There's at least two other options.

This is Paludis with 1019 packages and 6 overlays, doing the slowest thing you're likely to see in regular use, equivalent to $(emerge -DpvuN --with-bdeps=y):
Code:
 ~ $ time cave resolve -c world
Done: 1 restarts, 5105 steps               

These are the actions I will take, in order:

(nothing to do)

 * No unread news items found

real   1m56.912s
user   1m10.908s
sys   0m1.474s

This is slow, but it's always been this way and it's not getting any slower.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum