Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
emerge gets stuck on every sync
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1323
Location: Sweden

PostPosted: Mon Jul 20, 2020 6:25 am    Post subject: emerge gets stuck on every sync Reply with quote

A few days ago eix-sync would start getting stuck on every sync.
emerge --sync finishes fine, and the eix-sync also updates packages, so I think it's when eix tries to generate it's "diff" at the end that something goes wrong.

I've tried using strace to see if I can work out what's happening, but I have no idea how to interpret the output:

Code:
$ sudo strace -p 4973
strace: Process 4973 attached
wait4(-1,


Last edited by Letharion on Wed Aug 05, 2020 3:57 pm; edited 1 time in total
Back to top
View user's profile Send private message
SpiderPigLoki
Tux's lil' helper
Tux's lil' helper


Joined: 22 Aug 2008
Posts: 84

PostPosted: Sat Aug 01, 2020 9:42 am    Post subject: Reply with quote

Well I think a test (maybe even dirty workaround) would be:
Code:

emerge --sync
eix-update


AFAIK eix-update is nothing else then emerge --sync && eix-update.
First the portage tree gets downloaded / updated / synced and then eix updates it's own database with available, installed, use-flags, bla bla bla.

Apart from that maybe try running:
Code:
 strace eix-update
Back to top
View user's profile Send private message
fturco
Veteran
Veteran


Joined: 08 Dec 2010
Posts: 1169
Location: Italy

PostPosted: Sat Aug 01, 2020 10:39 am    Post subject: Reply with quote

SpiderPigLoki wrote:
AFAIK eix-update is nothing else then emerge --sync && eix-update.

If that were the case, it would be an infinite recursive command... :)
Back to top
View user's profile Send private message
toralf
Developer
Developer


Joined: 01 Feb 2004
Posts: 3794
Location: Hamburg

PostPosted: Sat Aug 01, 2020 11:30 am    Post subject: Reply with quote

Waht's about
Code:
rm /var/cache/eix/portage.eix
? Or do you have few trees at NFS shares ?
Back to top
View user's profile Send private message
Tony0945
Advocate
Advocate


Joined: 25 Jul 2006
Posts: 3857
Location: Illinois, USA

PostPosted: Sat Aug 01, 2020 2:01 pm    Post subject: Reply with quote

Code:
 $ ls -lh  /var/cache/eix/portage.eix
-rw-rw-r-- 1 portage portage 15M Jul 31 19:38 /var/cache/eix/portage.eix
Wow! That's big. I'm having long delays too.
Back to top
View user's profile Send private message
toralf
Developer
Developer


Joined: 01 Feb 2004
Posts: 3794
Location: Hamburg

PostPosted: Sat Aug 01, 2020 2:08 pm    Post subject: Reply with quote

Tony0945 wrote:
Code:
 $ ls -lh  /var/cache/eix/portage.eix
-rw-rw-r-- 1 portage portage 15M Jul 31 19:38 /var/cache/eix/portage.eix
Wow! That's big. I'm having long delays too.
Nope, 15 M is common IMO.
Back to top
View user's profile Send private message
Tony0945
Advocate
Advocate


Joined: 25 Jul 2006
Posts: 3857
Location: Illinois, USA

PostPosted: Sat Aug 01, 2020 2:28 pm    Post subject: Reply with quote

Would it help to filter out dev-haskell and dev-ros that seem to take a lot of time and I have zero interest in? I like it that the tree has over 10,000 packages, but not if it materially adds to processing time.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6356

PostPosted: Sun Aug 02, 2020 4:59 am    Post subject: Reply with quote

Letharion, if you have no special configuration for eix-sync, it does the following:
  1. It copies /var/cache/eix/portage.eix to /var/cahce/eix/previous.eix
  2. It calls emerge --sync
  3. It calls eix-update
  4. It calls eix-diff
Try to do this steps manually and check whether it is really eix-diff that hangs. An strace directly on eix-diff might be more useful.
You might also try the eix-update and eix-diff with /etc/eix-sync.conf, /etc/eixrc, ~/.eixrc renamed to something "harmless" and with an empty environment (env -i "command ....") to make sure that it is not a special configuration or environment variable of yours which is interfering.

Tony0945: What exactly is slow with eix? The eix-diff itself? For the size, it should help to set DEP=false but then you will not have the dependency information, of course. But for the speed it will not help much except for eix-diff. The same thing for filtering categories. If you use eix from the mv overlay, you have more possibilities for optimization which might or might not help a lot.
Back to top
View user's profile Send private message
Tony0945
Advocate
Advocate


Joined: 25 Jul 2006
Posts: 3857
Location: Illinois, USA

PostPosted: Sun Aug 02, 2020 1:37 pm    Post subject: Reply with quote

mv wrote:
Tony0945: What exactly is slow with eix? The eix-diff itself? For the size, it should help to set DEP=false but then you will not have the dependency information, of course. But for the speed it will not help much except for eix-diff. The same thing for filtering categories. If you use eix from the mv overlay, you have more possibilities for optimization which might or might not help a lot.

Not bad at all on this quiet Sunday morning. I usually sync Friday or Saturday night, probably the worst times. I think my complaint is with eix-diff which usually has interminable lines of stuff about dev-ros and dev-haskell. I think once it was dev-robots (?). Probably those were undergoing major overhauls, but several weeks in a row?
Anyway, here are my times and I'm surprised there is so little difference between the the web and the LAN.
Code:
On the server (bulldozer) syncing over the web
        emerge --sync
                real    0m57.634s
                user    0m20.596s
                sys     0m2.062s
        eix-update
                real    0m3.678s
                user    0m3.444s
                sys     0m0.230s
        eix-diff
                real    0m0.996s
                user    0m0.953s
                sys     0m0.040s
On the k10 syncing from the server over the LAN
        emerge --sync
                real    0m39.830s
                user    0m26.211s
                sys     0m3.057s
        eix-update
                real    0m22.483s
                user    0m3.268s
                sys     0m1.372s
        eix-diff
                real    0m1.227s
                user    0m0.658s
                sys     0m0.120s
Back to top
View user's profile Send private message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1323
Location: Sweden

PostPosted: Mon Aug 03, 2020 6:22 pm    Post subject: Reply with quote

Ok, since I have no special eix config, I'll use
Code:
$ sudo bash -c "cp /var/cache/eix/portage.eix /var/cache/eix/previous.eix && emerge --sync && eix-update && eix-diff"

To sync next time, and we'll see what happens.
Back to top
View user's profile Send private message
Letharion
Veteran
Veteran


Joined: 13 Jun 2005
Posts: 1323
Location: Sweden

PostPosted: Wed Aug 05, 2020 3:58 pm    Post subject: Reply with quote

I know have this process tree:

Code:
24091 pts/1    S+     0:00      |       \_ sudo bash -c cp /var/cache/eix/portage.eix /var/cache/eix/previous.eix && emerge --sync && eix-update && eix-diff
24092 pts/1    S+     0:00      |           \_ bash -c cp /var/cache/eix/portage.eix /var/cache/eix/previous.eix && emerge --sync && eix-update && eix-diff
24094 pts/1    S+     0:00      |               \_ /usr/bin/python3.8 -b /usr/lib/python-exec/python3.8/emerge --sync
24290 pts/1    S+     0:00      |                   \_ /usr/bin/python3.8 -b /usr/lib/python-exec/python3.8/emerge --sync


So it appears eix isn't involved at all, and emerge --sync on it's own just happened to work when I tried it.

The last output is:
Code:
=== Sync completed for gentoo

but then nothing happens.

Code:
$ sudo strace -p 24290
strace: Process 24290 attached
futex(0x7f31e196d930, FUTEX_WAIT_PRIVATE, 2, NULL
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum