Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Throttle Emerge Fetch?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
Palhoto
Tux's lil' helper
Tux's lil' helper


Joined: 27 Feb 2003
Posts: 105
Location: Iberian Peninsula

PostPosted: Tue Mar 17, 2020 6:31 pm    Post subject: Throttle Emerge Fetch? Reply with quote

For some odd reason, when emerge is fetching packages, my ISP's router flips out and I have to reset it. Is there a way to throttle emerge fetch, by bandwidth and/or number of simultaneous connections?
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6491

PostPosted: Tue Mar 17, 2020 6:37 pm    Post subject: Reply with quote

Check `emerge --info --verbose` for $FETCHCOMMAND (and $RESUMECOMMAND), you can tweak it in make.conf.
Back to top
View user's profile Send private message
Palhoto
Tux's lil' helper
Tux's lil' helper


Joined: 27 Feb 2003
Posts: 105
Location: Iberian Peninsula

PostPosted: Tue Mar 17, 2020 7:36 pm    Post subject: Reply with quote

Ant P. wrote:
Check `emerge --info --verbose` for $FETCHCOMMAND (and $RESUMECOMMAND), you can tweak it in make.conf.

Thanks Ant P., that is useful. From the current configuration, could be due to passive FTP being selected. In the meantime I also found out that my clock is skewed several hours; possibly a Hyper-V issue, I'm not sure if it is related. I'll still have to check that issue first.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1879

PostPosted: Wed Mar 18, 2020 8:29 pm    Post subject: Reply with quote

Yeah, you can set limit via wget params.

However, there is a really neat trick using firewall rules... Leveraging the fact that portage fetches sources using portage user.
The magic goes like this:
Code:
# Generated by iptables-save v1.6.1 on Wed Mar 18 21:21:52 2020
*mangle
:PREROUTING ACCEPT [54239:30201618]
:INPUT ACCEPT [31895:11202941]
:FORWARD ACCEPT [18191:13048228]
:OUTPUT ACCEPT [32740:10673359]
:POSTROUTING ACCEPT [51471:23761339]
:PORTAGE - [0:0]
-A INPUT -j CONNMARK --restore-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A INPUT -m mark --mark 0xfa -j PORTAGE
-A OUTPUT -m owner --uid-owner 250 -j MARK --set-xmark 0xfa/0xffffffff
-A OUTPUT -j CONNMARK --save-mark --nfmask 0xffffffff --ctmask 0xffffffff
-A PORTAGE -m limit --limit 30/sec -j ACCEPT
-A PORTAGE -j DROP
COMMIT

So, we mark connections initiated by user "portage" (in my case: uid 250), then mark the whole connection, and then restore mark on incoming packets within the same connection and then limit the number of packets accepted. Dropping the excess packets informs the sender that your connection is saturated and it should slow down.
Good stuff :D

Obviously, this trick is absolutely generic, so it can be used with any service running on top of TCP.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum