Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Rsync for portage tree and distfiles on a local network
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
HomeUser
n00b
n00b


Joined: 22 Jan 2006
Posts: 51

PostPosted: Fri Apr 26, 2013 3:01 pm    Post subject: Rsync for portage tree and distfiles on a local network Reply with quote

Hello,


I have more than one Gentoo machine and tried in the past not to load the same files 2 times. To spare bandwidth, time and/or avoid surcharging the servers, depending on my different ADSL contracts. Sometimes a little exaggerating.

It is probably me, who did not look very good, didn't understand it very well.
But as I now understand it, I can use rsync not only for emerge --sync, but also for emerge -f -uD world. So if you set it up for the portage tree, not also setting it up for the distfiles?

I done things in the past with NFS and with FTP. It gave always some problems with my iptables settings and gave some complexity.

I suppose sharing some directory's over the network is a very good solution Sharing Portage over NFS, (Under Ubuntu, linking a few directory's to the distribution on an other partition works also very well, I noticed. As using chroot to get everything updated from one script.), but using rsync also for distfiles when you did already set it up for the portage tree, seems me an easy and fast alternative.

Emerge seems to accept rsync in the mirror variable. Only a few files will still get searched on some third party servers on the internet. If the internet was closed at the moment of the emerge, a script can identify them from Portage's output and could try to get them from an other internal machine, but that starts to be complex. Also because the machines are not completely identical and the second needs some files from the internet for packages not installed on the first machine.

I had a problem with it rsync because my distfiles are links to an other folder structure.

I did until now not succeed in setting all the options in make.conf so that it all is checked in one pass without timeouts trying to access sites not mentioned in the mirror variable. (With the network down the first time it goes faster.)

If it doesn't exist, wouldn't it be useful to extent a document like Gentoo Linux rsync Mirrors Policy and Guide with the option to also get the distfiles in a secure, fast and simple (once configured) way?
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 31373
Location: 56N 3W

PostPosted: Fri Apr 26, 2013 6:25 pm    Post subject: Reply with quote

HomeUser,

I run a Gentoo rsync server for my home network. It syncs once a day and my other systems sync to it instead of the outside world.
For distfiles mirroring, I use http-replicator, which is a a caching proxy for distfiles. The first fetch goes to the outside world, subsequent fetches of the same file are served from cache.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Naib
Advocate
Advocate


Joined: 21 May 2004
Posts: 4078
Location: Removed by Neddy

PostPosted: Fri Apr 26, 2013 7:04 pm    Post subject: Reply with quote

Same here, main home NAS is the main sync box and then my other gentoo boxes have SYNC="rsync://192.168.0.2/gentoo-portage" in make.conf on the other boxes
_________________
A free press is the unsleeping guardian of every other right that free men prize; it is the most dangerous foe of tyranny. Where men have the habit of liberty, the Press will continue to be the vigilant guardian of the rights of the ordinary citizen.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum