Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Multiple SYNC sources in make.conf
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
kipibenkipod
n00b
n00b


Joined: 15 Apr 2010
Posts: 53
Location: Israel

PostPosted: Thu Apr 15, 2010 8:16 am    Post subject: Multiple SYNC sources in make.conf Reply with quote

Hi,
I would like to use two sync sources in make.conf like this:
Code:
SYNC="rsync://MYLOCASERVER/gentoo-portage rsync://PUBLICSERVER/gentoo-portage/"

This setup don't work if MYLOCALSERVER is down. It will just abort after 3 tries, and will not try to use PUBLICSERVER.
Why there is no support for redundant SYNC mirrors?

Regards,
Kfir
Back to top
View user's profile Send private message
John R. Graham
Administrator
Administrator


Joined: 08 Mar 2005
Posts: 9414
Location: Somewhere over Atlanta, Georgia

PostPosted: Thu Apr 15, 2010 10:41 am    Post subject: Reply with quote

It's true that Portage doesn't support fallback sync servers. I don't know why. However, if you're not averse to a little scripting, this is trivial to correct:
my_sync.sh:
#!/bin/bash

if ! SYNC="rsync://local_server_uri_or_ip" emerge --sync ; then
    SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" emerge --sync
fi
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters.
Back to top
View user's profile Send private message
d2_racing
Moderator
Moderator


Joined: 25 Apr 2005
Posts: 13047
Location: Ste-Foy,Canada

PostPosted: Thu Apr 15, 2010 12:08 pm    Post subject: Reply with quote

Indeed, I found that one too lately.
Back to top
View user's profile Send private message
kipibenkipod
n00b
n00b


Joined: 15 Apr 2010
Posts: 53
Location: Israel

PostPosted: Thu Apr 15, 2010 12:20 pm    Post subject: Reply with quote

John R. Graham wrote:
It's true that Portage doesn't support fallback sync servers. I don't know why. However, if you're not averse to a little scripting, this is trivial to correct:
my_sync.sh:
#!/bin/bash

if ! SYNC="rsync://local_server_uri_or_ip" emerge --sync ; then
    SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" emerge --sync
fi
- John


Ok thanks,
I did this in my sync script:
Code:

SYNC_SERVERS="rsync://HOME/gentoo-portage rsync://FIRST_INTERNET_SERVER/gentoo-portage/ rsync://SECOND_INTERNET_SERVER/gentoo-portage/"

for server in $SYNC_SERVERS
do
        SYNC="$server" eix-sync -q && break
done


Thanks for your help.
Now I would like to know, why there are no redundant sync servers in make.conf

Thanks,
Kfir
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9170
Location: beyond the rim

PostPosted: Thu Apr 15, 2010 2:37 pm    Post subject: Reply with quote

Because so far noone had a need for it, most people are happy with the round-robin DNS solution.
Back to top
View user's profile Send private message
kipibenkipod
n00b
n00b


Joined: 15 Apr 2010
Posts: 53
Location: Israel

PostPosted: Thu Apr 15, 2010 6:31 pm    Post subject: Reply with quote

Genone wrote:
Because so far noone had a need for it, most people are happy with the round-robin DNS solution.

How does the DNS solution work?
Back to top
View user's profile Send private message
Genone
Retired Dev
Retired Dev


Joined: 14 Mar 2003
Posts: 9170
Location: beyond the rim

PostPosted: Thu Apr 15, 2010 10:18 pm    Post subject: Reply with quote

The rsync.*.gentoo.org entries are actually groups of servers, so if one goes down you're just redirected to another one transparently. Of course that doesn't work if you want to use a private mirror as primary source.
Back to top
View user's profile Send private message
Timmer
Guru
Guru


Joined: 24 Aug 2004
Posts: 344
Location: Bloomington, IN, USA

PostPosted: Wed Aug 11, 2010 10:44 pm    Post subject: Reply with quote

Is there a bug for this yet to vote for?

I have my desktop set up as a private rsync for my laptop, but then if I'm not home (on vacation, for example), i can't sync to my home server. It's got a different address in the outside world. This would be a useful feature.
Back to top
View user's profile Send private message
cappellaiomatto
n00b
n00b


Joined: 04 Sep 2010
Posts: 10

PostPosted: Sat Sep 04, 2010 5:59 pm    Post subject: Reply with quote

Actually there are two bug reports: the first is about the documentation that incorrectly reported that was possible to specify multiple servers in the SYNC variable (maybe in the beginning it was allowed) bug 43778

The second is more relevant, but the poster missed the point, and it was closed as invalid: bug 159740

There are other places where this idea is stressed: http://gentoo-what-did-you-say.blogspot.com/2010/04/multiple-portage-sync-servers.html.

Gentoo users are encouraged to set up their own rsync mirror, but there isn't a trivial way to specify a fall-back for sync mirrors...
Back to top
View user's profile Send private message
someone19
n00b
n00b


Joined: 26 Sep 2006
Posts: 51

PostPosted: Sun Sep 05, 2010 2:25 am    Post subject: Reply with quote

Timmer wrote:
Is there a bug for this yet to vote for?

I have my desktop set up as a private rsync for my laptop, but then if I'm not home (on vacation, for example), i can't sync to my home server. It's got a different address in the outside world. This would be a useful feature.


My solution is this - My home router is also an rsync server, and I use dnsmasq as my DNS forwarding to local clients.

The laptop make.conf is set to rsync.us.gentoo.org, and dnsmasq.conf has

Code:
address=/rsync.gentoo.org/192.168.9.1


When I'm home all the gentoo machines hit the local mirror for the tree, when I'm remote it hits the normal round-robbin list of servers.

Similar can be done with a re-direct depending on your home network setup.
Back to top
View user's profile Send private message
cappellaiomatto
n00b
n00b


Joined: 04 Sep 2010
Posts: 10

PostPosted: Sun Sep 05, 2010 9:15 am    Post subject: Reply with quote

someone19 wrote:


My solution is this - My home router is also an rsync server, and I use dnsmasq as my DNS forwarding to local clients.

The laptop make.conf is set to rsync.us.gentoo.org, and dnsmasq.conf has

Code:
address=/rsync.gentoo.org/192.168.9.1


When I'm home all the gentoo machines hit the local mirror for the tree, when I'm remote it hits the normal round-robbin list of servers.

Similar can be done with a re-direct depending on your home network setup.


This solution is really neat, no rsync errors, no actual fall-back.
It is really far better than a client-side script, everything is managed server-side in a transparent way.

I'm already testing it! :)

BTW IIUC it should be
Code:
address=/us.gentoo.org/192.168.9.1

or
Code:
address=/rsync.us.gentoo.org/192.168.9.1

otherwise the client requests won't be intercepted.
Back to top
View user's profile Send private message
Bober2000
n00b
n00b


Joined: 07 Jun 2015
Posts: 2

PostPosted: Sun Jun 07, 2015 8:14 pm    Post subject: Reply with quote

Any updates here? Can I do a rollback to official SYNC server if my local is not aviable using new /etc/portage/repo.conf system?
Back to top
View user's profile Send private message
Apheus
Guru
Guru


Joined: 12 Jul 2008
Posts: 401

PostPosted: Mon Jun 08, 2015 7:02 am    Post subject: Reply with quote

Th sync type is in repos.conf nowadays, but I think portage honors make.conf's GENTOO_MIRRORS anyway. Regardless of sync-type. So you should be able to use GENTOO_MIRRORS for fallbacks.
Back to top
View user's profile Send private message
Bober2000
n00b
n00b


Joined: 07 Jun 2015
Posts: 2

PostPosted: Wed Jun 10, 2015 8:38 am    Post subject: Reply with quote

No portage says that GENTOO_MIRRORS is deprecated, and didn't respect any setting in it.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 5759

PostPosted: Wed Jun 10, 2015 9:20 am    Post subject: Reply with quote

i think every sync define in repos.conf are run
so you could sync two time, if first fail, it will certainly try the second, and if first has work, it will bail out the tree is already sync.
try copying your gentoo.conf to some alt.conf and setup your sync uri in it, enable auto-sync and it should do what you wish.

[alt-sync]
location = /usr/portage
sync-uri = the alt uri
auto-sync = yes

give it a try.
Back to top
View user's profile Send private message
Apheus
Guru
Guru


Joined: 12 Jul 2008
Posts: 401

PostPosted: Wed Jun 10, 2015 10:31 am    Post subject: Reply with quote

Bober2000 wrote:
No portage says that GENTOO_MIRRORS is deprecated


Where does it say that? I have portage 2.2.20 here, and have not seen any message reagarding GENTOO_MIRRORS. And emerge --info lists it.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum