Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Emerge --Sync stopped working.
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Installing Gentoo
View previous topic :: View next topic  
Author Message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Tue Aug 21, 2012 2:49 pm    Post subject: Emerge --Sync stopped working. Reply with quote

All, I have seen a number of posts that mention if rsync does not work for you (emerge --sync) that "emerge-webrsync" can get you a portage snapshot.

My Gentoo servers at home used to "emerge --sync" once a night, and at some time ago (a few months) it stopped working.

Now I use a Cisco 1841 with reflexive ACL's as my firewall, with a dedicated static IP for the main server. I have visibility over the network bits, what I would like to do is understand why the "emerge --sync" using rsync stopped working.

I saw a post somewhere that the newer rsync client changed it's behavior, but posts were not specific as to what the behavior was?

Anyone take a deeper look as to what changed with rsync?

Nick
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Tue Aug 21, 2012 3:29 pm    Post subject: Reply with quote

What does it do when you run it manually instead of in a cron job?

I have my server emerge --sync every few days, not every night - but it still seems to be working... All my other machines sync off of my server... I have a static IP with "consumer/home" networking equipment... I don't firewall off rsync or web ports...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Tue Aug 21, 2012 9:18 pm    Post subject: Reply with quote

eccerr0r wrote:
What does it do when you run it manually instead of in a cron job?

I have my server emerge --sync every few days, not every night - but it still seems to be working... All my other machines sync off of my server... I have a static IP with "consumer/home" networking equipment... I don't firewall off rsync or web ports...


Just the excessive retries... then bails.

/etc/make.conf
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
PORTAGE_RSYNC_OPTS="--progress --recursive --links --safe-links --compress --perms --times --force --whole-file --itemize-changes --delete --delete-during --stats --timeout=1800 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/overlay"
GENTOO_MIRRORS="http://gentoo.osuosl.org/ http://gentoo.gossamerhost.com http://gentoo.llarian.net/"


gubbie ~ # emerge --sync
>>> Starting rsync with rsync://208.100.4.53/gentoo-portage...
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(549) [Receiver=3.0.9]
>>> Retrying...


>>> Starting retry 1 of 18 with rsync://64.59.140.91/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(549) [Receiver=3.0.9]
>>> Retrying...


>>> Starting retry 2 of 18 with rsync://209.59.138.21/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(549) [Receiver=3.0.9]
>>> Retrying...


>>> Starting retry 3 of 18 with rsync://209.221.142.124/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(549) [Receiver=3.0.9]
>>> Retrying...
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Tue Aug 21, 2012 9:45 pm    Post subject: Reply with quote

I think rsync uses port 873, is this blocked in your firewall?
I don't recall there being any changes...then again I have a fairly unrestrictive outgoing firewall. Incoming is a bit different...
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Tue Aug 21, 2012 9:52 pm    Post subject: Reply with quote

eccerr0r wrote:
I think rsync uses port 873, is this blocked in your firewall?


No, it is not.

My firewall is a Cisco 1841, so I am using reflexive ACL's that premit the outbound request, and create a reverse rule that get's added to the inbound ACL.

DSL-Router#sho access-list traffic | inc 134.161.116.17
permit tcp host 134.161.116.17 eq 873 host 69.30.73.18 eq 36935 (11 matches) (time left 295)

In this case, the poorman's state engine (reflexive list entry) was created, and even saw 11 packets come back from this particular gentoo rsync server..

Nick
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Tue Aug 21, 2012 9:54 pm    Post subject: Reply with quote

nellson wrote:
eccerr0r wrote:
I think rsync uses port 873, is this blocked in your firewall?


No, it is not.

My firewall is a Cisco 1841, so I am using reflexive ACL's that premit the outbound request, and create a reverse rule that get's added to the inbound ACL.

DSL-Router#sho access-list traffic | inc 134.161.116.17
permit tcp host 134.161.116.17 eq 873 host 69.30.73.18 eq 36935 (11 matches) (time left 295)

In this case, the poorman's state engine (reflexive list entry) was created, and even saw 11 packets come back from this particular gentoo rsync server..

Nick


And if I look at the RSYNC port:

DSL-Router#sho access-list traffic | inc eq 873
permit tcp host 64.59.140.91 eq 873 host 69.30.73.18 eq 40563 (8 matches) (time left 299)
permit tcp host 208.70.246.16 eq 873 host 69.30.73.18 eq 36811 (12 matches) (time left 291)
permit tcp host 216.194.64.133 eq 873 host 69.30.73.18 eq 58534 (17 matches) (time left 298)
permit tcp host 216.165.129.134 eq 873 host 69.30.73.18 eq 47525 (13 matches) (time left 287)
permit tcp host 209.59.138.21 eq 873 host 69.30.73.18 eq 60070 (13 matches) (time left 280)
permit tcp host 128.175.60.112 eq 873 host 69.30.73.18 eq 48501 (12 matches) (time left 283)
permit tcp host 129.21.171.98 eq 873 host 69.30.73.18 eq 43443 (14 matches) (time left 299)
permit tcp host 208.100.4.53 eq 873 host 69.30.73.18 eq 36078 (14 matches) (time left 268)
permit tcp host 209.221.142.124 eq 873 host 69.30.73.18 eq 52969 (14 matches) (time left 267)
permit tcp host 128.61.111.9 eq 873 host 69.30.73.18 eq 44216 (14 matches) (time left 261)
permit tcp host 134.161.116.17 eq 873 host 69.30.73.18 eq 36935 (14 matches) (time left 227)
permit tcp host 128.10.252.13 eq 873 host 69.30.73.18 eq 43154 (3 matches) (time left 128)
permit tcp host 156.56.247.193 eq 873 host 69.30.73.18 eq 33529 (14 matches) (time left 233)
permit tcp host 67.212.64.3 eq 873 host 69.30.73.18 eq 39016 (14 matches) (time left 196)

You see that they all got replies...

And this used to work. I have not mucked with my router in quite a while..


Nick
Back to top
View user's profile Send private message
eccerr0r
Watchman
Watchman


Joined: 01 Jul 2004
Posts: 9601
Location: almost Mile High in the USA

PostPosted: Tue Aug 21, 2012 10:14 pm    Post subject: Reply with quote

Did you setup firewall stuff on your local machine recently?

After reading your post I went to look at my firewall and found it was a bit more relaxed than I thought, and my local gentoo rsync mirror is actually visible from the outside. While it's not a real big problem, I didn't expect it...

Might want to run tcpdump or wireshark on your ether port and see if anything's getting back to your machine...

Does other stuff work (web, etc.)? is it only rsync that's broken?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Thu Aug 23, 2012 12:01 am    Post subject: Reply with quote

eccerr0r wrote:
Did you setup firewall stuff on your local machine recently?

After reading your post I went to look at my firewall and found it was a bit more relaxed than I thought, and my local gentoo rsync mirror is actually visible from the outside. While it's not a real big problem, I didn't expect it...

Might want to run tcpdump or wireshark on your ether port and see if anything's getting back to your machine...

Does other stuff work (web, etc.)? is it only rsync that's broken?


Well.. So far just rsync.

<sigh> Wow... so I set up TCPDUMP, and in another session started the "emerge --sync" and the first site it hit just worked.. :-\

I really wanted to dog this down and post most details of a way to fix this. But if it won't stay broke...

I will watch it for a few days... if it breaks I will post the tcpdump.

I have never set up my local gentoo firewall (ipchains??) so if it got updated in a previous emerge world, I would have missed that.. good idea to check though!

Nick


Nick
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Fri Aug 24, 2012 1:41 pm    Post subject: Reply with quote

OK, so next day the problem is back. Only this time i have TCPDUMP working for me.

So from my server point of view, it never sees a reply.

Here is the server sending out the request:

Code:

gubbie ~ # tcpdump -i eth0 tcp port 873
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
06:32:50.351026 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [S], seq 2730467521, win 5840, options [mss 1460,sackOK,TS val 1034299160 ecr 0,nop,wscale 6], length 0
06:32:50.407702 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [S.], seq 4117327491, ack 2730467522, win 14480, options [mss 1460,sackOK,TS val 39555318 ecr 1034299160,nop,wscale 9], length 0
06:32:50.407899 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 1, win 92, options [nop,nop,TS val 1034299174 ecr 39555318], length 0
06:32:50.408266 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 1:15, ack 1, win 92, options [nop,nop,TS val 1034299174 ecr 39555318], length 14
06:32:50.457666 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [.], ack 15, win 29, options [nop,nop,TS val 39555331 ecr 1034299174], length 0
06:32:50.912943 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39555445 ecr 1034299174], length 14
06:32:50.913053 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034299300 ecr 39555445], length 0
06:32:50.913510 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034299300 ecr 39555445], length 15
06:32:51.161180 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39555507 ecr 1034299174], length 14
06:32:51.161276 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034299362 ecr 39555507,nop,nop,sack 1 {1:15}], length 0
06:32:51.165686 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034299363 ecr 39555507], length 15
06:32:51.656830 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39555631 ecr 1034299174], length 14
06:32:51.656926 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034299486 ecr 39555631,nop,nop,sack 1 {1:15}], length 0
06:32:51.670605 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034299489 ecr 39555631], length 15
06:32:52.648684 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39555879 ecr 1034299174], length 14
06:32:52.648780 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034299734 ecr 39555879,nop,nop,sack 1 {1:15}], length 0
06:32:52.676009 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034299741 ecr 39555879], length 15
06:32:54.637273 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39556376 ecr 1034299174], length 14
06:32:54.637384 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034300231 ecr 39556376,nop,nop,sack 1 {1:15}], length 0
06:32:54.694934 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034300245 ecr 39556376], length 15
06:32:58.612848 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39557370 ecr 1034299174], length 14
06:32:58.612910 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034301225 ecr 39557370,nop,nop,sack 1 {1:15}], length 0
06:32:58.726412 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034301253 ecr 39557370], length 15
06:33:02.209303 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [F.], seq 30, ack 15, win 92, options [nop,nop,TS val 1034302124 ecr 39557370], length 0
06:33:06.572702 IP 140.211.166.189.rsync > 10.0.0.22.35819: Flags [P.], seq 1:15, ack 15, win 29, options [nop,nop,TS val 39559360 ecr 1034299174], length 14
06:33:06.572836 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [.], ack 15, win 92, options [nop,nop,TS val 1034303215 ecr 39559360,nop,nop,sack 1 {1:15}], length 0
06:33:06.792342 IP 10.0.0.22.35819 > 140.211.166.189.rsync: Flags [P.], seq 15:30, ack 15, win 92, options [nop,nop,TS val 1034303269 ecr 39559360], length 15


Here is my 1841 router with "debug ip packet 123" where ACL 123 is

access-list 123 permit tcp any any eq 873
access-list 123 permit tcp any eq 873 any

ip nat inside source static 10.0.0.22 69.30.73.18 <- for reference.

It tracks the flow going out....

Code:

Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, Stateful Inspection(4), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, CCE Input Classification(5), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, QoS Drop(6), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, Ingress-NetFlow(17), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, Virtual Fragment Reassembly(21), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, Virtual Fragment Reassembly After IPSec Decryption(32), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, Policy Routing(58), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189, len 60, input feature, MCI Check(63), rtype 0, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, CCE Output Classification(5), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Post-routing NAT Outside(17), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Stateful Inspection(20), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Firewall (NAT)(33), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, IPsec or interface ACL checked on pre-encrypted cleartext packets(34), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Firewall (inspect)(38), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Post-Ingress-NetFlow(52), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, output feature, Egress-Netflow(56), rtype 1, forus FALSE, sendself FALSE, mtu 0
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), g=69.30.73.1, len 60, forward
Aug 24 06:32:50: IP: s=69.30.73.18 (FastEthernet0/0.1), d=140.211.166.189 (BVI1), len 60, sending full packet


And the reflexive ACL get's built and sees the return hits...

Code:

DSL-Router#sho access-list traffic | inc 873
     permit tcp host 140.211.166.189 eq 873 host 69.30.73.18 eq 35819 (10 matches) (time left 299)


and when I get clever and try just port testing:

Code:

gubbie ~ # telnet 140.211.166.189 873
Trying 140.211.166.189...
Connected to 140.211.166.189.
Escape character is '^]'.
@RSYNCD: 30.0
Welcome to bobolink.gentoo.org / rsync.gentoo.org

Server Address : 140.211.166.189
Contact Name   : mirror-admin@gentoo.org
Hardware       : 4 x Intel(R) Atom(TM) CPU D510 @ 1.66GHz, 3960MB RAM
Sponsor        : Gentoo Linux, Open Source Lab, Corvallis, OR, USA

Please note: common gentoo-netiquette says you should not sync more
than once a day.  Users who abuse the rsync.gentoo.org rotation
may be added to a temporary ban list.

MOTD autogenerated by update-rsync-motd on Sun Apr  1 01:06:12 UTC 2012


A TCP connect from my server also generates the same traces above, and appears to be good!

So there is something going wonky in the rsync protocol that it failing... anyone got a way to debug RSYNC?

Nick
Back to top
View user's profile Send private message
nellson
n00b
n00b


Joined: 24 Jun 2004
Posts: 66

PostPosted: Fri Aug 31, 2012 2:47 pm    Post subject: Reply with quote

Not sure why, but for the time being rsync is working reliably again. Wish I have something more concrete to report. :-\
Back to top
View user's profile Send private message
jrussia
Tux's lil' helper
Tux's lil' helper


Joined: 29 Aug 2012
Posts: 89
Location: Chicago

PostPosted: Fri Aug 31, 2012 3:56 pm    Post subject: Reply with quote

I noticed you have SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" in make.conf...

I don't get any results from pinging rsync.namerica.gentoo.org, but rsync.us.gentoo.org and rsync.ca.gentoo.org are pinging fine. Maybe choose one of those and replace. Maybe the namerica name was changed a few months ago (I don't see it in mirrorselect.)

-js


edit: now it pinged fine. Maybe it's just overloaded and that's causing your intermittent problems. I am using the rsync.us.gentoo.org link w/o problems, so maybe it's worth trying.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Installing Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum