Portage sync fails every time, io.c(458)

Message

Sajes · Post by **Sajes** » Sun Jun 08, 2008 7:11 am

I went to sync yesterday, and got the following error:

>>> Starting rsync with rsync://134.68.220.97/gentoo-portage...
>>> Checking server timestamp ...
rsync: connection unexpectedly closed (0 bytes received so far) [receiver]
rsync error: error in rsync protocol data stream (code 12) at io.c(458) [receiver=2.6.9]

I googled around and searched the Gentoo forums for this error, and I couldn't find anything on it so I dismissed it as a server error. I went to sync today, got the same error. Every time it tries to sync.

It seems like something on my system isn't allowing rsync to even check the timestamp. I don't have iptables even compiled at the moment (I'm on a fresh install of Gentoo 2007. I had an extreme amount of configuration errors the first time

)

So I did "emerge-webrsync -v", which did it perfectly, but I really prefer "emerge --sync", because each time I do webrsync it has to download the whole 30MB portage tree as opposed to the few hundred KB of select ebuilds.

After webrsync'ing, I thought that rsync might possibly be borked, so I updated it. It's now version 2.6.9, and still having the same problems.

EDIT: Here's my emerge --info:

Code: Select all

Portage 2.1.4.4 (default-linux/x86/2007.0/desktop, gcc-4.1.2, glibc-2.5-r0, 2.6.24-gentoo-r8 i686)
=================================================================
System uname: 2.6.24-gentoo-r8 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz
Timestamp of tree: Sun, 08 Jun 2008 01:45:01 +0000
app-shells/bash:     3.2_p33
dev-lang/python:     2.4.3-r4
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.9
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.61
sys-devel/automake:  1.9.6-r2, 1.10
sys-devel/binutils:  2.16.1-r3
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.22
virtual/os-headers:  2.6.17-r2
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/X11/xkb"
CONFIG_PROTECT_MASK="/etc/env.d /etc/gconf /etc/revdep-rebuild /etc/terminfo"
CXXFLAGS="-O2 -march=pentium4 -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks metadata-transfer sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LINGUAS="fr de es en"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="X acl acpi alsa arts berkdb cairo cdr cli cracklib crypt css cups dbus dri dvd dvdr dvdread eds emboss encode esd evo fam firefox fortran gdbm gif gnome gpm gstreamer gtk hal iconv isdnlog jpeg kde kerberos ldap mad midi mikmod mp3 mpeg mudflap ncurses nls nptl nptlonly ogg opengl openmp oss pam pcre pdf perl png pppd python qt3 qt3support qt4 quicktime readline reflection sdl session spell spl ssl svg tcpd tiff truetype unicode vorbis win32codecs x86 xml xorg xv zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="fr de es en" USERLAND="GNU" VIDEO_CARDS="apm ark chips cirrus cyrix dummy fbdev glint i128 i740 i810 imstt mach64 mga neomagic nsc nv r128 radeon rendition s3 s3virge savage siliconmotion sis sisusb tdfx tga trident tseng v4l vesa vga via vmware voodoo"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, MAKEOPTS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY

Does anyone have any suggestions?

desultory · Post by **desultory** » Sun Jun 08, 2008 8:45 am

This is due to recent changes in the mirror structure, including the loss of one of the main rotation servers. The infrastructure team is in the process of tuning the remaining servers in the main rotation, though even with tuning the capacity will remain somewhat reduced from its prior state unless another server can be set up to replace it. The most practical solution is to not use the main rotation by switching to the local rotation nearest you, at least so far as the network is concerned.

To that end the recommendation in the handbook is to use mirrorselect (app-portage/mirrorselect), mirrorselect -i -r should suffice.

Made topic sticky.

Sajes · Post by **Sajes** » Sun Jun 08, 2008 4:58 pm

Worked excellently. Thanks for responding so quickly

NorthGoingZax · Post by **NorthGoingZax** » Sun Jun 22, 2008 6:51 pm

I did mirrorselect -i -r and selected the NorthAmerica server, and it did not solve the problem for me.

desultory · Post by **desultory** » Mon Jun 23, 2008 7:39 am

Please post the output from emerge --info and the error message, unless it is identical to the error message posted by Sajes.

NorthGoingZax · Post by **NorthGoingZax** » Tue Jun 24, 2008 1:47 am

I am getting a different error message:

Code: Select all

 * Removing old portage-cache in /var/cache/edb/dep
 * Running emerge --sync
>>> Starting rsync with rsync://150.135.81.231/gentoo-portage...
>>> Checking server timestamp ...
rsync: failed to connect to 150.135.81.231: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 1 of 3 with rsync://141.219.155.230/gentoo-portage
>>> Checking server timestamp ...
rsync: failed to connect to 141.219.155.230: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 2 of 3 with rsync://209.221.142.124/gentoo-portage
>>> Checking server timestamp ...
rsync: failed to connect to 209.221.142.124: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 3 of 3 with rsync://156.56.247.193/gentoo-portage
>>> Checking server timestamp ...
rsync: failed to connect to 156.56.247.193: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]
>>> Exceeded PORTAGE_RSYNC_RETRIES: 3

here is my emerge --info

Code: Select all

Portage 2.1.4.4 (default-linux/x86/2006.1, gcc-4.1.2, glibc-2.6.1-r0, 2.6.24-tuxonice-r5 i686)
=================================================================
System uname: 2.6.24-tuxonice-r5 i686 Genuine Intel(R) CPU T2050 @ 1.60GHz
Timestamp of tree: Sun, 22 Jun 2008 01:45:01 +0000
app-shells/bash:     3.2_p17
dev-java/java-config: 1.3.7, 2.1.6
dev-lang/python:     2.4.4-r5
dev-python/pycrypto: 2.0.1-r6
sys-apps/baselayout: 1.12.11.1
sys-apps/sandbox:    1.2.18.1-r2
sys-devel/autoconf:  2.13, 2.61-r2
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.6-r2, 1.10.1
sys-devel/binutils:  2.18-r1
sys-devel/gcc-config: 1.4.0-r4
sys-devel/libtool:   1.5.26
virtual/os-headers:  2.6.23-r3
ACCEPT_KEYWORDS="x86"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=prescott -pipe -msse3 -fomit-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/kde/4.0/env /usr/kde/4.0/share/config /usr/kde/4.0/shutdown /usr/share/X11/xkb /usr/share/config"
CONFIG_PROTECT_MASK="/etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/terminfo /etc/udev/rules.d"
CXXFLAGS="-O2 -march=prescott -pipe -msse3 -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="distlocks metadata-transfer parallel-fetch sandbox sfperms strict unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="3dnow X acpi alsa bash-completion berkdb cairo cli cracklib crypt cups dbus dri dvd dvdread firefox fortran gdbm gif gpm hal iconv isdnlog javascript jpeg kde kde4 midi mmx mmxext mp3 mudflap mysql ncurses nls nptl nptlonly nsplugin ogg opengl openmp oss pam pcre pdf perl png ppds pppd python qt3 qt4 quicktime readline real reflection rtc ruby session spl sqlite sqlite3 sse sse2 ssl tcpd truetype unicode vorbiswin32codecs x86 xcomposite xorg xv zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="evdev synaptics keyboard mouse" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" USERLAND="GNU" VIDEO_CARDS="i810"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, LDFLAGS, LINGUAS, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

desultory · Post by **desultory** » Tue Jun 24, 2008 8:27 am

NorthGoingZax wrote:

Code: Select all

rsync: failed to connect to 150.135.81.231: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]

That could be due to firewall restrictions. Does rsync work with any external host?

NorthGoingZax · Post by **NorthGoingZax** » Tue Jun 24, 2008 4:09 pm

good question... I don't use rsync for anything else, is there another host of some sort you can recommend so I can test it ?

desultory · Post by **desultory** » Wed Jun 25, 2008 8:41 am

For the sake of expediency, utility and laziness, instead of trying to find public rsync test service just use a different mirror to do the test, like so: rsync -nv rsync.us.gentoo.org::gentoo-portage/metadata/timestamp.chk.

NorthGoingZax · Post by **NorthGoingZax** » Thu Jun 26, 2008 3:09 am

Code: Select all

# rsync -nv rsync.us.gentoo.org::gentoo-portage/metadata/timestamp.chk
rsync: failed to connect to rsync.us.gentoo.org: Connection refused (111)
rsync error: error in socket IO (code 10) at clientserver.c(122) [receiver=3.0.2]

desultory · Post by **desultory** » Thu Jun 26, 2008 4:22 am

Were you previously able to use the main rotation? I ask because this appears to be a problem with your firewall.

NorthGoingZax · Post by **NorthGoingZax** » Thu Jun 26, 2008 6:17 pm

I have been connecting to the main server for the last 2-3 years. I will ask our admin if something has changed.

queen · Post by **queen** » Sun Jul 06, 2008 10:21 pm

In my case emerge sync from belnet give me this:

Code: Select all

file has vanished: "sys-block/megamgr/metadata.xml" (in gentoo-portage)
file has vanished: "sys-block/ms-sys/metadata.xml" (in gentoo-portage)
file has vanished: "sys-block/afacli/afacli-4.1.ebuild" (in gentoo-portage)
file has vanished: "sys-boot/aboot/aboot-1.0_pre20040408.ebuild" (in gentoo-portage)
file has vanished: "sys-boot/arcboot/arcboot-0.3.8.6-r1.ebuild" (in gentoo-portage)
file has vanished: "sys-boot/arcload/files/arcload-0.43-tweaks1.patch" (in gentoo-portage)

And that's snipped, due to the length. It starts from a and continues.

desultory · Post by **desultory** » Mon Jul 07, 2008 4:11 am

It seems that is no longer a [bug=203437]valid[/bug] mirror, it has been dropped from the rotation due to related [bug=214928]problems[/bug]. Please post the output of either emerge --info or at least portageq envvar SYNC so that the problem can be properly isolated and resolved if it is not already.

queen · Post by **queen** » Mon Jul 07, 2008 8:20 am

desultory wrote:It seems that is no longer a [bug=203437]valid[/bug] mirror, it has been dropped from the rotation due to related [bug=214928]problems[/bug]. Please post the output of either emerge --info or at least portageq envvar SYNC so that the problem can be properly isolated and resolved if it is not already.

I managed to sync with germany. Couldn't sync at belnet, which I was very satisfied all the time. Here are the links:

Code: Select all

SYNC="rsync://linux.rz.ruhr-uni-bochum.de/gentoo-portage/"
#SYNC="rsync://rsync.belnet.be/gentoo-portage"

desultory · Post by **desultory** » Tue Jul 08, 2008 5:45 am

In most cases, essentially unless you have a local mirror, you should use a rotation instead of a specific server. Not only does it help to balance the load on the servers it helps to avoid this sort of problem.

In your case rsync.lu.gentoo.org and rsync.de.gentoo.org seem to be the best matches, considering that rsync.be.gentoo.org is presently an alias of rsync.us.gentoo.org.

magowiz · Post by **magowiz** » Thu Aug 07, 2008 11:08 am

queen wrote:
desultory wrote:It seems that is no longer a [bug=203437]valid[/bug] mirror, it has been dropped from the rotation due to related [bug=214928]problems[/bug]. Please post the output of either emerge --info or at least portageq envvar SYNC so that the problem can be properly isolated and resolved if it is not already.
I managed to sync with germany. Couldn't sync at belnet, which I was very satisfied all the time. Here are the links:
Code: Select all
SYNC="rsync://linux.rz.ruhr-uni-bochum.de/gentoo-portage/"
#SYNC="rsync://rsync.belnet.be/gentoo-portage"

switching to Bochum University mirror worked here,
before I was using rsync.europe.org , and this was mine error :

Code: Select all

 # emerge --sync
>>> Starting rsync with rsync://137.226.34.228/gentoo-portage...
>>> Checking server timestamp ...
Welcome to rsync.informatik.rwth-aachen.de (137.226.34.228).
This server is part of the SunSITE Central Europe and is located
in Aachen, Germany (http://sunsite.informatik.rwth-aachen.de).

receiving file list ... timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(541) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 1 of 3 with rsync://88.198.224.205/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(541) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 2 of 3 with rsync://88.156.78.16/gentoo-portage
>>> Checking server timestamp ...

contact: ftp@vectranet.pl

receiving incremental file list
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(541) [receiver=3.0.2]
>>> Retrying...


>>> Starting retry 3 of 3 with rsync://194.97.4.250/gentoo-portage
>>> Checking server timestamp ...
I-Node Gentoo rsync service
Gigabit Interface
2x10GBps co-lo connectivity
Ath64 2Ghz - 2GB RAM
Location: Duesseldorf

rsync.i-no.de aka
rsync5.de.gentoo.org

Contact: rsyncadm (swirly thing) schwarzvogel.de


Total rsync traffic yesterday: 7632.00 Mb
Number of rsync connections  : 4054

timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(541) [receiver=3.0.2]
>>> Exceeded PORTAGE_RSYNC_RETRIES: 3

Tadeas · Post by **Tadeas** » Mon Mar 09, 2009 12:08 pm

Now it doesn't work for me

Code: Select all

 # emerge --sync
>>> Starting rsync with rsync://194.97.4.250/gentoo-portage...
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(544) [receiver=3.0.5]
>>> Retrying...


>>> Starting retry 1 of 3 with rsync://88.198.224.205/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(544) [receiver=3.0.5]
>>> Retrying...


>>> Starting retry 2 of 3 with rsync://83.164.192.205/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(544) [receiver=3.0.5]
>>> Retrying...


>>> Starting retry 3 of 3 with rsync://62.149.9.12/gentoo-portage
>>> Checking server timestamp ...
timed out
rsync error: received SIGINT, SIGTERM, or SIGHUP (code 20) at rsync.c(544) [receiver=3.0.5]
>>> Exceeded PORTAGE_RSYNC_RETRIES: 3

I try to sync with the common European rotation:

Code: Select all

SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"

I've tried the Bochum University mirror as suggested above, but I've got the same problems. It never worked in last say month - it's how long I have Gentoo.
emerge-webrsync works well though, so it's nothing all that bad, but I write it to here so somebody can to something about that. Or shall I file a bug?

Xamindar · Post by **Xamindar** » Thu Mar 19, 2009 8:15 pm

I'm getting that same error today too. What is the problem this time?

iGentoo900 · Post by **iGentoo900** » Tue May 12, 2009 10:01 pm

You may need to use app-portage/mirrorselect. I am sorry if this has been referenced before.

BTW, I just synced, no problems for me using normal round-robin. Try removing the rsync mirrors from your make.conf and try emerge --sync again.

danulive · Post by **danulive** » Fri Aug 21, 2009 3:47 pm

Does That "Updating the Portage tree" Installing part we must have to do? or Not? when we are going to install gentoo 2008 x86 in to USB pen DRIVE.....

Need a quick reply from some one.............................................

pandaxiongmao · Post by **pandaxiongmao** » Fri Nov 20, 2009 2:05 pm

It seems that Georgia Tech rsync mirror is having problem today. I was connected, but then I never received the file. I was stuck on the "receiving file list..." message for a long time.

Darklapin · Post by **Darklapin** » Mon Nov 23, 2009 6:50 pm

It also fails *almost* everytime for me.
It's not due to firewall problems as it happened before I set it up, and because there's no drop packets in logs.

I succeed for some ports then stuck on one (randomly). Then it retries, but always fail after 3 times.
I used mirrorselect to select my mirror, and it's European one.