emerge hanging after ">>> Emerging (1 of 1) foo" line

Message

mhill · Post by **mhill** » Sat Jul 02, 2011 4:24 pm

I have a main system that runs "emerge sync" to update /usr/portage/distfiles daily. The rest of my systems then NFS-mount that directory from the main system. I just upgraded the kernel and some packages on the main system yesterday. Today, two laptops (that connect via wifi) can't emerge anything. Sample:

Code: Select all

#  emerge -u autoconf
Calculating dependencies... done!

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-devel/autoconf-2.68

It hangs after printing that, until I interrupt it. No CPU, disk or network activity that I can see. "pstree PID" shows only "emerge---emerge".

A desktop client, that connects via Ethernet, has no such problem, so I don't think it's probably related to the kernel/package upgrades on the main system.

I rebooted one of the laptops and tried again, in case it was something that was cached, but no dice; the output above is from after the reboot.

From the rebooted client:

# emerge --info
Portage 2.1.10.3 (default/linux/x86/10.0/desktop, gcc-4.4.4, glibc-2.11.3-r0, 2.6.36-gentoo-r8 i686)
=================================================================
System uname: Linux-2.6.36-gentoo-r8-i686-Intel-R-_Core-TM-2_Duo_CPU_T7500_@_2.20GHz-with-gentoo-1.12.14
Timestamp of tree: Sat, 02 Jul 2011 12:00:01 +0000
ccache version 2.4 [enabled]
app-shells/bash: 4.1_p9
dev-java/java-config: 2.1.11-r3
dev-lang/python: 2.6.6-r2, 3.1.3-r1
dev-util/ccache: 2.4-r9
dev-util/cmake: 2.8.1-r2
dev-util/pkgconfig: 0.25-r2
sys-apps/baselayout: 1.12.14-r1
sys-apps/sandbox: 2.4
sys-devel/autoconf: 2.13::<unknown repository>, 2.65-r1
sys-devel/automake: 1.8.5-r4, 1.9.6-r3, 1.10.3, 1.11.1
sys-devel/binutils: 2.20.1-r1
sys-devel/gcc: 3.4.6-r2, 4.3.4, 4.4.4-r2
sys-devel/gcc-config: 1.4.1-r1
sys-devel/libtool: 2.2.10
sys-devel/make: 3.82
sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers)
sys-libs/glibc: 2.11.3
Repositories: gentoo
ACCEPT_KEYWORDS="x86"
ACCEPT_LICENSE="* -@EULA"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=nocona -fomit-frame-pointer -pipe"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/openvpn/easy-rsa"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -march=nocona -fomit-frame-pointer -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
LINGUAS="en"
MAKEOPTS="-j3"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
SYNC="rsync://shadowfax/portage"
USE="X a52 aac acl acpi aiglx alsa apm assistant audacious bash-completion berkdb branding bzip2 cairo cdda cddb cdinstall cdparanoia cdr cli colordiff consolekit crypt css cups curl cxx dbus dedicated dell device-mapper divx dri dts dvd dvdr emboss emerald encode escreen exif fam fbcon fbcondecor fftw firefox flac gdbm gif glibc-omitfp glitz gnutls gpm gtk gtk2 hou iconv imlib jabber jack jpeg jpeg2k lame laptop lcms logrotate mad mbox mmx mmxext mng modules mp3 mp4 mpeg mudflap multiuser ncurses nptl nptlonly nsplugin ogg ole openal opengl oss pam pango pcre pdf pmu png policykit ppds python qt3support qt4 readline sdl session smp sou spell sse sse2 ssl startup-notification svg sysfs tcpd tiff tk truetype udev usb videos vim-syntax vorbis win32codecs wmp x264 x86 xcb xorg xscreensaver xulrunner xv xvid zlib" ALSA_CARDS="hda-intel" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="braindump flow karbon kexi kpresenter krita tables words" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="nvidia vga" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS

Timestamps of the gentoolkit and portage ebuilds:

Code: Select all

-rw-r--r-- 1 root root 220137 Jun 24 18:26 gentoolkit-0.3.0.4.tbz2
-rw-r--r-- 1 root root 878017 Jun 15 23:44 portage-2.1.10.3.tbz2

I have emerged about 30 packages since the upgrade of portage, and nine since June 24, so it wasn't broken as of June 27:

Code: Select all

-rw-r--r-- 1 root root       4810 Jun 27 21:56 perl-Test-Simple-0.980.0-r1.tbz2
-rw-r--r-- 1 root root     204103 Jun 24 18:37 neon-0.29.6.tbz2
-rw-r--r-- 1 root root     318711 Jun 24 18:33 libidn-1.22.tbz2
-rw-r--r-- 1 root root     220137 Jun 24 18:26 gentoolkit-0.3.0.4.tbz2
-rw-r--r-- 1 root root     106047 Jun 24 18:26 dnspython-1.9.4.tbz2
-rw-r--r-- 1 root root    2992770 Jun 24 18:26 bind-tools-9.7.3.tbz2
-rw-r--r-- 1 root root     246174 Jun 24 18:24 sip-4.12.1.tbz2
-rw-r--r-- 1 root root      46355 Jun 24 18:24 argparse-1.2.1.tbz2
-rw-r--r-- 1 root root      25982 Jun 24 18:23 unifdef-2.3.tbz2

Since emerge isn't a shell script, I don't know how to trace its execution (like I would with "sh -x"). "emerge -uvvv" didn't yield anything after the "Emerging" line.

Thanks for any help resolving this.

Post by Hu » Sat Jul 02, 2011 5:15 pm

What are the last lines out of dev-util/strace? Have you tried emerge --debug?

mhill · Post by **mhill** » Sat Jul 02, 2011 5:27 pm

I hadn't tried the --debug flag; here's the output.

Code: Select all

# emerge --debug -u autoconf
myaction None
myopts {'--debug': True, '--update': True, '--buildpkg': True}

myparams {'selective': True, 'recurse': True}

Calculating dependencies     ebuild: sys-devel/autoconf-2.68
installed: sys-devel/autoconf-2.65-r1
   ebuild: sys-devel/autoconf-2.13
installed: sys-devel/autoconf-2.13
   ebuild: sys-devel/autoconf-2.68
installed: sys-devel/autoconf-2.65-r1
   ebuild: sys-devel/autoconf-2.13
installed: sys-devel/autoconf-2.13

      Arg: autoconf
     Atom: sys-devel/autoconf
   ebuild: sys-devel/autoconf-2.68
installed: sys-devel/autoconf-2.65-r1
      Arg: autoconf
     Atom: sys-devel/autoconf:2.1
   ebuild: sys-devel/autoconf-2.13
installed: sys-devel/autoconf-2.13

Parent:    (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)
Depstring: >=sys-apps/texinfo-4.3 >=sys-devel/m4-1.4.6 dev-lang/perl
Priority:  buildtime

Parent:    (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)
Depstring: >=sys-apps/texinfo-4.3 >=sys-devel/m4-1.4.6 dev-lang/perl
Priority:  buildtime
Candidates: ['dev-lang/perl', '>=sys-devel/m4-1.4.6', '>=sys-apps/texinfo-4.3']
   ebuild: dev-lang/perl-5.12.3-r1
installed: dev-lang/perl-5.12.2-r6
   ebuild: sys-devel/m4-1.4.15
installed: sys-devel/m4-1.4.15
   ebuild: sys-apps/texinfo-4.13
installed: sys-apps/texinfo-4.13
Exiting... (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)

Parent:    (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)
Depstring: >=sys-apps/texinfo-4.3 >=sys-devel/m4-1.4.6 dev-lang/perl >=sys-devel/autoconf-wrapper-10
Priority:  runtime

Parent:    (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)
Depstring: >=sys-apps/texinfo-4.3 >=sys-devel/m4-1.4.6 dev-lang/perl >=sys-devel/autoconf-wrapper-10
Priority:  runtime
Candidates: ['dev-lang/perl', '>=sys-devel/m4-1.4.6', '>=sys-apps/texinfo-4.3', '>=sys-devel/autoconf-wrapper-10']
   ebuild: dev-lang/perl-5.12.3-r1
installed: dev-lang/perl-5.12.2-r6
installed: sys-devel/m4-1.4.15
installed: sys-apps/texinfo-4.13
   ebuild: sys-devel/autoconf-wrapper-10-r1
installed: sys-devel/autoconf-wrapper-10-r1
Exiting... (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)

Parent:    (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge)
Depstring: emacs? ( app-emacs/autoconf-mode )
Priority:  runtime_post

digraph:

autoconf depends on
  (sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge) (soft)
  (sys-devel/autoconf-2.13::__unknown__, installed) (soft)
(sys-devel/autoconf-2.68::gentoo, ebuild scheduled for merge) depends on
  (sys-devel/m4-1.4.15::gentoo, installed) (buildtime)
  (sys-apps/texinfo-4.13::gentoo, installed) (buildtime)
  (sys-devel/autoconf-wrapper-10-r1::gentoo, installed) (runtime)
(sys-devel/autoconf-wrapper-10-r1::gentoo, installed) (no children)
(sys-devel/autoconf-2.13::__unknown__, installed) (no children)
(sys-devel/m4-1.4.15::gentoo, installed) (no children)
(sys-apps/texinfo-4.13::gentoo, installed) (no children)

   ebuild: dev-lang/python-2.7.1-r1
   ebuild: dev-lang/python-2.6.6-r2
installed: dev-lang/python-2.6.6-r2
   ebuild: dev-lang/python-3.1.3-r1
installed: dev-lang/python-3.1.3-r1
   ebuild: dev-lang/python-3.1.3-r1
installed: dev-lang/python-3.1.3-r1

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sys-devel/autoconf-2.68
+ dyn_clean
+ '[' -z /var/tmp/portage/sys-devel/autoconf-2.68 ']'
+ '[' '!' -d /var/tmp/portage/sys-devel/autoconf-2.68 ']'
+ hasq chflags assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch
+ [[  assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch  == *\ \c\h\f\l\a\g\s\ * ]]
+ rm -rf /var/tmp/portage/sys-devel/autoconf-2.68/image /var/tmp/portage/sys-devel/autoconf-2.68/homedir
+ rm -f /var/tmp/portage/sys-devel/autoconf-2.68/.installed
+ [[ ebuild = binary ]]
+ hasq keeptemp assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch
+ [[  assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch  == *\ \k\e\e\p\t\e\m\p\ * ]]
+ hasq keepwork assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch
+ [[  assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch  == *\ \k\e\e\p\w\o\r\k\ * ]]
+ rm -rf /var/tmp/portage/sys-devel/autoconf-2.68/temp
+ [[ ebuild = binary ]]
+ hasq keepwork assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch
+ [[  assume-digests binpkg-logs buildpkg ccache collision-protect distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch  == *\ \k\e\e\p\w\o\r\k\ * ]]
+ rm -f /var/tmp/portage/sys-devel/autoconf-2.68/.ebuild_changed /var/tmp/portage/sys-devel/autoconf-2.68/.logid /var/tmp/portage/sys-devel/autoconf-2.68/.pretended /var/tmp/portage/sys-devel/autoconf-2.68/.setuped /var/tmp/portage/sys-devel/autoconf-2.68/.unpacked /var/tmp/portage/sys-devel/autoconf-2.68/.prepared /var/tmp/portage/sys-devel/autoconf-2.68/.configured /var/tmp/portage/sys-devel/autoconf-2.68/.compiled /var/tmp/portage/sys-devel/autoconf-2.68/.tested /var/tmp/portage/sys-devel/autoconf-2.68/.packaged /var/tmp/portage/sys-devel/autoconf-2.68/.die_hooks /var/tmp/portage/sys-devel/autoconf-2.68/.ipc_in /var/tmp/portage/sys-devel/autoconf-2.68/.ipc_out /var/tmp/portage/sys-devel/autoconf-2.68/.ipc_lock /var/tmp/portage/sys-devel/autoconf-2.68/.exit_status
+ rm -rf /var/tmp/portage/sys-devel/autoconf-2.68/build-info
+ rm -rf /var/tmp/portage/sys-devel/autoconf-2.68/work
+ '[' -f /var/tmp/portage/sys-devel/autoconf-2.68/.unpacked ']'
+ rm -rf /var/tmp/portage/sys-devel/autoconf-2.68/distdir
+ cd /var/tmp/portage/sys-devel/autoconf-2.68/../..
+ rmdir /var/tmp/portage/sys-devel/autoconf-2.68
+ true
+ set +x

Of course, it hangs after the "set +x" line. I'll try strace and see if it shows anything useful. Thanks.

mhill · Post by **mhill** » Sat Jul 02, 2011 5:40 pm

strace produced a 2.2MB file, almost 30k lines long. Here's the tail end:

Code: Select all

lstat64("/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat64("/usr/portage", {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0
lstat64("/usr/portage/rpm", 0xbfffd2fc) = -1 ENOENT (No such file or directory)
access("/usr/portage/sys-devel/autoconf/autoconf-2.68.ebuild", R_OK) = 0
stat64("/usr/portage/sys-devel/autoconf/autoconf-2.68.ebuild", {st_mode=S_IFREG|0644, st_size=1411, ...}) = 0
open("/usr/portage/metadata/cache/sys-devel/autoconf-2.68", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=527, ...}) = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=527, ...}) = 0
_llseek(3, 0, [0], SEEK_CUR)            = 0
_llseek(3, 0, [0], SEEK_CUR)            = 0
read(3, ">=sys-apps/texinfo-4.3 >=sys-dev"..., 8192) = 527
read(3, "", 8192)                       = 0
fstat64(3, {st_mode=S_IFREG|0644, st_size=527, ...}) = 0
close(3)                                = 0
ioctl(1, SNDCTL_TMR_TIMEBASE or TCGETS, 0xbfffd6dc) = -1 EINVAL (Invalid argument)
pipe([3, 5])                            = 0
fcntl64(3, F_GETFL)                     = 0 (flags O_RDONLY)
fcntl64(3, F_SETFL, O_RDONLY|O_NONBLOCK) = 0
fcntl64(3, F_GETFL)                     = 0x800 (flags O_RDONLY|O_NONBLOCK)
fstat64(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb70c3000
_llseek(3, 0, 0xbfffd68c, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
fstat64(3, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
munmap(0xb70c3000, 4096)                = 0
open("/var/tmp/portage/sys-devel/autoconf-2.68/temp/build.log", O_WRONLY|O_CREAT|O_APPEND|O_LARGEFILE, 0666) = 6
fstat64(6, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb70c3000
fstat64(6, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
_llseek(6, 0, [0], SEEK_SET)            = 0
fstat64(6, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
stat64("/var/tmp/portage/sys-devel/autoconf-2.68/temp/build.log", {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
chown32("/var/tmp/portage/sys-devel/autoconf-2.68/temp/build.log", 250, 250) = 0
chmod("/var/tmp/portage/sys-devel/autoconf-2.68/temp/build.log", 0660) = 0
dup(1)                                  = 7
fcntl64(7, F_GETFL)                     = 0x1 (flags O_WRONLY)
fstat64(7, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb70c2000
_llseek(7, 0, 0xbfffd68c, SEEK_CUR)     = -1 ESPIPE (Illegal seek)
fstat64(7, {st_mode=S_IFIFO|0600, st_size=0, ...}) = 0
clone(child_stack=0, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0xb7481728) = 16310
close(5)                                = 0
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)
poll([{fd=3, events=POLLIN|POLLERR|POLLHUP|POLLNVAL}], 1, 3000) = 0 (Timeout)

mhill · Post by **mhill** » Tue Jul 05, 2011 2:00 pm

New update: It does appear to be related to the NFS upgrade on the main system. I tried unmounting the NFS mount on /usr/portage/distfiles and "emerge -u portage" launched right into the download. I am filtering the wifi traffic, but have had the NFS traffic tuned and working for years. The problem is fairly subtle, though, as I can read files on the NFS mount without trouble.

mhill · Post by **mhill** » Sun Jul 10, 2011 3:05 am

Another update: The only NFS-related package that had changed recently was libtirpc. So about an hour ago I reinstalled the old one. Same problem; emerge hangs on the two wifi laptops, works fine on the wired NFS client.

I also upgraded the kernel on or around July 1 (2.6.36 to 2.6.37). I checked NFS settings and the only diff is CONFIG_NFSD_DEPRECATED=y in the new kernel. I have rebuilt with that undefined and will test after the next reboot. Other than that, I'm out of ideas. :^(

gkmac · Post by **gkmac** » Tue Oct 25, 2011 7:42 pm

Did you manage to solve this problem?

I share my portage tree from my server using NFS and I've recently been hit by exactly the same symptoms about a week ago, namely emerge will stop dead after printing the ">>> Emerging (1 of 1)" line, with no errors anywhere to be seen.

I can't put my finger on what is causing this to happen; I'm running portage-2.2.0_alpha70

What is very strange is only portage seems to be showing this symptom. If I cd to /usr/portage/distfiles I can do all sorts of write operations, whether it be touch'ing a test file or running wget inside it or even permission changing, it all works flawlessly.

Reading /usr/portage/distfiles isn't a problem for portage; if I manually download the package .tar.bz2 myself into that directory, portage reads it and goes on to compile & install it.

So it seems something within portage is causing this to happen, although as I've said before all sorts of write-tests I can do are working normally, so I can't think of what portage is doing that causes it to hang...

Aquous · Post by **Aquous** » Tue Oct 25, 2011 8:15 pm

I've had this problem when an incorrectly stopped emerge command left a stale lockfile in /var/tmp/portage. So try\

Code: Select all

rm -rf /var/tmp/portage/.*

(usual 'rm -rf is dangerous' disclaimers apply)

mhill · Post by **mhill** » Tue Oct 25, 2011 8:19 pm

gkmac wrote:Did you manage to solve this problem?

I did... unfortunately I can't recall exactly what the solution was.

gkmac wrote:I share my portage tree from my server using NFS and I've recently been hit by exactly the same symptoms about a week ago, namely emerge will stop dead after printing the ">>> Emerging (1 of 1)" line, with no errors anywhere to be seen.

I can't put my finger on what is causing this to happen; I'm running portage-2.2.0_alpha70

What is very strange is only portage seems to be showing this symptom. If I cd to /usr/portage/distfiles I can do all sorts of write operations, whether it be touch'ing a test file or running wget inside it or even permission changing, it all works flawlessly.

Agreed; my problem didn't seem to be with NFS access proper.

gkmac wrote:Reading /usr/portage/distfiles isn't a problem for portage; if I manually download the package .tar.bz2 myself into that directory, portage reads it and goes on to compile & install it.

Interesting... I thought I remembered that even if the source file was present, the client would still hang. But of course it's been 3.5 months....

gkmac wrote:So it seems something within portage is causing this to happen, although as I've said before all sorts of write-tests I can do are working normally, so I can't think of what portage is doing that causes it to hang...

Haha! History to the rescue!

My original post: July 2. I remember it took maybe as long as a week before I got it figured out.
/usr/portage/packages/All from early July: A bunch of activity on July 1 (when I first struggled with the problem, before I tried the forums), 1 on July 3, then several on July 9. Also coming back to me: It wasn't a package rebuild (I tried most/all of the NFS-related packages), but a kernel option.
Check /boot/config* ... paydirt!

Code: Select all

# ll /boot/config-2.6.37*
-rw-r--r-- 1 root root 58441 Jul  9 21:07 /boot/config-2.6.37-gentoo-r4
-rw-r--r-- 1 root root 58430 Jul  6 17:37 /boot/config-2.6.37-gentoo-r4.old
# diff /boot/config-2.6.37-gentoo-r4{.old,}
4c4
< # Wed Jul  6 17:32:42 2011
---
> # Sat Jul  9 20:53:26 2011
1880c1880
< CONFIG_NFSD_DEPRECATED=y
---
> # CONFIG_NFSD_DEPRECATED is not set

So that was it. It appears that in 2.6.37 they deprecated some NFSd code. Sorry that, in the excitement of the moment and the busy catching-up on my clients, I never did get around to getting back here with the solution.

I thought I'd mark this topic as [solved], but I think I should wait for you to see if that fixes your problem first. HTH!

gkmac · Post by **gkmac** » Tue Oct 25, 2011 10:16 pm

Aquous wrote:I've had this problem when an incorrectly stopped emerge command left a stale lockfile in /var/tmp/portage. So try\
Code: Select all
rm -rf /var/tmp/portage/.*
(usual 'rm -rf is dangerous' disclaimers apply)

Nothing was to be found in /var/tmp/portage, not even hidden files that would have been revealed by ls -a

mhill wrote:Check /boot/config* ... paydirt!
Code: Select all
# ll /boot/config-2.6.37*
-rw-r--r-- 1 root root 58441 Jul  9 21:07 /boot/config-2.6.37-gentoo-r4
-rw-r--r-- 1 root root 58430 Jul  6 17:37 /boot/config-2.6.37-gentoo-r4.old
# diff /boot/config-2.6.37-gentoo-r4{.old,}
4c4
< # Wed Jul  6 17:32:42 2011
---
> # Sat Jul  9 20:53:26 2011
1880c1880
< CONFIG_NFSD_DEPRECATED=y
---
> # CONFIG_NFSD_DEPRECATED is not set
So that was it. It appears that in 2.6.37 they deprecated some NFSd code. Sorry that, in the excitement of the moment and the busy catching-up on my clients, I never did get around to getting back here with the solution.

Sadly my server runs on openvz-sources which are stuck at 2.6.32, thus upgrading the kernel to find that setting is not an option for me.

krinn · Post by **krinn** » Tue Oct 25, 2011 10:49 pm

try with FEATURES="-distlocks"

mhill · Post by **mhill** » Tue Oct 25, 2011 11:53 pm

gkmac wrote:Sadly my server runs on openvz-sources which are stuck at 2.6.32, thus upgrading the kernel to find that setting is not an option for me.

Well, that means the new kernel config option wasn't the source [sic] of your problem, either. Sorry that didn't help you.

gkmac · Post by **gkmac** » Wed Oct 26, 2011 8:52 am

krinn wrote:try with FEATURES="-distlocks"

Bingo!

Now emerge is downloading files again into that NFS share again.

Thanks a lot krinn! ...and to mhill for suggesting other things to try.

krinn · Post by **krinn** » Wed Oct 26, 2011 1:06 pm

Glad that work but you should be aware of what is happening and consequences to you then.

- your nfs export is somehow invalid/buggy, as the user doesn't match the uid on the client, same for guid, or your export it as anonymous and the anon uid/gid doesn't match, or you use nfs4 without proper security, or you lack acl in your kernel or nfs server, nfsv4 server with strange implementation and a strick nfsv4 client, or too strict rules to access it...
Well, it could be really many many things, but to sum up, portage cannot create a new directory in it, as portage try to create an hidden one in your distfiles dir and write a file in it for each package you will emerge, so emerge twice time the same package will be handle... any parallel operations.

- now that the feature is disable, portage simply stop try to write to that directory, so you bypass the reason, but portage now cannot see if it is already downloading or emerge the file...
It's not really a big lost, if you're aware of what you loose, but per example, <emerge vlc & emerge world> and vlc is in your world won't be handle properly now. --jobs might be affect as well

Just to let you know

gkmac · Post by **gkmac** » Thu Oct 27, 2011 9:05 pm

krinn wrote:your nfs export is somehow invalid/buggy

Well it all worked smoothly for several years and suddenly stopped last week. I never changed anything at all on the NFS side of things on either my server or any of my clients. The only change I can put my finger on is the ever-shapeshifting nature of the portage-2.2.0_alpha code.

krinn wrote:as the user doesn't match the uid on the client, same for guid

The portage UID and GID are and have always been the same on my server and all my clients (250).

krinn wrote:or you lack acl in your kernel or nfs server

Admittedly I don't have ACL in the kernel or NFS server, but it wasn't a problem for several years... until last week.

krinn wrote:or you use nfs4 without proper security ... nfsv4 server with strange implementation and a strick nfsv4 client

I'm using NFS v3. The kernel options for NFS v4 have a big EXPERIMENTAL next to them, so I don't use it.

krinn wrote:or your export it as anonymous and the anon uid/gid doesn't match

The share is exported as the portage UID and GID of 250, and I shall repeat that that number is consistent across all my machines.

krinn wrote:Well, it could be really many many things, but to sum up, portage cannot create a new directory in it, as portage try to create an hidden one in your distfiles dir and write a file in it for each package you will emerge

Hmmm... portage can't create a directory in the NFS share, meaning it can't write. And yet somehow it seems to download a file in that same NFS share, meaning it can write. That doesn't add up at all. Just now I've tried touching files and creating directories in the shell and it all works hunky dory, so it must be a fault in portage, though I can't imagine what.

adamf663 · Post by **adamf663** » Sat Sep 12, 2015 4:52 pm

I had something similar after messing around with the '--jobs' option. No emerge would run after that.
When I went to clear out /var/tmp/portage, I discovered a lockfile.
Ran lsof and found an orphaned emerge. Killed it and emerges started running properly again.

ravloony · Post by **ravloony** » Fri Jan 15, 2016 9:38 am

adamf663 wrote:I had something similar after messing around with the '--jobs' option. No emerge would run after that.
When I went to clear out /var/tmp/portage, I discovered a lockfile.
Ran lsof and found an orphaned emerge. Killed it and emerges started running properly again.

Just happened to me. Thanks for the writeup!

emerge hanging after ">>> Emerging (1 of 1) foo" line

emerge hanging after ">>> Emerging (1 of 1) fo

a solution

Re: a solution