distcc fails on several libs, no apparent reason

Message

chaoscommander · Post by **chaoscommander** » Tue Dec 17, 2013 8:17 pm

I've set up distcc correctly on two machines, it usually works too. Both have the same problem: on some libraries, emerge will abort saying something along the lines of
"/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/4.2.3/../../../../lib/libXt.a(Alloc.o): relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC"
This is not actually my error, just an example for the error message. I received it with libXt, glibc and several others, but since it doesn't happen when I deactivate distcc it can't really be a bug with the packages, it can't be about use flags, cflags etc. (tried setting -fPIC or USE="pic" anyway, helped for some, didn't help for all); it always happens to compile threads on localhost so the host gcc version can't be an issue either (they are the same anyway). By now I'm out of ideas where to look for the error. Any suggestions..?

szatox · Post by **szatox** » Wed Dec 18, 2013 10:07 am

1) some packages simply don't like distcc (and disable it with ebuild)
2) different versions might couse errors
3) distcc doesn't like ipv6,even though it claims it's supported
3) it's rainy day in UK
5) I've just used 3 twice
and possibly many more.

Anyway, as long as distcc fails only on several packages it's not a big problem, since those are supposed to be recompiled locally. Just make sure compiler versions match. For some issues you might also try pump mode.
For me distcc usually works and it's cool when it works, but better don't get attached to this, as it sometimes fails, and it seems not much can be done about it.

chaoscommander · Post by **chaoscommander** » Wed Dec 18, 2013 1:13 pm

If it just retried failed packages without distcc I'd be happy, do you or does anyone know how to make it do that? Because now it just aborts the entire emerge, meaning I have to sit in front of the computer all the time and manually switch off distcc every few minutes. Quite simply, just not using it sucks as well, because compiling userspace on Intel Atom is no fun. I tried to figure out how to automate that, but no luck yet.

So, if anyone has a suggestion how to make emerge retry failed packages without distcc and then turn it back on, that would be awesome.

As I wrote above, the failure always occurs building, _on localhost_, it can't be a network issue or something about gcc versions.

krinn · Post by **krinn** » Wed Dec 18, 2013 3:00 pm

It looks like you are cross compling using distcc, something many users think it just work because amd64 can build 32bits binaries too.

But you'll get trouble at doing it if you don't build a real cross toolchain.
- amd64 cannot build shared lib without -PIC, so any attempt at cross building for x86 without -PIC will fail
- for that same reason, avoid using pump mode, it's a bad idea to have a 64bits host handling linking phase with its own tools and libraries something for an x86 host.
- you cannot build a program that will be reuse itself to build the final tool : like gcc that build xgcc to build itself. But it's something generally handle in the ebuild that disable distcc, but i suppose not all ebuild are aware of that.
If you want full cross compiling, you cannot just use a 64bits gcc with -m32

And distcc will retry building at localhost automatically ONLY if the error is from distcc :
- distcc cannot reach helper host, so a distcc error : building on localhost
- distcc reach host, but host gcc report an error, so not a distcc error : failure.

You can temporary disable distcc if you want : <FEATURES="-distcc" emerge package> to build problematic package.

chaoscommander · Post by **chaoscommander** » Wed Dec 18, 2013 7:59 pm

krinn wrote:It looks like you are cross compling using distcc, something many users think it just work because amd64 can build 32bits binaries too.

No, I am not cross-compiling. Both machines are amd64 and both run 64bit Gentoo, same gcc version, same architecture (core2).

You can temporary disable distcc if you want : <FEATURES="-distcc" emerge package> to build problematic package.

Hum, that would still require me to do it manually for each and every failing package..

krinn · Post by **krinn** » Wed Dec 18, 2013 9:05 pm

Then you should report failing ebuild so.

And i think it would be more appropriate for you to buildpkg the package you want and use that host as binhost for the atom, if your systems are identical, it would be the fastest way to build your package for the atom.

chaoscommander · Post by **chaoscommander** » Wed Dec 18, 2013 9:41 pm

krinn wrote:Then you should report failing ebuild so.

And i think it would be more appropriate for you to buildpkg the package you want and use that host as binhost for the atom, if your systems are identical, it would be the fastest way to build your package for the atom.

Hmm, never heard of that, definitely need to take a look at it, interesting - thanks!

Post by Hu » Wed Dec 18, 2013 9:56 pm

If you still want to pursue this, please post the output of emerge --info from both machines.

chaoscommander · Post by **chaoscommander** » Thu Dec 19, 2013 5:46 pm

nb: currently I removed all the distcc-relevant options from both make.confs, because I wanted to compile stuff without constant failures.

#1

Code: Select all

Portage 2.2.7 (default/linux/amd64/13.0/desktop/kde, gcc-4.7.3, glibc-2.16.0, 3.10.7-gentoo-r1 x86_64)
=================================================================
System uname: Linux-3.10.7-gentoo-r1_chaosbox64-x86_64-Intel-R-_Core-TM-2_Duo_CPU_P8400_@_2.26GHz-with-gentoo-2.2
KiB Mem:     3974756 total,   1623300 free
KiB Swap:    4192960 total,   3966508 free
Timestamp of tree: Tue, 17 Dec 2013 20:30:01 +0000
ld GNU ld (GNU Binutils) 2.23.1
distcc[29729] (dcc_trace_version) distcc 3.1 x86_64-pc-linux-gnu; built Dec 15 2013 12:14:28 [disabled]
app-shells/bash:          4.2_p45
dev-java/java-config:     2.1.12-r1
dev-lang/python:          2.7.5-r3, 3.2.5-r3, 3.3.2-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.11.6, 1.12.6, 1.13.4
sys-devel/binutils:       2.23.1
sys-devel/gcc:            4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.16.0
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=core2 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
USE="X a52 aac acl acpi alsa amd64 berkdb bluetooth branding bzip2 cairo cdda cdr cli consolekit cracklib crypt css cups cxx dbus declarative dri dts dvd dvdr emboss encode exif fam ffmpeg firefox flac fortran ftp gdbm gif gpm gtk handbook iconv icu ipv6 jpeg kde kipi lastfm lcms ldap libnotify mad mikmod mmx mng modules mozilla mp3 mp4 mpeg mtp mudflap multilib musicbrainz ncurses nls nptl nsplugin offensive ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds qt3support qt4 quicktime raw readline sdl semantic-desktop session spell sse sse2 ssl startup-notification svg symlink tcpd tiff truetype udev udisks unicode upower usb vcd vorbis win32codecs wxwidgets x264 xcb xcomposite xft xinerama xml xscreensaver xv xvid zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="intel vesa fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON

#2

Code: Select all

Portage 2.2.7 (hardened/linux/amd64, gcc-4.7.3, glibc-2.16.0, 3.10.17 x86_64)
=================================================================
System uname: Linux-3.10.17-gentoo x86_64-Intel-R-_Atom-TM-_CPU_330_@_1.60GHz-with-gentoo-2.2
KiB Mem:     1792212 total,     92624 free
KiB Swap:    2097148 total,   2068088 free
Timestamp of tree: Thu, 19 Dec 2013 17:00:01 +0000
ld GNU ld (GNU Binutils) 2.23.1
distcc 3.1 x86_64-pc-linux-gnu [disabled]
app-shells/bash:          4.2_p45
dev-lang/python:          2.7.5-r3, 3.3.2-r2
dev-util/cmake:           2.8.11.2
dev-util/pkgconfig:       0.28
sys-apps/baselayout:      2.2
sys-apps/openrc:          0.12.4
sys-apps/sandbox:         2.6-r1
sys-devel/autoconf:       2.13, 2.69
sys-devel/automake:       1.12.6, 1.13.4
sys-devel/binutils:       2.23.1
sys-devel/gcc:            4.7.3-r1
sys-devel/gcc-config:     1.7.3
sys-devel/libtool:        2.4.2
sys-devel/make:           3.82-r4
sys-kernel/linux-headers: 3.9 (virtual/os-headers)
sys-libs/glibc:           2.16.0
Repositories: gentoo
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="*"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-march=core2 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs candy config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://de-mirror.org/gentoo/"
LANG="de_DE.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY=""
USE="aac acl acpi alsa amd64 berkdb bindist branding bzip2 cdda cdr cli cracklib crypt css cxx dbus dri dvd dvdr ftp gdbm hardened iconv icu ipv6 justify mmx modules mtp mudflap multilib ncurses nls nptl openmp pam pax_kernel pcre readline session sse sse2 ssl symlink tcpd unicode urandom vcd xcb xft zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" LINGUAS="de" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-5" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_3" RUBY_TARGETS="ruby19 ruby18" USERLAND="GNU" VIDEO_CARDS="vesa fbdev" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, SYNC, USE_PYTHON

edit: missed some lines while copying

Post by Hu » Thu Dec 19, 2013 9:15 pm

You are using hardened on one machine and not on the other. That can influence the exact meaning of a simple gcc invocation, which in turn might influence your distcc results. Use the same profile on all machines.

chaoscommander · Post by **chaoscommander** » Sat Dec 21, 2013 10:03 am

Hmm, does that explain why the failures always happen on localhost, even if the remote distcc host is not running?