Growing list of packages failing with distcc

[Ralphred]

Recently the size of my package.env/nodistcc has been growing, without new packages being added to world, but just through updates.
It's got to the point that the most recent plasma/qt updates from 5.12/5.9 to 5.13/5.11 (~70 package updates) added 6 new atoms to the list (6 may not sound like a lot, but a 15% increase is a lot)
.
Most additions arise from somefile.h not found errors (oddly, mostly after demotion to plain mode too), and some are well known to not play well with distcc anyway.
So, the question is, is my list* below reasonable, or should I start posting config files and see where the root cause is?

[eccerr0r]

numpy, mysql and ffmpeg have been behaving for me and I don't recall distcc sourced failures. For me, I believe mythtv, and even llvm and rust also builds properly as well if I remember correctly.

cargo and a lot of rust do not work very well under distcc because they run...rust. Of course, since there is no distrust...

I agree with samba and firefox (which also runs rust) having problems with distcc. I don't run kde, so I don't know what it's like on that side. I do recall having problems with talloc as well.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Ralphred]

The talloc tdb and tevent entries are very recent, they used to work fine.
It's getting to the point that unless you nurse -uDNav updates, it actually takes longer than just letting them compile on a 2 core xeon at 2GHz.

I'm tempted to open a bug on some rust/clang developers git, but my trollgame is not that strong, yet.

I'm also on the verge of setting up /etc/portage/package.env/ as a git repo just to keep this nonsense in check.

I should probably report failures upstream more, but it seems kinda petty when a line in package.env/nodistcc fixes it.

[eccerr0r]

Samba has been on my angry list for quite a while, luckily not all my machines has samba installed. Talloc has also been on and off my angry list for a while now too, never consistently failing, but now it fails consistently.

I'm not sure if talloc, tdb, ldb, and tevent had a recent change but as they appear to likewise waf now, so I'm not surprised (I should write that I'm not surprised these four use waf, since they are related to the samba project).

BTW - are these with distcc or distcc-pump?
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Ralphred]

[eccerr0r]

Okay, no problems, just curious, as I've had strange problems with pump mode too, and want to make sure everyone's experiences are one way or the other, or at least indicate so... One of the main reasons I've had so many problems is the diverse computing environment as each machine of mine has a specific purpose instead of being generic - which kills pump mode apparently.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[yuyuyak]

I too have a growing list on several machines and architectures here of packages failing with distcc. I've been using distcc for > 5 years, never anything like this before. About a month or so ago most machines fail building the kernel and make oldconfig fails too unless I remove /usr/lib/distcc/bin from the path. Interestingly we have a skylake laptop here and it seems totally immune, distcc working normally for it. Also an armv7a device which doesn't have any problem. All of the core2 machines do though.
I have always found it amazing that samba, in my experience, has never compiled with distcc, aren't the samba devs the same ones who support distcc? As mentioned above now all the samba "friends" can't use distcc either.
For clarity I should mention I'm using Funtoo with gcc-7.3.1-r5. I wonder if the problem is with gcc or rather distcc + gcc > 7. A couple of days ago I updated gcc to r5 from the r3 version, the problem is worse. The r5 version uses -fPIE, one failure I saw yesterday complained about recompiling with -fPIC.
I don't have a clue, just tossing it out there. Oh, and I am using pump mostly.

[krinn]

[eccerr0r]

In this case, it almost seems there's a portage bug in using distcc ... doesn't make sense why waf should break with portage/distcc...

Time to investigate...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[artbx]

Strange a bit that they fails for you, because i use clang on arm64 with distcc and distcc server with intel core i7
and most except few paackages im able to compile using distcc
for example thise build without problems

kde-plasma/kde-cli-tools
kde-plasma/kscreenlocker
kde-plasma/kwin

on arm64 cortex-a72

[yuyuyak]

In my case it turned out to be the Skylake machine was the culprit (the one that wasn't failing ). The fastest, so of course it's the 1st entry in /etc/distcc/hosts on all the machines. The gcc on that machine was emerged without the pie use flag. All the others had pie. I apologize for the sidetrack.

[Maitreya]

[artbx]

May I ask You how with package env You disable distcc ?
The nodistcc in env ...

When i put FEATURES there without distcc it is ignored and global FETURES are used ..

[yuyuyak]

I have:

[yuyuyak]

@Maitreya:
A list of non-working packages will never work because sometimes it will work on one machine and not another, sometimes a package that used to work now doesn't and vice-versa, etc.
There is only one constant I would point out (and maybe it's just me):

[Maitreya]

[eccerr0r]

I was able to remove nodistcc from firefox. If I redo the LANG environment variable in the helper distccd machines, firefox now builds (of course the rust components cannot be distributed and are not affected by the new ability to distribute the C/C++ portions of firefox.)

qlop -g firefox results: (Core-i7 as builder, Core2Quad as helper)

Before:
firefox: Thu Nov 1 15:33:48 2018: 2648 seconds

After:
firefox: Mon Nov 19 02:03:06 2018: 1503 seconds

I also verified that my VM x86 box now is able to compile firefox with FEATURES=distcc where it had been failing before the hack.

==========

WORKAROUND CHEAT SHEET (Note this ONLY helps firefox and likely thunderbird, USE AT YOUR OWN RISK - MAY BREAK OTHER PACKAGES, though so far it looks okay for me - no other packages get broken!):
For systemd distccd boxes:
edit /etc/systemd/system/distccd.service.d/00gentoo.conf
add

[Ralphred]

At the risk of thread necromancy*, WTF guys, tar and wget get added to the list of programs that fail to pass the ./configure stage with distcc enabled?

Now, whilst experience has taught me that the first thing to do when encountering failures like this is to "nodistcc" them, and just except the consequences when it works, really, two programs so entwined in the usability of Linux as a whole, two programs that I would consider part of "@system" for even the most frugal of installs failing in this way?

As much as I am disinclined to engage in hyperbole etc. how long is it before we start using compatibility white lists for distcc instead of incompatibility blacklists?
As always, if my conclusions and therefore config files are unique, please let me know how so....

*IMHO a thread such as this has no redundant information, just growth in the face of a new environment, so again if you disagree post how/why.

[eccerr0r]

Don't care either.

But something's really wrong... thought wget and tar distcc file last time I checked.

However I found that dev-qt/qtmultimedia-5.12.3 fails distcc as well, this one does not pass the architecture to the remote workers during configure too ... The result of a mixed distcc environment of amd64 and x86 machines, and my big iron machines building both...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[eccerr0r]

More thread necromancy:

My nodistcc list has shrunk ... to nothing once more! Samba distcc's just fine once more.

Seems that either things cleaned up or the Gentoo devs cleaned up a lot of the potential distcc issues. So those disabling distcc, try it once more...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Massimo B.]

If there are still failing packages with distcc or distcc-pump, please report at least to https://wiki.gentoo.org/wiki/Talk:Distcc

I once again started with distcc, followed the wiki. Is pump mode still discouraged to use?
_________________
HP ZBook Power 15.6" G8 i7-11800H|HP EliteDesk 800G1 i7-4790|HP Compaq Pro 6300 i7-3770

[eccerr0r]

For the longest while, samba was distcc'ing fine for me... However it started failing again.

Had to stop distcc on ldb, sometimes tevent, and samba itself. Seems it's very non-deterministic or depends on some secret condition...
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[Phoenix591]

samba has been working fine for me with distcc. Currently at 4.15.5-r1 for me. with USE: acl, client, pam, python 3.9, regedit, and system-mitkrb5 and systemd.

I'm even cross-distccing from my amd64 desktop to my arm/arm64 ( just switched it back to an arm64 install yesterday).

I'm not using pump mode, but just lzo to compress things

[eccerr0r]

Yes, seems something nondeterministic, sometimes it works - and sometimes it doesn't, depending on environment... just don't know what in the environment that causes something to go out of order... sigh. My x86 was using amd64 helpers in this case, but had some other amd64 helping amd64 fail too.
_________________
Intel Core i7 2700K/Radeon R7 250/24GB DDR3/256GB SSD
What am I supposed watching?

[logrusx]

Pump mode is not recommended, if you're still using it, try without it.

Regards,
Georgi