Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
please help troubleshoot distcc config?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
endtransmission
n00b
n00b


Joined: 16 Nov 2003
Posts: 48

PostPosted: Tue Feb 17, 2004 11:11 pm    Post subject: please help troubleshoot distcc config? Reply with quote

I seem to continue to have problems - or it feels like I do.

The farm.
My gentoo box = the client (slow box)
My helper box = the host (fast box - booted into knoppix 3.3 from CD)

My client is running a first generation k7 Athlon 700mhz "Socket A" chip on an old asus k7m board. Thus the term (slow box)

My host is generously loaned to me by my roomie who has windows XP but for the purposes of this, is booted into Knoppix 3.3. We're both using the same gcc version 3.3.2. Ergo the term (fast box)

Perhaps this isn't the forum to ask this question, but in the event it's something silly, I thought I'd ask it here first.

While it looks like the host is doing work, it appears to be doing only preprocessing work while my client still slogs through the heavy compiling - wish the reverse could be true - where slow client does the preprocessing work and the host do the serious crunching.

I've done everything I can think of to try to work the host harder but to no avail. Its a 1.4Ghz Pentium 4 and it sits on 95% idle for the most part even though the gnome monitor shows it doing preprocessing work. It appears to do almost no compiling.

I've even gone so far as to take localhost out of the list of hosts in an effort to see an improvement - nada. i've tried booting up with different -jN variables (both in the host daemon and my own make.conf file), i've tried adjusting the number of jobs in the host config (ip 192.168.1.101/4 etc...), have tried switching the order of the hosts in the host config file... zilch.

Have been compiling Gnome for the past 24 hours (not unexpectedly) but this post comes after futzing with it for the past 72 trying a series of different items to compile.

edit: After having decided that compile times are actually taking longer using the host to "help" than without it, I have taken the host computer (192.168.1.101) out of the list of hosts leaving only "localhost". Things are compiling fine that way - obviously this is a little self defeating but it was a quick way to take the other computer out of the farm and still keep distcc up and clicking - silly but i like to look at the monitors and doing it this way allows me to. Incidentally when doing this the monitor shows my client as doing NO preprocessing work. It's all compiling "green-bars".

My two hypothesis's for this currently are

1. I need a cross-compiler - (hers being a Pentium 4 chip on a ***gak*** Dell and mine being an old AMD chip). I have no idea what a cross-compiler is precisely or how to get one but theres enough documentation I'm not totally grasping for me to think that I'm missing something in this arena.

or

2. The purple bars indicating preprocessing work actually includes network time/activity - that distcc is spending ALOT of effort to send a little bit of data, which, once the 1.4Ghz Dell recieves it, cruches through it like a spoon full of sugarpops quick quick - and then spends alot of time sending it back thus defeating the purpose of using her box to help as it's taking more time to transfer data then it is to compile it. Now, this shouldn't be the case as we're both connected to the same router and I was pretty sure I had it configured properly, but I'm new enough to not want to rule it out.


Any help would be welcome. Thanks guys. Sorry for the lengthy post.


Last edited by endtransmission on Wed Feb 18, 2004 12:16 am; edited 2 times in total
Back to top
View user's profile Send private message
adaptr
Watchman
Watchman


Joined: 06 Oct 2002
Posts: 6730
Location: Rotterdam, Netherlands

PostPosted: Tue Feb 17, 2004 11:30 pm    Post subject: Reply with quote

No, you don't need a cross-compiler - whether the resultant code is for a P-4 or an Athlon has nothing to do with it.
The platform, or processor architecture, is still Intel x86 throughout.

What you do have to be certain of is that the distcc versions are also relatively the same, and that networking is up and working correctly - the transfer problems you are having may not be bandwidth related at all.

When distcc is not compiling it is mostly doing nothing, so I'm not sure what you mean here - unless you mean preprocessing, which is never done by the hosts: they just run the pure gcc compiler since they don't necessarily have the header files needed to do the preprocessing.

On my P-3 733 / Tualatin 1133 combo absolutely everything is distributed as far as possible - that already starts as soon as the tests compiled by the configure script, which checks for certain functionality of the compiler.
Still, it is the compiler that decides whether a task can be run in parallel, not you.
Some things will never be built in parallel, either for stability or because it simply isn't possible - notable examples are the kernel and XFree86.

In my case, all I needed to do to get it up and running was start the distccd daemon on my Debian server, and set the number of threads on that box to 2.
Then I upped the total number of threads (in make.conf) to 5, and just let it do its thing - which it did perfectly, albeit not always logically...

My distccd.conf lists the host first, then localhost.

The most remarkable result (for me, anyway) was that many sizeable builds will be shunted to the host exclusively, since it apparently figured out that that machine is almost twice as fast as the localhost!

In these cases it is doing exactly what you mean - preprocessing is done on the slow box, while the fast box handles the raw crunching.
Note again that this is by no means a scientific fact - in totally unpredictable circumstances (unpredicatble by me, that is) it will not do this, or even reverse this logic!
_________________
>>> emerge (3 of 7) mcse/70-293 to /
Essential tools: gentoolkit eix profuse screen
Back to top
View user's profile Send private message
endtransmission
n00b
n00b


Joined: 16 Nov 2003
Posts: 48

PostPosted: Tue Feb 17, 2004 11:45 pm    Post subject: Reply with quote

adaptr wrote:
No, you don't need a cross-compiler - whether the resultant code is for a P-4 or an Athlon has nothing to do with it.
The platform, or processor architecture, is still Intel x86 throughout.

What you do have to be certain of is that the distcc versions are also relatively the same, and that networking is up and working correctly - the transfer problems you are having may not be bandwidth related at all.

When distcc is not compiling it is mostly doing nothing, so I'm not sure what you mean here - unless you mean preprocessing, which is never done by the hosts: they just run the pure gcc compiler since they don't necessarily have the header files needed to do the preprocessing.

On my P-3 733 / Tualatin 1133 combo absolutely everything is distributed as far as possible - that already starts as soon as the tests compiled by the configure script, which checks for certain functionality of the compiler.
Still, it is the compiler that decides whether a task can be run in parallel, not you.
Some things will never be built in parallel, either for stability or because it simply isn't possible - notable examples are the kernel and XFree86.

In my case, all I needed to do to get it up and running was start the distccd daemon on my Debian server, and set the number of threads on that box to 2.
Then I upped the total number of threads (in make.conf) to 5, and just let it do its thing - which it did perfectly, albeit not always logically...

My distccd.conf lists the host first, then localhost.

The most remarkable result (for me, anyway) was that many sizeable builds will be shunted to the host exclusively, since it apparently figured out that that machine is almost twice as fast as the localhost!

In these cases it is doing exactly what you mean - preprocessing is done on the slow box, while the fast box handles the raw crunching.
Note again that this is by no means a scientific fact - in totally unpredictable circumstances (unpredicatble by me, that is) it will not do this, or even reverse this logic!


Preprocessing. That was the term I was looking for. Thank you.

The monitor shows my hosts doing all the preprocessing and my client aka the slow box, my main box, my gentoo box as doing all the compiling. So for me, it's precisely the opposite and if what you say is true, my Hosts shouldn't be doing any of the preprocessing, just the compiling, yes? My gentoo box (the client) should be doing all the preprocessing.

Well thats my problem. Whats do you think a solution is?

PS I think it's important that everyone who wants to contribute to this thread is savvy to the fact that when dealing with distcc, the client is the main gentoo box and the helper boxes are the hosts - so; backwards from how we usually refer to client and hosts. ie one client can have several hosts helping crunch data. As long as everyones up to speed on this, then fine. Just thought I'd mention something about it early on to make sure everyones on the same page.
Back to top
View user's profile Send private message
adaptr
Watchman
Watchman


Joined: 06 Oct 2002
Posts: 6730
Location: Rotterdam, Netherlands

PostPosted: Tue Feb 17, 2004 11:52 pm    Post subject: Reply with quote

Hmm I'm afraid I answered in haste.
Of course, if the distcc client can send .c files to the host, it can also send header files - so the preprocessing can just as well be done remotely :oops:

As for a solution - who knows ?
You can for a start peruse th distccd logfiles (on the host) - this logs every file processed or compiled, and every network transfer as well, with timestamps, so you can see how long everything took.

Maybe examining those logs (on the gentoo box as well!) will give you a clue or two...
_________________
>>> emerge (3 of 7) mcse/70-293 to /
Essential tools: gentoolkit eix profuse screen
Back to top
View user's profile Send private message
endtransmission
n00b
n00b


Joined: 16 Nov 2003
Posts: 48

PostPosted: Wed Feb 18, 2004 12:04 am    Post subject: Reply with quote

adaptr wrote:
Hmm I'm afraid I answered in haste.
Of course, if the distcc client can send .c files to the host, it can also send header files - so the preprocessing can just as well be done remotely :oops:

As for a solution - who knows ?
You can for a start peruse th distccd logfiles (on the host) - this logs every file processed or compiled, and every network transfer as well, with timestamps, so you can see how long everything took.

Maybe examining those logs (on the gentoo box as well!) will give you a clue or two...


Ah, okay. Well scouring the logs don't seem to do me much good as they simply indicate "preprocessing". Is this something for bugzilla? It doesn't feel like it but I've never submitted anything to bugzilla before, and again I'm not totally convinced that I am not at fault here in my configuration.
Back to top
View user's profile Send private message
Shar
n00b
n00b


Joined: 07 Jan 2004
Posts: 4
Location: Escondido, CA USA

PostPosted: Mon Feb 23, 2004 10:11 pm    Post subject: distcc will do either job on either machine. Reply with quote

Over the weekend I re-installed my slow machine system and set up distcc for it. Had a few issues with my fast machine so booted it with the knoppix cd following the ] PAINLESS distcc setup instructions and the DISTCC guide and voilla. While it was crunching away, I got freaked out :? since it looked like JUST the pre-processing was happening on the fast machine.

But then I played with the distccmon-text {seconds} and noticed that a lot of jobs were being sent back and forth AND they were both pre and compile jobs. I also noticed that it worked VERY well. :D

Might also note that the slow machine had the stock live-cd gcc, glibc installed, but I upgraded the distcc to the latest before I started. The knoppix cd I used was the latest (2004-02-16 ?) that I downloaded just the evening before. If you are having problems, you might want to check your versions at least up to date with those.
_________________
Linux user since 11/2001 # 345056
Gentoo user since 12/2003
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum