| View previous topic :: View next topic |
| Author |
Message |
seansmr
n00b
Joined: 04 Dec 2011
Posts: 52
Location: Bangkok
|
 Posted: Sun Dec 04, 2011 8:17 am Post subject: Distcc Pump: Can't get pump mode working, distcc works great |
 |
|
My first question!
I have been using Gentoo as a LAMP server for years and recently as a desktop. It is a great system, I love it. Now I am trying to get to know it better.
I have read a lot and tried a lot, but I just can't get distcc pump mode working. Some solutions reported in the forums do not produce the same results for me. There are 3 main ways that I have tried:
1) Set ",cpp" in the conf.d/hosts file or set the variable DISTCC_HOSTS in make.conf. When I try to use pump --startup or pump emerge, either one gets me the error: | Code: | | /usr/bin/pump: error: pump mode requested, but distcc hosts list does not contain any hosts with ',cpp' option |
2) Tried at the command line: | Code: | | DISTCC_POTENTIAL_HOSTS="192.168.2.4,cpp 192.168.2.7,cpp" pump --startup |
This seems to produce ok output, below, but does not do anything different than normal distcc.
| Code: |
export INCLUDE_SERVER_PID='2042'
export INCLUDE_SERVER_DIR='/tmp/distcc-pump.flghAS'
export INCLUDE_SERVER_PORT='/tmp/distcc-pump.flghAS/socket'
export DISTCC_HOSTS='--randomize '
export PATH='/usr/lib/distcc/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gcc-bin/4.5.3'
|
3) Also, tried at the command line:
| Code: | | DISTCC_POTENTIAL_HOSTS="192.168.2.4,cpp 192.168.2.7,cpp" pump emerge screen -pv |
This give the result below:
| Code: | __________Using distcc-pump from /usr/bin
__________Found 0 available distcc servers
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] app-misc/screen-4.0.3-r4 USE="pam -debug -multiuser -nethack (-selinux)" 0 kB
Total: 1 package (1 reinstall), Size of downloads: 0 kB
__________Shutting down distcc-pump include server
|
Normal distcc works great for me. Also, I haven't seen anything in the logs when I try to use pump mode.
Using DISTCC_HOSTS does not work:
| Code: | | DISTCC_HOSTS="192.168.2.4,cpp 192.168.2.7,cpp" pump emerge screen -pv |
Produces this error:
| Code: | distcc[3916] (dcc_get_protover_from_features) ERROR: pump mode (',cpp') requires compression (',lzo')
distcc[3916] (dcc_parse_options) ERROR: invalid host options: ,cpp 192.168.2.7,cpp
distcc[3916] (dcc_show_hosts) CRITICAL! Failed to get host list
/usr/bin/pump: error: pump mode requested, but distcc hosts list does not contain any hosts with ',cpp' option
|
OK, so then I tried this:
| Code: | | DISTCC_HOSTS="192.168.2.4,cpp,lzo 192.168.2.7,cpp,lzo" pump emerge screen -pv |
And it seems to work:
| Code: | __________Using distcc-pump from /usr/bin
__________Using 2 distcc servers in pump mode
These are the packages that would be merged, in order:
Calculating dependencies... done!
[ebuild R ] app-misc/screen-4.0.3-r4 USE="pam -debug -multiuser -nethack (-selinux)" 0 kB
Total: 1 package (1 reinstall), Size of downloads: 0 kB
__________Shutting down distcc-pump include server
|
However, adding ",lzo" to the hosts file or make.conf does not work. I must be missing something obvious, or I completely misunderstand the whole pump configuration. How can I get pump to take the ",cpp" option from the hosts file or the variable in make.conf?
Also, I do not want compression, is it possible to get pump working without ,lzo?
And, I haven't seen anything in the documentation, is there any setup for pump required on the remote host?
I hope that this is understandable, let me know if you need more information. |
|
| Back to top |
|
 |
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Dec 04, 2011 10:19 pm Post subject: |
|
|
Well, it's just adding lzo,cpp to your hosts list.
and if you don't use DISTCC_POTENTIAL_HOSTS, it will default to find them in /etc/distcc/hosts & DISTCC_HOSTSnano
And don't ask me why cpp doesn't simply imply lzo, but you cannot run cpp without lzo, so both are needed for pump.
And you can ask distcc to show you if your hosts are configure fine using distcc --show-hosts
| Code: | distcc --show-hosts
@beleg/8,lzo,cpp
@hurin/4,lzo,cpp
localhost/2,lzo,cpp
|
on running you might also check the hosts are handle, but except informative, don't take it as a proof the host is valid, just one the host will be tried.
| Code: | __________Using distcc-pump from /usr/bin
__________Using 3 distcc servers in pump mode
|
and on the running host, ls /tmp/distcc-pump* |
|
| Back to top |
|
|
seansmr
n00b
Joined: 04 Dec 2011
Posts: 52
Location: Bangkok
|
| Posted: Wed Dec 07, 2011 3:42 am Post subject: |
|
|
Thanks for your help. I have added ,lzo to the hosts file and it seems to start ok now.
I can't really tell if it is using pump mode or not. I have checked the logs, but not the monitor.
One more problem, I can't get a separate log file, it still logs to syslog.
I have tried this:
| Code: | | DISTCCD_OPTS="${DISTCCD_OPTS} --log-file /var/log/distcc.log" |
|
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed Dec 07, 2011 4:38 pm Post subject: |
|
|
Try gave it a level too.
| Code: | | DISTCCD_OPTS="${DISTCCD_OPTS} --log-level info --log-file /var/log/distcc.log" |
And i don't remember if it need a valid file or not, anyway it won't kill you to touch it in case it's a need. |
|
| Back to top |
|
|
seansmr
n00b
Joined: 04 Dec 2011
Posts: 52
Location: Bangkok
|
| Posted: Thu Dec 08, 2011 5:56 am Post subject: Maybe pump is working |
|
|
I tried log level info, with and without an existing file. Neither worked.
I now have some messages about pump mode. I tried it with today's upgrade, scite-2.28
| Code: | | pump emerge --oneshot scite |
This gives me this result:
| Code: | * GNU info directory index is up-to-date.
__________Warning: 4 pump-mode compilation(s) failed on server, but succeeded locally.
__________Distcc-pump was demoted to plain mode. See the Distcc Discrepancy Symptoms section in the include_server(1) man page.
__________Shutting down distcc-pump include server
|
I did read the man page, not something that I know about.
Distccmon shows this:
| Code: | 16030 Compile ContractionState.cxx 192.168.2.7[0]
16046 Preprocess localhost[0]
16078 Compile RunStyles.cxx 192.168.2.7[0]
16129 Compile LexPS.cxx 192.168.2.7[0]
16142 Compile LexHTML.cxx 192.168.2.7[0]
16162 Compile LexBash.cxx 192.168.2.7[0]
16190 Compile LexTxt2tags.cxx 192.168.2.7[0]
16214 Compile LexD.cxx 192.168.2.7[0]
16238 Compile LexModula.cxx 192.168.2.7[0]
16258 Compile LexCLW.cxx 192.168.2.7[0]
16283 Compile LexAU3.cxx 192.168.2.7[0]
16311 Compile LexMMIXAL.cxx 192.168.2.7[0]
16348 Compile LexErlang.cxx 192.168.2.7[0]
16372 Compile LexScriptol.cxx 192.168.2.7[0]
16396 Compile LexPerl.cxx 192.168.2.7[0]
16416 Compile LexOthers.cxx 192.168.2.7[0]
16436 Compile LexPascal.cxx 192.168.2.7[0]
16480 Blocked [0]
16478 Blocked [0]
16455 Compile Widget.cxx 192.168.2.7[0]
16454 Compile GUIGTK.cxx 192.168.2.7[2]
16453 Compile SciTEGTK.cxx localhost[1]
16457 Compile SciTEBase.cxx localhost[2]
16480 Blocked [0]
16481 Blocked [0]
16482 Blocked [0]
16478 Compile SciTEIO.cxx 192.168.2.7[0]
16498 Compile SciTEProps.cxx 192.168.2.7[1] |
However, each time I run it, distccmon has a different output.
One time I go this:
| Code: | ... similar lines left out...
11403 Compile LexLout.cxx 192.168.2.7[0]
11427 Compile LexFortran.cxx 192.168.2.7[0]
11479 Blocked [0]
11471 Preprocess GUIGTK.cxx 192.168.2.4[1]
11473 Preprocess FilePath.cxx 192.168.2.4[3]
11476 Preprocess SciTEBase.cxx 192.168.2.4[4]
11470 Preprocess SciTEGTK.cxx 192.168.2.7[0]
11472 Preprocess Widget.cxx 192.168.2.7[2]
11479 Blocked [0]
11525 Preprocess DirectorExtension.cxx 192.168.2.4[0]
11528 Preprocess SciTEProps.cxx 192.168.2.4[1]
11476 Compile SciTEBase.cxx 192.168.2.4[4]
11516 Compile PropSetFile.cxx 192.168.2.7[0]
11470 Compile SciTEGTK.cxx 192.168.2.7[0]
|
It appears that it worked correctly, at least sometimes. Is this the expected output for a properly working pump mode?
Thanks for all of your help. |
|
| Back to top |
|
|
seansmr
n00b
Joined: 04 Dec 2011
Posts: 52
Location: Bangkok
|
| Posted: Fri Dec 09, 2011 6:50 am Post subject: |
|
|
I have tried distcc pump again, this time with firefox-8.0. Here is what happened
My hosts: | Code: | 192.168.2.7/3,cpp,lzo
192.168.2.4/4,cpp,lzo |
The distcc monitor outputs in batches like code below. The first 10 or so batched were like this:
| Code: |
23872 Blocked [0]
23870 Blocked [0]
23861 Blocked [0]
23868 Blocked [0]
23902 Compile jsfriendapi.cpp 192.168.2.7[0]
23888 Compile jsemit.cpp 192.168.2.7[0]
23833 Compile jsapi.cpp 192.168.2.7[2]
|
At this point nothing was done on remote the server.
After this, I got activity on the server and the distcc monitor showed like this:
| Code: |
24422 Preprocess ExecutableAllocatorWin.cpp 192.168.2.4[0]
24341 Compile FastOps.cpp 192.168.2.4[1]
24417 Compile ExecutableAllocatorPosix.cpp 192.168.2.4[1]
24349 Compile MonoIC.cpp 192.168.2.4[3]
24338 Compile FastArithmetic.cpp 192.168.2.7[0]
24379 Compile TrampolineCompiler.cpp 192.168.2.7[0]
24352 Compile PolyIC.cpp 192.168.2.7[4]
|
Then for a few minutes there was no output from distccmon. Emerge continued working, but nothing happened on the server. After this blank period, it continued as in the above code black for the rest of the emerge. The local machine was usually at 100% cpu use and the remote was 60% - 90% About 10% of the status lines were Preprocess, the rest were Compile.
At the end of the, successful, emerge I go this message:
| Code: | __________Warning: 4 pump-mode compilation(s) failed on server, but succeeded locally.
__________Distcc-pump was demoted to plain mode. See the Distcc Discrepancy Symptoms section in the include_server(1) man page.
__________Shutting down distcc-pump include server
|
So, what does this output mean?
"Distcc-pump was demoted to plain mode." Was distcc running in plain mode for the rest of the emerge? What about the status lines that show Preprocess.
Does the fallback only apply to one job or to the rest of the emerge?
The question is: Is pump mode working correctly or does it need to be fixed?
I also tried it with DISTCC_FALLBACK="1"
It didn't stop and there was no output from the server. |
|
| Back to top |
|
|
Goverp
Veteran
Joined: 07 Mar 2007
Posts: 1999
|
| Posted: Fri Dec 09, 2011 9:19 am Post subject: |
|
|
This is probably either incorrect or irrelevant, but I suspect distcc/pump reverts to local mode if there's a transient network error. I've noticed that I get long compilations (the ones that I want distcc to handle!) reverting to local if I use my netbook too far from my desktop distcc server. My hypothesis is that transient WiFi problems cause distcc to mark the connection as broken.
I see KDE 4.7.3 is about. When I summon up the courage to try it, I'll use a wire connection to my router rather than WiFi and see if that works better.
_________________
Greybeard |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Fri Dec 09, 2011 2:09 pm Post subject: |
|
|
Pump or not, distcc always default to localhost on a compile failure.
And because pump need to upload to the server the headers and everything to let the linking goes, the preprocess is of course slower than a non-pump running distcc.
It's again, upto you to finetune your farm and decide if it's good or not to let another computer handle the linking for another.
It depend on network/cpu/hdd speed of the slave and its master, and of course the ebuild in use. |
|
| Back to top |
|
|
seansmr
n00b
Joined: 04 Dec 2011
Posts: 52
Location: Bangkok
|
| Posted: Mon Dec 12, 2011 6:54 am Post subject: |
|
|
Thanks for your responses to my questions.
Earlier I had seen promises of 3X or more increase in speed when used with pump mode. However, now I have just found a readme, in with the distcc source code. It says that the greatly increased speed only applies to networks with 100 CPUs. It says that one computer can keep up to 10 pcs busy without pump mode.
I am just going to add in one more fast computer and not worry about perfecting pump mode.
Since a few compilations must be done on localhost, it does not appear suitable for a very slow PC. I have Gentoo installed on one of these: http://www.norhtec.com/products/mcjrdx |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
