| View previous topic :: View next topic |
| Author |
Message |
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
 Posted: Sat Aug 01, 2015 12:10 pm Post subject: crossdev with distcc speed problems |
 |
|
I 'm experimenting with an AMD Athlon XP1700+(Gentoo) and a 2core Toshiba (Gentoo)
I am trying to compile the kernel of the Athlon with the help of the Toshiba.
I can see distcc appearing in top of the Toshiba, but why is the cpu load so low(Toshiba).
I want the Toshiba go to 100% for compiling the Athlon kernel. Now I see only peaks of 40% max.
Is my configuration wrong? What can I do.. I'm already using pump. |
|
| Back to top |
|
 |
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Aug 01, 2015 1:02 pm Post subject: |
|
|
skorefish,
distcc cowes with a monitor whicd you need to run on the AMD Athlon XP1700+ after you tave invoked a build using distcc.
It tells what in being built where.
The kernel build is special. It does not use the distcc as defined in make.conf. Rather you need a wrapper to make or you need te invoxe distcc yourself.
What do you have in /etc/distcc/hosts on the Athlon?
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Sat Aug 01, 2015 1:34 pm Post subject: |
|
|
hi Neddy,
| Quote: | | What do you have in /etc/distcc/hosts on the Athlon |
| Code: | | 192.168.2.4,lzo,cpp |
and I type
| Code: |
export DISTCC_HOSTS="192.168.2.4,lzo,cpp localhost"
pump make CC="distcc" -j100
|
| Quote: | | distcc comes with a monitor whicd |
you mean distccmon-text 10 |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sat Aug 01, 2015 3:12 pm Post subject: |
|
|
skorefish,
thats it. If it shows nothing you need to tell it the DISTCCDIR,
| /etc/distcc/hosts: | | 192.168.2.4,lzo,cpp |
only allows four jobs to be sent to the helper.
See the /LIMIT option in man distcc.
| Code: | | pump make CC="distcc" -j100 |
I'm not sure if the order of parameters is importont here.
The -j100 is for make and CC="distcc" sets the environment.
As I've already said, only four jobs will be sent to the helper concurrently.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Sun Aug 02, 2015 12:19 pm Post subject: |
|
|
It's not because you allow 100 jobs to be run that 100 will be run, if any job depend on other jobs to finish, they must wait them to finish first, making the total running jobs limit to X (the ones without dep) than 1 (the one that now have his dep built).
Remember also that a one job busying a 1 core cpu, would just made a 2 cores cpu busy for 50% |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Sun Aug 02, 2015 12:26 pm Post subject: |
|
|
krinn,
The kernel will use a huge number of parallel makes, as will glibc. However what you say is generally correct.
Don't try either of these with just make -j Thats an unlimited number of maxe jobs.
skorefish,
-j100 is excessive. The total number of cores+1 is still a good rule of thumb for a distcc build farm.
You count all the cores in the build farm and set /LIMIT on each helper.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Mon Aug 03, 2015 8:16 am Post subject: |
|
|
Hi Neddy,
I use the following commands now on the Athlon
| Code: | export DISTCC_HOSTS="192.168.2.4,lzo,cpp/LIMIT 20"
pump make -j4 CC="distcc"
|
I've seen up to 8 distcc's in 2core top screen. It peaks at 40-60% cpu now. but sometimes idle.
Seems the Athlon should distribute faster. How can this be done?
Sometimes I see this error
| Code: | distcc[15547] ERROR: failed to connect to 0.0.0.20:3632: Invalid argument
distcc[15547] Warning: failed to distribute fs/jbd2/commit.c to 20, running locally instead
|
2core ip: 192.168.2.4
Athlon ip 192.168.2.20 |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Mon Aug 03, 2015 11:11 am Post subject: |
|
|
It mean you mistake the format and that you didn't do like NeddySeagoon told you to do : | NeddySeagoon wrote: | | See the /LIMIT option in man distcc. |
Because of that, distcc think 20 is an hostname and try to distribute a job to 20, that of course fail.
| Code: | | DISTCC_HOSTS="192.168.2.4/20,lzo,cpp |
Note that setting a limit to 20 on that host and using -j4 will limit it to 4 anyway.
What you have done good? Using -j4 you limit your make to 4 jobs only (and not that silly -j100), and as it is the default for any host distribution, it try to distribute the 4 jobs primary to your 192.168.2.4 host, avoiding to distribute some to localhost, making it work better so.
If your host is a bit slow at processing new jobs and distribute them, increasing jobs could help, it is no more the same trick as the cpu+1 (you overfeed your cpu, to make sure it has always something to do), this mean try using -j5, so you will have 5 running jobs, if one end, while it is processing and sending the new one, you still have 4 running, keeping the cpu busy. And unlike the cpu+1 trick, the gap between two jobs is not only the waiting time for your hdd to read the datas to process, but also waiting time to distribute it on your network, so even with +x the gap will always still be there, and your helper will rarelly be 100% busy.
From experience, i could tell you, the correct -j is between quickest helper number of cores to total number of cores of all helper * 2, and its real value depend on your configuration: helpers number & power and specially "who" is asking help. |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Wed Aug 05, 2015 1:09 pm Post subject: |
|
|
yes, this is it !! thx I see the cpu peak at 100% now.
I plugged in another computer but it doesn't get a lot of traffic. should I use the random function or is there another limitation?
I 'm talking about emerge...
How can one use --randomize for emerge ?
I try this...
| Code: | /usr/bin/distcc-config --randomize --set-hosts "192.168.2.4/20 192.168.2.50/20 192.168.2.20/1"
!!! Error: --randomize is an invalid option.
|
| Code: | /usr/bin/distcc-config --set-hosts DISTCC_HOSTS "--randomize 192.168.2.4/20 192.168.2.50/20 192.168.2.20/1"
!!! Error: --randomize 192.168.2.4/20 192.168.2.50/20 192.168.2.20/1 is an invalid option.
|
How is this done ? |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed Aug 05, 2015 2:11 pm Post subject: |
|
|
| just edit the file, not all options may be support in distcc-config, that is a poor wrapper. |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Wed Aug 05, 2015 3:22 pm Post subject: |
|
|
krinn,
| Quote: | | just edit the file |
What file do I need to edit ? vi /etc/conf.d/distccd ? |
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Wed Aug 05, 2015 3:41 pm Post subject: |
|
|
| /etc/distcc/hosts |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Wed Aug 05, 2015 4:06 pm Post subject: |
|
|
is this legal ?
| Code: | | --randomize 192.168.2.4/20,lzo,cpp 192.168.2.50/20,lzo,cpp 192.168.2.20/1,lzo,cpp |
to put in /etc/distcc/hosts |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Wed Aug 05, 2015 5:59 pm Post subject: |
|
|
skorefish,
Where is your sense of adventure?
distcc will tell you and you can tell everyone else.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
skorefish
Apprentice
Joined: 21 Jun 2015
Posts: 285
|
| Posted: Wed Aug 05, 2015 6:36 pm Post subject: |
|
|
| Quote: | Where is your sense of adventure?
|
Well, I say it works for 192.168.2.4 (it works around 90% with is very good)
but 192.168.2.50 gets very small loads
I 'm still searching what's causing that !!! maybe it has something to do with crossdev, ...??? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54237
Location: 56N 3W
|
| Posted: Wed Aug 05, 2015 8:40 pm Post subject: |
|
|
skorefish,
Lets start from the beginning.
You have a client host, that wants things built and you have one or more helpers.
The helpers only need to be set up to listen for help. Their make.conf plays no part in the number of concurrent jobs they will execute.
On the client, for emerge, set MAKEOPTS= to the number of concurrent jobs the helpers can process, plus 1.
For the kernel, use make -jX where X is the same number.
On the client in /etc/distcc/hosts set /LIMIT on a per helper basis to the number of jobs that that host will be allowed to process concurrently.
distcc is not a magic bullet. The client has to unpack the tarballs, send work to helpers, get the results back, do the linking and installing.
In pump mode, compression is used too. That ods to the client workload.
For some clients and networks, the client may not be able to keep helpers busy. Thats expected
Further, some jobs, like gcc cannot be distributed.
I use -j8 on a 700MHz Raspberry Pi B, with 256Mb RAM, (its an early one). It can get 8 jobs running on my AMD64 helper but equally, there are times when its doing things that cannot be distributed and the helper is idle.
The Pi only has a 100Mbit ethernet.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
|
Gentoo Chat
