Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Distcc, zeroconf and Xcompilation question
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Sun Feb 01, 2015 5:56 pm    Post subject: Distcc, zeroconf and Xcompilation question Reply with quote

Hello,

It seems to me, distcc exposes only its native compiler : is it any way to expose all compiler present on the machine ?

Thanks

Laurent
Back to top
View user's profile Send private message
Hu
Moderator
Moderator


Joined: 06 Mar 2007
Posts: 14276

PostPosted: Sun Feb 01, 2015 6:37 pm    Post subject: Reply with quote

Distccd exposes whatever you tell it to expose. If you want to use a cross-compiler, the distcc client must be called with the fully qualified name of the cross-compiler, just like if you want to run a non-distributed cross-compile.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Sun Feb 01, 2015 7:07 pm    Post subject: Reply with quote

Quote:
Distccd exposes whatever you tell it to expose.

But how do you achieve that ? And is it any interest for client distcc ?

Quote:
. If you want to use a cross-compiler, the distcc client must be called with the fully qualified name of the cross-compiler, just like if you want to run a non-distributed cross-compile.

On my ARM machines, I have already those symlinks created :
Code:
armv7a-hardfloat-linux-gnueabi-c++ -> /usr/bin/distcc
armv7a-hardfloat-linux-gnueabi-g++ -> /usr/bin/distcc
armv7a-hardfloat-linux-gnueabi-gcc -> /usr/bin/distcc


But when I'm launching a compilation on those ARM boxes, only others ARM are participating, not x86 ones on which an Xcompiler is installed.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43985
Location: 56N 3W

PostPosted: Sun Feb 01, 2015 7:20 pm    Post subject: Reply with quote

destroyedlolo,

I used crossdev to set up the toolchain on my AMD64. With distcc it builds for my Rasberry Pi.
You don't have to do anything special with distcc.

What do you have in /etc/distcc/hosts on the ARM box needing the help (the client)?
What do you have in its MAKEOPTS and in its CFLAGS?

How do you know the x86 boxes are not helping.
What does distccmon on the client show.?
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Sun Feb 01, 2015 10:23 pm    Post subject: Reply with quote

NeddySeagoon wrote:
What do you have in /etc/distcc/hosts on the ARM box needing the help (the client)?

Code:
torchwood ~ # cat /etc/distcc/hosts
+zeroconf


NeddySeagoon wrote:
What do you have in its MAKEOPTS and in its CFLAGS?

Code:
MAKEOPTS="-j15 -l3.0"
CFLAGS="-O2 -pipe -march=armv7-a -mfpu=vfpv3-d16 -mfloat-abi=hard"


NeddySeagoon wrote:
How do you know the x86 boxes are not helping.
What does distccmon on the client show.?

It's because distccmon shows only my BananaPIes :
Code:
  1991  Blocked                                                            [4]
  2020  Blocked                                                            [5]
  2021  Blocked                                                            [6]
  2022  Blocked                                                            [6]
  2023  Blocked                                                            [7]
  1961  Preprocess  config.c                                   192.168.0.14[1]
  1969  Preprocess  util_script.c                              192.168.0.14[3]
  1973  Compile     util_md5.c                                 192.168.0.14[5]
  2014  Preprocess  listen.c                                   192.168.0.66[0]
  1959  Compile     gen_test_char.c                            192.168.0.66[0]
  1960  Compile     log.c                                      192.168.0.66[2]


*.14 and *.66 are my 2 BananaPIs ... None of my PC is ever present and I noticed this strange "Blocked" entries.
Now if I add specifically my PCs
Code:
torchwood ~ # cat /etc/distcc/hosts
pset.local piv.local +zeroconf

they are used and the compilation is successful (so it's not a problem of permission).
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43985
Location: 56N 3W

PostPosted: Sun Feb 01, 2015 11:40 pm    Post subject: Reply with quote

destroyedlolo,

distccmon only shows 8 jobs but its allowed to start 15 with the -j15.
Excess jobs would run locally. They would show in distccmon too, if they were there.

This suggests that your client cannot distribute work fast enough, as the default number of jobs per helper, when none is specified is four.

Work is assigned to helpers in the order they appear in /etc/distcc/hosts
Code:
pset.local piv.local +zeroconf

From that, I conclude that the client cannot process jobs fast enough to send any to +zeroconf, as its last in the list.

To test, change /etc/distcc/hosts to read pset.local/1 piv.local/1 +zeroconf
That will send one job each to the bananapis before zeroconf hosts get any jobs, then they will get up to four each.
Blocked in distccmon means that the job is waiting for something else to complete.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Mon Feb 02, 2015 12:16 am    Post subject: Reply with quote

Quote:
distccmon only shows 8 jobs but its allowed to start 15 with the -j15.
Excess jobs would run locally. They would show in distccmon too, if they were there.

In fact, it's due to the load limitation : The Banana is dual core so I limited its load to 3.
Here we can see 4 per host only as none of them reach this load.

Quote:
Work is assigned to helpers in the order they appear in /etc/distcc/hosts
Code:
pset.local piv.local +zeroconf

From that, I conclude that the client cannot process jobs fast enough to send any to +zeroconf, as its last in the list.

It was only an example to show the compilation succeeds on PCs if I explicitly add them in the hosts list. The distccmon grab I've put is when only +zeroconf is provided. And as shown, only *.14 and *.66 (the Bananas) are serving. Not the PCs, even if they're having lower number on my network.

In addition, it's not a network issue as with "pset.local piv.local +zeroconf" list, the compilation is spread on PSET, PIV and also on both Banana.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7195

PostPosted: Mon Feb 02, 2015 1:16 pm    Post subject: Reply with quote

You shouldn't just tune your helpers for their cores, the power of the cores makes a real diff, and using zeroconf leave distcc without hint on who will do the job better.

Here's why with two cpu sample.
suppose you have a core2 at 1ghz and another core2 at 4ghz (i take the same cpu with higher freq so you don't need to guess who is stronger)
both could handle 2 jobs, but it is easy to see the 4ghz core2 will takes less time to build one job.
In this case it is counter productive to feed the two hosts with 2 jobs each, as if you build a 5 jobs program, 2 jobs will be end by the 4ghz while 2 jobs remain compiling and the 5rd job is waiting the 4 jobs end to start.
While if you had given the 4jobs to the 4ghz host, you would had start 5rd job faster.

So instead of setting 1ghzhost/4 4ghzhost/4 a better setup is 4ghzhost/8 1ghzhost/2 hinting distcc 4ghzhost should be more used.

Without hinting distcc about job power handling, distcc assume each host is set to 4 (no matter the number of cores as distcc cannot get that value).
So first host get 4 jobs, next host get 4... at 8 jobs few makefile exists already that could produce more jobs.

zeroconf usage is also limit as even you will not use them with zeroconf or not use the helpers as client themselves, they should be run with zeroconf too, so if your pset and piv host don't have zeroconf set because you don't need it, this will prevent a client using zeroconf itself to use them.
another limit is that pump mode is not doable in zeroconf.
You can add randomize to higher balance between your host so distcc doesn't balance it on "suppose" power but number of host available.

So: you better just not use zeroconf as it sucks bad at doing a good job, if you want use it, use it with randomize and make sure all hosts are using zeroconf even they are just helper.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Thu Feb 05, 2015 5:14 pm    Post subject: Reply with quote

Hi krinn,

I know my configuration isn't optimum but my goal is only to benefit from the present of big PC with XCompiler installed if available, without having dealing with distcc host list manually as I'm currently doing.
Having distcc advertising reach this objective ... as soon as I can made XCompiler discovered.

Unfortunately, as said above, it doesn't work for me :cry:

Bye

Laurent
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43985
Location: 56N 3W

PostPosted: Thu Feb 05, 2015 7:38 pm    Post subject: Reply with quote

destroyedlolo,

"doesn't work" does not help reduce the problem space.

Lets test if its zeroconf or distcc. Then we know what to look at next.
Put your big PC with the cross compiler into the distcc host list by IP address, as the first host in the list.

If it works now, its a zeroconf issue. If not, its a distcc or cross compiler issue.
This will help reduce the problem space.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Thu Feb 05, 2015 9:02 pm    Post subject: Reply with quote

NeddySeagoon wrote:
Lets test if its zeroconf or distcc. Then we know what to look at next.
Put your big PC with the cross compiler into the distcc host list by IP address, as the first host in the list.


Yes, it's working : the last sentence of my Fév 01, 2015 11:23 pm post.

It's a zeroconf issue :wink:
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 43985
Location: 56N 3W

PostPosted: Thu Feb 05, 2015 9:08 pm    Post subject: Reply with quote

destroyedlolo,

Now you have proved its a zeroconf issue, you may want to start a new thread with your findings.
I have never used zeroconf, so I'm out now.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
destroyedlolo
l33t
l33t


Joined: 17 Jun 2011
Posts: 703
Location: Close to Annecy (France)

PostPosted: Thu Feb 05, 2015 10:58 pm    Post subject: Reply with quote

As per my tests :
  • zeroconf distcc only select participants exposing the same compiler signature (I did some test with an Ubuntu machine : it is not select as GCC version differs, even if it share the same architecture).
  • it seems to me distcc exposes only the native compiler, not Cross ones
  • Hu said I can expose everything I want ... the question is how ?


By the way, why a new thread ?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum