Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Programs that can't get emerged with distcc & tips
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
zen_guerrilla
Guru
Guru


Joined: 18 Apr 2002
Posts: 343
Location: Greece

PostPosted: Mon Nov 11, 2002 5:32 pm    Post subject: Programs that can't get emerged with distcc & tips Reply with quote

Hello world,
I'm starting this thread in order to have a full list of programs that can't get compiled with distcc and/or possible solutions/suggestions.

.:: zen ::.
Back to top
View user's profile Send private message
IWBCMAN
Guru
Guru


Joined: 25 Jun 2002
Posts: 474

PostPosted: Tue Nov 26, 2002 11:49 pm    Post subject: Reply with quote

started using distcc two days ago....


the following apps would not compile with distcc sofar....

everything kde(3.04 and 3.1-r3)
gstreamer
libxml2
alsa-lib
giflib

on the plus side almost everything Gnome does work with distcc-I haven't run ay benchmarks(ie. timed it) but it sure seems faster, perhaps by as much %30...
heres a look at my distcc stuff in /etc/make.conf

Code:
CC="distcc"
CXX="distcc g++"        # may not be neccesary, and may serve only to break things - not sure yet
MAKEOPTS="-j3"          # replace 3 with a number slightly higher than the number of CPUs that will be working
DISTCC_HOSTS="192.168.0.3 localhost"
DISTCC_VERBOSE=1       # if you want distcc to spew extra output


Last edited by IWBCMAN on Sat Nov 30, 2002 2:15 pm; edited 1 time in total
Back to top
View user's profile Send private message
hertog
Tux's lil' helper
Tux's lil' helper


Joined: 28 Jun 2002
Posts: 138
Location: Enschede/The Netherlands/Europe

PostPosted: Fri Nov 29, 2002 5:53 pm    Post subject: Reply with quote

Am doing a full rebuild with distcc enabled from stage 2 up.

In Stage2 the following packages failed to build:
Netkit-base
psmisc
util-linux
bin86

And some hints:

export CC="distcc"
export CXX="distcc g++"

The first one causes the gcc compiles to be distributed, the second one causes the g++ compiles to be distributed, these are otherwise ignored.

Some makefiles don't use $(CC) var to define the compiler, they just plainly call gcc, on packages with such a makefile distcc will (obviously) not work, meaning, not distribute any work. This can be distributed anyway by making a softlink to distcc named gcc in a directory that is searched BEFORE the directory that gcc is in, I don't know if this is a very smart move tho, that way everything will be (tried to be) distributed, probably failing in some places...

I will expand the list of not working packages as I progress through the build...

Gr.

Hertog
Back to top
View user's profile Send private message
ravon
n00b
n00b


Joined: 24 Apr 2002
Posts: 15
Location: Stockholm, Sweden

PostPosted: Mon Dec 02, 2002 8:41 am    Post subject: Reply with quote

Add libjpeg to that one. It will only build the static libs.
Back to top
View user's profile Send private message
JeroenV
Guru
Guru


Joined: 16 Jul 2002
Posts: 447
Location: Amsterdam / Hamburg

PostPosted: Thu Dec 19, 2002 11:47 am    Post subject: Reply with quote

Hi,

maybe this thread comes in handy. My little script will (if everything works) also automagically produce a list of programs that failed to build with distcc :wink:
_________________
Cheers 8)
Jeroen
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
May The Source be with you!
Back to top
View user's profile Send private message
brunes69
n00b
n00b


Joined: 26 Dec 2002
Posts: 1
Location: NB, Canada

PostPosted: Thu Dec 26, 2002 7:37 pm    Post subject: How to make distcc work with *anything* Reply with quote

If you notice a pattern, it is that distcc works with pretty much any C program (like GNOME), but craps out on some C++ ones (like KDE). The reason distcc doesnt work with some C++ programs is because the Makefile scripts expect the name of the compiler to be one word, which is now two ("distcc g++"). Solution? I wrote a one line shell script as a "front end" to distcc

#!/bin/sh
distcc g++ $@

I called this "dist++" and put it in my /usr/bin, then set CXX to dist++ everywhere. Works like a charm, I have never had a single problem building anything with it, even building every app in KDE3. Try it out, get the benefits of distcc for every app you emerge!
Back to top
View user's profile Send private message
JeroenV
Guru
Guru


Joined: 16 Jul 2002
Posts: 447
Location: Amsterdam / Hamburg

PostPosted: Thu Dec 26, 2002 11:57 pm    Post subject: Reply with quote

Thanks, that's a good one :D
I incorporated that in my unofficial portage hacks
_________________
Cheers 8)
Jeroen
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
May The Source be with you!
Back to top
View user's profile Send private message
bonsaikitten
Apprentice
Apprentice


Joined: 01 Jan 2003
Posts: 213
Location: Shanghai, China

PostPosted: Wed Jan 01, 2003 6:12 pm    Post subject: Packages not working with default distcc install Reply with quote

Hi,

I had the following packages crap out on me using distcc:
* ncurses
* glibc
* ftp
they compile fine using normal gcc.
Back to top
View user's profile Send private message
jonner
n00b
n00b


Joined: 25 Jul 2002
Posts: 42

PostPosted: Wed Jan 15, 2003 10:23 am    Post subject: Re: How to make distcc work with *anything* Reply with quote

brunes69 wrote:
If you notice a pattern, it is that distcc works with pretty much any C program (like GNOME), but craps out on some C++ ones (like KDE). The reason distcc doesnt work with some C++ programs is because the Makefile scripts expect the name of the compiler to be one word, which is now two ("distcc g++"). Solution? I wrote a one line shell script as a "front end" to distcc

#!/bin/sh
distcc g++ $@

I called this "dist++" and put it in my /usr/bin, then set CXX to dist++ everywhere. Works like a charm, I have never had a single problem building anything with it, even building every app in KDE3. Try it out, get the benefits of distcc for every app you emerge!


I came to exactly the same conclusion and wrote exactly the same script when I noticed that a couple of C++ programs were failing with compile command lines that include "g++" as one of the options (the compiler complained it couldn't find the file "g++").

I suspect a better solution would be to make multiple links to distcc (distcc, distc++, distjavac, distf77, etc) and have distcc assume different compiler commands based on argv[0]. I couldn't find any reference to this technique in the distcc docs, but it probably would be trivial to add.

More generally, this is one of the pitfalls of using a purely textual command language. Of course, text-based command languages have proven themselves very powerful and general. I keep meaning to start using scsh (Scheme Shell) instead of /bin/sh or Bash to see if a more symbolic approach is advantageous.
Back to top
View user's profile Send private message
AlterEgo
Veteran
Veteran


Joined: 25 Apr 2002
Posts: 1619

PostPosted: Wed Jan 15, 2003 10:58 am    Post subject: Reply with quote

I can't get nmap (3.10_alpha6) to compile using distcc.
Back to top
View user's profile Send private message
bartonlp
n00b
n00b


Joined: 03 Apr 2005
Posts: 1
Location: CA

PostPosted: Sun Apr 10, 2005 1:42 am    Post subject: Re: How to make distcc work with *anything* Reply with quote

brunes69 wrote:
If you notice a pattern, it is that distcc works with pretty much any C program (like GNOME), but craps out on some C++ ones (like KDE). The reason distcc doesnt work with some C++ programs is because the Makefile scripts expect the name of the compiler to be one word, which is now two ("distcc g++"). Solution? I wrote a one line shell script as a "front end" to distcc

#!/bin/sh
distcc g++ $@

I called this "dist++" and put it in my /usr/bin, then set CXX to dist++ everywhere. Works like a charm, I have never had a single problem building anything with it, even building every app in KDE3. Try it out, get the benefits of distcc for every app you emerge!


I thought one was supposed to put the "/usr/lib/distcc/bin" at the start of your PATH and then you don't need to do the CC= stuff?
_________________
Barton Phillips
Back to top
View user's profile Send private message
ciaranm
Retired Dev
Retired Dev


Joined: 19 Jul 2003
Posts: 1719
Location: In Hiding

PostPosted: Sun Apr 10, 2005 4:22 am    Post subject: Reply with quote

Generally if things won't compile with distcc, it's because the Makefile doesn't handle parallel builds correctly. It'd be nice if certain upstream people learned how to write proper Makefile dep rules -- this kinda thing is a real pain in the ass when we're building release packages or testing on one of our 16-CPU build boxes... :evil:
Back to top
View user's profile Send private message
Kate454
n00b
n00b


Joined: 03 Aug 2011
Posts: 1

PostPosted: Wed Aug 03, 2011 9:40 am    Post subject: Reply with quote

I set up distcc cross-compile with two different machines and some packages wont emerge is there a way i can solve this problem?
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum