| View previous topic :: View next topic |
| Author |
Message |
secondshadow
Guru
Joined: 23 Jun 2003
Posts: 362
|
 Posted: Fri Nov 12, 2004 7:44 am Post subject: my plan to get distcc to use muliple cross-compiler sortof |
 |
|
there are a couple of bugs about distcc not using the proper compiler when cross-compiling such as:
https://bugs.gentoo.org/show_bug.cgi?id=18024
and
https://bugs.gentoo.org/show_bug.cgi?id=29558
The solution suggested therein (forcing the fully qualified compiler name) does seem to work, but unfortunately I noticed something someone else complained about. For some odd reason, configure takes a WHOLE lot longer to complete. So that, IMHO, is not good as it defeats the purpose of using distcc. So what is a guy to do when he has multiple archetectures to support? I mean, I have an iBook (G4) 1GHz, an Athlon64 3200+, and an AMD K6-2 450 to maintain gentoo on. Now obviously I really don't want to have to compile a new version of gcc for the K6-2 all on its lonesome (especially not on just 32MB RAM...the k6-2 is a NAT router, btw). So, my previous solution to the K6-2 was to have a 32-bit chroot on the Athlon64, tar it all up once a month, sftp it over and untar it. That was irritating. But I digress.
All in all, this seems to be a problem that, while being worked on, currently has to particularly elegant solution so I thought I'd just throw my 2 cents in and see how it goes. So whats my plan? Well. Simple. Run multiple distcc servers on the machines serving up distcc. And it turns out this is particularly easy to do too. All I've done is to rename the distccd files in /etc/conf.d and /etc/init.d to things like ppc-distccd and x86_64-distccd. Along with that I made a few alterations to the aforementioned files.
The original /etc/conf.d/distccd
| Code: |
# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.17/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file
DISTCCD_OPTS=""
# this is the distccd executable
DISTCCD_EXEC=/usr/bin/distccd
# this is where distccd will store its pid file
DISTCCD_PIDFILE=/var/run/distccd/distccd.pid
# set this option to run distccd with extra parameters
# Default port is 3632. For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3632"
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical
DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"
# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --allow or --listen options
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# example: --allow 192.168.0.0/24
# example: --allow 192.168.0.5 --allow 192.168.0.150
# example: --listen 192.168.0.2
#DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24 --listen 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="15"
|
MY /etc/conf.d/ppc-distccd
| Code: |
# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.17/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file
DISTCCD_OPTS=""
# this is the distccd executable
DISTCCD_EXEC=/usr/bin/distccd
# this is where distccd will store its pid file
DISTCCD_PIDFILE=/var/run/distccd/ppc-distccd.pid
# set this option to run distccd with extra parameters
# Default port is 3632. For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3635"
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical
DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"
# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --allow or --listen options
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# example: --allow 192.168.0.0/24
# example: --allow 192.168.0.5 --allow 192.168.0.150
# example: --listen 192.168.0.2
#DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24 --listen 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="15"
|
The two things that changed there were the listen port number and, slightly more important, the name of the DISTCCD_PIDFILE which is what allows the machine to distinguish between different distccd server and shut them down properly.
The original /etc/init.d/distccd
| Code: |
#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.17/init,v 1.1 2004/08/22 19:14:10 lisa Exp $
depend() {
need net
use ypbind
}
start() {
[ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null
ebegin "Starting distccd"
chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
TMPDIR="${TMPDIR}" \
PATH="$(gcc-config --get-bin-path):${PATH}" \
/sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
--pidfile ${DISTCCD_PIDFILE} -- \
--pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
${DISTCCD_OPTS}
eend $?
}
stop() {
ebegin "Stopping distccd"
start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
rm -f "${DISTCCD_PIDFILE}"
eend $?
}
|
MY /etc/init.d/ppc-distccd
| Code: |
#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.17/init,v 1.1 2004/08/22 19:14:10 lisa Exp $
depend() {
need net
use ypbind
}
start() {
[ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null
ebegin "Starting ppc-distccd"
chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
TMPDIR="${TMPDIR}" \
PATH="/home/crossdev/ppc/bin:/home/crossdev/ppc/powerpc-unknown-linux-gnu/bin" \
/sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
--pidfile ${DISTCCD_PIDFILE} -- \
--pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
${DISTCCD_OPTS}
eend $?
}
stop() {
ebegin "Stopping distccd"
start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
rm -f "${DISTCCD_PIDFILE}"
eend $?
}
|
What I've done here is 1) make it tell me that its starting the ppc-distcc (just makes my life a little easier instead of having 4 different lines that all say Starting distcc...) and 2) changed the path in which distcc looks for the compiler. In this case it will ONLY look where I placed the cross-compiler and since thats the only arch that this particular distccd needs to be concerned with I decided that that scheme made the most sense.
Before anyone asks, no I didn't actually use crossdev to generate my cross-compiler toolchain. I did it by hand following the instructions found here:
http://www.sable.mcgill.ca/~dbelan2/crossdev-powerpc-i686.html
It was actually pretty easy and painless and appears to work just fine. Right now I'm doing a little speed test compiling a kernel so we'll see how that goes, but otherwise I just thought I'd share my idea. Questions, comments?
Well, looks like the speedtest is actually done (proofreading gave it the time it needed). Using distcc and only the Athlon64 (running in 32-bit mode mind you) to compile the 2.6.9-r1 kernel took roughly 9:40 using make -j2. Doing the same only with the local gcc (completely bypassing distcc with $PATH) using just the iBook and make -j2 all took 23:57. So all in all I'd say thats pretty significant. Next I test it using both. I'm hoping for something close to 6-ish minutes. |
|
| Back to top |
|
 |
secondshadow
Guru
Joined: 23 Jun 2003
Posts: 362
|
| Posted: Fri Nov 12, 2004 8:19 am Post subject: |
|
|
update, well.... it appears that the Athlon64 combined with the iBook were only about 16 seconds faster than the Athlon64 alone. Very interesting indeed....
-EDIT-
Alright, well, I figured out what the major malfunction was. It turns out that my iBook was suffering from the "running at half speend but not saying that it wasn't" syndrome. Meaing the frequecy in /proc/cpuinfo looked right, but the bogomips were 1/2 of what they should have been (see the ppc forums if you want more info). Soooo, that say, using the following
distcc-config --set-hosts "192.168.0.xxx/2 localhost/2"
make -j4 all
the time is down to 6:51 which is about what I was figuring it should be. |
|
| Back to top |
|
|
iTux
Guru
Joined: 07 Sep 2004
Posts: 586
Location: Toronto
|
| Posted: Sat Nov 13, 2004 2:47 am Post subject: |
|
|
Very interesting!
I tried to do something similar (though with symlinks - like the net interfaces) some time ago but I failed.
Now I guess now I will able to cross-compile for powerpc-unknown-linux-gnu and powerpc-apple-darwin6.0 at the same time. 
Thanks,
iTux |
|
| Back to top |
|
|
iTux
Guru
Joined: 07 Sep 2004
Posts: 586
Location: Toronto
|
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Portage & Programming
