Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
HOWTO: Use a Windows box as a distcc server for linux.
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
ylon
n00b
n00b


Joined: 04 Jun 2003
Posts: 5

PostPosted: Sun Nov 23, 2003 1:29 am    Post subject: Reply with quote

Well Kick, I'm not sure I'd be able to help at this moment as I've not gotten PowerFactor's steps to work thus far with newer apps in the current portage tree. Perhaps we can get those problems that we currently may see worked out and then get this rolling, as I think its much more convenient as well as reliable than following a document each time. Also, perhaps the gentoo on cygwin project would be of help:

http://www.toso-digitals.de/gentoo/

If we could get a joint e-build going there that would be great. For folks with access to bunches of systems it would be rather easy for them to deploy something like this in which they could run the ebuild on the linux side to get the files necessary, then tftp or ssh over to that linux box to pull necessary libs and get the cygwin ebuild running and then have some sort of menu built on the linux side that would easily walk you though popping in the distcc and make flags for the number of processors on your network (and when you upgrade to more boxen)... Anyways, I could see some neat things coming of this. It'd be easy for many linux machines, but this would be handy any way you look at it. Also, we can incorporate Mac OS X into the mix.
Back to top
View user's profile Send private message
jago25_98
Apprentice
Apprentice


Joined: 23 Aug 2002
Posts: 180

PostPosted: Fri Dec 05, 2003 5:49 pm    Post subject: easy to make a do it all package? Reply with quote

Don't know about you but I find doing this stuff on Windows a bit of a pain; not what I have windows for.

A great big .exe or .zip that does it all would be a dream. Could it just be a case of combining:


    - cygwin,
    - gcc tarball
    - a distcc binary compiled by self


into one place (.exe / .zip)?
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Wed Dec 24, 2003 6:46 pm    Post subject: Reply with quote

I have created a new verstion of my my cross-linux toolkit dist for Cygwin. It is a complete toolchain and utilities for makign a Cygwin host a distcc host. It includes:

binutils-2.14.90.0.6
glibc-2.3.2
gcc-3.2.3 (from -r3)
distcc-2.11.1

This 27MB .tar.bz2 can be downloaded and untarred into /usr and it will create a local directory (or overlay your own). In /usr/local/bin you can find a script called 'mkservice' which will install distcc as a NT service (as long as you have cygrunsrv installed).

Once you've done this and started the distcc service (net start distccd) you should have a complete distcc host ready to run. I have tested it by compling the 2.4.22 kernel only on my Cygwin-Distcc hosts and it completes fine.

It can be found at ftp://ftp.dympna.com/pub

--
Veto
Back to top
View user's profile Send private message
jago25_98
Apprentice
Apprentice


Joined: 23 Aug 2002
Posts: 180

PostPosted: Wed Dec 24, 2003 7:59 pm    Post subject: can't seem to connect to ftp://ftp.dympna.com/pub ...? Reply with quote

veto:

can't seem to connect to ftp://ftp.dympna.com/pub ...?

That new cross compiler sounds handy. How's about a small httpd server if that makes things simpler.
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Thu Dec 25, 2003 7:00 am    Post subject: Reply with quote

Ugh, I've fixed this problem...the ftp server should be available now. Sorry about that.

--
Veto
Back to top
View user's profile Send private message
john.robinson
n00b
n00b


Joined: 21 Oct 2003
Posts: 24

PostPosted: Thu Jan 08, 2004 5:36 pm    Post subject: Reply with quote

[fx] *BANG* *BANG* *BANG* [/fx of head against wall]

Wish I'd read to the end of the thread. I did do a month or so ago, the last time I checked it out, but now I was feeling brave enough to have a go. gcc is now compiling, but sure as fate it'll be all wrong, and I'll be better off using the tarball.

Thanks very much for your efforts, Veto and PowerFactor, and for keeping things up to date!

I do wonder, though, since distcc only passes pre-processed C source over, and Cygwin's own gcc binary package is only 3Mb, whether we can get this down to 3Mb or so? Or am I being really stupid :? and we really do need all the other binutils, libs and includes?

Also, if I understand ylon's suggestion right, that means (i) building a cross-compiler from gentoo to cygwin targets, so that we can (ii) build the cross-compiler for cygwin? That might be quite cool, but just a little exhausting for our systems. OTOH once you've done it once, you'll have a distributed cc which can do it, so it'll be quicker the next time :D
Back to top
View user's profile Send private message
john.robinson
n00b
n00b


Joined: 21 Oct 2003
Posts: 24

PostPosted: Thu Jan 08, 2004 6:35 pm    Post subject: Reply with quote

Oh darn.

Code:
John@cygwin ~
$ gcc -o hello hello.c
/usr/local/cross-linux/lib/gcc-lib/i686-pc-linux-gnu/3.2.3/../../../../i686-pc-linux-gnu/lib/crt1.o(.text+0xc): In function `_start':
: undefined reference to `__libc_csu_fini'
/usr/local/cross-linux/lib/gcc-lib/i686-pc-linux-gnu/3.2.3/../../../../i686-pc-linux-gnu/lib/crt1.o(.text+0x11): In function `_start':
: undefined reference to `__libc_csu_init'
collect2: ld returned 1 exit status

John@cygwin ~
$


This is on an Athlon box running XP, on latest cygwin. :(
Back to top
View user's profile Send private message
john.robinson
n00b
n00b


Joined: 21 Oct 2003
Posts: 24

PostPosted: Fri Jan 09, 2004 1:18 pm    Post subject: Reply with quote

I whinged about gcc not working, but that doesn't seem to affect distcc. btw that gcc was i686-pc-linux-gnu-gcc, I had the path etc. set properly, and the same results happen when you say `i686-pc-linux-gnu-gcc --target=i686-pc-linux-gnu -o hello.o hello.c`, but I don't care; I think it's because hello.c says #include <stdio.h> which is of course done by distcc before the source is passed over. Anyway, ignorant musings over.

Happy New Year all!
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Wed Jan 21, 2004 6:50 am    Post subject: Distcc for Cygwin build environment updated Reply with quote

I just wanted to let you guys know that I've updated my Cygwin hosted Linux distcc/gcc toolchain to 1.2 (012104). This toolchain will allow you to use your Windows boxes as Distcc compile hosts for your Linux machines. It is specifically designed for Gentoo Stable with the addition of distcc-2.12.1. I would guess it will work on other distributions, but I don't have any feedback on this.

The toolchain contains:

binutils-2.14.90.0.7
glibc-2.3.2-r9 (Gentoo patched and built with linuxthreads)
gcc-3.2.3-r3 (Gentoo patched)
distcc-2.12.1

This 25MB distribution can be found at ftp://ftp.dympna.com/pub/cross-linux-012104-stripped.tar.bz2

To install just cd /usr/local && tar xvjf /some/where/cross-linux-012104-stripped.tar.bz2. Read the README for making distcc a Windows Service so it'll run transparently and startup.
Back to top
View user's profile Send private message
jago25_98
Apprentice
Apprentice


Joined: 23 Aug 2002
Posts: 180

PostPosted: Sun Jan 25, 2004 1:17 pm    Post subject: can we make it easier? Reply with quote

So ftp://ftp.dympna.com/pub/cross-linux-012104-stripped.tar.bz2 is all a Windows user needs to be a distccd server?

Cygwin still need to be installed?

Is there a way we can make it simpler? - just double click would be best, if not, .zip?
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Mon Jan 26, 2004 10:26 pm    Post subject: Reply with quote

You will still need a installed Cygwin install. I won't be doing an .exe or a .zip, but you are welcome to. 90% of the work is in actually getting the toolchain configured, built and installed properly and then into the tarball.

Downloading the file and reading the README might be a good start.
Back to top
View user's profile Send private message
Harri
n00b
n00b


Joined: 12 Apr 2002
Posts: 21
Location: Helsinki

PostPosted: Wed Jan 28, 2004 10:57 am    Post subject: Reply with quote

Veto,

Thanks a million for this. It works great, and because of you I installed Cygwin for the first time and looked into it also. :)
Back to top
View user's profile Send private message
warlock55
n00b
n00b


Joined: 25 Jan 2004
Posts: 8

PostPosted: Wed Jan 28, 2004 3:25 pm    Post subject: Reply with quote

Hi,
I got 3 box: 2 gentoo and 1 windows. I wanna run distcc on all of them.
On the Windows machine, I didnt use Veto file. I installed Cygwin and Gcc-3.2.3 and distcc.2.11.1 successfully... and distcc works fine on all of them. However, there're packages that the Windows machine will break the compilation like Python-2.3. It compile fine between 2 Gentoo systems. And there's packages like gaim-0.75 which distcc between 2 Gentoo systems also break the compilation.
the errors said: distcc on local host fail...
I'm wondering if the error is because of my wrong setup or distcc...
All the machine are using the same version of gcc and distcc.
Planning to compile openoffice tonight using distcc on 3 systems. :D
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Wed Jan 28, 2004 7:52 pm    Post subject: Reply with quote

warlock55,

Are you using a consistently patched gcc and a consistently patched glibc? I've seen similar issues with the Cygwin/DistCC compiles when I used the plain distribution of either gcc or glibc. These went away when I switched to using the patched Gentoo sources to build the tools.

Hari,

Glad to see you liked it! It's actually pretty handy having Cygwin on a Windows box anyways.

-Veto
Back to top
View user's profile Send private message
warlock55
n00b
n00b


Joined: 25 Jan 2004
Posts: 8

PostPosted: Thu Jan 29, 2004 4:23 am    Post subject: Reply with quote

Oh Thanks alot Veto,
That's the reason, I'm using the plain distribution of gcc and distcc on the Windows machine...
Back to top
View user's profile Send private message
gralves
Guru
Guru


Joined: 20 May 2003
Posts: 389
Location: Sao Paulo, Brazil

PostPosted: Fri Jan 30, 2004 1:23 pm    Post subject: Re: Distcc for Cygwin build environment updated Reply with quote

Veto wrote:
I just wanted to let you guys know that I've updated my Cygwin hosted Linux distcc/gcc toolchain to 1.2 (012104). This toolchain will allow you to use your Windows boxes as Distcc compile hosts for your Linux machines. It is specifically designed for Gentoo Stable with the addition of distcc-2.12.1. I would guess it will work on other distributions, but I don't have any feedback on this.

The toolchain contains:

binutils-2.14.90.0.7
glibc-2.3.2-r9 (Gentoo patched and built with linuxthreads)
gcc-3.2.3-r3 (Gentoo patched)
distcc-2.12.1

This 25MB distribution can be found at ftp://ftp.dympna.com/pub/cross-linux-012104-stripped.tar.bz2

To install just cd /usr/local && tar xvjf /some/where/cross-linux-012104-stripped.tar.bz2. Read the README for making distcc a Windows Service so it'll run transparently and startup.


Veto, do you have any scripts to build those? It's been a pain to try to manually extract and patch c++ + binutils to lastest versions...
Back to top
View user's profile Send private message
gralves
Guru
Guru


Joined: 20 May 2003
Posts: 389
Location: Sao Paulo, Brazil

PostPosted: Fri Jan 30, 2004 2:40 pm    Post subject: Reply with quote

I've followed the instructions and I'having this error when I try to emerge something:

Code:

cc1: warning: -fPIC ignored for target (all code is position independent)
 distcc -DPACKAGE_NAME=\"lcms\" -DPACKAGE_TARNAME=\"lcms\" -DPACKAGE_VERSION=\"1.12\" "-DPACKAGE_STRING=\"lcms 1.12\"" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -DSIZEOF_INT=4 -DSIZEOF_UNSIGNED_INT=4 -DSIZEOF_LONG=4 -DSIZEOF_UNSIGNED_LONG=4 -DSIZEOF_LONG_LONG=8 -DSIZEOF_UNSIGNED_LONG_LONG=8 -DHasJPEG=1 -DHasZLIB=1 -DHasTIFF=1 -DHAVE_TIFFCONF_H=1 -I. -I. -I../include -I../include -O3 -march=athlon-xp -O3 -pipe -fomit-frame-pointer -ffast-math -mmmx -m3dnow -msse -mfpmath=sse,387 -c cmsvirt.c -o liblcms_la-cmsvirt.o >/dev/null 2>&1
/bin/sh ../libtool --mode=link distcc  -march=athlon-xp -O3 -pipe -fomit-frame-pointer -ffast-math -mmmx -m3dnow -msse -mfpmath=sse,387   -o liblcms.la -rpath /usr/lib -no-undefined -version-info 1:12:0 liblcms_la-cmscnvrt.lo liblcms_la-cmserr.lo liblcms_la-cmsgamma.lo liblcms_la-cmsgmt.lo liblcms_la-cmsintrp.lo liblcms_la-cmsio1.lo liblcms_la-cmslut.lo liblcms_la-cmsmatsh.lo liblcms_la-cmsmtrx.lo liblcms_la-cmspack.lo liblcms_la-cmspcs.lo liblcms_la-cmswtpnt.lo liblcms_la-cmsxform.lo liblcms_la-cmssamp.lo liblcms_la-cmscam97.lo liblcms_la-cmsnamed.lo liblcms_la-cmsps2.lo liblcms_la-cmscam02.lo liblcms_la-cmsvirt.lo -lm
distcc -shared  .libs/liblcms_la-cmscnvrt.o .libs/liblcms_la-cmserr.o .libs/liblcms_la-cmsgamma.o .libs/liblcms_la-cmsgmt.o .libs/liblcms_la-cmsintrp.o .libs/liblcms_la-cmsio1.o .libs/liblcms_la-cmslut.o .libs/liblcms_la-cmsmatsh.o .libs/liblcms_la-cmsmtrx.o .libs/liblcms_la-cmspack.o .libs/liblcms_la-cmspcs.o .libs/liblcms_la-cmswtpnt.o .libs/liblcms_la-cmsxform.o .libs/liblcms_la-cmssamp.o .libs/liblcms_la-cmscam97.o .libs/liblcms_la-cmsnamed.o .libs/liblcms_la-cmsps2.o .libs/liblcms_la-cmscam02.o .libs/liblcms_la-cmsvirt.o  -lm  -march=athlon-xp -mmmx -m3dnow -msse -mfpmath=sse,387 -Wl,-soname -Wl,liblcms.so.1 -o .libs/liblcms.so.1.0.12
.libs/liblcms_la-cmserr.o: file not recognized: File format not recognized
collect2: ld returned 1 exit status
distcc[13247] ERROR: compile on localhost failed
distcc[13246] ERROR: compile on localhost failed
make[1]: *** [liblcms.la] Error 1
make[1]: Leaving directory `/var/tmp/portage/lcms-1.12/work/lcms-1.12/src'
make: *** [all-recursive] Error 1


Everything works ok if I disable distcc.

Anyideias?


--------------

I will not triple post so I'm just editing this one...

You have to
Code:

ln -s /usr/local/cross-linux/gcc.exe /usr/local/cross-linux/cc.exe


Or some ebuilds will fail
Back to top
View user's profile Send private message
Veto
Tux's lil' helper
Tux's lil' helper


Joined: 04 Jul 2003
Posts: 83

PostPosted: Sat Jan 31, 2004 1:17 am    Post subject: Reply with quote

Thanks for the tip gralves, I'll add that to the next go at my toolchain.

Also, I don't have any scripts to do it, I do it by hand each time...and yes, it can be a pain in the ass. It's a 3-4hour ordeal each time with at least one recompile and some breakages to fix. I think it was the last go that I found that the plain gcc would compile fine, but the -r3 wouldn't. I knew that I had to make the -r3 work since a plain gcc will puke on some code and on top of that, it's the 'right thing (tm)' to do.

-Veto
Back to top
View user's profile Send private message
rk187
Tux's lil' helper
Tux's lil' helper


Joined: 14 Apr 2003
Posts: 136
Location: France - Paris

PostPosted: Mon Feb 02, 2004 7:44 pm    Post subject: Reply with quote

maybe colinux with Gentoo...

http://www.colinux.org
Back to top
View user's profile Send private message
Trejkaz
Guru
Guru


Joined: 14 Nov 2002
Posts: 479
Location: Sydney, Australia

PostPosted: Thu Feb 05, 2004 1:30 am    Post subject: Reply with quote

I'm wondering if there are any pitfalls to watch out for with respect to multiple architectures (in the GCC -march sense of the word.)

The install instructions specify i686-pc-linux-gnu, now... GCC accepts -march=athlon-xp supposedly so would it make more sense if my Gentoo install were setup for that architecture, to setup things differently? Or does the setup descripted result in the Gentoo box passing -march=athlon-xp to the Cygwin daemon?

Also, what is the deal if you _do_ have multiple architectures around the network? Does a single distccd installation need to cater for just one sort? Or does it switch between the directories depending on what the client machines ask it to perform?
Back to top
View user's profile Send private message
dewhite
n00b
n00b


Joined: 16 Mar 2003
Posts: 72
Location: Houston, Texas, USA

PostPosted: Thu Feb 05, 2004 9:37 pm    Post subject: Reply with quote

Wouldn't that be the point of cross compiling? ALSO - does anyone have a copy of this bz mirrored?
_________________
Work FS: i7-4790k || 16Gb DDR3 || nVidia 7200GS || 5x 1Tb SATA Drives in RAID5

Home FS: FX-8320e || 8Gb DDR3 || nVidia 8600GTS || 4x 2TB SATA Drives in RAID5

Home Desktop: i5-6500 || 16Gb DDR4 || 275Gb NVMe w/ FDE
Back to top
View user's profile Send private message
dewhite
n00b
n00b


Joined: 16 Mar 2003
Posts: 72
Location: Houston, Texas, USA

PostPosted: Fri Feb 06, 2004 3:22 am    Post subject: Reply with quote

reading the readme a few times over really made this more clear to me. I think I've got it done and working now proper.

Is there a good way to know if it's working? Should I just watch task manager to see if its using process time?
_________________
Work FS: i7-4790k || 16Gb DDR3 || nVidia 7200GS || 5x 1Tb SATA Drives in RAID5

Home FS: FX-8320e || 8Gb DDR3 || nVidia 8600GTS || 4x 2TB SATA Drives in RAID5

Home Desktop: i5-6500 || 16Gb DDR4 || 275Gb NVMe w/ FDE
Back to top
View user's profile Send private message
odessit
Apprentice
Apprentice


Joined: 01 Feb 2004
Posts: 180
Location: Current Residency - Server Room - Caution - Frostbite Imminent!

PostPosted: Fri Feb 06, 2004 10:04 am    Post subject: Reply with quote

You can just look at the task manager in Windows.
distcc has monitoring utility but I never used it.

What -j is everybody using?
I have 2 CPUS on 100-BT switched) LAN
localhost P3-866m and the cygwin host with XP1925MHz

-j3 barely touches the XP, ~30% with rare spikes to ~85%
-j4 does it better with ~60% util. and spikes up to ~90%

Should I try -j5/6 to load the cygwin-distcc even more?
Can this break anything? (<-linux nub paranoiah)

BTW, Thank you PowerFactor and Veto and other developers for working on this project :!: :lol:
Back to top
View user's profile Send private message
odessit
Apprentice
Apprentice


Joined: 01 Feb 2004
Posts: 180
Location: Current Residency - Server Room - Caution - Frostbite Imminent!

PostPosted: Sat Feb 07, 2004 1:27 am    Post subject: Bootable distcc CD Reply with quote

here's the link I found.
Looks like somebody created knoppix bootable CD with distcc on it.
fairly handy for the systems without cygwin or the hard drive

http://opendoorsoftware.com/cgi/http.pl?p=distccKNOPPIX
Back to top
View user's profile Send private message
dewhite
n00b
n00b


Joined: 16 Mar 2003
Posts: 72
Location: Houston, Texas, USA

PostPosted: Sat Feb 07, 2004 3:55 am    Post subject: Reply with quote

2 questions...

1. Will that knoppix cd boot with nvnet support for my NF7-S mobo? Or, is there a way to make it work with said adapter?

2. If not, is there anyway you could repackage the bz2 image with gcc 3.3.2 instead of 3.2.3 for us ~x86 users veto?
_________________
Work FS: i7-4790k || 16Gb DDR3 || nVidia 7200GS || 5x 1Tb SATA Drives in RAID5

Home FS: FX-8320e || 8Gb DDR3 || nVidia 8600GTS || 4x 2TB SATA Drives in RAID5

Home Desktop: i5-6500 || 16Gb DDR4 || 275Gb NVMe w/ FDE
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 2 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum