"Ultrapeer" for Gentoo?

mikenerone · Posted: Tue Feb 08, 2005 1:21 pm Post subject:

Relevant discussion at Bug 78407.

Arkham · Posted: Tue Feb 08, 2005 2:02 pm Post subject:

I think the "Gentoo way" is the distcc aproach... sharing binary packages with "popular" flags will led people to standarize theyr flags and you will be punished for selecting EXTREME tweaks
My thought only..

Can someone post numbers here??

Assuming there're enough IDLE machines, and those machines have the tarballs, how long would it take to compile KDE FULL (with split ebuilds of course

)

2 hours? 1? 30 min??

Anyway, I think the benefits for the comunity are huuuuuge. It may worth a try

?

LucaSpiller · Posted: Tue Feb 08, 2005 3:07 pm Post subject:

Since I saw this topic last night I decided to do a bit of research into this to see what has been discussed before. Rather than discussing just sharing specific packages most of these focused on sharing the whole of whatever you have in /usr/portage/distfiles/.

There was also some discussion of sharing the ebuilds themselves, but some sort of security signature would have to be added by the developers so fake ebuilds cannot be made that could damage your system - patches and whatever would also have to have this. The list of the public keys would have to be hosted on the Gentoo site or mirrors so that it is known to be secure. Have a look at this post for more info (the whole topic is actually worth a read).

I really don't think a BT type system would work, it is designed for people to share single (and usually) large files easily without having to do much work. You would have to set up a webserver to share the .torrents, ok so that wouldn't be that much but it will take up bandwidth, and if the current mirrors themselves are seeding the files they will take most of the brunt. Ok so every kb saved helps them but it won't make a huge difference.

I believe a serverless p2p service such as Kad(emlia) would be much better suited to the job because there is no single point of redudancy. Also rather than sending the whole file if somebody already has an older version a Deltup type system (see my previous post in this topic) could be used to send out just the changes. I believe one of the devs (see the topic I linked to above) was working on a p2p Portage but nothing new has really been announced so I don't know if it is still going.
_________________
:: Luca :: Mac Fag :: Original Macbook, 2g RAM :: Closet Linux user (seasoned with salt and pepper) :: C2D E4400 @ 2ghz, 4g RAM (only 3.2g detected under 64bit...), Nvidia 9600GSO ::

Lews_Therin · l33t Joined: 03 Oct 2003 Posts: 657 Location: Banned

TazG · Guru Joined: 22 May 2004 Posts: 320 Location: Canada

LucaSpiller · Posted: Tue Feb 08, 2005 9:50 pm Post subject:

A dictcc thing over the internet would be extremely slow, let alone insecure. I guess what distcc does is send out different source files to be compiled to different computers which then send back the .o file. Then when compilation of all of them has been done the linker on the main computer puts them all together.

The reason why I say this would be extremely slow is because just having a quick look the .o files are around 10x bigger than the corresponding .c and .h files put together. Although it may not take as long to compile these files you still have to get them back to the main system to be linked. Here in Britiain the norm upload speed for broadband is 256k and download is 512k, so if you are compiling something such as OOo and lets say the source compiled into .o files is 200mb (just a guess) you would be waiting a hell of a long time for it to get to you.
_________________
:: Luca :: Mac Fag :: Original Macbook, 2g RAM :: Closet Linux user (seasoned with salt and pepper) :: C2D E4400 @ 2ghz, 4g RAM (only 3.2g detected under 64bit...), Nvidia 9600GSO ::

southsider · Guru Joined: 05 Jul 2004 Posts: 358

Also you could have people sending you infected .o's.

No thanks.

Arkham · Posted: Wed Feb 09, 2005 12:15 pm Post subject: