Portage with p2p

nmcsween · Guru Joined: 12 Nov 2003 Posts: 381

This has all been said before... Why not change the way you think why should compiler flags even be touched by the average user? Why not distribute binaries that have been compiled with simply -02 and -i686 and create deltas of sse and 3dnow or even profiled feedback?
_________________
Great Resources

Fluid Portage

Matteo Azzali · Retired Dev Joined: 23 Sep 2004 Posts: 1133

Matteo Azzali · Retired Dev Joined: 23 Sep 2004 Posts: 1133

Hypnos · Posted: Wed May 31, 2006 9:28 am Post subject:

My major beef with binary distros were ABI dependency headaches and having to reconfigure binary packages specs when I needed a new option.

The two key objections to having an extensive binary infrastructure in Gentoo seem to be security and USE flags.

Regarding USE flags, every hardware host has a set of default USE flags defined /usr/portage/profiles -- why not just use those as the default? If a user wants to deviate, he can build from source. This would give the best of both worlds between source- and binary-based.

Regarding security, isn't Portage moving to GPG signatures anyway? If there is a central, trusted binary repository, they can authenticate binary packages just like vendors authenticate source packages.

However, I don't know what to do about ABI hell. Even if you fix gcc, glibc, etc. versions, USE flags and CFLAGS can alter compatbility (AFAIK). This is a general Linux annoyance, more than Gentoo's fault, but the problem remains ...
_________________
Personal overlay | Simple backup scheme

rafo · Posted: Wed May 31, 2006 9:44 am Post subject:

Hi Matteo,

As I see it these kind of "binary" initiatives are just adding choice to Gentoo: You can trade trust, optimization and customization for installation speed. Other friendly initiatives that are also driven by the sometimes perceived need for speedy installation are Kororaa (http://kororaa.org/), Chinstrap (http://chinstrap.alternating.net/), and RR4 Linux (http://www.lxnaydesign.net/). Gentoo takes no responsibility for them, but they are there if you trust them and find them useful. One thing they have in common is to underline that the end result is a valid Gentoo installation: you can tweak flags and re-emerge just like Gentoo promises.

I just helped a colleague with a Kororaa installation. My judgment is that he will benefit from being able to "emerge" this and that, and that he would not have tolerated me fiddling with his machine for several days. Maybe he will take further steps into Gentoo land, maybe not. Maybe in the end he will find that Gentoo is too demanding and go for a commercial boxed distribution. But even if that happens he has already learned a little about what Gentoo is.

rafo · Posted: Wed May 31, 2006 9:53 am Post subject:

Hi Hypnos,

just tossing in a link that explains ABI a little (I have not stumbled over the term before): http://en.wikipedia.org/wiki/Application_binary_interface

rafo · Posted: Sat Jun 03, 2006 12:14 pm Post subject:

So far I have been thinking of a p2p scheme for sharing trustable binaries. However, there could really be different distribution mechanisms, suitable for different contexts.

Suppose that you want to deploy Gentoo within an organization (a company, a campus, a hospital, ..). Then it may in fact be cost effective to keep produced binaries on a central server, or a network of servers. These servers would of course be trusted within the organization. Fingerprints could still be checked "just in case".

When a host is being installed or updated it will send requests for binaries (produced under specified preconditions) to the server. If the binary is already available it is of course downloaded. If it is not yet produced the requesting host may handle the "no" response in two ways: Either fall back to local compilation and installation, or hang on to the request until the server has produced the binary and download it then. The latter mode may be implemented in different ways, perhaps a polling scheme is sufficient.

Such a scheme could work very well for, say, a school. Schools are often low-budget operations and may be reusing "outdated" and hence slow hardware. For the administrator of school computers it would be a pain to wait for every box to compile its own software. With a binaries server the first box would take some time to install (still better off than classical Gentoo since binaries are produced on a faster central server), and subsequent boxes would be install rapidly using binaries from the central server.

I am aware that putting binaries on a central server is nothing new--it can be done today, and is maybe done too. The nice thing is that it could be one mode of operation of the "bonus binaries" scheme; whereas p2p sharing could be another. What they have in common is the demand-driven "binaries factory" that allows payback on the investment in compile time while at the same time preserving the full flexibility of Gentoo.

barrymac · Tux's lil' helper Joined: 15 Jul 2004 Posts: 87

How about a p2p portage system so no-one on the planet ever compiles something that's been compiled before by another gentooist?

I'm sure it wouldn't be the most difficult system to set up with bittorrent and some clever way to name torrents after their architectures and use flag options.

Probably the most popular stuff should be permanently seeded.

nixnut · Posted: Sat Jul 28, 2007 5:26 pm Post subject:

merged above post here.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand

sugar

It would save on having to depend on mirrors.
_________________
He who calls for full employment calls for war!

sirdilznik · l33t Joined: 28 Apr 2005 Posts: 731

Maybe because then it would force someone to also have a torrent client. I think minimum requirements is a good thing for portage. Also which client would you choose as default? I guess it could be added with a USE flag.

Just my thoughts.

sugar · Posted: Sun Aug 05, 2007 4:44 am Post subject:

Sadako · Posted: Sun Aug 05, 2007 6:55 am Post subject:

It would not save on depending on mirrors, it would just lighten the load on them.
You would still have to ensure that there are seeds for each tarball available at all times.

It would also involve setting up a dedicated tracker, and you would need to publish a new torrent for every single version bump or new ebuild (although I'm sure this could be automated).

Another point is that the majority of files/tarballs on the mirrors are quite small, and in such cases only making them available as torrents simply wouldn't make sense.

There is also the issue of the amount of work it would take to add it to portage.

It's not a terrible idea, and I'd be surprised if it wasnt at least discussed before, but overall I don't think it would really be worthwhile implementing.
_________________
"You have to invite me in"

codergeek42 · Posted: Sun Aug 05, 2007 7:26 am Post subject:

Moved from Off the Wall to Gentoo Chat.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF

djdunn · l33t Joined: 26 Dec 2004 Posts: 810

I think that certain files say over 50 megs a torrent could be an option, much like using FTP is an option.

I'm sure some of the mirrors would be able to handle seeding also,

for example when new versions of KDE or OOo or whatnot come out having a torrent as an option would probably help lessen the demand on the mirrors.

but again i don't think that every single file would benefit from being a torrent but some could benefit greatly.
_________________
“Music is a moral law. It gives a soul to the Universe, wings to the mind, flight to the imagination, a charm to sadness, gaiety and life to everything. It is the essence of order, and leads to all that is good and just and beautiful.”

― Plato

Telexen · Apprentice Joined: 09 Oct 2005 Posts: 189

I don't like it. There are things torrents are good for and there are things they're not good for.

You're not going to lighten the load on portage servers that much with torrents - even from big ones. And I'll be able to download the large files I need MUCH faster with the side effect of a little extra server load than I would with torrents.

I'm not sure why people think torrents are the answer to everything. I've downloaded my share linux distros and such with torrents, and have never seen a single download max out my 3mbit cable connection - yet I always do downloading large files from a portage server.

nixnut · Posted: Sun Aug 05, 2007 10:43 am Post subject:

merged above seven posts here.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand

sugar · Posted: Mon Aug 06, 2007 4:01 am Post subject:

nwmcsween · n00b Joined: 25 May 2007 Posts: 41

For something that can be attained without a large overhaul why not just have solid releases such as kernel 2.6.22 and create deltas for any of the updates to said kernel? KDE does it SUSE does it.

Corona688 · Veteran Joined: 10 Jan 2004 Posts: 1204

nwmcsween · n00b Joined: 25 May 2007 Posts: 41

Roman_Gruber · Posted: Sat Aug 11, 2007 12:15 am Post subject:

ever heard of sabayon linux?

minervaix · Posted: Sat Sep 15, 2007 6:43 am Post subject: gentella network

After recently installing gentoo on one of my boxes using all the packages from another gentoo machine via emerge -k $package with my PKGDIR set, and installing a full system with xorg and kde in a couple of hours. I decided all those gentoo users out there with buildpkg in their make.conf should set up some P2P software so we can all make use of pre built packages, especially good for the i need it NOW!! situations, we can call it the GENTELLA network. whatda think?

omp · Posted: Sat Sep 15, 2007 7:07 am Post subject: Re: gentella network

bunder · Bodhisattva Joined: 10 Apr 2004 Posts: 5934

Merged the above two threads.
_________________