Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Portage with p2p
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
nmcsween
Guru
Guru


Joined: 12 Nov 2003
Posts: 381

PostPosted: Wed May 31, 2006 2:37 am    Post subject: Reply with quote

This has all been said before... Why not change the way you think why should compiler flags even be touched by the average user? Why not distribute binaries that have been compiled with simply -02 and -i686 and create deltas of sse and 3dnow or even profiled feedback?
_________________
Great Resources
Back to top
View user's profile Send private message
Matteo Azzali
Retired Dev
Retired Dev


Joined: 23 Sep 2004
Posts: 1133

PostPosted: Wed May 31, 2006 9:06 am    Post subject: Reply with quote

rafo wrote:

But think for a moment about a Gentoo "emerge". It is open source, yet we use it. By following sound operating procedures we can be confident that we don't run a hacked version of "emerge". The same reasoning goes for an install program that you download from some other organization: There are ways to be sure that what you downloaded is the program that they provide, and not a hacked version.

Emerge has nothing to do with p2p, it seems to me that any rsync or distfile server is identified and well-known , so server admin have some kind
of "responsibility" on what they do (and you can track back they if it's needed), if they would crack users machines, their server could be "removed" from
the list.

Quote:

The package-building machinery of the Bonus Binary Foundry should be automated and demand driven. If many users have a certain flag combination then the BBF will be hit by many requests for binary packages with that flag combination, and so those packages will tend to be produced early on. Packages for less common flag combinations will also be produced, but it will take longer. There is no need to try and predict which flag combination is the most typical; after a while it is however possible to gather some interesting statistics.

Yes, but in this way don't you tend to "convince" users to use the already built combinations? I mean, X or DE emersion can be quite long,
will not users decide to use all the same flags (and that's quite bad for testing...)
_________________
Every day a new distro comes to birth. Every day a distro "eats" another.
If you're born distro, no matter what, start to run.
---- http://www.linuxprinting.org/ ---- http://tuxmobil.org/
Back to top
View user's profile Send private message
Matteo Azzali
Retired Dev
Retired Dev


Joined: 23 Sep 2004
Posts: 1133

PostPosted: Wed May 31, 2006 9:16 am    Post subject: Reply with quote

nmcsween wrote:
This has all been said before... Why not change the way you think why should compiler flags even be touched by the average user? Why not distribute binaries that have been compiled with simply -02 and -i686 and create deltas of sse and 3dnow or even profiled feedback?


IMHO you are trying to move from gentoo to another distro: some of these just stick with -O2 -i386 without any delta....
Why not try directly another distro, then?
_________________
Every day a new distro comes to birth. Every day a distro "eats" another.
If you're born distro, no matter what, start to run.
---- http://www.linuxprinting.org/ ---- http://tuxmobil.org/
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2887
Location: Omnipresent

PostPosted: Wed May 31, 2006 9:28 am    Post subject: Reply with quote

My major beef with binary distros were ABI dependency headaches and having to reconfigure binary packages specs when I needed a new option.

The two key objections to having an extensive binary infrastructure in Gentoo seem to be security and USE flags.

Regarding USE flags, every hardware host has a set of default USE flags defined /usr/portage/profiles -- why not just use those as the default? If a user wants to deviate, he can build from source. This would give the best of both worlds between source- and binary-based.

Regarding security, isn't Portage moving to GPG signatures anyway? If there is a central, trusted binary repository, they can authenticate binary packages just like vendors authenticate source packages.

However, I don't know what to do about ABI hell. Even if you fix gcc, glibc, etc. versions, USE flags and CFLAGS can alter compatbility (AFAIK). This is a general Linux annoyance, more than Gentoo's fault, but the problem remains ...
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
rafo
Apprentice
Apprentice


Joined: 18 Nov 2003
Posts: 160
Location: Sollentuna, Sweden

PostPosted: Wed May 31, 2006 9:44 am    Post subject: Reply with quote

Hi Matteo,

As I see it these kind of "binary" initiatives are just adding choice to Gentoo: You can trade trust, optimization and customization for installation speed. Other friendly initiatives that are also driven by the sometimes perceived need for speedy installation are Kororaa (http://kororaa.org/), Chinstrap (http://chinstrap.alternating.net/), and RR4 Linux (http://www.lxnaydesign.net/). Gentoo takes no responsibility for them, but they are there if you trust them and find them useful. One thing they have in common is to underline that the end result is a valid Gentoo installation: you can tweak flags and re-emerge just like Gentoo promises.

I just helped a colleague with a Kororaa installation. My judgment is that he will benefit from being able to "emerge" this and that, and that he would not have tolerated me fiddling with his machine for several days. Maybe he will take further steps into Gentoo land, maybe not. Maybe in the end he will find that Gentoo is too demanding and go for a commercial boxed distribution. But even if that happens he has already learned a little about what Gentoo is.
Back to top
View user's profile Send private message
rafo
Apprentice
Apprentice


Joined: 18 Nov 2003
Posts: 160
Location: Sollentuna, Sweden

PostPosted: Wed May 31, 2006 9:53 am    Post subject: Reply with quote

Hi Hypnos,

just tossing in a link that explains ABI a little (I have not stumbled over the term before): http://en.wikipedia.org/wiki/Application_binary_interface
Back to top
View user's profile Send private message
rafo
Apprentice
Apprentice


Joined: 18 Nov 2003
Posts: 160
Location: Sollentuna, Sweden

PostPosted: Sat Jun 03, 2006 12:14 pm    Post subject: Reply with quote

So far I have been thinking of a p2p scheme for sharing trustable binaries. However, there could really be different distribution mechanisms, suitable for different contexts.

Suppose that you want to deploy Gentoo within an organization (a company, a campus, a hospital, ..). Then it may in fact be cost effective to keep produced binaries on a central server, or a network of servers. These servers would of course be trusted within the organization. Fingerprints could still be checked "just in case".

When a host is being installed or updated it will send requests for binaries (produced under specified preconditions) to the server. If the binary is already available it is of course downloaded. If it is not yet produced the requesting host may handle the "no" response in two ways: Either fall back to local compilation and installation, or hang on to the request until the server has produced the binary and download it then. The latter mode may be implemented in different ways, perhaps a polling scheme is sufficient.

Such a scheme could work very well for, say, a school. Schools are often low-budget operations and may be reusing "outdated" and hence slow hardware. For the administrator of school computers it would be a pain to wait for every box to compile its own software. With a binaries server the first box would take some time to install (still better off than classical Gentoo since binaries are produced on a faster central server), and subsequent boxes would be install rapidly using binaries from the central server.

I am aware that putting binaries on a central server is nothing new--it can be done today, and is maybe done too. The nice thing is that it could be one mode of operation of the "bonus binaries" scheme; whereas p2p sharing could be another. What they have in common is the demand-driven "binaries factory" that allows payback on the investment in compile time while at the same time preserving the full flexibility of Gentoo.
Back to top
View user's profile Send private message
barrymac
Tux's lil' helper
Tux's lil' helper


Joined: 15 Jul 2004
Posts: 87

PostPosted: Sat Jul 28, 2007 5:18 pm    Post subject: P2P portage anyone? Reply with quote

How about a p2p portage system so no-one on the planet ever compiles something that's been compiled before by another gentooist?

I'm sure it wouldn't be the most difficult system to set up with bittorrent and some clever way to name torrents after their architectures and use flag options.

Probably the most popular stuff should be permanently seeded.
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Sat Jul 28, 2007 5:26 pm    Post subject: Reply with quote

merged above post here.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
sugar
Guru
Guru


Joined: 07 Aug 2004
Posts: 579
Location: Morrinsville, New Zealand

PostPosted: Sun Aug 05, 2007 4:38 am    Post subject: Why not enable portage to download tarballs as torrents? Reply with quote

It would save on having to depend on mirrors.
_________________
He who calls for full employment calls for war!
Back to top
View user's profile Send private message
sirdilznik
l33t
l33t


Joined: 28 Apr 2005
Posts: 731

PostPosted: Sun Aug 05, 2007 4:41 am    Post subject: Reply with quote

Maybe because then it would force someone to also have a torrent client. I think minimum requirements is a good thing for portage. Also which client would you choose as default? I guess it could be added with a USE flag.

Just my thoughts.
Back to top
View user's profile Send private message
sugar
Guru
Guru


Joined: 07 Aug 2004
Posts: 579
Location: Morrinsville, New Zealand

PostPosted: Sun Aug 05, 2007 4:44 am    Post subject: Reply with quote

sirdilznik wrote:
Maybe because then it would force someone to also have a torrent client. I think minimum requirements is a good thing for portage. Also which client would you choose as default? I guess it could be added with a USE flag.

Just my thoughts.


libtorrent, and rtorrent if necessary, and you'd start portage as a service I suppose, so it would act as a client.
_________________
He who calls for full employment calls for war!
Back to top
View user's profile Send private message
Sadako
Advocate
Advocate


Joined: 05 Aug 2004
Posts: 3788
Location: sleeping in the bathtub

PostPosted: Sun Aug 05, 2007 6:55 am    Post subject: Reply with quote

It would not save on depending on mirrors, it would just lighten the load on them.
You would still have to ensure that there are seeds for each tarball available at all times.

It would also involve setting up a dedicated tracker, and you would need to publish a new torrent for every single version bump or new ebuild (although I'm sure this could be automated).

Another point is that the majority of files/tarballs on the mirrors are quite small, and in such cases only making them available as torrents simply wouldn't make sense.

There is also the issue of the amount of work it would take to add it to portage.

It's not a terrible idea, and I'd be surprised if it wasnt at least discussed before, but overall I don't think it would really be worthwhile implementing.
_________________
"You have to invite me in"
Back to top
View user's profile Send private message
codergeek42
Bodhisattva
Bodhisattva


Joined: 05 Apr 2004
Posts: 5142
Location: Anaheim, CA (USA)

PostPosted: Sun Aug 05, 2007 7:26 am    Post subject: Reply with quote

Moved from Off the Wall to Gentoo Chat.
_________________
~~ Peter: Programmer, Mathematician, STEM & Free Software Advocate, Enlightened Agent, Transhumanist, Fedora contributor
Who am I? :: EFF & FSF
Back to top
View user's profile Send private message
djdunn
l33t
l33t


Joined: 26 Dec 2004
Posts: 751
Location: Arrakis

PostPosted: Sun Aug 05, 2007 9:22 am    Post subject: Reply with quote

I think that certain files say over 50 megs a torrent could be an option, much like using FTP is an option.

I'm sure some of the mirrors would be able to handle seeding also,

for example when new versions of KDE or OOo or whatnot come out having a torrent as an option would probably help lessen the demand on the mirrors.

but again i don't think that every single file would benefit from being a torrent but some could benefit greatly.
_________________
A process cannot be understood by stopping it. Understanding must move with the flow of the process, must join it and flow with it.

-The First Law of Mentat
Back to top
View user's profile Send private message
Telexen
Apprentice
Apprentice


Joined: 09 Oct 2005
Posts: 189

PostPosted: Sun Aug 05, 2007 9:57 am    Post subject: Reply with quote

I don't like it. There are things torrents are good for and there are things they're not good for.

You're not going to lighten the load on portage servers that much with torrents - even from big ones. And I'll be able to download the large files I need MUCH faster with the side effect of a little extra server load than I would with torrents.

I'm not sure why people think torrents are the answer to everything. I've downloaded my share linux distros and such with torrents, and have never seen a single download max out my 3mbit cable connection - yet I always do downloading large files from a portage server.
Back to top
View user's profile Send private message
nixnut
Bodhisattva
Bodhisattva


Joined: 09 Apr 2004
Posts: 10974
Location: the dutch mountains

PostPosted: Sun Aug 05, 2007 10:43 am    Post subject: Reply with quote

merged above seven posts here.
_________________
Please add [solved] to the initial post's subject line if you feel your problem is resolved. Help answer the unanswered

talk is cheap. supply exceeds demand
Back to top
View user's profile Send private message
sugar
Guru
Guru


Joined: 07 Aug 2004
Posts: 579
Location: Morrinsville, New Zealand

PostPosted: Mon Aug 06, 2007 4:01 am    Post subject: Reply with quote

nixnut wrote:
merged above seven posts here.


Thanks nixnut. I didn't realise that this seems to come up regularly. I should browse Gentoo Chat more often eh.
_________________
He who calls for full employment calls for war!
Back to top
View user's profile Send private message
nwmcsween
n00b
n00b


Joined: 25 May 2007
Posts: 41

PostPosted: Wed Aug 08, 2007 12:17 am    Post subject: Reply with quote

For something that can be attained without a large overhaul why not just have solid releases such as kernel 2.6.22 and create deltas for any of the updates to said kernel? KDE does it SUSE does it.
Back to top
View user's profile Send private message
Corona688
Veteran
Veteran


Joined: 10 Jan 2004
Posts: 1204

PostPosted: Wed Aug 08, 2007 5:27 am    Post subject: Reply with quote

nwmcsween wrote:
For something that can be attained without a large overhaul why not just have solid releases such as kernel 2.6.22 and create deltas for any of the updates to said kernel? KDE does it SUSE does it.
That could get really complicated as more and more patches get added on, patching the patched patch of the patch; they already get dozens as is. Suse doesn't have to maintain more than one kernel, either.

It'd save in downloading kernel images if that could be conquered, but really, how many kernel tarballs do users download? Might there be better savings to be had improving something else? Maybye those ginormous KDE tarballs could be diffed.
_________________
Petition for Better 64-bit ATI Drivers - Sign Here
http://www.petitiononline.com/atipet/petition.html
Back to top
View user's profile Send private message
nwmcsween
n00b
n00b


Joined: 25 May 2007
Posts: 41

PostPosted: Wed Aug 08, 2007 6:40 am    Post subject: Reply with quote

Corona688 wrote:
nwmcsween wrote:
For something that can be attained without a large overhaul why not just have solid releases such as kernel 2.6.22 and create deltas for any of the updates to said kernel? KDE does it SUSE does it.
That could get really complicated as more and more patches get added on, patching the patched patch of the patch; they already get dozens as is. Suse doesn't have to maintain more than one kernel, either.

It'd save in downloading kernel images if that could be conquered, but really, how many kernel tarballs do users download? Might there be better savings to be had improving something else? Maybye those ginormous KDE tarballs could be diffed.

You don't patch a patch of a patch you patch the base with the update not patch the patch with the patch. Simply have a base and patch it every new release . e.g xorg 1.1.1.1 -> xorg 1.1.1.1-rc1-patch then xorg 1.1.1.1 -> xorg 1.1.1.1-rc2-patch simple.
Back to top
View user's profile Send private message
Roman_Gruber
Advocate
Advocate


Joined: 03 Oct 2006
Posts: 3796
Location: Austro Bavaria

PostPosted: Sat Aug 11, 2007 12:15 am    Post subject: Reply with quote

ever heard of sabayon linux?
Back to top
View user's profile Send private message
minervaix
n00b
n00b


Joined: 03 Aug 2007
Posts: 9
Location: Melbourne Australia

PostPosted: Sat Sep 15, 2007 6:43 am    Post subject: gentella network Reply with quote

After recently installing gentoo on one of my boxes using all the packages from another gentoo machine via emerge -k $package with my PKGDIR set, and installing a full system with xorg and kde in a couple of hours. I decided all those gentoo users out there with buildpkg in their make.conf should set up some P2P software so we can all make use of pre built packages, especially good for the i need it NOW!! situations, we can call it the GENTELLA network. whatda think?
Back to top
View user's profile Send private message
omp
Retired Dev
Retired Dev


Joined: 10 Sep 2005
Posts: 1018
Location: Glendale, California

PostPosted: Sat Sep 15, 2007 7:07 am    Post subject: Re: gentella network Reply with quote

minervaix wrote:
whatda think?

That it would be easy to inject malicious code into the source prior to building something.
_________________
meow.
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5288

PostPosted: Sat Sep 15, 2007 7:10 am    Post subject: Reply with quote

Merged the above two threads.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6  Next
Page 5 of 6

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum