Binary packages for stable users [long]

Message

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 12:04 pm

It is a fairly long post and I took some time to think about it and write it. So please read it to the end before trolling me. Thanks.

I'd like to see the possiblity for users that don't want to build every package to get packages off a binary repository. This would typically be people who has actual work to be done and can't afford the system to be unavailable for an extended period. Servers and average workstations users come to mind.

Some server administrators are already building their own binary repositories to cover their own needs. Some even open and contribute their repositories to the community.
http://forums.gentoo.org/viewtopic.php?t=87151

We would need to introduce a concept I call a PROFILE. Your gentoo profile would consist of your ARCH, CHOST, CFLAGS and USE variables.
Now if you had the same profile as some other user should be able to exchange binary packages without any problems. So obviously users who want to take advantage of my little scheme would have to gravitate around common profiles. With the users I have in mind I'd guess that you'd have to stick to more conservative settings. The settings used to produce the gentoo stage 3 system, could be used as the common ground.

Now you'd use the -k flag in emerge as it was intended. If a binary package for a matching profile exists, emerge would just download that and install it. If no matching package exist you'd just build it locally as you do now.

There are of course some technical issues that should be resolved before this could become a reality.

1) Unlimited number of profiles makes space requirements explode on server

We might have to limit the profiles to a small select handful. I don't belive that there will be a noticable difference between the performance of "all cflags i could find mentioned at the forums" and best practise flags. Of course we would have to have a profile for each processor family. Athlon-XP, P4, IA-64, Athlon64 and so forth, but apart from that someone insightful would have to deside what flags can stay and which will have to go from the cflags. For the USE flags I think it is pretty obvious that we would have to go with just one for each machine arch. However there shouldn't be any limits. If you change your profile to something clever, and can get enough people to do the same you have a community in your own right.
Plus 200GB discs are more than reasonably prized and can fit an amazing amount of data.

2) Binary packages would fill the server over time

A program would have to analize the hits the binary packages get on the server. Once a package hasn't gotten a hit for a while it should be deleted. So if somebody would force install e.g. gnome 2.0 they'd have to wait for a manual build.

3) With all the packages on the server they'd run out of bandwidth. Trying to serve all the users and sync each other at the same time.

They would if we relied on ftp or rsync technology. For large files something like BitTorrent would be a much better match. We would just need one central mirror to host the torrents. Then all who downloads would benefit from each others bandwidth. Gentoo users like to be up to date, so there would be a fair chance that they'd all download the same stuff at the same time (e.g. after the gnome 2.6 release) which is just the kind of situation that BT is created for.

4) Who would maintain all these binary packages.

I am open to suggestions. You and me, mostly. ArchLinux has an open ftp server where you can upload binary packages for review. Maybe gentoo could have the same thing. Perhaps emerge could be expanded to generate the binary packages (it already does this part), couple it with the profile metadata and upload it to your preferred upload mirror. Later a maintainer could approve the package and publish the torrent to the portage tree. Placements in the portage tree and naming conventions should leverage existing portage mechanisms for torrent discovery.

5) What if you want binary packages, but don't have a matching profile.

Like you do now, you could temporarily alter your USE and CFLAGS to adhere to a certain profil of your choice -- at your own risk of course!
A "fallback" i686 or even i586 repository could also be maintained, I don't know if that is really needed though.

6) How could profiles evolve ?

If the profile flag was adopted as a standard part of the make.conf you could have it point to a file with the flags in. Like athlon_xp_server.gentoo.profile.conf pentium_4_workstation.gentoo.profile.conf and so forth. Those files should be updated as part of the standard emerge update process then.

Thanks for taking the time to read my post. I am looking forward to hear your feedback

Regards
Lasse

klieber · Post by **klieber** » Sat Jan 03, 2004 12:38 pm

Lazlo wrote:We would need to introduce a concept I call a PROFILE. Your gentoo profile would consist of your ARCH, CHOST, CFLAGS and USE variables.
Now if you had the same profile as some other user should be able to exchange binary packages without any problems. So obviously users who want to take advantage of my little scheme would have to gravitate around common profiles.

There are something like 175 USE variables (never mind local use variables) and at least 2 dozen -march/-mcpu settings that I can think of off the top of my head. Then there are lots of other various GCC flags as well. (-O2, etc.) I tried calculating all the possible permutations that could be used there, but my computer melted before I could get a number...

If you restrict yourself to "common profiles" like you suggest, then you've just negated one of the primary benefits of Gentoo and turned it in to Just Another Binary Distribution. So why use Gentoo at that point? Why not use Debian or Fedora or Mandrake or Suse or any of the other 16 gazillion binary distros?

I personally think what you're trying to do is largely impractical if not impossible. But assuming for the sake of argument that it is possible, why would you do such a thing?

--kurt

madchaz · Post by **madchaz** » Sat Jan 03, 2004 12:40 pm

Hello Lazlo
Binary packadges have been a "hot button" topic for a bit on the forum here. Some people are agains it, some are for. However, I think you might want to look at a few things that are comming.

Some of the talk about binaries in portage is going on from the dev side as well, from what I understand. I think portage-ng that is comming will have the capabilities to introduce binary packadges.

As you pointed out, the bigest draw back of binaries is optimisation. You can't have ALL the possible use flags and cflags for everyone. Standards will have to be set in order for this to become possible.

Personaly, I find the idea of bin packadges interesting, but it as to remain an option, not something you have to use. Check out the portage dev page to see what's comming for this

klieber · Post by **klieber** » Sat Jan 03, 2004 12:51 pm

madchaz wrote:Some of the talk about binaries in portage is going on from the dev side as well, from what I understand. I think portage-ng that is comming will have the capabilities to introduce binary packadges.

Portage already supports binary packages via the -k option and the BINHOST variable in make.conf. That's been there for a while. There are also plans to offer limited mirrors of GRP packages. However, GRP is only intended for the initial installation of a computer -- there is no support (now or planned) for migrating an existing computer from KDE 3.1 to KDE 3.2, for example. Except of course by compiling from source.

There are no plans at the moment to offer any sort of official binary mirrors outside of what I mentioned above.

--kurt

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 1:10 pm

I want to make absolutly clear, that this is an enhancement, not a replacement of emerge.

As kurt mentions emerge already has the -k flags and the BINHOST setting.
The trick is that is doen't work as no BINHOST mirrors exists currently.

My proposal just seeks to clarify some of the technical issues there might be before it could actually work.

Of course if you do not call emerge with -k you do not "pollute" your system with prebuild binaries. So it is totally optional.

Regarding the enormous variation of USE and CFLAGs. I don't belive that we see all permutations like you claim. I'll be willing to bet on that 80%+ users running gentoo stable would fit into maybe 10 different profiles. I don't claim that you did. But I and a lot of other users roaming the forums just copied some sensible looking USE and CFLAGs and went on with that. Like I said, the settings used for building the GRP could be a good starting point to identifying the most common profiles.

If there isn't plans to improve this situation as kurt says. I think we as a community should think hard about how it could be improved.

So basically I belive it is a 80/20 thing. If you are part of the 20% that won't benefit, then be content that you are already content with the system at it works now. If you are part of the 80% that might benefit. Then look forward to upgrading KDE in 20 minutes instead of 20 hours.

I think the advantage to that is pretty obvious.

/Lasse

EvilTwinSkippy · Post by **EvilTwinSkippy** » Sat Jan 03, 2004 1:30 pm

Well, my answer it to cheat and create a new distro, based on Gentoo. Thus eliminating the flamewar issues, and leaving one free to experiment and modify as needed.

Gentoo is really intended to be a meta-distribution anyway. Why pollute the purity that is Gentoo with the troubles of maintaining a binary system. The developers don't want to hear about it. Most hard-core users don't want to hear about it.

The folks who are interested in it fall into 2 camps: those who need to maintain a butload of machines and don't want to have to reinvent the wheel, and people with crufty old computer that lack the cahoonas to perform a full gentoo build. Many just want something to unstuff onto a box that has half a chance of being kept up to date.

Neither of these camps are as interested in performance as they are in compadibility and relative stability. Rapid deployment and a simplified install process would be a plus.

My misguided effort is being documented at http://www.etoyoc.com/tao-linux

klieber · Post by **klieber** » Sat Jan 03, 2004 1:31 pm

Lazlo wrote:I'll be willing to bet on that 80%+ users running gentoo stable would fit into maybe 10 different profiles.

I can tell you for a fact this is wrong. From my experience, more folks run -march than -mcpu, so let's start there:

Code: Select all

-march=pentium3
-march=pentium4
-march=athlonXP
-march=athlon

There are also ppc, i686, etc., but I'd say 80% of users use one of those 4 -march settings. Then there are CFLAGS. Here, I'd agree that perhaps 50% of users use fairly vanilla ones. However, it's not 80% as you claim. Even of those 50% that do use vanilla ones, there can still be differences. The defaults used to be something like:

Code: Select all

-O3 -funroll-loops -fprefetch-loop-arrays -pipe

I know a number of people don't like -funroll-loops and -fprefetch-loop-arrays, so they get removed. Still more people change -O3 to -O2. (and, in fact, the defaults eventually got changed to -O2 as well). Some folks prefer to add -fomit-frame-pointer because they don't like useful debugging information.

Then, let's talk about USE flags. That's where I think you're really going to get hosed. Some folks prefer PHP, others prefer perl. Still more will be offended because I didn't mention python. So you either have to force folks to use +perl +php +python or you've severely limited the number of people who will find that particular binary package of any use. That's just one example -- remember there are over 175 USE flags at the moment, with more being added all the time. Most people prefer using either gnome OR kde. Are you going to force them to have USE flags for both environments? Or are you going to have separate packages for each environment? (in which case you've just all-but-doubled the amount of disk space that you'll need)

My point is there are *too many* variables to assume/hope/pray that 80% of your users are going to be using compatible settings. They won't be. So you would have to force people to use a subset of the settings available to them, which brings me back to my original question: Why Gentoo, then? Why not Debian, et al? Quite honestly, Debian does a far better job of being a binary distribution than we can (IMO) ever hope to be.

--kurt

klieber · Post by **klieber** » Sat Jan 03, 2004 1:36 pm

EvilTwinSkippy wrote:The folks who are interested in it fall into 2 camps: those who need to maintain a butload of machines and don't want to have to reinvent the wheel, and people with crufty old computer that lack the cahoonas to perform a full gentoo build.

Anyone who needs to maintain a "buttload" of machines is probably also going to have some specific wants and needs in terms of software. This is going to affect their USE flags at the very least and probably also their CFLAGS. Portage already offers support for these folks. You can create the package once and distribute it to all your machines via the -k option and BINHOST. It's not reinventing the wheel. It's setting up one web server and making a change to /etc/make.conf. That's it.

EvilTwinSkippy wrote:Many just want something to unstuff onto a box that has half a chance of being kept up to date.

There are already distributions that do this far better than Gentoo.

--kurt

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 2:20 pm

You see debian does a pretty bad job of optimizing their binaries for my (sub)architecture. I just tried the latest and greatest, and I was not impressed. ArchLinux offers some of what I want, but has a very little community. I simply don't see a reason that gentoo couldn't have a binary repository too. And I certainly don't see a reason to create another distribution to add such a relativly small feature.

You might be right that we'd be hard pressed to fit everyone into 10 categories. That was a random number anyway. I wish we could do something more scientific about it that just throw mud though. Take the most common archtectures, use a proven vanilla CFLAGs. A little too heavy USE flags isn't going to kill anyone.

Is there a place where stats are kept on gentoo users in general. Does emerge send any info back to the portage servers ? Isn't AthlonXP and P4 way the most popular ?

How much space does a compiled portage tree take up ?
You could go the other way and see how many binary repositories would fit on a $100 disc. P2P could take care of most of the bandwidth issue.

If gentoo had a binary edge I am sure more users would be attracted. Debian would have very little to offer against us then it seems to me. And as gentoo growth out of its little niece. More joe-average-user is attracted. People who are happy to stick within a predefined profile.

EvilTwinSkippy · Post by **EvilTwinSkippy** » Sat Jan 03, 2004 2:54 pm

Lazlo wrote: If gentoo had a binary edge I am sure more users would be attracted. Debian would have very little to offer against us then it seems to me. And as gentoo growth out of its little niece. More joe-average-user is attracted. People who are happy to stick within a predefined profile.

I think there is something to be said for Gentoo being content with the burgeoning user base it has already. I'm hooked with it in it's present state. Besides, I can't help but envision sliced flesh and a few gallons of blood when I envision what "binary edge' is supposed to mean.

I agree with Klieber, I was oversimplifying matters a bit. So, let me backtrack from a buttload and state enough to make it worth the hassle of setting up. Perhaps you could do the same with your Sub Architecture. Gentoo is the starting point, it doesn't have to be the ending point.

EvilTwinSkippy · Post by **EvilTwinSkippy** » Sat Jan 03, 2004 2:56 pm

klieber wrote:
EvilTwinSkippy wrote:The folks who are interested in it fall into 2 camps: those who need to maintain a butload of machines and don't want to have to reinvent the wheel, and people with crufty old computer that lack the cahoonas to perform a full gentoo build.
Anyone who needs to maintain a "buttload" of machines is probably also going to have some specific wants and needs in terms of software. This is going to affect their USE flags at the very least and probably also their CFLAGS. Portage already offers support for these folks. You can create the package once and distribute it to all your machines via the -k option and BINHOST. It's not reinventing the wheel. It's setting up one web server and making a change to /etc/make.conf. That's it.

EvilTwinSkippy wrote:Many just want something to unstuff onto a box that has half a chance of being kept up to date.
There are already distributions that do this far better than Gentoo.

--kurt

My apologies Kurt. I was having DotCom style pangs of "if you build it they will come...' '...PROFIT!'

--Sean

QPegasus · Post by **QPegasus** » Sat Jan 03, 2004 3:04 pm

With this debate, along with others, a popular arguement seems to be "well, if you want binary packages, go away and use another distro".

Well, I don't use gentoo because its source based, and I'm pretty sure I'm not alone. Infact, I'd rather use a binary distro, but none exist that match up with portage imo. Few that I've tried have been as easy to use and problem free as portage, and those that are either have a poor selection of available packages, or a heavily outdated selection of packages.

Thats the reason I use gentoo....along with the large and helpful community.

I dont see how it can hurt to have a binary repository with packages for the popular CPU's and sensible/common USE/CFLAGS. Obviously its impossible to have a binary available for every combination under the sun, and many would still compile things themself, but how are they losing out? Nothing will change for them, it'll still be the gentoo they know and love.

On the bandwidth issue - surely, adding binaries wont add as much as some say? I know that syncing would certainly use more, but the actual packages? When someone downloads one of the new binaries, arent they just downloading that instead of the source they would have downloaded anyway? Also (someone correct me if I'm wrong), arent sources usually larger than the binaries?

The way I see it, it would make gentoo appeal to a much larger userbase, aswell as making things easier for many current users. The rest wouldnt be affected and would continue the way they always have.

imo, its not a matter of "would it benefit gentoo?", its "is this possible?". The problems are, disk space, bandwidth and any costs associated with those. With so many potential contributers and packages, security would also be an issue. Probably the biggest one imo.

Just my 2 cents....but what do I know.

klieber · Post by **klieber** » Sat Jan 03, 2004 3:56 pm

QPegasus wrote:The problems are, disk space, bandwidth and any costs associated with those. With so many potential contributers and packages, security would also be an issue. Probably the biggest one imo.

You left out "support". That's probably as big as security.

As for disk space issues, we already have a 40GB+ repository. If you just picked one combination of architecture, cflags, use flags, etc. to support and compiled everything for that, I'd expect to see that take up another 30GB (rough swag).

Also, regarding the security aspect...imo, anyone who blindly installs binary packages created by unknown "community members" is quite simply asking for their machine to be trojaned. That's not a very feasible idea -- there is no way to ensure the integrity of the packages.

--kurt

DrNick · Post by **DrNick** » Sat Jan 03, 2004 4:17 pm

I for one, having been through rpm hell of redhat 6 and mandrake 6/7/8 chose gentoo for being the number one source based distro in the world IMHO. Not because I can chose whatever uber cflags when compiling, hell I'd use gentoo if it was the slowest distro out there, but just because doesn't break (...that easy

) as binary distros tend to do.

But maybe there's a middle road to this discussion, there're probably relative few packages that are a real pain to compile and maybe they could be stored somewhere or why not a comunity effort among those interested. Having a common naming scheme for the packages including the USE-flags and p2p sharing.

# emerge_p2p package_useflag1_useflag2 || emerge package

klieber · Post by **klieber** » Sat Jan 03, 2004 4:21 pm

DrNick wrote:But maybe there's a middle road to this discussion, there're probably relative few packages that are a real pain to compile and maybe they could be stored somewhere or why not a comunity effort among those interested.

That is already being done via the GRP. With the first release in 2004, GRP packages will be available for installation over the network.

However, GRP is only intended for new installations. Not for upgrading existing installations.

--kurt

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 10:00 pm

It seems like Daniel Robbins is on my side here, if that counts for anything.
http://www.gentoo.org/main/en/philosophy.xml

I can't seem to find the any plans on what portage-ng holds for the future. I'd be happy for a url, if anyone has it. But reading the philosophy page, I find it hard to belive that better support for binary packages is not planned. It is only THE single largest complaint from people who has looked at gentoo.

As for security and binary packages. In the end it is all about trust. You can assume that the binary packages are safe just as safe as you assume that nobody has injected backdoors into the code you are manually building. Like the recent case, that was fortunatly caught in the kernel development.
http://slashdot.org/article.pl?sid=03/1 ... 06&tid=185

It is a matter of procedures and engineering to ensure that the system stays as safe as possible from intruders. I am confident it can be done so it is both safe and a small hassle to add new binary packages and an even smaller hassle for maintainers to test and approve these packages.

With the amount of data P2P already serves today, I can't see how a 3-400Gb binary repository would be impossible to support at all. Hard drives are damn cheap. P2P allows for massive downloads without killing the server.
So it would require P2P rather than ftp and rsync and automation rather than manual labour, but impossible? I don't think so!

/Lasse

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 10:20 pm

Woops, I was a bit too sloppy seaching there. I found the portage-ng plans myself.
http://www.gentoo.org/proj/en/portage/p ... emspec.xml

They are very highlevel and seems to mention that it is a rewrite of the current version. Nothing about why the current is so bad, but we'll just have to assume that it doesn't adhere to the architechtural requirements outlined in the document.

The only thing it really say about features is this:

7. Delivery a feature set that addresses the needs of user community

Careful attention should be paid to the needs of our user community. What features are our users looking for in portage-ng? What kinds of problems are they interested in having portage-ng solve? These needs should be taken into account in the design.

-- which isn't a lot :)

Given the amount of talk about the missing binary packages and the philosophy page for gentoo, I still find it hard to belive that support wouldn't be added somewhere down the road.

Having read the requirements I'd be surprised if it got into the first version though. First version after a rewrite usually have less features than the old one :([/i]

EvilTwinSkippy · Post by **EvilTwinSkippy** » Sat Jan 03, 2004 10:42 pm

Lazlo wrote:It seems like Daniel Robbins is on my side here, if that counts for anything.
http://www.gentoo.org/main/en/philosophy.xml

I can't seem to find the any plans on what portage-ng holds for the future. I'd be happy for a url, if anyone has it. But reading the philosophy page, I find it hard to belive that better support for binary packages is not planned. It is only THE single largest complaint from people who has looked at gentoo.

As for security and binary packages. In the end it is all about trust. You can assume that the binary packages are safe just as safe as you assume that nobody has injected backdoors into the code you are manually building. Like the recent case, that was fortunatly caught in the kernel development.
http://slashdot.org/article.pl?sid=03/1 ... 06&tid=185

It is a matter of procedures and engineering to ensure that the system stays as safe as possible from intruders. I am confident it can be done so it is both safe and a small hassle to add new binary packages and an even smaller hassle for maintainers to test and approve these packages.

With the amount of data P2P already serves today, I can't see how a 3-400Gb binary repository would be impossible to support at all. Hard drives are damn cheap. P2P allows for massive downloads without killing the server.
So it would require P2P rather than ftp and rsync and automation rather than manual labour, but impossible? I don't think so!

/Lasse

In regards to tossing binary packages on P2P networks, I can name an awful lot of times where I pulled up a binary named something tantelizing, only to have a the download never complete because some key part was missing, or for it to turn out to be something completely different than what I was expecting. (No dear I was looking for information about the Hilton hotel in Paris... I swear!)

Running a large network let me tell you that the hard drive you throw in your computer at home are NOT the same thing one drops in a server. Server hard drives are almost exclusive SCSI. SCSI drives cost a good deal more, and are often installed in pairs for redundency. (Sore subject. I have users try to tell me all the time how easy and cheap it would be to all more drive space.) And then there is the whole issue of tape backup...

You have some good ideas, but at this point there are other fish to fry.

Lazlo · Post by **Lazlo** » Sat Jan 03, 2004 11:09 pm

Well the difference between the legal and illegal use of P2P is that in the legal version which i belive BitTorrent is a proponent of you have a backbone server that holds the actual file. So if you are alone in the world you'd still be able to pull the file off that one server albeit slower. If more than one server participates in the BT network you'd be able to download a bit from each server at the same time.
So using a P2P technology where there is an actual server holding the entire file the whole time is essential.

As to disc space. Ok, maybe running the operation off IDE discs are not a good idea. So maybe it is a bit more expensive that I would have it sound like. Still servers like ibiblio must have enormous capacities available.

If pressed hard the mainstray of packages would have to be just i686 optimized and more CPU intensive programs like most of the desktop stuff would be more fine tuned for your sub-architechture. I am not sure anyway that I can defend maintaining 15 super optimized packages of the vi editor anyway :P

I'll still see before I belive that the space we need can't be found though.

klieber · Post by **klieber** » Sun Jan 04, 2004 1:52 am

Lazlo wrote:But reading the philosophy page, I find it hard to belive that better support for binary packages is not planned. It is only THE single largest complaint from people who has looked at gentoo.

Perhaps you haven't read my previous posts. Portage *already* supports binary packages. You can install binary packages using the -k option of emerge. It works. It's supported. Now and today.

What is missing and *not* planned is a massive repository of Gentoo-produced binary packages.

--kurt

Lazlo · Post by **Lazlo** » Sun Jan 04, 2004 2:00 am

Perhaps you didn't read the philosophy page I linked to.
Here's an interesting quote:

It's important that our tools support binary packages, because binary packages are widely used and widely in demand in the Linux community. If our tools don't support binary packages, then we can't claim that our tools are designed to allow a user to do anything he or she might want to do. If we purposely choose to exclude binary support, then we are attempting to interfere with how users might choose to approach particular problems, by instead imposing our own will or view of how they should approach a problem. And if we do not build binary packages, then we are not taking any steps to ensure that our tools actually work well with binary packages, nor are we taking steps to ensure that others can build binary packages, nor are we able to *demonstrate* that our tools work well with binary packages. Besides these philosophical reasons, there are many practical reasons to create binary packages.

I realize that portage supports binary packages. In all my assumptions above you will see that I wish to build upon that functionality.

However that support isn't worth a dime without a repository. And one, I might add, that is updated more than once every year!

Kihaji · Post by **Kihaji** » Sun Jan 04, 2004 2:30 am

klieber wrote:There are something like 175 USE variables (never mind local use variables) and at least 2 dozen -march/-mcpu settings that I can think of off the top of my head. Then there are lots of other various GCC flags as well. (-O2, etc.) I tried calculating all the possible permutations that could be used there, but my computer melted before I could get a number...

If you restrict yourself to "common profiles" like you suggest, then you've just negated one of the primary benefits of Gentoo and turned it in to Just Another Binary Distribution. So why use Gentoo at that point? Why not use Debian or Fedora or Mandrake or Suse or any of the other 16 gazillion binary distros?

Actually, it seems as if most people dont care that much about optimization, but instead ease of use, so a common profile could most definately be created that would satisfy the majority of the users.

Cerement · Post by **Cerement** » Sun Jan 04, 2004 3:40 am

Several others have already mentioned that portage can already use a binary package, but this item popped out:

Lazlo wrote:This would typically be people who has actual work to be done and can't afford the system to be unavailable for an extended period. Servers and average workstations users come to mind.

Your system isn't unavailable during emerges. It's even available during most of the initial install. Between virtual consoles, terminal emulators, su and sudo and nfs and ssh and whatever, the compiling can easily happen in the background without interrupting anything else going on. Checking through the forums and people running stable systems have cronned "emerge sync && emerge -uUD world", people have sshed into their machines from across the continent and ran an emerge ...

EvilTwinSkippy · Post by **EvilTwinSkippy** » Sun Jan 04, 2004 4:18 am

Let me try to assemble what we all have said here so far:

First the issue is not about binary package support, portage has that, it's about creating a binary repository. Gentoo Alumni (myself included) generally cast a disparaging eye on the whole concept. We have a hard enough time keeping the sources straight, let alone figure out what canonical package was linked with what version of what library.

In order to successfully create and maintain a binary repository would require such a generalization of features that the result would be almost, but not entirely, unlike Gentoo. The maintenance of the binary repository would require the same time and resources required to maintain a standard binary Linux distro. You are just using Gentoo for a build environment.

Now, I do see merits to the "sub-distro" approach. I maintain my own for my server cluster. With a little communication, and some forethought a small community could come up with a microdistro that serves their specific needs.

Which brings us back to the original intent of Gentoo. It's not a distribution. It's a way to come up with your own. Now, if someone is particularly queer for a distribution that knits in KDE, Cups, Alsa, MySQL, and TCL with the default editor being EMACS, my little 'tao-linux' distro is perfect for you. Some other VI

Lazlo · Post by **Lazlo** » Sun Jan 04, 2004 7:35 pm

I certainly don't see why how this discussion can be summed up to that we should all create our own specialized binary repositories if that is needed.

I would sum it up as follows.

A binary repository would be a great addition to the gentoo world. Users would likely flock to join the gentoo banners, when given the prospect that an updated system could be installed in a matter of hours, rather than days.
The situation as it is now is absurd. Granted, you have the LiveCD that can provide you with a working system in fast. But since the first thing everybody does is to run emerge sync && emerge update. They soon discover that getting an updated system from a stage3 install is slower than it would have been from a stage2 install.
Also you see people like this guy:
http://forums.gentoo.org/viewtopic.php?t=120853
try to do twists in order not having his main CPU occupied with building all the time.
Or - I suddenly find my self urging to watch a divX movie. How long did you say I have to wait before totem and dependencies is ready ???

The reason the binary distos isn't good enough is that they are not compiled for my machine architechture, but rather one 5 or 10 years old. And if a binary install fails on one of the usual distros they can't handle a natural fallback to a build from source.

I outlined a few ideas off the top of my head on how some of the technical difficulties could be handled. Good response back on why it wasn't all as easy as I originally imagined it. So experiments will have to show how many binary repositories it would be feasable to handle and how many % of packages they should hold. My take is that when a package is marked as stable, a binary build should be provided.

Some claim that if I want binary packages I have chosen the wrong distro or that no support for binary packages is planned for the future.
I belive that drobbins proves the first statement wrong if the philosophy page is to be trusted.
http://www.gentoo.org/main/en/philosophy.xml

If this:
http://www.gentoo.org/proj/en/portage/p ... emspec.xml
is truly all the plans that currently exists of plans for portage-ng, then it is far to early to conclude anything on what amounts of support is or is not planned for improvements on binary packages.

But the plan does state that it is open for suggestions. So I hereby formally suggest that portage-ng should have excellent support for building and maintaining binary repositories.

Now I think I will go and see what thoughts the portage-ng developer group has on the concept.

Has anyone found links to further plans for portage-ng development or even the mailing list archives for the portage-dev mailing lists ?

Thanks for the feedback and the civilized tone so far.

Binary packages for stable users [long]

Binary packages for stable users [long]

Re: Binary packages for stable users [long]

Cheaters Answer

Re: Cheaters Answer

Binary Edges...

Re: Cheaters Answer

Binaries over P2P

Re: Binary packages for stable users [long]

Re: Binary packages for stable users [long]

Don't get in the way of a good rant...