The Future of Gentoo

[mrknowitall]

as i'm a big fan of gentoo, i wonder how it will prevail in the future. some days ago, i read a blog about gentoo's killing the world, if everybody in the world would run gentoo on their pc. i'm not a environmentalist, but he's got a point.

i'm aware of binhosts, but i this concept makes only sense if you have machines of similar hardware and setup. for all the gentoo people out there, this would be like cutting their balls, if we're reduced to the power of regular binary distros. but on the other hand, they do have some comfort. it took me about 20 minutes to install kubuntu on a laptop. every one of us knows, it takes quite a lot more time and cpu power to do that with gentoo. on the contrary, we have our use-flags and cflags, which eventually provide us with a well tailored system. but if you purchased one of those fancy netbooks as i did, gentoo is a real pain in the ass (but it works nicely, once done). i use my big pc to do the compiling, but it's just to complicated to move it all over. (i know, there is binhost and so forth, but it's not ment to do that, i think).

if you watch distrowatch on a regular basis, you will see, that gentoo is dropping. maybe due to the heavy competition among the countless distros. but i think the concept of use-flags and portage is still a great thing. but we need, i consider myself a power user, not developer nor admin, put some more magic in the process. so this is basically my idea:

* combine binhosts with userdefinend useflags in a cached way
* make use of peer-to-peer networking (unlikely, nobody else compiled the same package as i'm about to do)
* provide users with standard setups (if someone can write a wiki about setting up gentoo on some laptop, he could provide you with the config files)

is there a concept like this on the way? would anybody have some interest in participating?


just sharing thoughts...

alex

[AllenJB]

Gentoo is a from-source distro. Using it as the basis for a binary distro has been discussed many times already - to sumarise: It doesn't really work. The way Gentoo uses use flags and other settings (CFLAGS/LDFLAGS/etc) a binary distro would be a massive amount of work. I'm not going to go over the issues with this in detail - there are plenty of threads lying around that already do that.

To my knowledge, the only metric distrowatch uses for ranking distros is hits on their own site. It's a really bad way to measure the popularity of distros if you ask me. Personally I ignore it completely. You also have to be very careful about their reporting - they've proven in the past to be really bad at this - basically they're an ad-supported site that wants to get hits, and they're not afraid of sensationalist misreporting to do so.

I don't think you can possibly determine any kind of standard setups of Gentoo users - the power of Gentoo is the choice of exactly what gets installed. This means that every users system is different. In addition, this would again be a massive amount of work to create, continually update and distribute the stages (I guess you're basically talking about preconfigured stage4 images). I just don't see any value in this when you can head to the web and find the configuration for things which don't work "out of the box" 90% of the time.

Distributing even just config files doesn't work well because they change from version to version - and especially with X you have issues of different versions of drivers within the same version of X, as well as different choices of drivers for the same hardware (eg. nvidia vs nv vs nouveau).

[monsm]

Environmentalists have some strange ideas some times, don't listen to them. Haven't seen this blog, but presumably the idea was that by compiling all the the software yourself you use more electricity.

Use of electricity is a sign of a healthy society. We will need more power generation, not less in the future. If they are so afraid of pollution they should concentrate getting more nuclear power built (at least UK and USA has deficits in power generation, others are in a similar situation). Old coal and oil powered electricity generation pollutes, thats where the source is. Gentoo people might understand this better; we go to the source. Typical environmentalists being more concerned about indiviual behaviour than actually fixing a problem...

Mons

[d2_racing]

If they are so afraid of pollution they should concentrate getting more nuclear power built (at least UK and USA has deficits in power generation, others are in a similar situation)

Environmentalists don't like nuke power at all.

[MaximeG]

Hi,



What's the point with gentoo though ? I don't think Gentoo will never be the responsible of climat collapsing. Nor I think that Gentoo will ever be used by all the people worldwide. There are so many distributions around because each of them fits different needs, and I don't think everyone will become compiling enthusiastics before the next zillion years.

I totally agree with the fact that we should protect environment but I don't think putting us back in the stone age is the solution.
We need not to stop producing electricity but to produce it greenly, as I don't want to reduce my electricity consumption ; although, of course, I'm keen to reduce my waste of energy.

I hate ecologists' extremism.
What about :

- " Hey, why don't you stop using X server for it's using electricity more than your console ? Do it, or you will kill your own children !"

Isn't it a great idea to reduce Gentoo's use of power?

Regards,
Maxime

[MaximeG]

If they are so afraid of pollution they should concentrate getting more nuclear power built (at least UK and USA has deficits in power generation, others are in a similar situation)

Environmentalists don't like nuke power at all.

Yes, they want to replace them by wind or sun power. Which is a good idea, but stopping nuclear power plant right is stupid.
Because, well ... wait ... we need to build alternative power sources for wind/sun/stuff first ?

Regards,
Maxime

[d2_racing]

In fact, we need a solution for replacement of nuke power.

But, we have a lot of fuel in the North Pole and a lot of other stuff too.

[mrknowitall]

hi folks,

thanks for your thoughts!

the reason for throwing in the argument about destroying nature was merely a thing to point out, that gentoo, due to it's nature as source based distro needs a lot of cpu-power and thus energy for it's compiling process. i don't say gentoo responsible for global warming, since it's impact is just to little and i won't switch to another distro due to that fact, but nowadays, it's at least a thing to think about for some moments... that's all. sorry if i mislead you!

the thing i was trying to get at was basically answered by AllenJB.

he's right, no one should take distrowatch to seriously. due to the way they measure. And does it really matter if many people prefer distro A over B. But nevertheless, the more popular a distro is, the more community work is done, from which the quality of the whole experience with this distro benefits. this is a fact. and much more people are aware of ubuntu, fedora and opensuse. gentoo seems to be a real insider thing. which may be a good thing, too. i don't know...

but my main focus was to take a look at the compile or emerge process itself. if you could somehow combine the power of useflags and cflags with the speed of binary packages, this would just be an amazing experience. i use ccache and distcc to speed up the process. but still, it takes some time. on the other hand, for me it's worth, since i really appreciate the gentoo-experience. if we just could put more magic into portage or related tools like peer-to-peer-awareness, multi-source-download (like bittorrent), binary-caching-over-peers, then portage could, before starting the compile process, ask the other peers if there is already a finished build around fitting the local setup... so much for my basic theory. but maybe it's just to complicated...

@AllenJB: if you are aware of any of those discussions and if there is still one alive you know, i'd appreciate a link. thanks!


cheers!

alex

[djdunn]

not too long ago it was cow farts,

and tomorrow it will be dolphin farts or something... i donno

[loki99]

the reason for throwing in the argument about destroying nature was merely a thing to point out, that gentoo, due to it's nature as source based distro needs a lot of cpu-power and thus energy for it's compiling process.

Same thing can be said for...

* playing games
* video editing
* image editing
* rendering
* surfing the web
* watching porn

... and I could go on and on and on....

[slackline]

If they are so afraid of pollution they should concentrate getting more nuclear power built (at least UK and USA has deficits in power generation, others are in a similar situation)

Environmentalists don't like nuke power at all.

James Lovelock (the co-proposer along with Lyn Marguilis of Gaia Theory) is something of an environmentalist and is also a proponent of nuclear power (wrote an open letter to the UK government on just such a topic a few years back).

Besides which there are two forms of nuclear power, most environmentalists ignorantly lump them together and think its all bad, whilst acutally nuclear fusion is very clean (its fission that has lots of nasty by-products). Its just a shame that despite the various Taurus projects at Oxford and other places you currently need more energy put into the system than you get out of the fusion.

slack

[timeBandit]

1. A torus (a topological surface resembling a donut) describes the shape of the reaction chamber of a tokamak-style fusion reactor. Taurus (Latin for bull) is the name of a zodiacal constellation and a Ford Motor Company product.
2. More important, please return this discussion to the original topic. Otherwise, it will split, and one by-product of its fission will land in OTW.

[poly_poly-man]

I hate the environment... besides, by the time I see any real effects of what I'm doing, I'll be dead, so I won't care

[arch_dude]

Sure, gentto installation an maintanance require more CPU cycles. However, it should result in less CPU cycles during operation. This means that the tradeoff will depend on how often you do an emerge.

If we really cared, we could dramatically increase the efficiency of the build portion of the ebuild process by somehow cacheing the autoconf results. As of now, every build does its own separate scan for the system's capabilities. This is a horrid waste.

Gentoo may actually save resources overall, since it permits you to continue to use an old PC for a few extra months. if this effect extends the average time between purchases, it means each of us buys computers less often, which reduces the manufacturing load on the environment.

With Gentoo, I am muchmore likely to keep my system up to date and therefore I am not subject to old seurity vulnerabilities. a computer that is a victim of a successful security exploit contribute to that exploits impact on the environment, and that impact can be very large indeed.

The biggest negative impact of Gentoos for me is the fact that the power-save feature for prebuilt distros tends to work cosistently, while the powersave features for my Gentoo systems are flakey enough that I end up turning them off. This is especially true for Nvidia drivers. They work for awhile, and then some new 'emerge <options> world' causes the system to freeze up and I get frustrated and turn them off for another six months.

[mrknowitall]

sorry guys,

maybe we started off the wrong foot...

what i was trying to get at, that there should be some way of preserving gentoos power as a source based distro whilst saving compile time by optimizing the compile process. binary distris do the compilation for you, hence you have a fast install/update but to the cost of flexibility we all appreciate so much.

in your local network you can use distcc and ccache to speed up the process but not everyone as a server-farm at home (not to mention the energy you need ). therfore, if portage or some sort of plugin make use of the community, we could, that was the idea, benefit from each others cpu-time and power.

although i'm not really into portage-programming, it should be possible to pass the compile request first to a peer-to-peer network, where somebody probably already build such a package (with the same cflags and useflags you want) and download it using something like bittorrent. if there is nothing out there, then portage will resume as it does now, plus storing somewhere the information what it just compiled, in case someone on the network requests it later.

so again, the thing with the enviroment was ment to be something like an eyecatcher (of course, i do care about nature, but that wasn't main focus). but in times where people have netbooks (as i do now) you have the demand for less compile intensive linux. of course, you could switch to moblin (when it's done) or ubuntu netbook remix. maybe android or google's new thing chrome-os (wich will be again based on the linux-kernel) would be an option too.

but i really love gentoo. and after i did the compilation on my big pc i'm more than satisfied with the performance.

but if gentoo could make use of all cpus and pcs you guys have, this would be just a real cool thing. that is what i'm trying to get at.


cheers!

alex


btw.: and yes, i will keep my gentoo and if it burns earth! muhaha!!!

[krinn]

i think what you're looking for is sabayon (it's a gentoo fork with bin if i'm right).
or a light linux for notebook as http://www.thinkgos.com/index.html

You're idea isn't bad, but build should be sign or face security breach... At first view, the idea isn't really bad, at a 2nd one, i don't like the idea of using a pre-build package from someone else i don't know/trust.

I use distcc, and i don't really care about earth, if someone care, he could use a binary distro and remove xorg package to save energy. There also the good old and earth safe solution : http://pagesperso-orange.fr/jmonnet/mat ... inois2.jpg

[ScarletPimpFromHell]

If you were to add some options like this ...

Code: Select all

FEATURES="buildpkg"
PKGDIR="/home/binaries"

... to the make.conf file on the machine that does the majority of your compilation. For the sake of explaination lets call this host packagesvr.mydomain.au.

Then add some retrival options to the make.conf of new machines to be built ...

Code: Select all

FEATURES="getbinpkg"
PORTAGE_BINHOST="ftp://[user]:[password]@packagesvr.mydomain.au/../home/binaries/"

You would be cutting down on a lot of unnessessary compilation.

[krinn]

could also set distcc and promote your fastest computer to help the slowest one, distcc distribute jobs, but jobs could be distribute unfairly to computers...
as
@fastcomputer/4 localhost/1
set 4 jobs to fastcomputer and 1 to localhost, didn't test, but i think you could even remove localhost so only fastcomputer do the build (distcc will default to localhost on failure)

[d2_racing]

@poly_poly-man, off the record, your signature is so true, since I'm French Canadian hahahah

[mrknowitall]

@krinn: you trully got a point concerning the trust level. haven't thought about that yet. i don't think, i will go for a slide rule, since it only supports 1 bit color . but i do like this one: http://www.gjlay.de/UliStein-Rechner.jpg
translation: salesman->the newest computing generation, 16 colors on a hard drive including mouse.... | advisor: you should wait - in 6 months it will be half the money!

@ScarletPimpFromHell / @krinn: when the next kde comes out, i will probably go for the solution one of you proposed. but, again, the binhost can only provide packages according to it's own make.conf, right? but of course the make.conf on my big box differs by far from the one on my netbook... so i think distcc would be the most appropriate

@d2_racing / @poly_poly-man: sorry for you guys... on the other hand, british cooking isn't that bad, i survived there for some months

[slackline]

1. A torus (a topological surface resembling a donut) describes the shape of the reaction chamber of a tokamak-style fusion reactor. Taurus (Latin for bull) is the name of a zodiacal constellation and a Ford Motor Company product.

Ooops, fingers instead of brain doing the thinking there!

sorry guys,


what i was trying to get at, that there should be some way of preserving gentoos power as a source based distro whilst saving compile time by optimizing the compile process. binary distris do the compilation for you, hence you have a fast install/update but to the cost of flexibility we all appreciate so much.

Isn't this inherent/implicit in Gentoo, since only the USE flags that are required trigger the compilation of the features that are desired, rather than building packages with all options included?

[mrknowitall]

@slack---line:

cflags set the compiler options specific to your system. they can be more or less generic. also you control optimization among other things. on a distri like ubunutu you would have something like this:

Code: Select all

CFLAGS="-march=i686 -mtune=generic -O2 -pipe -fomit-frame-pointer"

since they need to support many platforms. on gentoo, you can compile for your specific needs. i just found out that you can use

Code: Select all

CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"

, where as GCC > 4.2.3 is required. hence leading to a build only supported by the same plattforms, but best suited, therefore better performance. see http://en.gentoo-wiki.com/wiki/Safe_Cflags for details.

useflags determine, what features of the software should be used/enabled. for example, kopete by default only supports jabber and irq (in kde 4.2), therefore you need to add oscar to your useflags in order to have the aim and icq enabled. portage will also retrieve dependencies for you. (wich is really cool).

intensive use of clfags and useflags result in a very specialized system, just for your needs. so gentoo can cut out all the overhead other distris need to support different plattforms and user needs, resulting in a rather thin and efficient system. (but don't forget to spare some memory and disk space for the compilation process)

today i played a bit with tmpfs mounting my /var/tmp/portage to ramdisk using this entry in fstab

Code: Select all

tmpfs     /var/tmp/portage    size=1G    0 0

. first results where amazing. it compiled amarok in almsot half the time. (tmpfs: 2min 40sec / without: 4min 20sec). i also think, that your harddrive can benefit from this in terms of less usage thus prolonged life.

but this is all done on your machine. in my theory, i would like to make use of the whole community instead of using soley distcc (wich has some restrictions, too. you cannot set the perfect cflags if you have two different systems). i asume, that in a big community this compilations already exists. but maybe sabayon is optimized enough. i never meassured the effency of my cflags over a more generic setting. maybe it's really not worth talking about and instead use sabayon for low performing plattforms...


but's really nice sharing thought's with you!!! thanks!

alex

[slackline]

@slack---line:

cflags set the compiler options specific to your system. they can be more or less generic. also you control optimization among other things. on a distri like ubunutu you would have something like this:

Code: Select all

CFLAGS="-march=i686 -mtune=generic -O2 -pipe -fomit-frame-pointer"

since they need to support many platforms. on gentoo, you can compile for your specific needs. i just found out that you can use

Code: Select all

CFLAGS="-march=native -O2 -pipe -fomit-frame-pointer"

, where as GCC > 4.2.3 is required. hence leading to a build only supported by the same plattforms, but best suited, therefore better performance. see http://en.gentoo-wiki.com/wiki/Safe_Cflags for details.

You must have missed that then since its in the handbook, and one of the fundamental steps in setting up your system

but this is all done on your machine. in my theory, i would like to make use of the whole community instead of using soley distcc (wich has some restrictions, too. you cannot set the perfect cflags if you have two different systems). i asume, that in a big community this compilations already exists. but maybe sabayon is optimized enough. i never meassured the effency of my cflags over a more generic setting. maybe it's really not worth talking about and instead use sabayon for low performing plattforms...

Are you getting at people sharing their compiled binaries with their choice of CFLAGS/USE? That would results in hundreds if not thousands of the same package built slightly differently, and how would you co-ordinate things such that no one ever duplicates the same package with the same configuration.

Or are you advocating the use of distcc over t'internet?

Either way both are going to increase network traffic which has a small amount of energy requirements and this would (likely) eventually offset any saving you are making by not having to compile everything on your own optimised machine by virtue of the increased bandwidth!!!

slack

[mrknowitall]

well, this is what discussion is all about...

i'm sure, i was repeating a bit common knowledge there... just for the sake of completness.

now, i tend to accept, that either you make yourself a nice and costumized gentoo on and for your box (maybe using various optimaztation techniques) or use something binary. at least sabayon seems to support core2 explicitly, so it doesn't have to be ubuntu, fedora, etc.

after sharing thought's with you guys, i understand now, why such a mechanismn as i proposed isn't alread part of gentoo. it would probably cause to much overhead and not worth the trouble. but a man can dream sometimes...


at least we had a little nice chat here.


thank you all!!!

(let's burn earth! )

alex

[el_Salmon]

Well, I'm afraid you suggest too many changes in Gentoo. Let's go to do it easy. Firstly, the basics. Gentoo Release Team could to begin to release a 2009 Installation CD or USB and then support a Wiki for the community.