Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Gentoo is speed ... at least it use to be
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
barureddy
n00b
n00b


Joined: 07 Oct 2003
Posts: 48

PostPosted: Thu Aug 23, 2012 3:47 pm    Post subject: Gentoo is speed ... at least it use to be Reply with quote

There has always been a compromise of having a system that is fast vs one that is stable. As a long time gentoo user (2002, wow I just realized it has been 10 years since I built my first gentoo box from stage 1), the main reason I use gentoo vs any other distro was that gentoo was FAST, lean, and can be optimized for your particular system. Its founding is based on using an experimental fork of gcc (EGCS).

Today I feel Gentoo seems to have lost its speed heritage. Last night I was reading a Phoronix article about GCC 4.7 link-time optimization performance benchmarks and I was amazed of the potential this feature might have. Currently the stable gcc version is gcc 4.5.4 whose base was released over 2 years ago. GCC 4.6.3 is currently marked unstable. Even if GCC 4.6 were to be stabilized tomorrow, we would still be a major release behind with GCC 4.7.

For Gentoo to return to its speed roots, I think Gentoo needs to stay current with the stable release of GCC and have the unstable branch be based on the upcoming branch of GCC. GCC seems to have picked up its game and seems to be approaching ICC in terms of speed. Gentoo needs to better position itself to take advantage of GCC gains.

I know it is easy to spout all these idea and Gentoo is a small project vs Redhat or Ubuntu. I will be the first to admit I am no expert in linux especially in its programing. I am a structural biologist and use computers (even clusters) to figure out protein structures in order to solve many problems afflicting our lives. However, I am far from a linux novice.

I am appreciative of all the work the current and past developers do in maintaining and adding to the Gentoo distribution. I do not plan on leaving Gentoo anytime soon nor do I plan on stop sending in my occasional bug reports. Like many of you, I want Gentoo to be the best distro it can be.
Back to top
View user's profile Send private message
asturm
Developer
Developer


Joined: 05 Apr 2007
Posts: 8931

PostPosted: Thu Aug 23, 2012 5:07 pm    Post subject: Reply with quote

If you are prepared to try out LTO, you might as well add gcc to your package.keywords and package.unmask files or even add the toolchain overlay. Because it is heavily experimental.

Putting the upcoming gcc release into ~arch would mean to break builds for a huge part of the tree on a regular basis.
Back to top
View user's profile Send private message
aCOSwt
Bodhisattva
Bodhisattva


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Thu Aug 23, 2012 5:39 pm    Post subject: Reply with quote

Which GNU package is reported faster than anyone of its competitors ?
_________________
Back to top
View user's profile Send private message
PaulBredbury
Watchman
Watchman


Joined: 14 Jul 2005
Posts: 7310

PostPosted: Thu Aug 23, 2012 5:44 pm    Post subject: Reply with quote

See similar thread.
Back to top
View user's profile Send private message
Goverp
Veteran
Veteran


Joined: 07 Mar 2007
Posts: 1966

PostPosted: Fri Aug 24, 2012 9:06 am    Post subject: Reply with quote

This comment isn't intended to argue with anything else in this thread or related threads, just an aside.

A while back Phorinx ran a benchmark which showed Gentoo not significantly faster than other distributions. In particular they were testing on AMD64 cpus, which were pretty new then. Which probably explains the results. Since there was only one or maybe two versions of AMD64 about, tailoring gcc with march=native would actually have no effect. The only way say Ubuntu could compile the code would be with march=athlon or whatever was valid then, and that would be pretty much the same as native. As the hardware platform gets more mature, and therefore more versions arrive (phenom, Intel i7, etc) there's more mileage in march=native. I'm not too sure how granular the precompiled Linux distros go.

Of course, there's mileage in stripping out cruft from the kernel and desktop, which will save on memory and maybe application loading performance. With memory coming in Gb chunks, there's not a lot of paging about, and application loading is, of course, usually a one-time issue and generally factored out of benchmarks.
_________________
Greybeard
Back to top
View user's profile Send private message
ppurka
Advocate
Advocate


Joined: 26 Dec 2004
Posts: 3256

PostPosted: Fri Aug 24, 2012 1:41 pm    Post subject: Reply with quote

Gentoo used to be about speed long ago ~2000-2002 because of some gcc patches that it used by default. Today Gentoo is more about configurability, and in a sense hassle free upgrades.

A recent anecdote. I saw the kde packages from 4.9.0 in portage, and I noticed that the okular release can now annotate pdfs and save the annotations inside the pdf. Sure, enough, all of it "just worked" once I upgraded okular to the latest release. I attempted the same on ubuntu-12.04 - first added a backports ppa, then upgraded the okular package. Sure enough, it doesn't work even after the upgrade. And why is that? They decided to not compile okular against poppler-0.20, and there doesn't seem to be any sane way to get poppler-0.20 and a working ubuntu system at the same time.

Another recent anecdote. I suddenly decided I want samba on Gentoo. Enabled the USE flag, recompiled world, and there it is. Attempted the same on ubuntu-12.04 and guess what? The package is broken on ubuntu-12.04, and it's been so for several months already.

So yes, you can cry for gcc-4.7, but it is useless to me if half of what I try to do doesn't work. That said, there are reasons why gcc can not just be upgraded to the latest and greatest in Gentoo. The health of your Gentoo system crucially depends on it. It takes months and months before the problems with many packages are ironed out. If you want to follow the progress, look at the gcc porting bugs: gcc-4.6, gcc-4.7
_________________
emerge --quiet redefined | E17 vids: I, II | Now using kde5 | e is unstable :-/
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 6920

PostPosted: Fri Aug 24, 2012 6:32 pm    Post subject: Reply with quote

Speed is a by-product of flexibility, not of cflags ricing.
Back to top
View user's profile Send private message
dol-sen
Retired Dev
Retired Dev


Joined: 30 Jun 2002
Posts: 2805
Location: Richmond, BC, Canada

PostPosted: Sun Aug 26, 2012 8:04 pm    Post subject: Reply with quote

As already mentioned. It takes time to test most of the pkgs in the tree that they will work with a new compiler. gcc-4.7 is in the tree and has been going through some tinderbox runs to find the pkgs that fail to compile. There have already been some bugs submitted and fixed for 4.7 compatibility. After they have been fixed they still need proper testing to ensure they actually work, not just compile error free.

Best advice, if you want to test out the newest and greatest. Get a chroot or VM setup to do some testing and help get things fixed so it can make it's way into the unmasked testing stage and subsequently stabilized. When you find it stable enough for your liking, you can migrate your primary system to it.
_________________
Brian
Porthole, the Portage GUI frontend irc@freenode: #gentoo-guis, #porthole, Blog
layman, gentoolkit, CoreBuilder, esearch...
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Aug 27, 2012 6:21 am    Post subject: Re: Gentoo is speed ... at least it use to be Reply with quote

barureddy wrote:
There has always been a compromise of having a system that is fast vs one that is stable. As a long time gentoo user (2002, wow I just realized it has been 10 years since I built my first gentoo box from stage 1), the main reason I use gentoo vs any other distro was that gentoo was FAST, lean, and can be optimized for your particular system. Its founding is based on using an experimental fork of gcc (EGCS).

Today I feel Gentoo seems to have lost its speed heritage. Last night I was reading a Phoronix article about GCC 4.7 link-time optimization performance benchmarks and I was amazed of the potential this feature might have. Currently the stable gcc version is gcc 4.5.4 whose base was released over 2 years ago. GCC 4.6.3 is currently marked unstable. Even if GCC 4.6 were to be stabilized tomorrow, we would still be a major release behind with GCC 4.7.

For Gentoo to return to its speed roots, I think Gentoo needs to stay current with the stable release of GCC and have the unstable branch be based on the upcoming branch of GCC. GCC seems to have picked up its game and seems to be approaching ICC in terms of speed. Gentoo needs to better position itself to take advantage of GCC gains.

I know it is easy to spout all these idea and Gentoo is a small project vs Redhat or Ubuntu. I will be the first to admit I am no expert in linux especially in its programing. I am a structural biologist and use computers (even clusters) to figure out protein structures in order to solve many problems afflicting our lives. However, I am far from a linux novice.

I am appreciative of all the work the current and past developers do in maintaining and adding to the Gentoo distribution. I do not plan on leaving Gentoo anytime soon nor do I plan on stop sending in my occasional bug reports. Like many of you, I want Gentoo to be the best distro it can be.


If you want to make Gentoo fast, I suggest installing it on ZFS. The Adaptive Replacement Cache algorithm does wonders for performance.

Also, we fixed the main GCC 4.6 stabilization blocker a month or two ago. It will likely be stabilized as soon as the toolchain team signs off on it. Until then, you could keyword it locally on stable. That is what I did.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Mon Aug 27, 2012 3:45 pm    Post subject: Reply with quote

Unless you run a real critical system, it is no problem to use gcc-4.7.1 with -flto
Sure, many packages (about half of them) will not compile with -flto, but you realize this when you compile them :lol:
There is also some packages failing with gcc-4.7, so you should still keep a copy of gcc-4.6 for these. Currently, for me these are firefox and libreoffice. firefox had been fixed with gcc-4.7, but now is broken again, and libreoffice compilation fails nonreproducible at random places - I was trying to nail it down for hours and was surprised that with gcc-4.6 there were not such issues.
Back to top
View user's profile Send private message
jdhore
Retired Dev
Retired Dev


Joined: 13 Apr 2007
Posts: 106

PostPosted: Mon Aug 27, 2012 8:06 pm    Post subject: Reply with quote

Perhaps GCC 4.7.x shouldn't be hardmasked, but IMO, SO MUCH STUFF breaks with it, even without -flto that maybe it should. with all those breakages, it certainly shouldn't be in "stable". IMO, the fact that Gentoo "stable" is quite up-to-date and yet quite stable, but I *CAN* install single unstable or hardmasked packages is one of Gentoo's biggest selling points.

IMO, unless you're on god-awful hardware (which would make using Gentoo a pain unless you're using a binhost) or you NEED insane performance (protip: unless you're a bank or a stock trading company you DON'T), using Gentoo for the speed means you're using Gentoo for the wrong reasons.

Gentoo is about the flexibilitiy and the cleanliness, not the speed (IMO). Sure the speed is a nice bonus, but that's not what it's about.
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Mon Aug 27, 2012 10:10 pm    Post subject: Reply with quote

Here are the tracker bugs outlining all known/reported issues with GCC 4.6 and GCC 4.7:

https://bugs.gentoo.org/show_bug.cgi?id=gcc-4.6
https://bugs.gentoo.org/show_bug.cgi?id=gcc-4.7
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Mon Sep 03, 2012 2:52 pm    Post subject: Reply with quote

I have 1,746 packages (including firefox and libreoffice) installed on my laptop, everything against gcc-4.7.1 without LTO. If anything fails to compile, it most probably needs a
Code:
CFLAGS="${CFLAGS} -fpermissive"
in /etc/make.conf.

Ok, I admit it, I am too busy right now to test with LTO, but for now the message is clear:
grep -A 1 lto /usr/portage/sys-devel/gcc/metadata.xml wrote:
<flag name="lto">Add support for link-time optimizations (unsupported, use
at your own risk
).</flag>

And finally, the performance benchmarks on phoronix do not really show any benefit of LTO. Quite the contrary. The Dhrystone 2 test show a better result, but the rest are nearly equal or even worse than without LTO.
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6747

PostPosted: Mon Sep 03, 2012 7:11 pm    Post subject: Reply with quote

Yamakuzure wrote:
I have 1,746 packages (including firefox and libreoffice) installed on my laptop, everything against gcc-4.7.1 without LTO.

firefox-15.0 now works again, but libreoffice-3.5.6.2 still fails. (This is not related with lto which I switched off for such huge binaries/libraries).
Quote:
If anything fails to compile, it most probably needs a
Code:
CFLAGS="${CFLAGS} -fpermissive"
in /etc/make.conf.

Maybe, but I do not like such hacks.
Quote:
And finally, the performance benchmarks on phoronix do not really show any benefit of LTO.

For most packages, the main advantage will be space-saving. IIRC the eix binary shrinks to 1/3 (or was it only 2/3? I forgot...) of its size with LTO, and this was rather typical.
Quote:
Quite the contrary. The Dhrystone 2 test show a better result, but the rest are nearly equal or even worse than without LTO.

Never trust a benchmark you haven't cheated by yourself: Every optimization can just break some pipeline and make things worse. But it will improve things on the average.
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Thu Sep 06, 2012 12:56 pm    Post subject: Reply with quote

mv wrote:
Never trust a benchmark you haven't cheated by yourself
Of course. I didn't mean to say "It's better without lto!", I was more like "Stop to hype lto, please, it's not worth the hassle (, yet)!" ;)
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
psycho
Guru
Guru


Joined: 22 Jun 2007
Posts: 534
Location: New Zealand

PostPosted: Sat Sep 15, 2012 1:45 pm    Post subject: Reply with quote

barureddy wrote:
Gentoo is speed ... at least it used to be

It still is...on this Core 2 Duo, at least. Having spent the last few years running mostly Slackware and Debian, I'm amazed at how much faster some of the larger applications are in Gentoo. I'm assuming it's march=native that achieves this, because I built generic x86_64 binaries in Slackware (for copying to various machines) and obviously Debian's packages are all generic. With monsters like libreoffice it may also be that useless libraries aren't being loaded, while they're all being dragged in by one-size-fits-all builds in other distros. In any case, whatever the reasons, libreoffice in Gentoo launches so quickly I don't get any time to admire Larry on the splash screen...easily four or five times faster than Debian (amd64 testing)...and it remains much more responsive (snappier feeling), as does the whole desktop, especially apps like inkscape and so on.

Oh, and Debian's using gcc 4.7.1. They're probably using it very conservatively, but the point is, my Gentoo ~amd64 systems with no rice at all (just plain old -O2) are a heck of a lot faster than equivalent Debian amd64 testing boxes; and slightly faster than Slackware64. So in addition to the flexibility and ease of maintenance and other advantages that Gentoo enjoys regardless of speed, Gentoo is still faster, too.
Back to top
View user's profile Send private message
bastibasti
Guru
Guru


Joined: 27 Nov 2006
Posts: 581

PostPosted: Tue Sep 18, 2012 12:08 pm    Post subject: Reply with quote

Speed is a subjective thing. I noticed that my system gets slower and slower, but I think big parts of it are also caused by the software, not the optimization.
Kernel gets bigger, Desktop gets heavier, of course there's nothing the gentoo developers can do about it.

I tried many distros. And believe me it doesnt take a day to come back to gentoo :-) Even Slackware which is really fast when installing as few as possible packages. (Xorg~300mb) but who wants 1 year old software :-))
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sat Sep 22, 2012 3:56 pm    Post subject: Reply with quote

bastibasti wrote:
Speed is a subjective thing. I noticed that my system gets slower and slower, but I think big parts of it are also caused by the software, not the optimization.
Kernel gets bigger, Desktop gets heavier, of course there's nothing the gentoo developers can do about it.

No they can't: they spend what little free time they can devote to the project maintaining a vast package tree, some of them doing some rather excellent work on toolchains and crossdev (which beats the pants off any other cross-compiler setup imo), and others usually helping maintain the packages upstream, officially or not. Gentoo gives the best bug-reports, since it's as near to vanilla upstream as possible by convention, and being source-based, the developers also have a lot more expertise with things like autotools, and how to do things the right way wrt patches (such as fixing the problem in the build-system and sending the patch upstream), than most.

Users can however do a lot about the way things are going, if motivated. Examples that spring to mind are Gentoo hardened, which a few years ago was dead in the water til Xake, Zorry, kernelOfTruth and a few others put effort into making it work on their own, and Walter's work on mdev replacement for udev.

Personally I'd love to get rid of consolekit and policykit on my boxen, and just have pam. A mate who's tried it said KDE really didn't like it, but I'd gladly put some effort in if someone who knows the software (I don't, nor do I code C++, but I do know C) wanted to take it on, and ofc they couldn't do something like that alone: they'd need testers and admin/network support, as well as at least one more person to help with code.

Quote:
I tried many distros. And believe me it doesnt take a day to come back to gentoo :-)

Heh, yeah: there's countless "I'm leaving gentoo" threads: standard practice is to say "Good luck, and see you when you come back" and they usually do. :)
Back to top
View user's profile Send private message
ryao
Retired Dev
Retired Dev


Joined: 27 Feb 2012
Posts: 132

PostPosted: Fri Oct 05, 2012 5:26 am    Post subject: Reply with quote

Yamakuzure wrote:
I have 1,746 packages (including firefox and libreoffice) installed on my laptop, everything against gcc-4.7.1 without LTO. If anything fails to compile, it most probably needs a
Code:
CFLAGS="${CFLAGS} -fpermissive"
in /etc/make.conf.


That is enough to block stabilization.
Back to top
View user's profile Send private message
Yamakuzure
Advocate
Advocate


Joined: 21 Jun 2006
Posts: 2280
Location: Adendorf, Germany

PostPosted: Tue Oct 09, 2012 10:19 am    Post subject: Reply with quote

ryao wrote:
Yamakuzure wrote:
I have 1,746 packages (including firefox and libreoffice) installed on my laptop, everything against gcc-4.7.1 without LTO. If anything fails to compile, it most probably needs a
Code:
CFLAGS="${CFLAGS} -fpermissive"
in /etc/make.conf.


That is enough to block stabilization.
It just means that the now strictly used rules for Name lookup (http://gcc.gnu.org/onlinedocs/gcc/Name-lookup.html) are somewhat relaxed to pre-4.7 standard.
_________________
Important German:
  1. "Aha" - German reaction to pretend that you are really interested while giving no f*ck.
  2. "Tja" - German reaction to the apocalypse, nuclear war, an alien invasion or no bread in the house.
Back to top
View user's profile Send private message
Caladon
n00b
n00b


Joined: 13 Oct 2012
Posts: 15
Location: Vasa/Vörå

PostPosted: Sat Oct 20, 2012 9:51 pm    Post subject: Reply with quote

I'm gaming on wine on gentoo and of course it's the unstable wine version and it's blown my socks off. It's speedy. Fglrx driver.

Gonna optimize gallium kms when I get back to my box and that is nice and stable. Darn thing I gotta feed kernel the radeon ucode again though.

So in my opinion Gentoo got best of both worlds. The speedy unstable stuff if you feel like having it or the bit slower stable stuff if you wanna have that instead.

Btw what does -fpermissive do? I know nothing.
Back to top
View user's profile Send private message
LiamOS
n00b
n00b


Joined: 06 Jun 2012
Posts: 61
Location: Ireland

PostPosted: Sun Oct 21, 2012 8:38 pm    Post subject: Reply with quote

Caladon wrote:
Btw what does -fpermissive do? I know nothing.

-fpermissive
Downgrade some diagnostics about nonconformant code from errors to warnings. Thus, using -fpermissive will allow some nonconforming code to compile.


It essentially makes the compiler less strict about what constitutes a valid program. Sometimes it can be a very good flag, but if you're using it, it can be pretty dodgy unless you know exactly why you're using it. As far as building a Gentoo system goes, I'd be of the opinion that it shouldn't ever be necessary, but I'm a noob, so take that with a mountain of salt.
_________________
CFLAGS=" -O999999"
Back to top
View user's profile Send private message
ArneBab
Guru
Guru


Joined: 24 Jan 2006
Posts: 429
Location: Graben-Neudorf, Germany

PostPosted: Thu Jan 03, 2013 10:18 am    Post subject: testing 4.7 with LTO Reply with quote

I just tested activating LTO on GCC 4.7 and I only had to disable it for 72 out of 1800 packages (I run a KDE desktop with quite many multimedia useflags enabled).

But to be frank: LTO is expected to become really useful with GCC 4.8, so my testing is more out of curiosity than anything else. I want to try using GCC with LTO at work, though. If you have a model which runs for a week, 10% better performance can be a lot :). That’s also why I tested it at home: I wanted to see it in a safe environment (well, my main home computer… the convenience of Gentoo sure dilluted my sense of “safe”) before hitting it with scientific code :)
_________________
Being unpolitical means being political without realizing it. - Arne Babenhauserheide ( http://draketo.de )

pkgcore: So fast that it feels unreal - by doing only what is needed.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum