View previous topic :: View next topic |
Author |
Message |
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Wed Sep 22, 2004 6:34 pm Post subject: Are we missing some easy optimizations (LDFLAGS)? |
|
|
I know how much everyone here loves optimized software. This is why I was surprised to read today on the GNOME mailing list that Ubuntu is taking advantage of some optimization opportunities that I think Gentoo is missing out on.
See these two posts:
http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00377.html
http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00381.html
The LDFLAG in question is -O1, which according to Ulrich Drepper's paper, "How to Write Shared Libraries" is an optimization that should decrease application start times. You can try this out for yourself by having a line in /etc/make.conf that reads:
Feel free to share your results with this, as I'm curious to know if it has any positive or negative effects. Anyone more familiar with ld and LDFLAGS know if this would actually be useful? I assume others would be interested in doing this for Gentoo.
See my bug report here:
https://bugs.gentoo.org/show_bug.cgi?id=65002
[edit] This post from last year might be interesting:
LDFLAGS Central
https://forums.gentoo.org/viewtopic.php?t=67777&highlight=ldflags
How come this idea seemed to fizzle out? Does it really have that little effect? I don't know how many people here have tried Ubuntu yet, but I was impressed with the speed, especially considering it's a binary distribution. I know I found myself curious as to how they made their binary distribution so snappy...
See lower in the thread for some information on how this works from Ulrich Drepper's report.
Last edited by aethyr on Wed Sep 22, 2004 10:49 pm; edited 6 times in total |
|
Back to top |
|
|
curtis119 Bodhisattva
Joined: 10 Mar 2003 Posts: 2160 Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.
|
Posted: Wed Sep 22, 2004 7:04 pm Post subject: |
|
|
aethyr,
it seems to me that you could very easily just put a line under the CFLAGS option and it should be passed to gcc. Try it out and let us know if it works. If it does then also let us know if it actually does improve startup times. I have steered clear of prelinking because of all the horror stories I hear on the forum. _________________ Gentoo: it's like wiping your ass with silk. |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Wed Sep 22, 2004 7:08 pm Post subject: |
|
|
curtis119 wrote: | aethyr,
it seems to me that you could very easily just put a line under the CFLAGS option and it should be passed to gcc. Try it out and let us know if it works. If it does then also let us know if it actually does improve startup times. I have steered clear of prelinking because of all the horror stories I hear on the forum. |
As far as I understand it, this isn't the same as prelinking [1]. Can you just add an LDFLAGS line to make.conf (which supposedly is supported in portage? I'm not sure)? Like I said, if this is a no brainer optimization (i.e. only benefits, no problems), then shouldn't this be made the default in Gentoo? Note, I'm not sure if this is a no brainer optimization, or even an optimization at all.
After reading that post from Jeff Waugh about how they do this on Ubuntu, I'm mainly looking for feedback from people more knowledgeable than myself on this issue. Ubuntu is surprisingly fast for something based on Debian, I'm wondering if this doesn't have something to do with it. If this optimization does effectively reduce application start up time, how is it not more well known amongst the Gentoo crowd? It seems that in all the worrying about CFLAGS people seemed to have glossed over LDFLAGS altogether. Perhaps they're not effective, or maybe people just really don't understand them or are not aware of them?
[1] From the first reference I gave in my original post: "it'd also be nice to get GNOME optimising the linker hash tables by default. For the majority going "huh?" out there, it's basically a way of making load times shorter without resorting to the evil of prelinking."
Last edited by aethyr on Wed Sep 22, 2004 7:57 pm; edited 3 times in total |
|
Back to top |
|
|
Syntaxis Guru
Joined: 28 Apr 2002 Posts: 511 Location: London, UK
|
Posted: Wed Sep 22, 2004 7:36 pm Post subject: |
|
|
Given Ubuntu's approach involves them contributing such improvements back to Debian as soon as possible, the easiest thing to do is probably just to keep an eye on the Gnome packages in Experimental.
Oh, and quite a few (all?) of the patches they submit to the Debian BTS are also indexed here. Probably worth watching that space as well... _________________ The Debian User Forums - help them grow! |
|
Back to top |
|
|
asph l33t
Joined: 25 Aug 2003 Posts: 741 Location: Barcelona, Spain
|
Posted: Wed Sep 22, 2004 7:51 pm Post subject: |
|
|
curtis: you mean we could use LDFLAGS="" in our make.conf? will gcc take those? _________________ gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep |
|
Back to top |
|
|
curtis119 Bodhisattva
Joined: 10 Mar 2003 Posts: 2160 Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.
|
Posted: Wed Sep 22, 2004 7:58 pm Post subject: |
|
|
nastassja wrote: | curtis: you mean we could use LDFLAGS="" in our make.conf? will gcc take those? |
Yes exactly. I wasn't very clear about it. But the link that aethyr posted at the top of this thread had that as an option posted by a dev so it should work. Of course that was a year ago and portage has changed drastically since then so who knows? _________________ Gentoo: it's like wiping your ass with silk. |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Wed Sep 22, 2004 8:11 pm Post subject: |
|
|
I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.
*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*
Yeah, it seems faster so now I will try it with Firefox later. |
|
Back to top |
|
|
Ruzbeh Apprentice
Joined: 23 Jun 2004 Posts: 223
|
Posted: Wed Sep 22, 2004 8:18 pm Post subject: |
|
|
Pwnz3r wrote: | I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.
*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*
Yeah, it seems faster so now I will try it with Firefox later. |
Are you serious? |
|
Back to top |
|
|
curtis119 Bodhisattva
Joined: 10 Mar 2003 Posts: 2160 Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.
|
Posted: Wed Sep 22, 2004 8:35 pm Post subject: |
|
|
Pwnz3r wrote: | I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.
*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*
Yeah, it seems faster so now I will try it with Firefox later. |
Cool Pwnz3r, let us know if it works with firefox. I would use it for mozilla for sure. Also, can you paste the line from the compilation that shows the LDFLAGS being used? _________________ Gentoo: it's like wiping your ass with silk. |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Wed Sep 22, 2004 8:47 pm Post subject: |
|
|
Yeah, I'll post what shows that it is being used.
Code: | # While compiling aterm
checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe -Wl,-O1 -Wl,-z,now) works... yes |
So it does indeed accept them, as it is GCC-based and not portage-based. |
|
Back to top |
|
|
curtis119 Bodhisattva
Joined: 10 Mar 2003 Posts: 2160 Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.
|
Posted: Wed Sep 22, 2004 9:28 pm Post subject: |
|
|
Pwnz3r wrote: | Yeah, I'll post what shows that it is being used.
Code: | # While compiling aterm
checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe -Wl,-O1 -Wl,-z,now) works... yes |
So it does indeed accept them, as it is GCC-based and not portage-based. |
sweet sweet sweet! I have to try this out now! _________________ Gentoo: it's like wiping your ass with silk. |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Wed Sep 22, 2004 9:45 pm Post subject: |
|
|
Pwnz3r wrote: | Code: | checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe -Wl,-O1 -Wl,-z,now) works... yes |
|
Which of those are from your LDFLAGS in /etc/make.conf?
The reason I ask is because "-z now" doesn't seem like it would be universally beneficial according to taviso's post.
Quote: | -z now
Lazy binding is really clever, rather than loading all shared code into memory at runtime, the dynamic loader locates them, and just keeps track of it, when a reference is made to the shared code, then it is loaded memory. clever eh? this saves some memory, and speeds up startup at the expense of runtime performance. Using -z now disables lazy binding, which means slower startup, possibly more (although consistent) memory usage, but better runtime performance. |
|
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Wed Sep 22, 2004 9:58 pm Post subject: |
|
|
Hmm.... I switched it to LDFLAGS="-Wl,-O1" because it also included $LDFLAGS before, so it must have a been a "feature creep" or something. heh |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Wed Sep 22, 2004 10:24 pm Post subject: |
|
|
Just accumulating info as I go along....
From here:
http://svn.debian.org/viewcvs/pkg-gnome/www/package-status.html?rev=1293
Quote: | Ensure libraries don't depend on undefined symbols by using LDFLAGS += -Wl,-z,defs and make things easier/faster for the dynamic loader by linking them using LDFLAGS += -Wl,-O1 (this makes sense for executables as well). See Ulrich Drepper's paper "how to write shared libraries" for background. |
Ulrich Drepper's paper can be found here:
http://people.redhat.com/drepper/dsohowto.pdf
And has this to say:
Quote: | Many linkers do not put special emphasis on selecting an appropriate table size. The GNU linker tries to optimize the hash table size for minimal lengths of the chains if it gets passed the -O option.
... snip ...
Changing any of the factors 'number of exported symbols', 'length of the symbol strings', 'number and length of common prefixes', 'number of DSOs', and 'hash table size optimization' can reduce the costs dramatically. In general the percentage spent on relocations of time the dynamic linker uses during startup is around 50-70% if the binary is already in the file system cache, and about 20-30% if the file has to be loaded from disk. It is therefore worth spending time on these issues and in the remainder of the text we will introduce methods to do just that. So far to remember: pass -O1 to the linker to generate the final product. |
Last edited by aethyr on Wed Sep 22, 2004 10:47 pm; edited 1 time in total |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Wed Sep 22, 2004 10:38 pm Post subject: |
|
|
Could you tell me what flags to use? I am lost and also have nothing installed/working to read PDF files since I rarely run into them. |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Wed Sep 22, 2004 10:40 pm Post subject: |
|
|
Here's what I believe it should be used based on what I've read so far. In your /etc/make.conf, have a line that reads:
I'll put this at my top post of the thread as well.
Last edited by aethyr on Wed Sep 22, 2004 10:59 pm; edited 2 times in total |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Wed Sep 22, 2004 10:42 pm Post subject: |
|
|
Ah okay, that is what I have in there now so I suppose I was not lost. |
|
Back to top |
|
|
Zepp Veteran
Joined: 15 Mar 2004 Posts: 1246 Location: Ontario, Canada
|
Posted: Wed Sep 22, 2004 11:59 pm Post subject: |
|
|
anyone tested this more and noticed anymore differences in performance... |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Thu Sep 23, 2004 12:05 am Post subject: |
|
|
I used it with Zsh and Mplayer and both load faster. |
|
Back to top |
|
|
Zepp Veteran
Joined: 15 Mar 2004 Posts: 1246 Location: Ontario, Canada
|
Posted: Thu Sep 23, 2004 1:32 am Post subject: |
|
|
interesting I should give this a shot. does this matter at all on cpu or anything? will this work for any cpu and/or architecture? |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
Posted: Thu Sep 23, 2004 4:06 am Post subject: |
|
|
Well, I just did some brief testing on nautilus 2.8.0. Using "time nautilus --browser" my cached times went from an average (over 10 startups) of about 0.8s to about 0.6s. Not too shabby. I'm going to do some more testing...
More results, totem over 10 startups went from an average of ~1.6s to ~1.45s. Again, not a ton, but still measureable.
More results from gftp. Startup time went from around 0.65s to just under 0.6s, a savings of 0.05s! ;)
I wonder what would happen if all the underlying dependencies were built with the flag? Would the improvement propogate down the toolchain?
Time to rebuild GTK!
[edit] Well, rebuilding gtk+ by itself didn't seem to help too much. I'm rebuilding gnome 2.8 to try to test a bit more...
[edit edit] It doesn't seem like changes propagate down the toolchain, so this might be a per-app thing.
[edit edit edit] Corrected benchmarks (Sharp eyes, dangle) ;)
Last edited by aethyr on Sat Sep 25, 2004 6:57 am; edited 5 times in total |
|
Back to top |
|
|
placeholder Advocate
Joined: 07 Feb 2004 Posts: 2500
|
Posted: Thu Sep 23, 2004 4:22 am Post subject: |
|
|
Well, Fluxbox loads noticeably faster now. Good stuff. |
|
Back to top |
|
|
ender11782 n00b
Joined: 10 Aug 2004 Posts: 31
|
Posted: Thu Sep 23, 2004 4:30 am Post subject: |
|
|
I'm gonna try adding this flag to my make.conf; if you don't hear anything else from me, it didn't work. |
|
Back to top |
|
|
nsahoo l33t
Joined: 17 Jul 2003 Posts: 618
|
Posted: Thu Sep 23, 2004 5:05 am Post subject: |
|
|
ender11782 wrote: | I'm gonna try adding this flag to my make.conf; if you don't hear anything else from me, it didn't work. |
heh .. i hear ya man |
|
Back to top |
|
|
Ari Rahikkala Guru
Joined: 02 Oct 2002 Posts: 370 Location: Finland
|
Posted: Thu Sep 23, 2004 5:12 am Post subject: |
|
|
If I understood http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00377.html correctly, the -Wl,-O1 basically speeds up doing the same thing - relocations - that prelinking makes it possible to skip completely... but does it in a cleaner way that doesn't break when you upgrade libraries. _________________ <laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|