Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Are we missing some easy optimizations (LDFLAGS)?
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Wed Sep 22, 2004 6:34 pm    Post subject: Are we missing some easy optimizations (LDFLAGS)? Reply with quote

I know how much everyone here loves optimized software. This is why I was surprised to read today on the GNOME mailing list that Ubuntu is taking advantage of some optimization opportunities that I think Gentoo is missing out on.

See these two posts:
http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00377.html
http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00381.html

The LDFLAG in question is -O1, which according to Ulrich Drepper's paper, "How to Write Shared Libraries" is an optimization that should decrease application start times. You can try this out for yourself by having a line in /etc/make.conf that reads:
Code:
LDFLAGS="-Wl,-O1"


Feel free to share your results with this, as I'm curious to know if it has any positive or negative effects. Anyone more familiar with ld and LDFLAGS know if this would actually be useful? I assume others would be interested in doing this for Gentoo.

See my bug report here:
http://bugs.gentoo.org/show_bug.cgi?id=65002

[edit] This post from last year might be interesting:
LDFLAGS Central
http://forums.gentoo.org/viewtopic.php?t=67777&highlight=ldflags

How come this idea seemed to fizzle out? Does it really have that little effect? I don't know how many people here have tried Ubuntu yet, but I was impressed with the speed, especially considering it's a binary distribution. I know I found myself curious as to how they made their binary distribution so snappy...

See lower in the thread for some information on how this works from Ulrich Drepper's report.


Last edited by aethyr on Wed Sep 22, 2004 10:49 pm; edited 6 times in total
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2159
Location: Toledo, OH, USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Wed Sep 22, 2004 7:04 pm    Post subject: Reply with quote

aethyr,
it seems to me that you could very easily just put a line under the CFLAGS option and it should be passed to gcc. Try it out and let us know if it works. If it does then also let us know if it actually does improve startup times. I have steered clear of prelinking because of all the horror stories I hear on the forum.
_________________
Please read the Forum Guidelines.
* | www.gayroughnecks.com | *
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Wed Sep 22, 2004 7:08 pm    Post subject: Reply with quote

curtis119 wrote:
aethyr,
it seems to me that you could very easily just put a line under the CFLAGS option and it should be passed to gcc. Try it out and let us know if it works. If it does then also let us know if it actually does improve startup times. I have steered clear of prelinking because of all the horror stories I hear on the forum.


As far as I understand it, this isn't the same as prelinking [1]. Can you just add an LDFLAGS line to make.conf (which supposedly is supported in portage? I'm not sure)? Like I said, if this is a no brainer optimization (i.e. only benefits, no problems), then shouldn't this be made the default in Gentoo? Note, I'm not sure if this is a no brainer optimization, or even an optimization at all.

After reading that post from Jeff Waugh about how they do this on Ubuntu, I'm mainly looking for feedback from people more knowledgeable than myself on this issue. Ubuntu is surprisingly fast for something based on Debian, I'm wondering if this doesn't have something to do with it. If this optimization does effectively reduce application start up time, how is it not more well known amongst the Gentoo crowd? It seems that in all the worrying about CFLAGS people seemed to have glossed over LDFLAGS altogether. Perhaps they're not effective, or maybe people just really don't understand them or are not aware of them?

[1] From the first reference I gave in my original post: "it'd also be nice to get GNOME optimising the linker hash tables by default. For the majority going "huh?" out there, it's basically a way of making load times shorter without resorting to the evil of prelinking."


Last edited by aethyr on Wed Sep 22, 2004 7:57 pm; edited 3 times in total
Back to top
View user's profile Send private message
Syntaxis
Guru
Guru


Joined: 28 Apr 2002
Posts: 511
Location: London, UK

PostPosted: Wed Sep 22, 2004 7:36 pm    Post subject: Reply with quote

Given Ubuntu's approach involves them contributing such improvements back to Debian as soon as possible, the easiest thing to do is probably just to keep an eye on the Gnome packages in Experimental.

Oh, and quite a few (all?) of the patches they submit to the Debian BTS are also indexed here. Probably worth watching that space as well...
_________________
The Debian User Forums - help them grow!
Back to top
View user's profile Send private message
asph
l33t
l33t


Joined: 25 Aug 2003
Posts: 737
Location: Barcelona, Spain

PostPosted: Wed Sep 22, 2004 7:51 pm    Post subject: Reply with quote

curtis: you mean we could use LDFLAGS="" in our make.conf? will gcc take those?
_________________
gentoo sex is updatedb; locate; talk; date; cd; strip; look; touch; finger; unzip; uptime; gawk; head; emerge --oneshot condom; mount; fsck; gasp; more; yes; yes; yes; more; umount; emerge -C condom; make clean; sleep
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2159
Location: Toledo, OH, USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Wed Sep 22, 2004 7:58 pm    Post subject: Reply with quote

nastassja wrote:
curtis: you mean we could use LDFLAGS="" in our make.conf? will gcc take those?


Yes exactly. I wasn't very clear about it. But the link that aethyr posted at the top of this thread had that as an option posted by a dev so it should work. Of course that was a year ago and portage has changed drastically since then so who knows?
_________________
Please read the Forum Guidelines.
* | www.gayroughnecks.com | *
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Wed Sep 22, 2004 8:11 pm    Post subject: Reply with quote

I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.

*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*

Yeah, it seems faster so now I will try it with Firefox later.
Back to top
View user's profile Send private message
Ruzbeh
Apprentice
Apprentice


Joined: 23 Jun 2004
Posts: 223

PostPosted: Wed Sep 22, 2004 8:18 pm    Post subject: Reply with quote

Pwnz3r wrote:
I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.

*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*

Yeah, it seems faster so now I will try it with Firefox later.


Are you serious?
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2159
Location: Toledo, OH, USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Wed Sep 22, 2004 8:35 pm    Post subject: Reply with quote

Pwnz3r wrote:
I'm going to check it out with Beep Media Player. I really don't have anything to compile that takes a long time to start, but this might give me some sort of idea.

*Time passes as the end of the compilation grows nearer and Pwnz3r eagerishly waits.*

Yeah, it seems faster so now I will try it with Firefox later.


Cool Pwnz3r, let us know if it works with firefox. I would use it for mozilla for sure. Also, can you paste the line from the compilation that shows the LDFLAGS being used?
_________________
Please read the Forum Guidelines.
* | www.gayroughnecks.com | *
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Wed Sep 22, 2004 8:47 pm    Post subject: Reply with quote

Yeah, I'll post what shows that it is being used.

Code:
# While compiling aterm
checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe  -Wl,-O1 -Wl,-z,now) works... yes


So it does indeed accept them, as it is GCC-based and not portage-based. :)
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2159
Location: Toledo, OH, USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Wed Sep 22, 2004 9:28 pm    Post subject: Reply with quote

Pwnz3r wrote:
Yeah, I'll post what shows that it is being used.

Code:
# While compiling aterm
checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe  -Wl,-O1 -Wl,-z,now) works... yes


So it does indeed accept them, as it is GCC-based and not portage-based. :)


sweet sweet sweet! I have to try this out now!
_________________
Please read the Forum Guidelines.
* | www.gayroughnecks.com | *
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Wed Sep 22, 2004 9:45 pm    Post subject: Reply with quote

Pwnz3r wrote:
Code:
checking whether the C compiler (gcc -O2 -march=athlon-xp -mfpmath=sse -msse -mmmx -m3dnow -pipe  -Wl,-O1 -Wl,-z,now) works... yes


Which of those are from your LDFLAGS in /etc/make.conf?

The reason I ask is because "-z now" doesn't seem like it would be universally beneficial according to taviso's post.
Quote:
-z now
Lazy binding is really clever, rather than loading all shared code into memory at runtime, the dynamic loader locates them, and just keeps track of it, when a reference is made to the shared code, then it is loaded memory. clever eh? this saves some memory, and speeds up startup at the expense of runtime performance. Using -z now disables lazy binding, which means slower startup, possibly more (although consistent) memory usage, but better runtime performance.
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Wed Sep 22, 2004 9:58 pm    Post subject: Reply with quote

Hmm.... I switched it to LDFLAGS="-Wl,-O1" because it also included $LDFLAGS before, so it must have a been a "feature creep" or something. heh
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Wed Sep 22, 2004 10:24 pm    Post subject: Reply with quote

Just accumulating info as I go along....

From here:
http://svn.debian.org/viewcvs/pkg-gnome/www/package-status.html?rev=1293
Quote:
Ensure libraries don't depend on undefined symbols by using LDFLAGS += -Wl,-z,defs and make things easier/faster for the dynamic loader by linking them using LDFLAGS += -Wl,-O1 (this makes sense for executables as well). See Ulrich Drepper's paper "how to write shared libraries" for background.


Ulrich Drepper's paper can be found here:
http://people.redhat.com/drepper/dsohowto.pdf

And has this to say:
Quote:
Many linkers do not put special emphasis on selecting an appropriate table size. The GNU linker tries to optimize the hash table size for minimal lengths of the chains if it gets passed the -O option.
... snip ...
Changing any of the factors 'number of exported symbols', 'length of the symbol strings', 'number and length of common prefixes', 'number of DSOs', and 'hash table size optimization' can reduce the costs dramatically. In general the percentage spent on relocations of time the dynamic linker uses during startup is around 50-70% if the binary is already in the file system cache, and about 20-30% if the file has to be loaded from disk. It is therefore worth spending time on these issues and in the remainder of the text we will introduce methods to do just that. So far to remember: pass -O1 to the linker to generate the final product.


Last edited by aethyr on Wed Sep 22, 2004 10:47 pm; edited 1 time in total
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Wed Sep 22, 2004 10:38 pm    Post subject: Reply with quote

Could you tell me what flags to use? I am lost and also have nothing installed/working to read PDF files since I rarely run into them.
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Wed Sep 22, 2004 10:40 pm    Post subject: Reply with quote

Here's what I believe it should be used based on what I've read so far. In your /etc/make.conf, have a line that reads:
Code:
LDFLAGS="-Wl,-O1"


I'll put this at my top post of the thread as well.


Last edited by aethyr on Wed Sep 22, 2004 10:59 pm; edited 2 times in total
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Wed Sep 22, 2004 10:42 pm    Post subject: Reply with quote

Ah okay, that is what I have in there now so I suppose I was not lost. :)
Back to top
View user's profile Send private message
Zepp
Veteran
Veteran


Joined: 15 Mar 2004
Posts: 1246
Location: Ontario, Canada

PostPosted: Wed Sep 22, 2004 11:59 pm    Post subject: Reply with quote

anyone tested this more and noticed anymore differences in performance...
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Thu Sep 23, 2004 12:05 am    Post subject: Reply with quote

I used it with Zsh and Mplayer and both load faster.
Back to top
View user's profile Send private message
Zepp
Veteran
Veteran


Joined: 15 Mar 2004
Posts: 1246
Location: Ontario, Canada

PostPosted: Thu Sep 23, 2004 1:32 am    Post subject: Reply with quote

interesting I should give this a shot. does this matter at all on cpu or anything? will this work for any cpu and/or architecture?
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 4:06 am    Post subject: Reply with quote

Well, I just did some brief testing on nautilus 2.8.0. Using "time nautilus --browser" my cached times went from an average (over 10 startups) of about 0.8s to about 0.6s. Not too shabby. I'm going to do some more testing...

More results, totem over 10 startups went from an average of ~1.6s to ~1.45s. Again, not a ton, but still measureable.

More results from gftp. Startup time went from around 0.65s to just under 0.6s, a savings of 0.05s! ;)

I wonder what would happen if all the underlying dependencies were built with the flag? Would the improvement propogate down the toolchain?

Time to rebuild GTK!

[edit] Well, rebuilding gtk+ by itself didn't seem to help too much. I'm rebuilding gnome 2.8 to try to test a bit more...

[edit edit] It doesn't seem like changes propagate down the toolchain, so this might be a per-app thing.

[edit edit edit] Corrected benchmarks (Sharp eyes, dangle) ;)


Last edited by aethyr on Sat Sep 25, 2004 6:57 am; edited 5 times in total
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Thu Sep 23, 2004 4:22 am    Post subject: Reply with quote

Well, Fluxbox loads noticeably faster now. Good stuff. :D
Back to top
View user's profile Send private message
ender11782
n00b
n00b


Joined: 10 Aug 2004
Posts: 31

PostPosted: Thu Sep 23, 2004 4:30 am    Post subject: Reply with quote

I'm gonna try adding this flag to my make.conf; if you don't hear anything else from me, it didn't work. 8O
Back to top
View user's profile Send private message
nsahoo
l33t
l33t


Joined: 17 Jul 2003
Posts: 618

PostPosted: Thu Sep 23, 2004 5:05 am    Post subject: Reply with quote

ender11782 wrote:
I'm gonna try adding this flag to my make.conf; if you don't hear anything else from me, it didn't work. 8O


heh .. :) i hear ya man
Back to top
View user's profile Send private message
Ari Rahikkala
Guru
Guru


Joined: 02 Oct 2002
Posts: 370
Location: Finland

PostPosted: Thu Sep 23, 2004 5:12 am    Post subject: Reply with quote

If I understood http://mail.gnome.org/archives/desktop-devel-list/2004-September/msg00377.html correctly, the -Wl,-O1 basically speeds up doing the same thing - relocations - that prelinking makes it possible to skip completely... but does it in a cleaner way that doesn't break when you upgrade libraries.
_________________
<laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page 1, 2, 3, 4, 5, 6, 7, 8  Next
Page 1 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum