Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Are we missing some easy optimizations (LDFLAGS)?
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  
Author Message
truekaiser
l33t
l33t


Joined: 05 Mar 2004
Posts: 801

PostPosted: Thu Sep 23, 2004 8:10 am    Post subject: Reply with quote

how much more memory does this eat up?
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Thu Sep 23, 2004 11:04 am    Post subject: Reply with quote

I notice no more RAM being eaten up on my box.
Back to top
View user's profile Send private message
wilho
Apprentice
Apprentice


Joined: 22 Jul 2002
Posts: 169

PostPosted: Thu Sep 23, 2004 11:31 am    Post subject: Reply with quote

So prelink is useless with this one?
Back to top
View user's profile Send private message
MaxDamage
l33t
l33t


Joined: 03 Jan 2004
Posts: 650
Location: Oviedo, Spain

PostPosted: Thu Sep 23, 2004 11:56 am    Post subject: Reply with quote

It's time of informing the gentoo devs about this?
_________________
La PDA de tungsteno
Back to top
View user's profile Send private message
neenee
Veteran
Veteran


Joined: 20 Jul 2003
Posts: 1786

PostPosted: Thu Sep 23, 2004 12:04 pm    Post subject: Reply with quote

this looks very interesting :wink:

i never used prelink because i did not want to run some separate
command to make it happen and indeed i was worried about things
messing up because i updated packages a lot.

i'll recompile some packages with it too.
Back to top
View user's profile Send private message
stahlsau
Guru
Guru


Joined: 09 Jan 2004
Posts: 584
Location: WildWestwoods

PostPosted: Thu Sep 23, 2004 12:16 pm    Post subject: Reply with quote

i´m doing an -e world atm (ca.50% done) with LDFLAGS="-Wl,-O1", and no problems so far. Seems to be secure enough to try ;-)
Back to top
View user's profile Send private message
placeholder
Advocate
Advocate


Joined: 07 Feb 2004
Posts: 2500

PostPosted: Thu Sep 23, 2004 12:28 pm    Post subject: Reply with quote

The original post about it was posted by a developer, so I think that they already know. Also, I notice a much higher speed increase with LDFLAGS over prelink. Compared to LDFALGS, prelink is unnoticable.

Also, this should not harm your programs at all.
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2160
Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Thu Sep 23, 2004 12:47 pm    Post subject: Reply with quote

aethyr wrote:
... Well, rebuilding gtk+ by itself didn't seem to help too much. I'm rebuilding gnome 2.8 to try to test a bit more...


I (think?) that this would only speed up apps not libraries. But I could be totally way off base. Anyone have any ideas about this?

By the way. I have put this in my make.conf and so far it has not caused any errors in the packages that have been built against it. It does seem to make the build time itself a little longer but that could be all in my head.

Good stuff so far. Thanks again for bringing this to our attention aethyr!
_________________
Gentoo: it's like wiping your ass with silk.
Back to top
View user's profile Send private message
Evangelion
Veteran
Veteran


Joined: 31 May 2002
Posts: 1087
Location: Helsinki, Finland

PostPosted: Thu Sep 23, 2004 1:33 pm    Post subject: Reply with quote

Pwnz3r wrote:
The original post about it was posted by a developer, so I think that they already know. Also, I notice a much higher speed increase with LDFLAGS over prelink. Compared to LDFALGS, prelink is unnoticable.

Also, this should not harm your programs at all.


Could one these optimizations together with prelinking? Would it make any sense?
_________________
My tech-blog | My other blog
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 1:47 pm    Post subject: Reply with quote

Evangelion wrote:
Could one these optimizations together with prelinking? Would it make any sense?


Well, I think the best way to learn would be to try it on a non-critical app and then prelink it and see if anything breaks. According to this article:
http://www.suse.de/~bastian/Export/linking.txt

from 2001, it seems like it would be compatible. Specifically:
Quote:
Symbol lookup is further optimized by the use of hash tables which makes the lookup of symbols very efficient.


I also found this post from 2001 that talks about optimizations:
http://mail.gnome.org/archives/gnome-hackers/2001-September/msg00035.html

Both of those are fairly interesting :)


Last edited by aethyr on Thu Sep 23, 2004 2:41 pm; edited 1 time in total
Back to top
View user's profile Send private message
Wedge_
Advocate
Advocate


Joined: 08 Aug 2002
Posts: 3614
Location: Scotland

PostPosted: Thu Sep 23, 2004 2:13 pm    Post subject: Reply with quote

If anyone is running an emerge -e world with the new LDFLAGS, you might want to remerge sys-apps/shadow after you're done. I caught this scrolling by while emerging with "-Wl,-O1":
Code:
QA Notice: Security risk /usr/bin/chage. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/chfn. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/chsh. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/expiry. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/gpasswd. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/newgrp. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /usr/bin/passwd. Please consider relinking with 'append-ldflags -Wl,-z,now' to fix.
QA Notice: Security risk /bin/su. Please consider relinking with 'append-ldflags-Wl,-z,now' to fix.

I don't know what the security risk actually is, but it's probably a good idea to do what it suggests :)
_________________
Per Ardua Ad Astra
The Earth is the cradle of the mind, but we cannot live forever in a cradle - Konstantin E. Tsiolkovsky
Gentoo Radeon FAQ
Back to top
View user's profile Send private message
neenee
Veteran
Veteran


Joined: 20 Jul 2003
Posts: 1786

PostPosted: Thu Sep 23, 2004 3:04 pm    Post subject: Reply with quote

thanks - i am running emerge -e world :wink:
Back to top
View user's profile Send private message
Wedge_
Advocate
Advocate


Joined: 08 Aug 2002
Posts: 3614
Location: Scotland

PostPosted: Thu Sep 23, 2004 3:20 pm    Post subject: Reply with quote

It looks like there are more packages which give similar warnings. Try grepping for something like "QA Notice: Security risk" in /var/log/portage. My emerge -e world is still running, but these are the ones that had already been compiled:
Code:
cups
glibc
pam
shadow
sudo
utempter
xterm

_________________
Per Ardua Ad Astra
The Earth is the cradle of the mind, but we cannot live forever in a cradle - Konstantin E. Tsiolkovsky
Gentoo Radeon FAQ
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 3:38 pm    Post subject: Reply with quote

Wedge_ wrote:
It looks like there are more packages which give similar warnings. Try grepping for something like "QA Notice: Security risk" in /var/log/portage.


Note, this error message appears to be unrelated to this addition to LDFLAGS:
Usenet Post


A User wrote:
"QA Notice: Security risk /usr/X11R6/bin/Xorg. Please consider relinking
with 'append-ldflags -Wl,-z,now' to fix."


Spider (the dev) wrote:
Security feature. suid binaries should not allow LD_{PRELOAD,DEBUG} and
therefore be relinked in "immediate" mode rather than "lazy". That
resolves the library links and function calls -before- running the
application rather than "as needed".

Somone (solar?) Decided it was a good idea to sharpen up the packages.
The fix is to fix the ebuild that installs a suid binary to axtually "do
right".


Last edited by aethyr on Thu Sep 23, 2004 3:47 pm; edited 3 times in total
Back to top
View user's profile Send private message
curtis119
Bodhisattva
Bodhisattva


Joined: 10 Mar 2003
Posts: 2160
Location: Toledo, Ohio,USA, North America, Earth, SOL System, Milky Way, The Universe, The Cosmos, and Beyond.

PostPosted: Thu Sep 23, 2004 3:43 pm    Post subject: Reply with quote

this thread got some attention on #gentoo-dev. Here is a transcript. I hope the devs who I'm quoting don't mind. I know quoting from irc is frowned upon.

Quote:
<brad[]> https://forums.gentoo.org/viewtopic.php?t=226909&highlight=
<brad[]> curious
<brad[]> It possible to add LDFLAGS= to make.conf?
<solar> brad[]" yes but you will have mixed results with it and packages
<brad[]> solar: Lots of things overriding?
<solar> KDE iirc hates LDFLAGS=
<solar> yep.
<brad[]> ah
<solar> well bad Makefile's ;/
<solar> I know a few neat little tricks however if you want to do it right.
<brad[]> Enlighten me
<solar> edit your gcc specs. Go to the *link_command: section at the end.
<brad[]> ah....
<solar> right after %(linker) %l
<solar> add %{!flagname: -flagname }
<brad[]> ah ha
<solar> another way is to add -Wl to CFLAGS
<solar> but also mixed results with that.
<solar> the *link_command: never fails :)


[edit]fixed a typo
_________________
Gentoo: it's like wiping your ass with silk.


Last edited by curtis119 on Thu Sep 23, 2004 3:51 pm; edited 2 times in total
Back to top
View user's profile Send private message
Wedge_
Advocate
Advocate


Joined: 08 Aug 2002
Posts: 3614
Location: Scotland

PostPosted: Thu Sep 23, 2004 3:44 pm    Post subject: Reply with quote

aethyr wrote:
Note, this error message appears to be unrelated to this addition to LDFLAGS

Ah, thanks for that.
_________________
Per Ardua Ad Astra
The Earth is the cradle of the mind, but we cannot live forever in a cradle - Konstantin E. Tsiolkovsky
Gentoo Radeon FAQ
Back to top
View user's profile Send private message
Evangelion
Veteran
Veteran


Joined: 31 May 2002
Posts: 1087
Location: Helsinki, Finland

PostPosted: Thu Sep 23, 2004 3:52 pm    Post subject: Reply with quote

Quote:
<solar> KDE iirc hates LDFLAGS=


How exactly? I was just about to run emerge -e world, when I noticed that. And since KDE and it's apps are what I run 95% of the time, it's pretty important to me
_________________
My tech-blog | My other blog
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 3:59 pm    Post subject: Reply with quote

curtis119 wrote:
<solar> I know a few neat little tricks however if you want to do it right.


Maybe Solar should address it then, since he was an original proponent:
https://bugs.gentoo.org/show_bug.cgi?id=23983

Solar wrote:
Not adding support for [LDFLAGS] would make some of the work I do for gentoo linux moot.


For what it's worth, I recompiled GNOME 2.8 with it and I had no problems, other people seem to be ok so far with "emerge -e system". Hopefully if there are problems (which I haven't heard about yet, aside from that 1 remark) with this feature in portage it can be resolved.

Evangelion wrote:
Quote:
<solar> KDE iirc hates LDFLAGS=


How exactly? I was just about to run emerge -e world, when I noticed that. And since KDE and it's apps are what I run 95% of the time, it's pretty important to me


Evangelion, why not emerge something like kdeutils and see if there's a problem? If worst comes to worst, just comment out the line in make.conf and emerge it again.

[edit] I think Solar might have been addressing this:
Usenet Post
Which does have to do with KDE and LDFLAGS, but has nothing at all to do with "-Wl,-O1". If that's the case, then I think you're OK.
Back to top
View user's profile Send private message
Evangelion
Veteran
Veteran


Joined: 31 May 2002
Posts: 1087
Location: Helsinki, Finland

PostPosted: Thu Sep 23, 2004 4:36 pm    Post subject: Reply with quote

aethyr wrote:
Evangelion, why not emerge something like kdeutils and see if there's a problem? If worst comes to worst, just comment out the line in make.conf and emerge it again.


As you wish sir. I'm now emerginh KDE-utils. I disabled prelinking. If KDE-utils work with these new optimizations, I will try enabling prelinking again. If everything works, I'll do emerge -e world, with or without prelinking (depending that does it help/work)

This is on AMD64 BTW.
_________________
My tech-blog | My other blog
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 4:45 pm    Post subject: Reply with quote

Evangelion wrote:
If everything works, I'll do emerge -e world, with or without prelinking (depending that does it help/work)


hehe, you seem very excited about doing "emerge -e world". If kdeutils works, why not try a few more KDE packages and see what the effects are? Then maybe you'll be able to post some benchmarks as well, with and without LDFLAGS. I tested about 6 packages before I reemerged GNOME and I still haven't done "emerge -e world". I'll probably just leave it in my make.conf and let the change propagate as I keep my system up to date.
Back to top
View user's profile Send private message
Evangelion
Veteran
Veteran


Joined: 31 May 2002
Posts: 1087
Location: Helsinki, Finland

PostPosted: Thu Sep 23, 2004 5:07 pm    Post subject: Reply with quote

aethyr wrote:
Evangelion wrote:
If everything works, I'll do emerge -e world, with or without prelinking (depending that does it help/work)


hehe, you seem very excited about doing "emerge -e world".


What can I say.... I love extra performance ;)

Quote:
If kdeutils works, why not try a few more KDE packages and see what the effects are? Then maybe you'll be able to post some benchmarks as well, with and without LDFLAGS.


I could do that, yes.
_________________
My tech-blog | My other blog
Back to top
View user's profile Send private message
Evangelion
Veteran
Veteran


Joined: 31 May 2002
Posts: 1087
Location: Helsinki, Finland

PostPosted: Thu Sep 23, 2004 6:49 pm    Post subject: Reply with quote

I tired it with Koffice (without prelinking). Now, benchmarking was really difficult since the apps loaded in 2-3 seconds to begin with. I think that performance is more or less the same. Certainly not worse than it was. And the apps work just fine. It _might_ be a bit better.

I guess I would have to recompile Qt and KDElibs to get any noticeable improvement
_________________
My tech-blog | My other blog
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Sep 23, 2004 6:57 pm    Post subject: Reply with quote

Evangelion wrote:
I tired it with Koffice (without prelinking). Now, benchmarking was really difficult since the apps loaded in 2-3 seconds to begin with. I think that performance is more or less the same. Certainly not worse than it was. And the apps work just fine. It _might_ be a bit better.

I guess I would have to recompile Qt and KDElibs to get any noticeable improvement


The way I've been testing is to run the app with "time application" and then immediately close it as soon as possible (basically hold the mouse pointer over where the close button appears and then click ASAP). There's probably much better ways to do it though...

If I was testing KOffice, I'd start it ~10 times without "-Wl,-O1", average out those 10 times, and then start it 10 times with "-Wl,-O1" and average out those times. The trickiest part is timing the window close. Sometimes you'll totally miss it, I just disregard those runs (i.e. only keep runs where the window was closed as soon as it popped up).

You might want to test with and without prelinking as well, for a total of 4 tests.
Back to top
View user's profile Send private message
Jefklak
l33t
l33t


Joined: 26 Oct 2003
Posts: 818
Location: Belgium

PostPosted: Thu Sep 23, 2004 7:41 pm    Post subject: Reply with quote

Holy sh*t 8O

A simple program like gvim:

Code:
# BEFORE using LDFLAGS
gvim  0.52s user ...
# AFTER
gvim  0.35s user ...


pretty intresting, huh!
MOD: lol. Please ignore this - I forgot I switched from 1,6 ghz to 3 (speedstepping while compiling), it gets only slower here (gvim WAS updated) - around 0,57.
_________________
{Linux User #333296}
Kernel 2.6.9-nitro4!
<< hurrah
Back to top
View user's profile Send private message
Muso
Veteran
Veteran


Joined: 22 Oct 2002
Posts: 1052
Location: The Holy city of Honolulu

PostPosted: Thu Sep 23, 2004 8:20 pm    Post subject: Reply with quote

added it to my make.conf..... now running emerge -e world...

I'll let you know in about 12 hours how it went.
_________________
"You can lead a horticulture but you can't make her think" ~ Dorothy Parker
2021 is the year of the Linux Desktop!
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 2 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum