View previous topic :: View next topic |
Author |
Message |
infirit l33t
Joined: 11 Jan 2003 Posts: 778 Location: Hoofddorp / The Netherlands
|
Posted: Wed Mar 30, 2005 9:53 pm Post subject: LDFLAG --as-needed / Speeding up of GNOME or KDE |
|
|
It is ok now to file bugs for packages failing with --as-needed. File the bug and provide all the ussual needed info (emerge --info and error message for example). And make it depend on bug# 129413
I was reading this article on osnews and wondered what one would gain from this and what the problems could when using it in gentoo?
How to set LDFLAGS
There are a couple of ways to set the LDFLAGS.
1: export LDFLAGS="-Wl,--as-needed"
2: Set it globally in /etc/make.conf by adding LDFLAGS="-Wl,--as-needed"
3: Setting them per package, see below section named: Per package {LD,C}FLAGS It should be set as LDFLAGS="-Wl,--as-needed" not as the article says as CFLAG!!
The story
Quote: | Today I thought about how I can make my very own application to only link against those libraries that it really requires and not those it get provided by PKG-CONFIG.
Problem-case:
When we use pkgconfig within our programs which of course makes sense, we also see that other libraries from other packages are being processed to our *_LIBS variables. So we end up in linking a lot of libraries to our application which we really do not need and which also causes (so I assume) a lot of overhead. After thinking for a while I looked some programs up by using:
readelf -d /usr/local/bin/gnome-terminal |grep NEEDED | wc -l
And saw that it was requiring over 52 libraries. I then recompiled gnome-terminal with
export CFLAGS = "-Os -s -Wl,--as-needed"
Pay attention to the --as-needed and the resulting binary required just 21 libraries afterwards instead of the 52 before because it only linked those really required to the final executable. I continued this with vte and saw that it required 29 libraries initially but after the --as-needed flag it went down to 7 libraries.
Now imagine this for the entire GNOME desktop or the entire KDE desktop and also package management systems which will reduce dependency tracking a lot. The problem with this method is that you surely don't keep track of API compatibility but a good package management system will certainly make sure that the packages packed belong to a certain desktop version. |
How to filter the flags in ebuilds
There is a way to have portage filter this flag at compile time:
Add the following to the top of the ebuild below IUSE
Code: | inherit flag-o-matic |
and put the below in src_compile at the top.
Code: | # This LDFLAG breaks "package name"
filter-ldflags "-Wl,--as-needed"
filter-cflags "-Wl,--as-needed" |
New LDFLAG --enable-new_ldflags
There is another LDFLAG which is more safe then plain --as-needed. The flag is --enable-new_ldflags and devsk eplains it best
defsk wrote: | when passed enable-new_ldflags, configure just checks if --as-needed works and if it does then it just assigns the variable LDFLAGS_AS_NEEDED="-Wl,--as-needed", which Makefiles have the liberty of using. Currently most kde Makefile's do. If a component finds that it doesn't work for them, they don't put it on the specific link line. Its manual in that sense.
one more thing is configure arg is --enable-new_ldflags and NOT --enable-new-ldflags as mentioned by the ./configure --help. |
I could not work out if this works or not, tested on konsole
LDFLAG Central
Some interesting ldflags in this post
Per package {LD,C}FLAGS
There is a way to have a per package ldflag. It involves copying an bashrc file into /etc/portage and creating a file package.ldflags also in /etc/portage. Get the script here and full explanation here. Thanks to thebigslide for creating the original bashrc and SoTired for adding the LDFLAGS support _________________ EASY TO INSTALL = Difficult to install, but instruction manual has pictures.
Join the adopt an unanswered post initiative today
Last edited by infirit on Fri May 14, 2010 11:40 am; edited 72 times in total |
|
Back to top |
|
|
i92guboj Bodhisattva
Joined: 30 Nov 2004 Posts: 10315 Location: Córdoba (Spain)
|
Posted: Wed Mar 30, 2005 10:08 pm Post subject: |
|
|
Mmmh! This would be awesome, but I dont think that this is really a so easy thing.
The differences in numbers are just too large. |
|
Back to top |
|
|
Flameeyes Retired Dev
Joined: 30 Mar 2005 Posts: 189 Location: London, Europe
|
Posted: Thu Mar 31, 2005 9:48 am Post subject: |
|
|
Well, actually KDE already uses --as-needed by default, if not disabled (for example on *BSD where it doesn't work too well).
This approach generally works well.. but still there ae cases in which packages got broken by --as-needed.. see for example x11-libs/gtk+-2.6.4-r1 which fails under amd64 for some conflicts between pic, amd64 relocations and --as-needed. |
|
Back to top |
|
|
lanius Retired Dev
Joined: 08 Dec 2002 Posts: 160
|
Posted: Thu Mar 31, 2005 11:31 am Post subject: |
|
|
I think it only breaks if you set CFLAGS="-Wl,--as-needed", but the correct way to do it is to set LDFLAGS="-Wl,--as-needed" |
|
Back to top |
|
|
fuoco Guru
Joined: 23 May 2004 Posts: 386 Location: Israel
|
Posted: Thu Mar 31, 2005 3:02 pm Post subject: |
|
|
anyone tried that on his gentoo box, and can share his experience? |
|
Back to top |
|
|
nxsty Veteran
Joined: 23 Jun 2004 Posts: 1556 Location: .se
|
Posted: Thu Mar 31, 2005 3:21 pm Post subject: |
|
|
fuoco wrote: | anyone tried that on his gentoo box, and can share his experience? |
No, but I will try it ASAP. |
|
Back to top |
|
|
Rapsey Tux's lil' helper
Joined: 01 May 2004 Posts: 87
|
Posted: Thu Mar 31, 2005 6:01 pm Post subject: |
|
|
I think there is a speed improvement... |
|
Back to top |
|
|
badgers l33t
Joined: 04 Sep 2003 Posts: 680 Location: Madison, WI
|
Posted: Thu Mar 31, 2005 6:28 pm Post subject: |
|
|
is this supposed to be a speed improvment in the time to compile or will it result in a faster binary when done? _________________ Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)
2.6.17 Suspend2 kernel with no scsi support |
|
Back to top |
|
|
Rapsey Tux's lil' helper
Joined: 01 May 2004 Posts: 87
|
Posted: Thu Mar 31, 2005 6:41 pm Post subject: |
|
|
reduced startup time.
edit: some more playing arround and it does work great. It cut the number of libraries loaded with gedit by half, startup time is much faster. Same thing for gnome-terminal. |
|
Back to top |
|
|
aethyr Veteran
Joined: 06 Apr 2003 Posts: 1085 Location: NYC
|
|
Back to top |
|
|
badgers l33t
Joined: 04 Sep 2003 Posts: 680 Location: Madison, WI
|
Posted: Thu Mar 31, 2005 8:22 pm Post subject: |
|
|
so if I wanted to recompile just one app with this option does it get added to the cflags variable or the ldflags variable? I have never changed the ldflags in make.conf _________________ Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)
2.6.17 Suspend2 kernel with no scsi support |
|
Back to top |
|
|
hielvc Advocate
Joined: 19 Apr 2002 Posts: 2805 Location: Oceanside, Ca
|
|
Back to top |
|
|
rhill Retired Dev
Joined: 22 Oct 2004 Posts: 1629 Location: sk.ca
|
|
Back to top |
|
|
badgers l33t
Joined: 04 Sep 2003 Posts: 680 Location: Madison, WI
|
Posted: Fri Apr 01, 2005 2:46 am Post subject: |
|
|
if this is to speed up the launch of a program, how does this differ from pre-linking? _________________ Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)
2.6.17 Suspend2 kernel with no scsi support |
|
Back to top |
|
|
foosh Apprentice
Joined: 11 Jan 2004 Posts: 231 Location: STL
|
Posted: Fri Apr 01, 2005 11:58 am Post subject: |
|
|
Maybe the title of this thread could have been "compiling '--as-needed' / speeding up Gnome". such a title would almost immediately tell me we're doing specific thing X in hopes of specific result Y
"story on OSnews.com" tells me absolutely nothing worthwhile. sorry to be so anal...but I feel it's the duty of a poster to make a concise, useful thread title to aid in the de-obfuscation of these forums, allowing people to search them more quickly--there is a TON of information to go through.
How many times have you seen threads with completely vague or ambiguous titles like "problem with X.org" or "stupid question..." or "help me please" ? those do a less than ideal job of directing people to a.) problems they could provide answers for, or b.) problems they need help with.
</soapbox> (infirit, no offense to you)
~~~
flameeyes said KDE compiles with --as-needed by default. I'm just curious why Gnome, for instance, couldn't? _________________ http://wustlog.blogspot.com |
|
Back to top |
|
|
Rapsey Tux's lil' helper
Joined: 01 May 2004 Posts: 87
|
Posted: Fri Apr 01, 2005 12:11 pm Post subject: |
|
|
badgers wrote: | if this is to speed up the launch of a program, how does this differ from pre-linking? |
With prelinking the program still loads the same amount of libraries when you run it, but it loads them faster. This eliminates loading of unnecessary libraries.
But it can cause problems for some programs apparently. I used it on programs that I find realy annoyingly slow to load: gedit, gnome-console, gaim, totem, rhythmbox. None of them have any problems.
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs? |
|
Back to top |
|
|
haven Tux's lil' helper
Joined: 19 Nov 2003 Posts: 141 Location: Belfast, Uk
|
Posted: Fri Apr 01, 2005 1:42 pm Post subject: |
|
|
Quote: | So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs? |
A simple wrapper script in bash might work - not nearly as automated as you would like I'm sure but its an option.
Code: |
#!/bin/bash
source /etc/make.conf
CFLAGS="${CFLAGS} -Wl,--as-needed" /usr/bin/emerge $1
|
Call it what you like and run it instead of emerge when you want to run "-Wl,--as-needed". I suspect you were looking at some sort of ebuild fix though |
|
Back to top |
|
|
suka Developer
Joined: 22 Apr 2002 Posts: 383
|
Posted: Fri Apr 01, 2005 1:45 pm Post subject: |
|
|
Rapsey wrote: |
But it can cause problems for some programs apparently. I used it on programs that I find realy annoyingly slow to load: gedit, gnome-console, gaim, totem, rhythmbox. None of them have any problems.
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs? |
I got into problems with quite a few programs: Totem, Sound-Juicer, basically all the GStreamer stuff, also gnome-system-tools and evince. On the other hand it seems to really speed up startup of some other stuff... |
|
Back to top |
|
|
Rapsey Tux's lil' helper
Joined: 01 May 2004 Posts: 87
|
Posted: Fri Apr 01, 2005 2:52 pm Post subject: |
|
|
I use xine library for totem and not gstreamer. Works fine. |
|
Back to top |
|
|
infirit l33t
Joined: 11 Jan 2003 Posts: 778 Location: Hoofddorp / The Netherlands
|
Posted: Fri Apr 01, 2005 3:54 pm Post subject: |
|
|
foosh wrote: | Maybe the title of this thread could have been "compiling '--as-needed' / speeding up Gnome". such a title would almost immediately tell me we're doing specific thing X in hopes of specific result Y
</soapbox> (infirit, no offense to you) | None taken
i have change the topic title to LDFLAGS --as-needed / speeding up Gnome"
I am going to take some measurements this wekeend for a couple of large program (large = bigger than gnome-terminal). I'll report back with some results. Any suggestions? _________________ EASY TO INSTALL = Difficult to install, but instruction manual has pictures.
Join the adopt an unanswered post initiative today |
|
Back to top |
|
|
Rapsey Tux's lil' helper
Joined: 01 May 2004 Posts: 87
|
Posted: Fri Apr 01, 2005 5:08 pm Post subject: |
|
|
Well anjuta is bigger, runs fine if you compile it with that flag. Also starts up quite a bit fater. |
|
Back to top |
|
|
Maedhros Bodhisattva
Joined: 14 Apr 2004 Posts: 5511 Location: Durham, UK
|
Posted: Fri Apr 01, 2005 5:56 pm Post subject: |
|
|
I tried it, and things stopped being able to find gtk+ to compile against. I'm going to wait a while before trying again, I think. _________________ No-one's more important than the earthworm. |
|
Back to top |
|
|
fridrik n00b
Joined: 18 Dec 2004 Posts: 51
|
Posted: Fri Apr 01, 2005 10:07 pm Post subject: |
|
|
i tried and recompiled terminal and panel
can't "see" any improvements
by reading this thread
aethyr wrote: | http://thread.gmane.org/gmane.linux.redhat.fedora.testers/24698 |
i think i won't use it anymore |
|
Back to top |
|
|
jdgill0 Veteran
Joined: 25 Mar 2003 Posts: 1366 Location: Lexington, Ky -- USA
|
Posted: Fri Apr 01, 2005 10:32 pm Post subject: |
|
|
I would like to point out that --as-needed is not only good for GNOME. Although it was said that KDE already uses --as-needed, I saw a difference in the number of libs linked against Konsole as shown below.
I have also built gimp with --as-needed and it seems to be working fine. I rebuilt KDE's konsole (I am using the splitted ebuilds). Using -Wl,--as-needed as a compiler flag did NOT work for Konsole, I had to use the following as was suggested above:
Code: | LDFLAGS="$LDFLAGS -Wl,--as-needed" |
Using the below code to compare the number of libs linked, I got the following results for Konsole:
Konsole WITHOUT --as-needed links to 63 libs.
Konsole WITH --as-needed links to 26 libs.
Code: | readelf -d `which konsole` | wc -l |
I haven't compared the number of libs linked for gimp. |
|
Back to top |
|
|
haven Tux's lil' helper
Joined: 19 Nov 2003 Posts: 141 Location: Belfast, Uk
|
Posted: Sat Apr 02, 2005 9:02 am Post subject: |
|
|
Is it worth compiling a list of packages that benefit from this vs packages that break with this ? And where would be the best place to do this ?
A simple script to exploit this list could be (excuse the simplicity as I'm not a coder):
Code: | #!/bin/bash
source /etc/make.conf
match=$(grep "^$1$" "/etc/portage/package_as_needed" )
if [[ "$match" = "$1" ]]
then
echo "emerging using -Wl,--as-needed"
LDFLAGS="${LDFLAGS} -Wl,-O1 -Wl,--as-needed" /usr/bin/emerge $1
else
echo "emerging using defaults"
/usr/bin/emerge $1
fi
exit 0
|
This just uses a flat file "/etc/portage/package_as_needed" to check for packages that work when compiled with "-Wl,--as-needed" and chooses the appropriate method.
From what I read, those applications that have a plugin architecture are doomed to fail with this optimisation (i.e. apache) whilst many desktop apps that are fairly self contained can be optimised quite heavily.
NB the other LDFLAG optimisations came from https://forums.gentoo.org/viewtopic.php?t=67777 |
|
Back to top |
|
|
|