Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
LDFLAG --as-needed / Speeding up of GNOME or KDE
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3 ... 9, 10, 11  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
infirit
l33t
l33t


Joined: 11 Jan 2003
Posts: 774
Location: Hoofddorp / The Netherlands

PostPosted: Wed Mar 30, 2005 9:53 pm    Post subject: LDFLAG --as-needed / Speeding up of GNOME or KDE Reply with quote

It is ok now to file bugs for packages failing with --as-needed. File the bug and provide all the ussual needed info (emerge --info and error message for example). And make it depend on bug# 129413

I was reading this article on osnews and wondered what one would gain from this and what the problems could when using it in gentoo?

How to set LDFLAGS
There are a couple of ways to set the LDFLAGS.
    1: export LDFLAGS="-Wl,--as-needed"
    2: Set it globally in /etc/make.conf by adding LDFLAGS="-Wl,--as-needed"
    3: Setting them per package, see below section named: Per package {LD,C}FLAGS
It should be set as LDFLAGS="-Wl,--as-needed" not as the article says as CFLAG!!

The story
Quote:
Today I thought about how I can make my very own application to only link against those libraries that it really requires and not those it get provided by PKG-CONFIG.

Problem-case:

When we use pkgconfig within our programs which of course makes sense, we also see that other libraries from other packages are being processed to our *_LIBS variables. So we end up in linking a lot of libraries to our application which we really do not need and which also causes (so I assume) a lot of overhead. After thinking for a while I looked some programs up by using:

readelf -d /usr/local/bin/gnome-terminal |grep NEEDED | wc -l

And saw that it was requiring over 52 libraries. I then recompiled gnome-terminal with

export CFLAGS = "-Os -s -Wl,--as-needed"

Pay attention to the --as-needed and the resulting binary required just 21 libraries afterwards instead of the 52 before because it only linked those really required to the final executable. I continued this with vte and saw that it required 29 libraries initially but after the --as-needed flag it went down to 7 libraries.

Now imagine this for the entire GNOME desktop or the entire KDE desktop and also package management systems which will reduce dependency tracking a lot. The problem with this method is that you surely don't keep track of API compatibility but a good package management system will certainly make sure that the packages packed belong to a certain desktop version.


How to filter the flags in ebuilds
There is a way to have portage filter this flag at compile time:

Add the following to the top of the ebuild below IUSE
Code:
inherit flag-o-matic

and put the below in src_compile at the top.
Code:
# This LDFLAG breaks "package name"
filter-ldflags "-Wl,--as-needed"
filter-cflags "-Wl,--as-needed"


New LDFLAG --enable-new_ldflags
There is another LDFLAG which is more safe then plain --as-needed. The flag is --enable-new_ldflags and devsk eplains it best ;)
defsk wrote:
when passed enable-new_ldflags, configure just checks if --as-needed works and if it does then it just assigns the variable LDFLAGS_AS_NEEDED="-Wl,--as-needed", which Makefiles have the liberty of using. Currently most kde Makefile's do. If a component finds that it doesn't work for them, they don't put it on the specific link line. Its manual in that sense.

one more thing is configure arg is --enable-new_ldflags and NOT --enable-new-ldflags as mentioned by the ./configure --help.

I could not work out if this works or not, tested on konsole :?

LDFLAG Central
Some interesting ldflags in this post

Per package {LD,C}FLAGS
There is a way to have a per package ldflag. It involves copying an bashrc file into /etc/portage and creating a file package.ldflags also in /etc/portage. Get the script here and full explanation here. Thanks to thebigslide for creating the original bashrc and SoTired for adding the LDFLAGS support
_________________
EASY TO INSTALL = Difficult to install, but instruction manual has pictures.
Join the adopt an unanswered post initiative today


Last edited by infirit on Fri May 14, 2010 11:40 am; edited 72 times in total
Back to top
View user's profile Send private message
i92guboj
Moderator
Moderator


Joined: 30 Nov 2004
Posts: 9894
Location: Córdoba (Spain)

PostPosted: Wed Mar 30, 2005 10:08 pm    Post subject: Reply with quote

Mmmh! This would be awesome, but I dont think that this is really a so easy thing.
The differences in numbers are just too large.
_________________
Gentoo Handbook | My website
Back to top
View user's profile Send private message
Flameeyes
Developer
Developer


Joined: 30 Mar 2005
Posts: 187
Location: Dublin, Ireland

PostPosted: Thu Mar 31, 2005 9:48 am    Post subject: Reply with quote

Well, actually KDE already uses --as-needed by default, if not disabled (for example on *BSD where it doesn't work too well).

This approach generally works well.. but still there ae cases in which packages got broken by --as-needed.. see for example x11-libs/gtk+-2.6.4-r1 which fails under amd64 for some conflicts between pic, amd64 relocations and --as-needed.
Back to top
View user's profile Send private message
lanius
Retired Dev
Retired Dev


Joined: 08 Dec 2002
Posts: 160

PostPosted: Thu Mar 31, 2005 11:31 am    Post subject: Reply with quote

I think it only breaks if you set CFLAGS="-Wl,--as-needed", but the correct way to do it is to set LDFLAGS="-Wl,--as-needed"
Back to top
View user's profile Send private message
fuoco
Guru
Guru


Joined: 23 May 2004
Posts: 386
Location: Israel

PostPosted: Thu Mar 31, 2005 3:02 pm    Post subject: Reply with quote

anyone tried that on his gentoo box, and can share his experience?
Back to top
View user's profile Send private message
nxsty
Veteran
Veteran


Joined: 23 Jun 2004
Posts: 1556
Location: .se

PostPosted: Thu Mar 31, 2005 3:21 pm    Post subject: Reply with quote

fuoco wrote:
anyone tried that on his gentoo box, and can share his experience?


No, but I will try it ASAP. :)
Back to top
View user's profile Send private message
Rapsey
Tux's lil' helper
Tux's lil' helper


Joined: 01 May 2004
Posts: 87

PostPosted: Thu Mar 31, 2005 6:01 pm    Post subject: Reply with quote

I think there is a speed improvement...
Back to top
View user's profile Send private message
badgers
l33t
l33t


Joined: 04 Sep 2003
Posts: 680
Location: Madison, WI

PostPosted: Thu Mar 31, 2005 6:28 pm    Post subject: Reply with quote

is this supposed to be a speed improvment in the time to compile or will it result in a faster binary when done?
_________________
Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)

2.6.17 Suspend2 kernel with no scsi support
Back to top
View user's profile Send private message
Rapsey
Tux's lil' helper
Tux's lil' helper


Joined: 01 May 2004
Posts: 87

PostPosted: Thu Mar 31, 2005 6:41 pm    Post subject: Reply with quote

reduced startup time.
edit: some more playing arround and it does work great. It cut the number of libraries loaded with gedit by half, startup time is much faster. Same thing for gnome-terminal.
Back to top
View user's profile Send private message
aethyr
Veteran
Veteran


Joined: 06 Apr 2003
Posts: 1085
Location: NYC

PostPosted: Thu Mar 31, 2005 7:38 pm    Post subject: Reply with quote

Judging by this post, I'd be careful of what you use it with. I wouldn't put it in make.conf and rebuild my system.

http://thread.gmane.org/gmane.linux.redhat.fedora.testers/24698
Back to top
View user's profile Send private message
badgers
l33t
l33t


Joined: 04 Sep 2003
Posts: 680
Location: Madison, WI

PostPosted: Thu Mar 31, 2005 8:22 pm    Post subject: Reply with quote

so if I wanted to recompile just one app with this option does it get added to the cflags variable or the ldflags variable? I have never changed the ldflags in make.conf
_________________
Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)

2.6.17 Suspend2 kernel with no scsi support
Back to top
View user's profile Send private message
hielvc
Advocate
Advocate


Joined: 19 Apr 2002
Posts: 2801
Location: Oceanside, Ca

PostPosted: Thu Mar 31, 2005 10:09 pm    Post subject: Reply with quote

Dumb responces . Removed
_________________
An A-Z Index of the Linux BASH command line
Back to top
View user's profile Send private message
rhill
Developer
Developer


Joined: 22 Oct 2004
Posts: 1629
Location: sk.ca

PostPosted: Fri Apr 01, 2005 12:23 am    Post subject: Reply with quote

there are reasons that --as-needed isn't used by default. it breaks many things in vague and creative ways.

http://sources.redhat.com/ml/binutils/2005-03/msg00103.html
_________________
by design, by neglect
for a fact or just for effect
Back to top
View user's profile Send private message
badgers
l33t
l33t


Joined: 04 Sep 2003
Posts: 680
Location: Madison, WI

PostPosted: Fri Apr 01, 2005 2:46 am    Post subject: Reply with quote

if this is to speed up the launch of a program, how does this differ from pre-linking?
_________________
Abit KD7-S
Athlon XP2500+
166mHz FSB
512 Meg PC3200 Ram running at 166mHz
LiteOn DVD dual Layer burner(hdc)

2.6.17 Suspend2 kernel with no scsi support
Back to top
View user's profile Send private message
foosh
Apprentice
Apprentice


Joined: 11 Jan 2004
Posts: 231
Location: STL

PostPosted: Fri Apr 01, 2005 11:58 am    Post subject: Reply with quote

Maybe the title of this thread could have been "compiling '--as-needed' / speeding up Gnome". such a title would almost immediately tell me we're doing specific thing X in hopes of specific result Y

"story on OSnews.com" tells me absolutely nothing worthwhile. sorry to be so anal...but I feel it's the duty of a poster to make a concise, useful thread title to aid in the de-obfuscation of these forums, allowing people to search them more quickly--there is a TON of information to go through.
How many times have you seen threads with completely vague or ambiguous titles like "problem with X.org" or "stupid question..." or "help me please" ? those do a less than ideal job of directing people to a.) problems they could provide answers for, or b.) problems they need help with.
</soapbox> (infirit, no offense to you)

~~~
flameeyes said KDE compiles with --as-needed by default. I'm just curious why Gnome, for instance, couldn't?
_________________
http://wustlog.blogspot.com
Back to top
View user's profile Send private message
Rapsey
Tux's lil' helper
Tux's lil' helper


Joined: 01 May 2004
Posts: 87

PostPosted: Fri Apr 01, 2005 12:11 pm    Post subject: Reply with quote

badgers wrote:
if this is to speed up the launch of a program, how does this differ from pre-linking?

With prelinking the program still loads the same amount of libraries when you run it, but it loads them faster. This eliminates loading of unnecessary libraries.
But it can cause problems for some programs apparently. I used it on programs that I find realy annoyingly slow to load: gedit, gnome-console, gaim, totem, rhythmbox. None of them have any problems.
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs?
Back to top
View user's profile Send private message
haven
Tux's lil' helper
Tux's lil' helper


Joined: 19 Nov 2003
Posts: 141
Location: Belfast, Uk

PostPosted: Fri Apr 01, 2005 1:42 pm    Post subject: Reply with quote

Quote:
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs?


A simple wrapper script in bash might work - not nearly as automated as you would like I'm sure but its an option.

Code:

#!/bin/bash

source /etc/make.conf

CFLAGS="${CFLAGS} -Wl,--as-needed" /usr/bin/emerge $1


Call it what you like and run it instead of emerge when you want to run "-Wl,--as-needed". I suspect you were looking at some sort of ebuild fix though :)
Back to top
View user's profile Send private message
suka
Developer
Developer


Joined: 22 Apr 2002
Posts: 383

PostPosted: Fri Apr 01, 2005 1:45 pm    Post subject: Reply with quote

Rapsey wrote:

But it can cause problems for some programs apparently. I used it on programs that I find realy annoyingly slow to load: gedit, gnome-console, gaim, totem, rhythmbox. None of them have any problems.
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs?


I got into problems with quite a few programs: Totem, Sound-Juicer, basically all the GStreamer stuff, also gnome-system-tools and evince. On the other hand it seems to really speed up startup of some other stuff... :?
Back to top
View user's profile Send private message
Rapsey
Tux's lil' helper
Tux's lil' helper


Joined: 01 May 2004
Posts: 87

PostPosted: Fri Apr 01, 2005 2:52 pm    Post subject: Reply with quote

I use xine library for totem and not gstreamer. Works fine.
Back to top
View user's profile Send private message
infirit
l33t
l33t


Joined: 11 Jan 2003
Posts: 774
Location: Hoofddorp / The Netherlands

PostPosted: Fri Apr 01, 2005 3:54 pm    Post subject: Reply with quote

foosh wrote:
Maybe the title of this thread could have been "compiling '--as-needed' / speeding up Gnome". such a title would almost immediately tell me we're doing specific thing X in hopes of specific result Y
</soapbox> (infirit, no offense to you)
None taken :wink:

i have change the topic title to LDFLAGS --as-needed / speeding up Gnome"

I am going to take some measurements this wekeend for a couple of large program (large = bigger than gnome-terminal). I'll report back with some results. Any suggestions?
_________________
EASY TO INSTALL = Difficult to install, but instruction manual has pictures.
Join the adopt an unanswered post initiative today
Back to top
View user's profile Send private message
Rapsey
Tux's lil' helper
Tux's lil' helper


Joined: 01 May 2004
Posts: 87

PostPosted: Fri Apr 01, 2005 5:08 pm    Post subject: Reply with quote

Well anjuta is bigger, runs fine if you compile it with that flag. Also starts up quite a bit fater.
Back to top
View user's profile Send private message
Maedhros
Bodhisattva
Bodhisattva


Joined: 14 Apr 2004
Posts: 5511
Location: Durham, UK

PostPosted: Fri Apr 01, 2005 5:56 pm    Post subject: Reply with quote

I tried it, and things stopped being able to find gtk+ to compile against. I'm going to wait a while before trying again, I think.
_________________
No-one's more important than the earthworm.
Back to top
View user's profile Send private message
fridrik
n00b
n00b


Joined: 18 Dec 2004
Posts: 51

PostPosted: Fri Apr 01, 2005 10:07 pm    Post subject: Reply with quote

i tried and recompiled terminal and panel
can't "see" any improvements
by reading this thread
aethyr wrote:
http://thread.gmane.org/gmane.linux.redhat.fedora.testers/24698

i think i won't use it anymore
Back to top
View user's profile Send private message
jdgill0
Veteran
Veteran


Joined: 25 Mar 2003
Posts: 1366
Location: Lexington, Ky -- USA

PostPosted: Fri Apr 01, 2005 10:32 pm    Post subject: Reply with quote

I would like to point out that --as-needed is not only good for GNOME. Although it was said that KDE already uses --as-needed, I saw a difference in the number of libs linked against Konsole as shown below.

I have also built gimp with --as-needed and it seems to be working fine. I rebuilt KDE's konsole (I am using the splitted ebuilds). Using -Wl,--as-needed as a compiler flag did NOT work for Konsole, I had to use the following as was suggested above:
Code:
LDFLAGS="$LDFLAGS -Wl,--as-needed"



Using the below code to compare the number of libs linked, I got the following results for Konsole:
Konsole WITHOUT --as-needed links to 63 libs.
Konsole WITH --as-needed links to 26 libs.
Code:
readelf -d `which konsole` | wc -l




I haven't compared the number of libs linked for gimp.
Back to top
View user's profile Send private message
haven
Tux's lil' helper
Tux's lil' helper


Joined: 19 Nov 2003
Posts: 141
Location: Belfast, Uk

PostPosted: Sat Apr 02, 2005 9:02 am    Post subject: Reply with quote

Is it worth compiling a list of packages that benefit from this vs packages that break with this ? And where would be the best place to do this ?

A simple script to exploit this list could be (excuse the simplicity as I'm not a coder):
Code:
#!/bin/bash

source /etc/make.conf
match=$(grep "^$1$" "/etc/portage/package_as_needed" )

if [[ "$match" = "$1" ]]
   then
      echo "emerging using -Wl,--as-needed"
      LDFLAGS="${LDFLAGS} -Wl,-O1 -Wl,--as-needed" /usr/bin/emerge $1
   else
       echo "emerging using defaults"
       /usr/bin/emerge $1
    fi
exit 0


This just uses a flat file "/etc/portage/package_as_needed" to check for packages that work when compiled with "-Wl,--as-needed" and chooses the appropriate method.

From what I read, those applications that have a plugin architecture are doomed to fail with this optimisation (i.e. apache) whilst many desktop apps that are fairly self contained can be optimised quite heavily.

NB the other LDFLAG optimisations came from http://forums.gentoo.org/viewtopic.php?t=67777
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page 1, 2, 3 ... 9, 10, 11  Next
Page 1 of 11

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum