LDFLAG --as-needed / Speeding up of GNOME or KDE

Message

hielvc · Post by **hielvc** » Sat Apr 02, 2005 3:27 pm

You might want to check out this thread /etc/portage/package.c{,xx}flags features - v1.0_rc1. Between thebigslide and haven's code it be should easy to impliment a fancier solution.

badgers · Post by **badgers** » Mon Apr 04, 2005 12:41 pm

I would like to say that I agree with the idea of generating a list of packages that use the as-needed option.
I think we could have the best of both worlds with that type of system. It may also allow some developers to rework their code to make it on the list.

It may not be simple, but I think it is one of the things that can really set a gentoo system apart from some of the more popular distributions out there. Everybody notices how fast a program takes to launch, from the new to the most experienced, nobody like waiting.

fuoco · Post by **fuoco** » Mon Apr 04, 2005 3:32 pm

Everybody notices how fast a program takes to launch

What do you mean? you notice lots of difference in startup times with --as-needed?
And if everybody who uses --as-needed sends a bug report or post on that package, maybe it will be fixed.
I for example couldn't compile libdv with --as-needed...[/quote]

badgers · Post by **badgers** » Mon Apr 04, 2005 4:01 pm

I am not sure what you mean?

I have never used as-needed.

when a user clicks on an icon they notice how long it takes to launch.
everybody who has ever used a computer notices that when you click on an icon it takes a certain amount of time for that program to launch. The amount of time seems shorter on some computers than others.

even my mother-in-law who uses a computer once a week noticed that when she got a new machine the "email goes much faster now"

If you used a program 10 times a day you will become intimatly aware of how fast it takes to launch(assuming you closed and launched it each time). Even a once a week user notices how fast it launches. It is just one of those basic things.

if you couldn't compile libdv with as-needed then most likely it shouldn't be on the compatible list

I don't know that I would say if as-needed doesn't work that it needs to be fixed. It seems some things will never really work with as-needed. That is what spawned the idea of a compatible list. The default would be to not use it.

infirit · Post by **infirit** » Mon Apr 04, 2005 5:56 pm

I can keep a list of aplications that work in my first post. Let me know if you have (un)succesfully compiled an app with --as-needed.

fuoco · Post by **fuoco** » Mon Apr 04, 2005 6:05 pm

Good idea. so far two failures:

media-libs/libdv-0.102 - ebuild fails to compile (or actually fails at the linking stage)
gnome-extra/gnome-media-2.10.0 - ebuild completes successfully but one binary, gnome-cd, has errors in the symbols and does not work.

Anyone compiled or started compiling his system with --as-needed? Anyone feels a change in stsartup times or anything else?

Lowspirit · Post by **Lowspirit** » Mon Apr 04, 2005 6:08 pm

Playing it safe so far I've only done :

x11-terms/gnome-terminal
x11-libs/vte

Both compiled succesfully, did it to get gnome-terminal to start faster but have seen no difference whatsoever, but they run just as before.

fuoco · Post by **fuoco** » Mon Apr 04, 2005 6:48 pm

I just tried to compile a small hello world C program. I compiled it each time with different options and checked

Code: Select all

ldd -u -r 'bin_name'

to see what it's compiled against.

First time I compiled it with:

Code: Select all

gcc -o hello_world hello_world.c

and it was only linked against libc - as it should be.

Second time I compiled it with:

Code: Select all

gcc -o hello_world hello_world.c -lX11

and it was linked against both libc and X11, even though it doesn't use or need X11.

Third time I compiled it with:

Code: Select all

gcc -Wl,--as-needed -o hello_world hello_world.c -lX11

and it was again only linked against libc. So --as-needed found out it doesn't need X11 and didn't link against it.

If you'll check your system 'with ldd -u -r' you'll see that most binaries have unneeded links.

I don't really have a point in this, just letting you know.

haven · Post by **haven** » Mon Apr 04, 2005 7:56 pm

Personally I feel this would be something to make ebuild maintainers aware of. If "safe" LDFLAGS could be included in the ebuild then it would mean a little extra effort for the maintainer to test as opposed to a whole lot of effort for everyone else to find out on their own.

That asside I think thebigslide's code is very good - it just needs another function with LDFLAGS adding and a corresponding file in /etc/portage.

simondo42 · Post by **simondo42** » Mon Apr 04, 2005 9:21 pm

In addition to the start-up benefits, has anyone else noticed that programs take less memory as well? I recompiled the kde 3.4 split ebuilds with the --as-needed flag and my starting memory consumption for booting into kde seems to have dropped by a good 10MB. Not to be sniffed at!

Also, I've found that the latest ~x86 ebuilds of torsmo and beep-media-player build but won't run with --as-needed. As someone else mentioned, I think anything with a plug-in architecture may have problems.

Cheers,
simondo

ravageNG · Post by **ravageNG** » Tue Apr 05, 2005 1:41 pm

Just noticed that kdevelop does _NOT_ work when compiled with -Wl,--as-needed.
But this seems so far the only app failed.

infirit · Post by **infirit** » Tue Apr 05, 2005 1:58 pm

ravageNG wrote:Just noticed that kdevelop does _NOT_ work when compiled with -Wl,--as-needed.
But this seems so far the only app failed.

I have been keeping a list of packages that do and do not work. Which packages work correctly for you with --as-needed?

I added kdevelop to the not working list.

fuoco · Post by **fuoco** » Tue Apr 05, 2005 2:12 pm

I have many working packages - but I only keep track of non working:

1. libdv - fails in link stage
2. gnome-media - unresolved symbols in gnome-cd
3. dbus - unresolved symbols in dbus-glib-tool symbols
4. gnome-system-tools - unresolved symbols in network-admin and shares-admin
5. samba - unresolved symbols in rpcclient

That's so far - but I'm considering if it's worth using --as-needed at all, since it breaks quite a few stuff, and I don't seem to feel much change or improvement in startup times. Anyone else got imrovements in startup times from this?

Sparker · Post by **Sparker** » Wed Apr 06, 2005 11:07 am

gtk-engines (2.6.2) doesn't configure. Can't find gtk+-1.2.10

ravageNG · Post by **ravageNG** » Wed Apr 06, 2005 11:18 am

After compiling kdevelop without --as-needed in crashes when creating new projects.
Recompiling cervisia helps.

So cervisia does _NOT_ work with --as-needed.

PRELINK:
After using --as-needed for example to compile konsole and kdesktop (which are working) _AND_ running prelink -aR
--> prelink warns
--> konsole / kdesktop won't start (unresolved symbol)

So it seems you can't use prelink and --as-needed together!?!?

infirit · Post by **infirit** » Wed Apr 06, 2005 11:39 am

Thanks for the feedback, i have updated the original post.

badgers · Post by **badgers** » Wed Apr 06, 2005 12:48 pm

ravageNG wrote:After compiling kdevelop without --as-needed in crashes when creating new projects.
Recompiling cervisia helps.

So cervisia does _NOT_ work with --as-needed.

PRELINK:
After using --as-needed for example to compile konsole and kdesktop (which are working) _AND_ running prelink -aR
--> prelink warns
--> konsole / kdesktop won't start (unresolved symbol)

So it seems you can't use prelink and --as-needed together!?!?

is it possible that it could be a prelink bug rather then a true incompatibility?

ikke · Post by **ikke** » Wed Apr 06, 2005 1:34 pm

suka wrote:
Rapsey wrote: But it can cause problems for some programs apparently. I used it on programs that I find realy annoyingly slow to load: gedit, gnome-console, gaim, totem, rhythmbox. None of them have any problems.
So I guess we should not put it into make.conf. Can someone tell me how to set it, so that portage will only use it on specific programs?
I got into problems with quite a few programs: Totem, Sound-Juicer, basically all the GStreamer stuff, also gnome-system-tools and evince. On the other hand it seems to really speed up startup of some other stuff...

This could be caused by the runtime-linking gstreamer uses (or something like that), to enable plugins etc. It is possible evolution wont work very well too because of some similar issue (you'll loose your calendar and todo list functionality).

I had this in my LDFLAGS some months ago and sometimes it was a real PITA.

kimchi_sg · Post by **kimchi_sg** » Wed Apr 06, 2005 2:16 pm

haven wrote:From what I read, those applications that have a plugin architecture are doomed to fail with this optimisation (i.e. apache)

I've proven this remark partly wrong!

Just finished compiling apache-2.0.53 with LDFLAGS="-Wl,--as-needed":

Code: Select all

shrek@maximus ~ $ emerge --info | grep LDFLAGS
LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common -Wl,--as-needed -s"

But to prevent possible aborts or other errors, I static-ed all the modules into apache so it wouldn't have to look for any:

Code: Select all

maximus shrek # equery uses apache
[ Searching for packages matching apache... ]
[ Colour Code : set unset ]
[ Legend    : Left column  (U) - USE flags from make.conf              ]
[           : Right column (I) - USE flags packages was installed with ]
[ Found these USE variables for net-www/apache-2.0.53 ]
 U I
 + + apache2        : Chooses Apache2 support when a package supports both Apache1 and Apache2
 - - debug          : Tells configure and the makefiles to build for debugging. Effects vary across packages, but generally it will at least add -g to CFLAGS. Remember to set FEATURES=nostrip too
 - - doc            : Adds extra documentation (API, Javadoc, etc)
 - - ldap           : Adds LDAP support (Lightweight Directory Access Protocol)
 - - mpm-leader     : Adds support for the leader Apache MPM
 - - mpm-peruser    : Adds support for the peruser Apache MPM
 - - mpm-prefork    : Adds support for the prefork Apache MPM
 - - mpm-threadpool : Adds support for the threadpool Apache MPM
 - - mpm-worker     : Adds support for the worker Apache MPM
 - - no-suexec      : Don't install suexec with apache
 + + ssl            : Adds support for Secure Socket Layer connections
 + + static-modules : Build modules into apache instead of having them load at run time
 + + threads        : Adds threads support for various packages. Usually pthreads

I've noticed no differences in the way apache functions after re-compiling it.

Now embarking on

Code: Select all

emerge -e system && emerge -e system && emerge -e world

with this little optimisation.

aethyr · Post by **aethyr** » Wed Apr 06, 2005 2:38 pm

kimchi_sg wrote:Now embarking on
Code: Select all
emerge -e system && emerge -e system && emerge -e world
with this little optimisation. :D

I'm still not convinced it's an optimization in anything except # of linked packages that you see when you check with ldd.

HardenCoonor · Post by **HardenCoonor** » Fri Apr 08, 2005 1:50 pm

At first, thanks for

Code: Select all

--as-needed

. My kdebase-metas seem to start faster with it, and the numbers of libraries certain apps are linked against has decreased.

Meanwhile, using newer binutils may also have advantages. I use the newest in portage (94.2.2) on my ~x86, and they work just fine. I have recompiled glibc, gcc, binutils and other stuff (KDE-3.4 of course), and all worked.

It also seems that --relax now works on x86, at least there are no error messages.

See http://forums.gentoo.org/viewtopic-t-67 ... relax.html

Prelinking also works with my computer.

Apps working with --as-needed (no stress test yet)

konsole, kicker, krusader and others, mainly from kdebase-meta.

infirit · Post by **infirit** » Fri Apr 08, 2005 8:49 pm

There is a easy way to strip -Wl,--as-needed from the LDFLAGS with the flag-o-matic eclass.

Add the following to the top of the ebuild below IUSE

Code: Select all

inherit flag-o-matic

and put

Code: Select all

# This LDFLAG breaks "bmp"
filter-ldflags "-Wl,--as-needed"

in src_compile

Now the flag can be set globaly in /etc/make.conf

Would devs add this to ebuilds for apps that break with -Wl,--as-needed if I submit bug reports for them?

omnicloud · Post by **omnicloud** » Fri Apr 08, 2005 9:18 pm

This is kinda weird, Gnome-terminal was broken until I compiled it with these ldflags

fifthcent · Post by **fifthcent** » Fri Apr 08, 2005 11:58 pm

I was just about to post some info about how to fix ebuilds that break with -Wl,--as-needed, but infirit beat me to it. Although, it may be useful to also have

Code: Select all

filter-flags "-Wl,--as-needed"

just incase a user does have it in their CFLAGS as well.

If people find ebuilds that do break, it would be best to file bugs it. I just filed a bug about beep-media-player, so hopefully that will get fixed.

rhill · Post by **rhill** » Sat Apr 09, 2005 12:42 am

please don't file bugs about things that break with experimental flags. they'll just get closed as invalid, and waste everybody's time. if you use experimental flags, you do it at your own risk.