LDFLAG --as-needed / Speeding up of GNOME or KDE

Message

nxsty · Post by **nxsty** » Sat Apr 09, 2005 4:49 pm

Glibc uses --as-needed by default if it's found, but this is disabled in gentoo. Do anybody know why?

EDIT:
This problem seems to have been fixed:
http://sources.redhat.com/ml/libc-alpha ... 00028.html

rhill · Post by **rhill** » Tue Apr 12, 2005 2:07 am

since some people can't listen.

latexer what's our stance on the LDFLAGS --as-needed silliness?
latexer can we rudely close tickets of users using it that have problems? (:
SpanKY ??
SpanKY users are putting '--as-needed' into their LDFLAGS !?
gustavoz --as-needed? --ass-delivered!
SpanKY if a user was enough of a tool to put --as-needed into their LDFLAGS you can mark it INVALID

if you're using --as-needed, DONT FILE BUGS IN BUGZILLA.

don't get me wrong, i'll experimenting with it too. but anything you break is your responsibility.

Jengu · Post by **Jengu** » Tue Apr 12, 2005 2:17 am

Someone care to actually post some benchmarks? "Seems faster" could just be placebo.

HardenCoonor · Post by **HardenCoonor** » Tue Apr 12, 2005 10:03 am

Someone care to actually post some benchmarks? "Seems faster" could just be placebo.

Unfortunately, i cannot post a benchmark. Apps seem to load a bit faster. And as long as they work, i will stick with them.

Again a listing of apps that work with my box(with binutils-2.15.94.0.2.2). See my other posting for other apps i have tested it.

kde-base/kcontrol
kde-base/ktts
kde-misc/krusader
media-video/kmplayer

Concerning KDE, setting LDFLAGS=-Wl,--as-needed becomes necessary when using the meta packages, in order not to loose speed compared to the normal packages and DO_NOT_COMPILE.

My complete LDFLAGS are

LDFLAGS="-Wl,-O2"
LDFLAGS="${LDFLAGS} -Wl,--as-needed"
LDFLAGS="${LDFLAGS} -Wl,--sort-common"
LDFLAGS="${LDFLAGS} -Wl,--relax"

Before it was -O1, and had no issues with none of them, although i did not compile my whole system with them.

Maedhros · Post by **Maedhros** » Tue Apr 12, 2005 1:22 pm

Rather belatedly, moved from Portage & Programming.

infirit · Post by **infirit** » Wed Apr 13, 2005 11:59 am

dirtyepic wrote:since some people can't listen.

latexer what's our stance on the LDFLAGS --as-needed silliness?
latexer can we rudely close tickets of users using it that have problems? (:
SpanKY ??
SpanKY users are putting '--as-needed' into their LDFLAGS !?
gustavoz --as-needed? --ass-delivered!
SpanKY if a user was enough of a tool to put --as-needed into their LDFLAGS you can mark it INVALID
if you're using --as-needed, DONT FILE BUGS IN BUGZILLA.

don't get me wrong, i'll experimenting with it too. but anything you break is your responsibility.

Who are "people"?

Anyway I updated my original post with a warning to not file bugs with --as-needed in the LDFLAGS.

rhill · Post by **rhill** » Fri Apr 15, 2005 3:03 am

sorry, that was pretty rude of me. i got a little testy sitting in #gentoo-bugs and watching five consecutive --as-needed bugs scroll by.

just a heads up for any brave souls out there testing the latest binutils-2.16.90.0.1 that's hard masked in portage, the --as-needed flag is stripping more than it should, causing a fair number of builds to break with undefined references. earlier versions of binutils in portage are fine.

the bug is here: http://sources.redhat.com/bugzilla/show_bug.cgi?id=815

i'll post some examples of the errors here to seed the search engine.

/lib/libpthread.so.0: undefined reference to `_dl_allocate_tls_init@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `_rtld_global@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `_dl_make_stack_executable@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `__libc_stack_end@GLIBC_2.1'
/lib/libpthread.so.0: undefined reference to `_rtld_global_ro@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `_dl_deallocate_tls@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `_dl_allocate_tls@GLIBC_PRIVATE'
/lib/libpthread.so.0: undefined reference to `_dl_get_tls_static_info@GLIBC_PRIVATE'

undefined reference to `_Unwind_DeleteException@GCC_3.0'
undefined reference to `_Unwind_Resume@GCC_3.0'
undefined reference to `_Unwind_RaiseException@GCC_3.0'
undefined reference to `_Unwind_GetRegionStart@GCC_3.0'
undefined reference to `_Unwind_GetDataRelBase@GCC_3.0'
undefined reference to `_Unwind_SetGR@GCC_3.0'
undefined reference to `_Unwind_GetLanguageSpecificData@GCC_3.0'
undefined reference to `_Unwind_Resume_or_Rethrow@GCC_3.3'
undefined reference to `_Unwind_GetIP@GCC_3.0'
undefined reference to `_Unwind_GetTextRelBase@GCC_3.0'
undefined reference to `_Unwind_SetIP@GCC_3.0'

/lib/libdl.so.2: undefined reference to `_rtld_global@GLIBC_PRIVATE'
/lib/libdl.so.2: undefined reference to `_rtld_global_ro@GLIBC_PRIVATE'
/lib/libdl.so.2: undefined reference to `_dl_rtld_di_serinfo@GLIBC_PRIVATE'

llsardonicll · Post by **llsardonicll** » Fri Apr 15, 2005 7:33 am

I just compiled totem with this ldflag, and I was amazed at how fast it started up.

infirit · Post by **infirit** » Fri Apr 15, 2005 7:51 am

dirtyepic wrote:sorry, that was pretty rude of me. i got a little testy sitting in #gentoo-bugs and watching five consecutive --as-needed bugs scroll by.

Thanks for explaining, i feel a bit responsible for it as It was my post that likely caused it

dirtyepic wrote:just a heads up for any brave souls out there testing the latest binutils-2.16.90.0.1 that's hard masked in portage, the --as-needed flag is stripping more than it should, causing a fair number of builds to break with undefined references. earlier versions of binutils in portage are fine.

the bug is here: http://sources.redhat.com/bugzilla/show_bug.cgi?id=815

I am not yet brave anough to try that

HardenCoonor · Post by **HardenCoonor** » Fri Apr 15, 2005 12:35 pm

just a heads up for any brave souls out there testing the latest binutils-2.16.90.0.1 that's hard masked in portage, the --as-needed flag is stripping more than it should, causing a fair number of builds to break with undefined references. earlier versions of binutils in portage are fine.

I guess i got such a breakage. I compiled glibc-2.3.5 with the binutils-2.16.90.0.1 and --as-needed and it failed. Compiled without --as-needed and only -Wl,-O1 and -Wl,--sort-common it compiled just fine, and it seems to work, no application has failed me yet(on the newer glibc).

rhill · Post by **rhill** » Fri Apr 15, 2005 7:53 pm

i posted a patch for binutils in the gcc 4.0 thread in Portage and Programming. i'm testing it now, but it comes straight from upstream so it should be fine.

[edit] nope, still doesn't work. i blame Cananda.

Cadorna · Post by **Cadorna** » Fri Apr 15, 2005 8:10 pm

guys, I'm compiling kde 3.4 splited ebuils, and about 97 packages of 187 have been complete already with this LDFLAGS

Code: Select all

LDFLAGS="-Wl,-O1 -Wl,--as-needed"

and I started a session with a test user, everything seems to be running fine, including CERVISIA (i noted that in the first post it's marked as a non-working application)

SnakeByte · Post by **SnakeByte** » Wed May 11, 2005 6:07 pm

guys, I successfully compiled some of the big apps using

gcc version 3.4.3-20050110 (Gentoo Hardened Linux 3.4.3.20050110-r2, ssp-3.4.3.20050110-0, pie-8.7.7
LDFLAGS="$LDFLAGS -Wl,-O2,--as-needed"

media-gfx/gimp-2.2.6-r1
24 -> 18

net-im/gnomemeeting-1.2.0
28 -> 18

Apllications work, no problems so far.

Next steps are "system" and openoffice ...

Gentree · Post by **Gentree** » Thu May 12, 2005 10:05 am

haven wrote:Is it worth compiling a list of packages that benefit from this vs packages that break with this ? And where would be the best place to do this ?

A simple script to exploit this list could be (excuse the simplicity as I'm not a coder):
Code: Select all
#!/bin/bash

source /etc/make.conf
match=$(grep "^$1$" "/etc/portage/package_as_needed" )

if [[ "$match" = "$1" ]]
   then
      echo "emerging using -Wl,--as-needed"
      LDFLAGS="${LDFLAGS} -Wl,-O1 -Wl,--as-needed" /usr/bin/emerge $1
   else
       echo "emerging using defaults"
       /usr/bin/emerge $1
    fi
exit 0
This just uses a flat file "/etc/portage/package_as_needed" to check for packages that work when compiled with "-Wl,--as-needed" and chooses the appropriate method.

From what I read, those applications that have a plugin architecture are doomed to fail with this optimisation (i.e. apache) whilst many desktop apps that are fairly self contained can be optimised quite heavily.

NB the other LDFLAG optimisations came from http://forums.gentoo.org/viewtopic.php?t=67777

Nice idea.

I called mine emopt and added a few more LDbits

Code: Select all

      LDFLAGS="${LDFLAGS} -Wl,-O1 -Wl,--as-needed -W1,--sort-common" /usr/bin/emerge $1

I use /etc/portage/package.ldflags for the file list

What would be good is some very simple parsing to cope with when I need to specify a pkg version to emerge , like I type emopt media-gfx/gimp-2.0.4 , it sees a match to gimp in the file list and does the optimised build.

You seem better at scripting that me (not hard) , what U think?

Lucacri · Post by **Lucacri** » Tue May 17, 2005 2:54 pm

Hi,
i tried to use the --as-needed and i get a system that is usable, faster, but sometimes I get errors in some applications.

May you post to me your list of apps that are working/not working with --as-needed?
And also, i'm tring to do an emerge -e system with these ldflags

Code: Select all

LDFLAGS="-Wl,-O2"
LDFLAGS="${LDFLAGS} -Wl,--as-needed"
LDFLAGS="${LDFLAGS} -Wl,--sort-common"
LDFLAGS="${LDFLAGS} -Wl,--relax"

Is it an unsane action? Is it correct?? Plz help me!!

Gentree · Post by **Gentree** » Tue May 17, 2005 6:19 pm

yes.

Read the thread.

Lucacri · Post by **Lucacri** » Tue May 17, 2005 6:36 pm

(sorry for my bad english but I'm italian)

I read it but I can't find the answer!! I'm doing a emerge -e system, don't let me waste time

Tell me if I need to stop it and reemerge without that ldflag!

pussi · Post by **pussi** » Tue May 17, 2005 8:19 pm

Well.. I compiled my whole world (including system) with following ldflags.

Code: Select all

LDFLAGS="-Wl,-O2 -Wl,--sort-common -s -Wl,--strip-debug -Wl,--enable-new-dtags"
LDFLAGS="${LDFLAGS} -Wl,--as-needed"
LDFLAGS="${LDFLAGS} -Wl,--relax"

I had problems with some gtk1 applications but mostly everything compiled fine. But after that I got lots of problems that I got fixed by re-emerging some system package without the --as-needed ldflag. --as-needed can really break something badly and I wouldn't do emerge system with it again. Also you won't get much performance gain by using it with system packages.

Cadorna · Post by **Cadorna** » Wed May 18, 2005 1:12 am

for system i don't use --as-needed, only for world packages, and I have too issues with gtk1 apps like xmms, just removed --as-needed for those packages and evertything is working great

Gentree · Post by **Gentree** » Wed May 18, 2005 7:56 am

Lucacri wrote:(sorry for my bad english but I'm italian)

I read it but I can't find the answer!! I'm doing a emerge -e system, don't let me waste time Tell me if I need to stop it and reemerge without that ldflag!

you did not read anything. The first post clearly states a whole bunch of things that dont work.

If you cant be bother the help yourself a minimum by reading the top of a thread you are posting to dont expect others to do it all for you. Unless of course you time is so much more valuable that the time you expect others to give for nothing.

Thanks to the last two posts for the info on gtk1.

This is the sort of minor tweek which is interesting to but of little real use. Probably best applied with a script on a case by case basis as detailed above rather putting anything in make.conf.

infirit · Post by **infirit** » Wed May 18, 2005 9:35 am

Cadorna wrote:for system i don't use --as-needed, only for world packages, and I have too issues with gtk1 apps like xmms, just removed --as-needed for those packages and evertything is working great

Almost all the packages i use are gtk2 and not gtk1. Is there a problem with gtk1 packages in general which I should mention in the first post?

Cadorna · Post by **Cadorna** » Wed May 18, 2005 2:36 pm

infirit wrote:Almost all the packages i use are gtk2 and not gtk1. Is there a problem with gtk1 packages in general which I should mention in the first post?

I've installed a machine with dual boot for a client that wants to test linux, and I did it with a KDE enviroment almost, just a few apps that are not qt, like audacity (wxgtk), xmms (gtk1), gftp (with gtk2), gimp (gtk2), firefox(gtk2), and one or 2 more apps with gtk1 that i don't remember right now. Again "system" it's not compiled with --as-needed (i don't think it worthy), only world apps, and they compiled and worked great, except for the gtk1 apps, those didn't compile, jus unset --as-needed for those and compiled and are working great, so maybe to mention that (not saying that gtk1 apps won't compile, just let know that maybe users will have problems compiling them) would be very usefull.

pussi · Post by **pussi** » Wed May 18, 2005 2:46 pm

Yeah, --as-needed don't seem work with gtk1 and gtk1 apps

teutzz · Post by **teutzz** » Sun May 22, 2005 11:38 am

personally i have no problems with gtk1 apps and the gtk 1 lib, when using the --as-needed flag, for those interested here is my emerge info, if any should ask of more info, i shall provide it

Code: Select all

Teutzz teutzz # emerge info
Portage 2.0.51.22-r1 (default-linux/x86/2005.0, gcc-3.4.3-20050110, glibc-2.3.5-r0, 2.6.11-morph6 i686)
=================================================================
System uname: 2.6.11-morph6 i686 AMD Athlon(tm) Processor
Gentoo Base System version 1.6.12
ccache version 2.4 [enabled]
dev-lang/python:     2.3.5
sys-apps/sandbox:    1.2.8
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r8
sys-devel/libtool:   1.5.18
virtual/os-headers:  2.6.11
ACCEPT_KEYWORDS="x86 ~x86"
AUTOCLEAN="yes"
CBUILD="i686-pc-linux-gnu"
CFLAGS="-O2 -march=athlon-tbird -pipe -fomit-frame-pointer -ftracer -fprefetch-loop-arrays -falign-functions=64 -momit-leaf-frame-pointer"
CHOST="i686-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O2 -march=athlon-tbird -pipe -fomit-frame-pointer -ftracer -fprefetch-loop-arrays -falign-functions=64 -momit-leaf-frame-pointer -fvisibility-inlines-hidden"
DISTDIR="/home/teutz/Documents/sys/linux/sys/Gentoo/distfiles"
FEATURES="autoconfig ccache distlocks sandbox sfperms strict"
GENTOO_MIRRORS="ftp://ftp.iasi.roedu.net/pub/mirrors/gentoo.org http://ftp.iasi.roedu.net/pub/mirrors/gentoo.org http://ftp.lug.ro/gentoo ftp://ftp.lug.ro/gentoo"
LANG="en_GB.UTF-8"
LC_ALL="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--enable-new-dtags -Wl,--sort-common -s -Wl,--as-needed"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/home/teutz/temp/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/bmg-main /usr/local/overlays/bmg-main.alternative"
SYNC="rsync://rsync.ro.gentoo.org/gentoo-portage"
USE="x86 3dnow 3dnowext X alsa avi bitmap-fonts cddb commercial dga divx4linux dlloader esd ffmpeg font-server gif gtk gtk2 hal imlib imlib2 ithreads jpeg kde kdeenablefinal kexec live mad mmx mmx2 mmxext mozilla mp3 mpeg ncurses nomalloccheck nomotif nptl nptlonly nvidia opengl pic png qt quicktime real sdl slang ssl tcpd theora tiff truetype truetype-fonts type1-fonts usb userlocales vcd win32codecs xanim xine xmms xprint xv xvid xvmc zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, LINGUAS

pussi · Post by **pussi** » Sun May 22, 2005 12:09 pm

I just upgraded to libtool-1.5.18 and after that gtk+-1.2.10-r11 compiled fine with --as-needed. Now I'm gonna try some more gtk1 apps if this has really fixed it.

EDIT: appearently I don't have many gtk1 apps but I tried gentoo and xzgv. xzgv compiled fine but gentoo failed in ./configure. I might try firefox later.