| View previous topic :: View next topic |
| Author |
Message |
devilheart
l33t
Joined: 17 Mar 2005
Posts: 848
Location: Villach, Austria
|
 Posted: Wed Sep 06, 2006 9:12 am Post subject: |
 |
|
| Code: | /usr/lib/gcc/x86_64-pc-linux-gnu/4.1.1/../../../../x86_64-pc-linux-gnu/bin/ld: unrecognized option '--hash-style=both'
|
??? what do I need to do to use --hash-style?
i'm using gcc-4.1.1 and binutils 2.17 |
|
| Back to top |
|
 |
reynolds531
Apprentice
Joined: 23 Apr 2005
Posts: 260
Location: Rochester, NY
|
| Posted: Wed Sep 06, 2006 10:55 am Post subject: |
|
|
| Phenax wrote: | | Elfan wrote: | | Is there anything specific to KDE in your guide or were you just using that as an example? |
Nope, you'll just see best results using KDE because it has so many shared libraries. |
But wouldn't gnome benefit from this as well? Or is there something different about the way kde uses shared libraries?
Guenther: Please keep on doing what you're doing. Your posts of the last few days have been a real revelation to me. As a computer amateur, I appreciate the fact that you explain the rationale of every step you take (and the purpose of the commands you use). I find that many computer professionals mistakenly assume that all of their readers share the same background knowledge and thus give explanations that are less than clear to amateurs.I suppose part of the reason you write that way is that you're working out things for yourself as you go -- and that in itself is a marvel to me since you seem to have progressed in leaps and bounds. |
|
| Back to top |
|
|
Zmyrgel
Apprentice
Joined: 31 Jan 2006
Posts: 181
Location: Finland / Ireland
|
| Posted: Wed Sep 06, 2006 1:49 pm Post subject: |
|
|
Damn this thread... makes me wanna go back to Finland to my Athlon 3800+ X2 
I only have my puny laptop with me with 32-bit yonah... I might try prelinking on that one too once I figure out how to encrypt whole system with LVM partitions.
Great info though. I was planning to use prelink after I get the encryption thing to work. I was planning to follow the Conrad installation procedure to get most out of the machine. |
|
| Back to top |
|
|
Kvetch
Guru
Joined: 29 Apr 2004
Posts: 318
Location: /dev/null, VA
|
| Posted: Wed Sep 06, 2006 11:51 pm Post subject: |
|
|
Great threads Guenther Brunthaler. Thanks for all the info.
So last night I read over this thread and the prelinking guide but I was wondering what does
| Code: |
#cat /etc/env.d/99kde-env
KDEDIRS=/usr
CONFIG_PROTECT=/usr/share/config
#KDE_IS_PRELINKED=1
#KDEWM=compiz-decorator |
The Prelinking line in env.d do?
Nick |
|
| Back to top |
|
|
wizard69
Apprentice
Joined: 22 Sep 2003
Posts: 178
Location: Berlin
|
| Posted: Thu Sep 07, 2006 12:03 am Post subject: |
|
|
The option KDE_IS_PRELINKED=1 or KDE_IS_PRELINKED=true enables prelinking in Kde.
Gentoo Prelink Guide |
|
| Back to top |
|
|
Guenther Brunthaler
Apprentice
Joined: 22 Jul 2006
Posts: 217
Location: Vienna
|
| Posted: Thu Sep 07, 2006 12:14 am Post subject: |
|
|
| Kvetch wrote: | I was wondering what does
| Code: |
#KDE_IS_PRELINKED=1 |
The Prelinking line in env.d do?
|
I think it has to do with the kdeinit process.
Before prelinking was available, the only way to reduce the number of necessary relocations was this: The kdeinit process preloaded as much shared libraries as possible into RAM, without actually requiring them.
While those SOs were loaded by kdeinit, they were also relocated.
All remaining KDE processes were then forked from kdeinit.
Forking means inheriting all memory pages from the parent process, in this case including the relocated code pages.
This allowed child processes to directly use and share the relocated SOs without a need to relocate them again.
The downside was a horrible long loading time for kdeinit itself, because it had to load and relocate so many SOs, whether or not they were actually used.
However, if prelinked SOs are used, relocations should not occur very often (or at all). So it is no longer necessary for kdeinit to load all those SOs, which means faster startup.
With the KDE_IS_PRELINKED=1 line in the config file, kdeinit is instructed to no longer preload all those SOs. |
|
| Back to top |
|
|
Joe
n00b
Joined: 01 Jul 2002
Posts: 70
Location: Europe
|
| Posted: Thu Sep 07, 2006 12:54 am Post subject: |
|
|
| piercey wrote: | | zxy wrote: |
I wonder what would happen with oppenoffice if it could be compiled for amd64, and prelinked then. |
Well no need to wonder, OpenOffice 2.0.4 compiles perfectly on amd64 and is already lightning fast compared to the -bin. |
Is this true - it actually compiles&runs on AMD64?? Damn, good job...
Regards, Joe |
|
| Back to top |
|
|
Kvetch
Guru
Joined: 29 Apr 2004
Posts: 318
Location: /dev/null, VA
|
| Posted: Sat Sep 09, 2006 1:13 am Post subject: |
|
|
So I finally had a chance to setup prelinking. I can't say positively that it is much faster than before. I didn't feel that it was slow before though so if it is then it is only a small bit.
I tried the following after I installed some new apps.
| Code: | SD6 lib # prelink -aqR
Segmentation fault
SD6 lib # env-update
>>> Regenerating /etc/ld.so.cache...
SD6 lib # prelink -aqR
Segmentation fault |
I re-ran | Code: | | SD6 lib # prelink -afR |
then just tried it again to see if it would squawk
| Code: | SD6 lib # prelink -aqR
prelink: /usr/bin/nasl: Could not find one of the dependencies
prelink: /usr/bin/lddlibc4: Using /lib32/ld-linux.so.2, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib32/misc/glibc/pt_chown: Using /lib32/ld-linux.so.2, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib32/misc/glibc/getconf/POSIX_V6_ILP32_OFF32: Using /lib32/ld-linux.so.2, not /lib/ld-linux.so.2 as dynamic linker
prelink: /usr/lib32/misc/glibc/getconf/POSIX_V6_ILP32_OFFBIG: Using /lib32/ld-linux.so.2, not /lib/ld-linux.so.2 as dynamic linker
|
Any thoughts on this?
Thanks,
Nick |
|
| Back to top |
|
|
Guenther Brunthaler
Apprentice
Joined: 22 Jul 2006
Posts: 217
Location: Vienna
|
| Posted: Sat Sep 09, 2006 7:57 pm Post subject: |
|
|
| Kvetch wrote: |
prelink: /usr/lib32/misc/glibc/getconf/POSIX_V6_ILP32_OFFBIG: Using /lib32/ld-linux.so.2, not /lib/ld-linux.so.2 as dynamic linker
[/code]
Any thoughts on this? |
Seems to be an issue with mixed 32 and 64-bit libraries.
I admit, I have written my guide with pure 64-bit systems in mind, i. e. without any 32-bit only libraries.
I have also written an alternative guide, which works well with smaller 32-bit installations, but has a fallback to the "standard method" (from the Gentoo prelink guide) in case the address space is exhausted.
This guide should also work with 64 bit systems, and is slightly different from the approach presented in the current thread. Perhaps it might work better in your case; just give it a try: "New prelinking approach (may be more effective for IA-32)" (https://forums.gentoo.org/viewtopic-t-495997-highlight-.html).
If that modified prelinking approach should also lead to crashes of the prelink tool as you have encountered it, it will automatically revert to the old standard approach, which might be less effective but should work always.
If that approach *still* crashes, it's very likely that the shared library may contain illegal/damaged code, making the "prelink" tool crash as it tries to relocate that library.
In this case it may be helpful to just re-emerge the faulty shared library again.
In order to find the name of the failing library, use the -v option to prelink.
Then use "equery belongs <library-pathname>" in order to find out to which package the library belongs, and ultimately do an "emerge --oneshot <package-name>" in order to re-emerge that package. (Hint: "equery" is part of the gentoolkit package.)
And let's hope the library can be prelinked then! |
|
| Back to top |
|
|
castorilo
Apprentice
Joined: 25 Dec 2002
Posts: 157
|
| Posted: Wed Sep 13, 2006 8:56 pm Post subject: |
|
|
| reynolds531 wrote: | | Phenax wrote: | | Elfan wrote: | | Is there anything specific to KDE in your guide or were you just using that as an example? |
Nope, you'll just see best results using KDE because it has so many shared libraries. |
But wouldn't gnome benefit from this as well? Or is there something different about the way kde uses shared libraries?
|
C++ object oriented code uses something called "virtual tables" which is basically a list of pointer to the functions that a class has (I could get into more detail, if anyone is interested). Virtual tables require a ton of relocations. Most C++ code requries orders of magnitude more relocations than an equivalent C code.
Doing the relocations take a long time and is done by the linker at application start up time. KDE is a bunch of C++ applications and is greatly affected by this. Gnome is mostly written in C and does not require a significant amount of relocations.
Prelink eliminates the need for the linker to do relocations (I could go into more detail if you want). So C++ applications (KDE) will see an improvement in startup time while C code (Gnome) wont see any significant difference.
You can see how much time you will save for a particular application, run this:
| Code: |
LD_DEBUG=statistics DISPLAY= time konqueror
|
In my machine, before prelinking:
| Code: |
4136:
4136: runtime linker statistics:
4136: total startup time in dynamic loader: 282328590 clock cycles
4136: time needed for relocation: 272830518 clock cycles (96.6%)
4136: number of relocations: 24768
4136: number of relocations from cache: 62746
4136: number of relative relocations: 39607
4136: time needed to load objects: 8829311 clock cycles (3.1%)
konqueror: cannot connect to X server
4136:
4136: runtime linker statistics:
4136: final number of relocations: 25093
4136: final number of relocations from cache: 62746
real 0m0.126s
user 0m0.104s
sys 0m0.015s
|
In my machine, after prelinking:
| Code: |
2433:
2433: runtime linker statistics:
2433: total startup time in dynamic loader: 12124156 clock cycles
2433: time needed for relocation: 3147342 clock cycles (25.9%)
2433: number of relocations: 0
2433: number of relocations from cache: 560
2433: number of relative relocations: 0
2433: time needed to load objects: 8240776 clock cycles (67.9%)
konqueror: cannot connect to X server
2433:
2433: runtime linker statistics:
2433: final number of relocations: 0
2433: final number of relocations from cache: 560
real 0m0.035s
user 0m0.004s
sys 0m0.007s
|
so I save about 0.1 seconds when starting up konqueror; instead of doing 24768 relocations, now the linker does 0
this is more noticiable on slower machines, and considering this is for each one of kde apps, it adds up.
Replace konqueror for whichever application you want to profile.
If you try it for a gnome app, you will not see anywhere near those 24768 relocations. |
|
| Back to top |
|
|
Bones McCracker
Veteran
Joined: 14 Mar 2006
Posts: 1611
Location: U.S.A.
|
| Posted: Tue Oct 17, 2006 10:59 am Post subject: |
|
|
Good stuff! I'm going to put gentoo on my x64 just so I can try this.
Meanwhile, can anyone offer insight into these errors, which I'm getting on a 686 machine?
| Code: | brendlerjg@tempest ~ $ sudo prelink -amR
prelink: /usr/bin/Xvfb: NOBITS section followed by non-NOBITS section in the same segment
prelink: /usr/bin/Xnest: NOBITS section followed by non-NOBITS section in the same segment |
I've got nVidia-legacy-drivers running (and am using a framebuffer splash), but I added the lib to the path_mask in /env.d/60prelink and it stopped complaining. Is this related?
Thanks. |
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2790
Location: Pittsburgh, PA, USA
|
| Posted: Tue Oct 17, 2006 3:04 pm Post subject: |
|
|
Just wanted to clarify something as I know a little about this...
| Quote: | | That would be nice, but the address space of amd64 at present is only 2^40 bytes (about a terabyte). I think em64t is similar, no idea about ppc or anything else. |
That is the physical address space (max amount of ram you could possibly connect if somehow you had a source of 128 GB or larger dimms) and not virtual. Virtual is currently 2^48 on amd64.
http://en.wikipedia.org/wiki/AMD64
_________________
John
My gentoo overlay
Instructons for overlay |
|
| Back to top |
|
|
mattb0611
n00b
Joined: 07 Apr 2006
Posts: 18
|
| Posted: Tue Oct 17, 2006 4:52 pm Post subject: |
|
|
I just finished the steps and restarted KDE. I've got a fully-installed KDE system on AMD64, and had no major errors running prelink with --force. Everything feels faster...when I start KDE, all of the programs that run on start-up are already there when my splash screen goes away--that never happened before.
Thank you! |
|
| Back to top |
|
|
treor
Apprentice
Joined: 03 May 2005
Posts: 174
Location: germany
|
| Posted: Tue Oct 17, 2006 5:12 pm Post subject: |
|
|
oh something to do for the weekend 
@ gunther:
your writting very good things. go on |
|
| Back to top |
|
|
michaeljc
n00b
Joined: 17 Oct 2006
Posts: 2
|
|
| Back to top |
|
|
drescherjm
Advocate
Joined: 05 Jun 2004
Posts: 2790
Location: Pittsburgh, PA, USA
|
|
| Back to top |
|
|
sonicbhoc
Veteran
Joined: 24 Oct 2005
Posts: 1805
Location: In front of the computer screen
|
| Posted: Sun Oct 29, 2006 2:34 am Post subject: |
|
|
| vipernicus wrote: | | Even better: prelink + --hash-styles=gnu |
So, how do I go about using these new hash styles? do I have to recompile my whole system? and will the performance gain also affect x86? |
|
| Back to top |
|
|
kernelOfTruth
Watchman
Joined: 20 Dec 2005
Posts: 6111
Location: Vienna, Austria; Germany; hello world :)
|
|
| Back to top |
|
|
trevormtb
n00b
Joined: 04 Dec 2004
Posts: 22
Location: Canada
|
| Posted: Fri Nov 03, 2006 3:46 am Post subject: |
|
|
As an aside, be sure to use the newest kernel possible if you have a SATA drive on amd64. There's been a large amount of development on SATA drivers, so much that the difference between loading firefox with a 2.6.16 kernel compared to a new 2.6.18 kernel a difference of about 20 seconds or so.
Add in this great advice on prelinking, and the load is around 2 seconds or less! |
|
| Back to top |
|
|
radagast
Apprentice
Joined: 20 Mar 2004
Posts: 217
Location: sydney, .au
|
| Posted: Mon Jan 15, 2007 1:36 am Post subject: which version of prelink |
|
|
ah yes
i bought a new system a year ago and reinstalled gentoo from scratch, but forgot about prelinking.
because i'm scared i've installed the latest AMD64-stable version of prelink, 20050314. i remember 2005, and i can't imagine how it could be more stable for us than the 20060712 version. i don't see anything scary in bugzilla. but i'm still scared.
i bet if you're reading a thread like this you're the sort of person who just emerges the latest version, stable or not... so i'd like to know if all you guys having such good success are using version 20060712.
thanks... |
|
| Back to top |
|
|
energyman76b
Advocate
Joined: 26 Mar 2003
Posts: 2048
Location: Germany
|
| Posted: Mon Jan 15, 2007 4:46 pm Post subject: |
|
|
I am using 20060712, for a looong time. No problems.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
I identify as a dirty penismensch. |
|
| Back to top |
|
|
Phenax
l33t
Joined: 10 Mar 2006
Posts: 972
|
| Posted: Mon Jan 15, 2007 6:04 pm Post subject: |
|
|
| energyman76b wrote: | | I am using 20060712, for a looong time. No problems. |
++;
System runs ~amd64  |
|
| Back to top |
|
|
devsk
Advocate
Joined: 24 Oct 2003
Posts: 2995
Location: Bay Area, CA
|
| Posted: Thu Mar 08, 2007 3:06 am Post subject: |
|
|
prelink makes my kde install swell by ~100MB i.e. up by 25% (tested on a freshly installed full kde before and after prelink with 'du -sk /usr/kde/3.5'). Now, that means you are doing that much more disk IO when applications load. So, the trade off is between:
1. extra disk IO when using prelink while loading the dependent libraries (remember, you have to load the dependent libraries from disk into memory no matter library/exe is prelinked or not).
2. extra relocations when not using prelink.
Relocations happen in memory. 25000 relocations will typically take less than 0.2 second on most modern CPUs (tested on many >= 3Ghz).
Disk IO is an expensive operation even in year 2007: hard drives haven't caught up, and they probably never will.
So, if you have a modern enough machine, by using prelink you have most likely slowed down your kde install. I know I did.
OTOH, if you have an older processor and slower memory, you relocations will be much slower and you will benefit from prelink a LOT.
PS: I forgot to mention that I am using amd64. |
|
| Back to top |
|
|
energyman76b
Advocate
Joined: 26 Mar 2003
Posts: 2048
Location: Germany
|
| Posted: Thu Mar 08, 2007 3:31 am Post subject: |
|
|
@devsk
astonishingly startup times for KDE can go down as much as over 90% when using prelink.
So your math is flawed or you are using wrong premisses.
One problem is that relocations in c++ apps are really, really slow. And 100mb is not much. Modern harddisks can deliver 100mb in less than 3 seconds.
_________________
Study finds stunning lack of racial, gender, and economic diversity among middle-class white males
I identify as a dirty penismensch. |
|
| Back to top |
|
|
wuzzerd
Guru
Joined: 05 Jan 2005
Posts: 466
Location: New Mexico
|
| Posted: Thu Mar 08, 2007 3:38 am Post subject: |
|
|
| Vielen Dank Guenther. This really helps Kde. I remember running it on a P-II with 128meg of ram |
|
| Back to top |
|
|
|
Desktop Environments
