Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Optimized replacement for glibc
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Off the Wall
View previous topic :: View next topic  

What do you think about an optimized replacement for glibc such as libfreevec NG?
This is highly important for Linux to become a serious competitor with other OS on different architectures
31%
 31%  [ 5 ]
I'm interested in that progress.
31%
 31%  [ 5 ]
glibc is stable and functional. Never touch a running system.
37%
 37%  [ 6 ]
Total Votes : 16

Author Message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1181
Location: PB, Germany

PostPosted: Fri Jan 28, 2011 11:38 am    Post subject: Optimized replacement for glibc Reply with quote

glibc/Linux is not optimized for most architectures, what about an optimized replacement?

Coming from gentoo-powerpc I started a discussion in "Gentoo on PPC" about a potential replacement for glibc: freevec, as of today in portage the ppc-only sys-libs/libfreevec.
Sorry for cross-posting... For not overlooking that topic down in the subforum, I would like to call attention to the main audience too and point to that topic.

While reading History of libfreevec and libfreevec NG, I began to reconsider how optimized Linux actually is.

Now since this issue is not all about PowerPC and not restricted to Altivec, it is about all architectures. Today GNU/Linux / glibc is not even optimized for x86 or amd64/x86_64. All architectures could benefit from libfreevec and that is what the next libfreevec NG-project is aiming at.
Other Operating Systems already make use of architecture specific features. When Apple still was developping on PPC they did use Altivec in OSX. I'm sure Microsoft with its binary OS does the same. How can we run benchmarks against those OS without even using all CPU features?

We at Gentoo-Linux have been always trying to adapt the software as much as possible to the underlying architecture. Hardware is not fast or slow, it is just the software that fits well or not. I guess that is what markos, the developer of libfreevec, thought too when he wrote about further plans of libfreevec NG. That was 2009. Today he told me, work on libfreevec NG is still in progress but silent and without priority due to being a side project. I think after what he said about the glibc there should be at least an optimized library for the main architectures in order to make Linux a reasonable OS for these.
I wonder that nobody in the past had a similar purpose like markos.

Moreover after having a stable libfreevec there is still the kernel not optimized in large areas, but that's another topic...

With regards,
Massimo
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67, 2G(dev.notes[pdf]) | amd64:Core2Duo,i5-3470 | amd64-prefix:OpenSuse@Core2Quad | Lila-Theme
Back to top
View user's profile Send private message
aCOSwt
Moderator
Moderator


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Fri Jan 28, 2011 12:21 pm    Post subject: Reply with quote

Well, considering the time I spent 30 years ago trying to optimize the libc and hashcoding strings in order to avoid string comparisons... I am...

Interested !

Particularily regarding the compared performances for mem* functions. Amazing ! And interesting indeed is this 48 bytes approx threshold 8O

Of course, (and the author admits this well regarding glibc for x86 and x86_64 str* functions), "due to [their] extended use in many applications, [str* functions are] usually assembly optimised for most architectures (actually most string functions are optimised)"

So the remaining margin in rather poor.
Moreover... are applications highly relying on str* functions actually caring for speed ?

So... interested indeed but not to a point of throwing the glibc away... :P
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1181
Location: PB, Germany

PostPosted: Fri Jan 28, 2011 1:03 pm    Post subject: Reply with quote

aCOSwt wrote:
Of course, (and the author admits this well regarding glibc for x86 and x86_64 str* functions), "due to [their] extended use in many applications, [str* functions are] usually assembly optimised for most architectures (actually most string functions are optimised)"

The quote actually is:
    strcmp() ....it most likely is optimised for x86 and x86_64,...In any case the glibc implementation compares 32-bit (or 64-bit depending on the arch) blocks where possible.... we also use modern SIMD units (AltiVec for PowerPC CPUs, and in the future SSE for x86 CPUs).
Use of SSE for example is missing in glibc as I understand.
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67, 2G(dev.notes[pdf]) | amd64:Core2Duo,i5-3470 | amd64-prefix:OpenSuse@Core2Quad | Lila-Theme
Back to top
View user's profile Send private message
aCOSwt
Moderator
Moderator


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Fri Jan 28, 2011 1:34 pm    Post subject: Reply with quote

Massimo B. wrote:
Use of SSE for example is missing in glibc as I understand.


To be honest, I do not know the status of this initiative but, taking into account platforms' specific instruction sets was planned for glibc 2.10 implementations thanks to the STT_GNU_IFUNC symbol type :arrow: http://udrepper.livejournal.com/20948.html
Back to top
View user's profile Send private message
Massimo B.
Veteran
Veteran


Joined: 09 Feb 2005
Posts: 1181
Location: PB, Germany

PostPosted: Fri Jan 28, 2011 3:20 pm    Post subject: Reply with quote

You're right, Re: Proposal for CPU dispatching in libc. I will look into that.
_________________
ppc:PowerBook5,8 15"(1440)-G4/1.67, 2G(dev.notes[pdf]) | amd64:Core2Duo,i5-3470 | amd64-prefix:OpenSuse@Core2Quad | Lila-Theme
Back to top
View user's profile Send private message
tomk
Administrator
Administrator


Joined: 23 Sep 2003
Posts: 7219
Location: Sat in front of my computer

PostPosted: Mon Jan 31, 2011 3:59 pm    Post subject: Reply with quote

Moved from Gentoo Chat to Off the Wall as it's not related to Gentoo.
_________________
Search | Read | Answer | Report | Strip
Back to top
View user's profile Send private message
hellbringer
Tux's lil' helper
Tux's lil' helper


Joined: 12 Feb 2003
Posts: 82

PostPosted: Mon Jan 31, 2011 4:13 pm    Post subject: Re: Optimized replacement for glibc Reply with quote

Massimo B. wrote:
glibc/Linux is not optimized for most architectures, what about an optimized replacement?

Coming from gentoo-powerpc I started a discussion in "Gentoo on PPC" about a potential replacement for glibc: freevec, as of today in portage the ppc-only sys-libs/libfreevec.
Sorry for cross-posting... For not overlooking that topic down in the subforum, I would like to call attention to the main audience too and point to that topic.

While reading History of libfreevec and libfreevec NG, I began to reconsider how optimized Linux actually is.

Now since this issue is not all about PowerPC and not restricted to Altivec, it is about all architectures. Today GNU/Linux / glibc is not even optimized for x86 or amd64/x86_64. All architectures could benefit from libfreevec and that is what the next libfreevec NG-project is aiming at.
Other Operating Systems already make use of architecture specific features. When Apple still was developping on PPC they did use Altivec in OSX. I'm sure Microsoft with its binary OS does the same. How can we run benchmarks against those OS without even using all CPU features?

We at Gentoo-Linux have been always trying to adapt the software as much as possible to the underlying architecture. Hardware is not fast or slow, it is just the software that fits well or not. I guess that is what markos, the developer of libfreevec, thought too when he wrote about further plans of libfreevec NG. That was 2009. Today he told me, work on libfreevec NG is still in progress but silent and without priority due to being a side project. I think after what he said about the glibc there should be at least an optimized library for the main architectures in order to make Linux a reasonable OS for these.
I wonder that nobody in the past had a similar purpose like markos.

Moreover after having a stable libfreevec there is still the kernel not optimized in large areas, but that's another topic...

With regards,
Massimo

You will be hard pressed to find a program in desktop or server environments whose bottleneck was an unoptimized Glibc (most programs today are IO bound). At the level in the stack Glibc operates, clarity and stability trumps a 10% gain in speed any time of the day.
_________________
There is a lot of novelty and truth in what you say, but that which is true is not novel and that which is novel is not true.
Back to top
View user's profile Send private message
aidanjt
Veteran
Veteran


Joined: 20 Feb 2005
Posts: 1101
Location: Rep. of Ireland

PostPosted: Mon Jan 31, 2011 8:18 pm    Post subject: Reply with quote

A shitty libc can slow down I/O due to overhead. And clarity and stability in glibc? :P
_________________
juniper wrote:
you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1563
Location: U.S.A.

PostPosted: Tue Feb 01, 2011 2:43 am    Post subject: Reply with quote

If architecture-specific optimizations can't be carried out by the compiler, then they are likely to result in non-portable code, as developers who don't care about platform Y or Z won't write code for it.
Back to top
View user's profile Send private message
aCOSwt
Moderator
Moderator


Joined: 19 Oct 2007
Posts: 2537
Location: Hilbert space

PostPosted: Tue Feb 01, 2011 7:47 am    Post subject: Reply with quote

BoneKracker wrote:
If architecture-specific optimizations can't be carried out by the compiler, then they are likely to result in non-portable code, as developers who don't care about platform Y or Z won't write code for it.

<joking>
Well... I understand that someone having a processor running at 17.4 Ghz is not in desperate need for code optimization... :wink:
</joking>
This being said, I do not understand your point as long as optimized assembly coded snippets are proposed as a non exlusive-or alternative to the standard C routine.
When I run OOO, I often wish the code would not have been written for the sole sake of portability... :evil:
And... At the end of the day... If there are people choosing their arch irrespective of the requirements of the apps they want to run... then... I will be sorry for them ! :twisted:
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 16102
Location: Colorado

PostPosted: Tue Feb 01, 2011 3:52 pm    Post subject: Reply with quote

aCOSwt wrote:
And... At the end of the day... If there are people choosing their arch irrespective of the requirements of the apps they want to run... then... I will be sorry for them ! :twisted:
Which becomes a different game when it is a purchased piece of software in an environment with entrenched vendors.
_________________
lolgov. 'cause where we're going, you don't have civil liberties.

In Loving Memory
1787 - 2008
Back to top
View user's profile Send private message
shickapooka800
Guru
Guru


Joined: 05 Dec 2004
Posts: 304
Location: no

PostPosted: Tue Feb 01, 2011 4:36 pm    Post subject: Reply with quote

how quickly optimism dies in OTW.

OP: check this out!
POST: hey that is pretty interesting, I'll look into it
POST: I am skeptical , but so far it looks good!
MOD: Moved from blah to OTW as it is not related to Gentoo
POST: this is useless because of this, this and this
POST: not to mention that it is more a problem with this
POST: it's obama's fault
Back to top
View user's profile Send private message
aidanjt
Veteran
Veteran


Joined: 20 Feb 2005
Posts: 1101
Location: Rep. of Ireland

PostPosted: Tue Feb 01, 2011 4:52 pm    Post subject: Reply with quote

shickapooka800 wrote:
how quickly optimism dies in OTW.

OP: check this out!
POST: hey that is pretty interesting, I'll look into it
POST: I am skeptical , but so far it looks good!
MOD: Moved from blah to OTW as it is not related to Gentoo
POST: this is useless because of this, this and this
POST: not to mention that it is more a problem with this
POST: it's obama's fault

All hope successfully squashed.
_________________
juniper wrote:
you experience political reality dilation when travelling at american political speeds. it's in einstein's formulas. it's not their fault.
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1563
Location: U.S.A.

PostPosted: Tue Feb 01, 2011 7:50 pm    Post subject: Reply with quote

"Not Gentoo-specific! Throw him to the dogs!"

Being a Moderator can be a thankless and boring job. They must be allowed their bear-baiting.
Back to top
View user's profile Send private message
Ant P.
Advocate
Advocate


Joined: 18 Apr 2009
Posts: 2272
Location: UK

PostPosted: Tue Feb 01, 2011 7:53 pm    Post subject: Reply with quote

AidanJT wrote:
shickapooka800 wrote:
how quickly optimism dies in OTW.

OP: check this out!
POST: hey that is pretty interesting, I'll look into it
POST: I am skeptical , but so far it looks good!
MOD: Moved from blah to OTW as it is not related to Gentoo
POST: this is useless because of this, this and this
POST: not to mention that it is more a problem with this
POST: it's obama's fault

All hope successfully squashed.

Hope and Chroot!
Back to top
View user's profile Send private message
Bones McCracker
Veteran
Veteran


Joined: 14 Mar 2006
Posts: 1563
Location: U.S.A.

PostPosted: Tue Feb 01, 2011 8:07 pm    Post subject: Reply with quote

:lol:
Back to top
View user's profile Send private message
cach0rr0
Moderator
Moderator


Joined: 13 Nov 2008
Posts: 4121
Location: Houston, Republic of Texas

PostPosted: Tue Feb 01, 2011 8:09 pm    Post subject: Reply with quote

shickapooka800 wrote:
how quickly optimism dies in OTW.

OP: check this out!
POST: hey that is pretty interesting, I'll look into it
POST: I am skeptical , but so far it looks good!
MOD: Moved from blah to OTW as it is not related to Gentoo
POST: this is useless because of this, this and this
POST: not to mention that it is more a problem with this
POST: it's obama's fault


genuinely laughed audibly. cheers for that.
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2025
Location: Germany

PostPosted: Wed Feb 02, 2011 6:23 pm    Post subject: Reply with quote

so much bullshit in the first post alone.


glibc is so fucking huge because it is filled with optimizations for x86/amd64 and other architectures.
_________________
AidanJT wrote:

Libertardian denial of reality is wholly unimpressive and unconvincing, and simply serves to demonstrate what a bunch of delusional fools they all are.

Satan's got perfectly toned abs and rocks a c-cup.
Back to top
View user's profile Send private message
pigeon768
l33t
l33t


Joined: 02 Jan 2006
Posts: 669

PostPosted: Wed Feb 02, 2011 7:03 pm    Post subject: Reply with quote

Another of the reasons glibc is so big and slow is because it supports everything that ever existed ever. There is no alternative to glibc. No libc that supports as many C standards, targets, platforms required by a complete linux distribution. You'll always have at least a few packages that depend on some obscure feature or quirk of glibc.
_________________
My political bias.
Back to top
View user's profile Send private message
energyman76b
Advocate
Advocate


Joined: 26 Mar 2003
Posts: 2025
Location: Germany

PostPosted: Wed Feb 02, 2011 7:45 pm    Post subject: Reply with quote

and glibc is slow? Really?
_________________
AidanJT wrote:

Libertardian denial of reality is wholly unimpressive and unconvincing, and simply serves to demonstrate what a bunch of delusional fools they all are.

Satan's got perfectly toned abs and rocks a c-cup.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Off the Wall All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum