Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Support for Function Multi-Versioning (FMV)
View unanswered posts
View posts from last 24 hours

Goto page 1, 2  Next  
Reply to topic    Gentoo Forums Forum Index Gentoo Chat
View previous topic :: View next topic  

Shall portage support Function Multi-Versioning?
yes
15%
 15%  [ 3 ]
no
40%
 40%  [ 8 ]
don't care
45%
 45%  [ 9 ]
Total Votes : 20

Author Message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 632
Location: granada, spain

PostPosted: Fri Apr 06, 2018 3:35 pm    Post subject: Support for Function Multi-Versioning (FMV) Reply with quote

[Moderator note: changed title from support for fmv to Support for Function Multi-Versioning (FMV) because FMV in computing commonly means "Full Motion Video", which is completely unrelated to the topic of this thread; changed poll question from shall portage support fmv? to Shall portage support Function Multi-Versioning? for the same reason. -Hu]

hey!
since i use my desktop (ivybridge) to build packages for my laptop (skylake) i would really like to see function multi-versioning support in portage...
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17215

PostPosted: Fri Apr 06, 2018 4:43 pm    Post subject: Reply with quote

Quote:
optimized for Intel
So, no.

Although I voted "don't care" as long as the extraneous code isn't clogging up my system.
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5008

PostPosted: Fri Apr 06, 2018 6:36 pm    Post subject: Reply with quote

Someone should remind Intel that GCC exists.

You're not going to get much benefit by building fat binaries for two *lake chips in any case. If you're that hard pressed for speed, better to use hardware without Meltdown.
_________________
*.ebuild // /etc/service/*
Back to top
View user's profile Send private message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 632
Location: granada, spain

PostPosted: Fri Apr 06, 2018 6:50 pm    Post subject: Reply with quote

pjp wrote:
Quote:
optimized for Intel
So, no.

Although I voted "don't care" as long as the extraneous code isn't clogging up my system.

i wonder if you've cared to read the whole article or just the title... :roll:
the concept is about building application that runs on old generation cpu's and may leverage on more modern cpu extensions as well.
in my case i would be able to build an application that may use avx2 extensions and run on both my desktop (without avx2) and my laptop (with avx2).
and yes, even amd or arm processors would benefit from such a technique, it's not just another intel thing...
i know the most benefit are for binary distributions, however it would be just another nice (imho) choice even for gentoo... :)
Back to top
View user's profile Send private message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 632
Location: granada, spain

PostPosted: Fri Apr 06, 2018 6:56 pm    Post subject: Reply with quote

Ant P. wrote:
Someone should remind Intel that GCC exists.

You're not going to get much benefit by building fat binaries for two *lake chips in any case. If you're that hard pressed for speed, better to use hardware without Meltdown.

uh? those are gcc-6 and glibc-2-26 new features... 8O
i'll be happy to buy a power9 desktop when they'll start to sell them at a reasonable price and a risc-v laptop when there will be one available...
no intel/amd/arm fanboy here, just someone who buys hardware that suits my needs... :wink:
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 40809
Location: 56N 3W

PostPosted: Fri Apr 06, 2018 7:52 pm    Post subject: Reply with quote

skunk,

I'm a don't care leaning towards no. This feature cannot be implemented without a performance penalty.
I can't spare the RAM and in some cases the HDD space for the bloat.
Being a Gentoo user everything is tailored to the individual target anyway.

I can see binary distros and purveyors of binaries liking it but in my opinion, its rather like systemd.
Use it if you want to, just don't force it on me.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17215

PostPosted: Fri Apr 06, 2018 9:19 pm    Post subject: Reply with quote

skunk wrote:
i wonder if you've cared to read the whole article or just the title... :roll:
I didn't presume that the title would be false or inaccurate, and as I don't use Intel CPUs, the rest seemed not relevant. So, yeah. :roll:
_________________
The whole system has to go. The modern criminal justice system is incompatible with Neuroscience. --Sapolsky
Back to top
View user's profile Send private message
Tony0945
Advocate
Advocate


Joined: 25 Jul 2006
Posts: 2457
Location: Illinois, USA

PostPosted: Fri Apr 06, 2018 10:22 pm    Post subject: Reply with quote

Don't see any need. Run the appropriate march/mtune in your make.conf. This is for binary packages not source packages. If you're building for distribution to Ubuntu or Arch, yeah! There may be a small set of Gentoo users building binaries for Windows or cross-comopling. So, no, no special effort. Developer effort should go into fixing bugs or speeding up portage.

EDIT: spelling


Last edited by Tony0945 on Mon Apr 09, 2018 1:02 am; edited 1 time in total
Back to top
View user's profile Send private message
geki
Advocate
Advocate


Joined: 13 May 2004
Posts: 2285
Location: Germania

PostPosted: Sat Apr 07, 2018 7:29 am    Post subject: Reply with quote

My POV as software developer:
Luckily, most (if not all) software, that cares for performance and is in need thereof, does this or similar already. AFAI watch compiles flying by (i.e.: ffmpeg, mesa and others). :o

So, nothing really to be done; from this POV.
_________________
boost|select libs to build|slotable|python-buildid

hear hear
Back to top
View user's profile Send private message
bunder
Bodhisattva
Bodhisattva


Joined: 10 Apr 2004
Posts: 5573

PostPosted: Sat Apr 07, 2018 9:31 am    Post subject: Reply with quote

I would rather upgrade the ivybridge to something newer than skylake and use distcc. 8)
_________________
overlay | patches
Neddyseagoon wrote:
The problem with leaving is that you can only do it once and it reduces your influence.
Back to top
View user's profile Send private message
The_Document
Apprentice
Apprentice


Joined: 03 Feb 2018
Posts: 235

PostPosted: Sun Apr 08, 2018 10:17 pm    Post subject: Reply with quote

No cause its likely to break things. Developers should do it right and implement modern instruction sets right into their code.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 6649

PostPosted: Mon Apr 09, 2018 4:41 am    Post subject: Reply with quote

wonderful option for binaries distro
totally useless for gentoo, space & memory eater for no benefits.
Back to top
View user's profile Send private message
Goverp
Guru
Guru


Joined: 07 Mar 2007
Posts: 554

PostPosted: Mon Apr 09, 2018 7:25 am    Post subject: Reply with quote

a) Gentoo is about choice
b) Gentoo is a meta-distribution
Ergo, people should be able to build FMV distributions using Gentoo.
But I voted "Don't care"
_________________
Greybeard
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Mon Apr 09, 2018 9:34 am    Post subject: Reply with quote

geki wrote:
My POV as software developer:

That's exactly what the whole story is about.
Quote:
Luckily, most (if not all) software, that cares for performance and is in need thereof, does this or similar already.

This is plainly false: Only software that cares for performance and has bundled all libraries (at least the ones which could gain from the optimization) can do this.
For libraries which are not bundled but which are pulled in as dependencies, this is exactly what portage would need to care about.
Typical example: numpy (which would profit from an optimized cblas/lapack)
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Mon Apr 09, 2018 9:51 am    Post subject: Reply with quote

NeddySeagoon wrote:
This feature cannot be implemented without a performance penalty.

A performance penalty only at startup (depending on the glibc implementation perhaps even only once at startup of the machine).
Quote:
I can't spare the RAM and in some cases the HDD space for the bloat.

RAM cost is only a few bytes in glibc. HDD cost is none except if you target multiple architectures. In the latter case, it is only the code for the additional architectures itself which take HDD space.
Quote:
I can see binary distros and purveyors of binaries

Gentoo users are such purveyors if they build binary packages for several machines (which not necessarily all have the same processor features). I think it is a rather frequent setting to have one "build" machine and to distribute the binaries to other systems. Currently, you have to choose the lowest common denominator on these systems.
Quote:
Use it if you want to, just don't force it on me.

Nothing would be forced: From the user side an implementation would probably be rather similar to how ABI_X86=.... currently behaves, i.e. you select "your" architecture. If you do not select a second/third/forth/... architecture nothing changes for you (also no disk space overhead), except that perhaps a different directory path is used for the generated library (lib/arch/ instead of lib/)

Edit: Typos and reformulations to make things clearer.
Back to top
View user's profile Send private message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 632
Location: granada, spain

PostPosted: Mon Apr 09, 2018 10:27 am    Post subject: Reply with quote

mv wrote:
Quote:
I can see binary distros and purveyors of binaries

Gentoo users are such purveyors if they build binary packages for several machines (which not necessarily all have the same processor features). I think it is a rather frequent setting to have one "build" machine and to distribute the binaries to other systems. Currently, you have to choose the lowest common denominator on these systems.

that's exactly what i mean as use case, moreover in my specifically case:
  • i need to build packages for a modern mobile weak but power efficient processor and i want to use an older but powerful processor for that for obvious reasons
  • my job consist in deploying private clouds for my customers using linux containers and want to be able to move those containers around between an heterogeneous set of hosts without having to rebuild the container images on the destination host
of course, as mv pointed out, i'm sure it could be absolutely opt in without jamming it into anybody's throat... :wink:
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 40809
Location: 56N 3W

PostPosted: Mon Apr 09, 2018 2:04 pm    Post subject: Reply with quote

skunk,

I use (cross) distcc in these situations.
My main amd64 /no-multilib/ system builds for arm, arm64, x86 and another low power amd64 system.

I understand the problem but adding bloat is not a part of the solution on a source based distro,

Going back to lowest common denominator builds, all systems wait at the same speed, so its only a few performance critical apps where in actually makes a difference.
e.g. libreoffice spends most of its time waiting for keystrokes.
Video processing can use all the speed it can get, but you don't do that on out of date hardware if you can avoid it.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Mon Apr 09, 2018 3:03 pm    Post subject: Reply with quote

@skunk: Do you plan to implement it, e.g. in a google SOC project?

Otherwise, I am afraid that this whole poll is rather useless. A poll alone will not inspire an implementation. An implementation is rather tedious; much harder than ABI_X86= since in a sense it is a strict superset of this.
When I think how many years ABI_X86 took... not only the pure implementation itself, but also the fights how it should be implemented and many competing half-finished implementation attempts...
Back to top
View user's profile Send private message
skunk
l33t
l33t


Joined: 28 May 2003
Posts: 632
Location: granada, spain

PostPosted: Wed Apr 11, 2018 2:30 pm    Post subject: Reply with quote

ok guys, i was just asking some opinions about something i've believed could have been useful...
i'll keep doing the lowest common denominator strategy with my servers farm and on my desktop/laptop.

thank you
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5039
Location: The Peanut Gallery

PostPosted: Fri May 11, 2018 2:41 pm    Post subject: Reply with quote

I think there's a definite use-case or three for it, but not as any sort of default, nor do I believe it should require any changes to portage.
eg: I definitely believe skunk should be able to build suitable binpkgs for his cloud, that might adjust to the local CPU.

I don't understand why it can't be a simple addition to CFLAGS or CPPFLAGS, LDFLAGS et al, for those applications which know how to use it.

I'd imagine most don't, so I don't see the point in it being anything that happens at mangler level, but rather in a ROOT, with suitable config-root.

What am I missing mv? (why do you say it needs as much work at mangler level, or more, than ABI_X86, which sucks imo.)
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Fri May 11, 2018 4:36 pm    Post subject: Reply with quote

steveL wrote:
What am I missing mv? (why do you say it needs as much work at mangler level, or more, than ABI_X86, which sucks imo.)

If you want that a library supports this thing, you have to compile it several times with different CFLAGS and install the result into various /usr/lib/$ARCH directories (or whatever the convention for the location is); the point is that the build system of the library has probably no special support for this.
Only the calling package must be aware of the mechanism, and it needs to be passed somehow at compile time the possible $ARCH values (and directory names) for which the libraries are available.

So a way to implement it would be analogous to ABI_X86: The ebuilds for the libraries inherit from some eclass which builds them multiple times, depending on some USE (or EXPAND_USE) variable, and the calling package has the same USE-variables and corresponding USE-dependencies on the relevant libraries.

I am not claiming that this is the only way how it can be done, but given that this is the approach chosen for multilib support and the underlying problem is in principle the same (different libraries are needed for different processor versions/modes), it is natural to use the same approach.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 6649

PostPosted: Fri May 11, 2018 6:45 pm    Post subject: Reply with quote

mv wrote:
you have to compile it several times with different CFLAGS

Maybe i didn't get it fully, but i think the idea is not building a cross compiler or different arch, but running optimizations (not gcc ones, but code specific ones).
So the idea is using the dumbest generic optimization for gcc, while getting optimizations from the libraries and enhancement from the code in use.

ie: building vlc with -march=i686 -mmmx -msse ending up with
vlc for any i686
vlc-lib normal (i686) code
vlc-lib-part-mmx-enable
vlc-lib-part-sse-enable
...
This way, the "package" in its binary form provide all the libs and the program (vlc), and depending on what cpu features the host have, allow it to use vlc-lib-part-"cpufeature" to enhance it.
And user could run vlc on any i686, i686 with mmx, i686 with mmx+sse ; as of today, binary packager has to make a trade off, if he build it with sse, non-sse cpu cannot run it.
This is easier for x86-64 as "root" code available is bigger (mmx, sse... anything that the "root" x86-64 was supporting, must be opteron, except the specific amd feature part), still it start to get odd even for x86-64 because of sse4, sse4.1, avx... support
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Fri May 11, 2018 8:02 pm    Post subject: Reply with quote

krinn wrote:
Maybe i didn't get it fully, but i think the idea is not building a cross compiler or different arch

$ARCH refers here to things like +ssl-ssl2+mmx (that's why there is a large number of possible $ARCH values and the thing with USE_EXPAND would not be so simple).
Quote:
vlc

Packages with bundled libs require no package manager involvement, of course: They do the multiple translations in their own build system if necessary.
The difficulty is when the libs are a separate package like the already mentioned blas or lapack libs which are used by e.g. numpy.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5039
Location: The Peanut Gallery

PostPosted: Sat May 12, 2018 12:01 am    Post subject: Reply with quote

mv wrote:
$ARCH refers here to things like +ssl-ssl2+mmx (that's why there is a large number of possible $ARCH values and the thing with USE_EXPAND would not be so simple).
Cheers for the explanation.
Like krinn, I thought this was about enabling extra code paths that can be "dynamically patched" in, equivalently to kernel NOP => traps.
I'm sure glibc does something similar (or did.)

I didn't realise it's about building X variants of the whole library (that sucks, afaic.)
Quote:
The difficulty is when the libs are a separate package like the already mentioned blas or lapack libs which are used by e.g. numpy.
I agree that's an important use-case, that we (Gentoo and its users, ie: users, in the medium-term) should address.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6069

PostPosted: Sat May 12, 2018 6:18 am    Post subject: Reply with quote

steveL wrote:
about enabling extra code paths that can be "dynamically patched"

How could one do this for an external library? Especially for blas and lapack (which are probably the main example) where more or less every function requires such an adapted code path?
The most "natural" way to solve this (which after the decision about the processor type needs no time at all) is simply to load the "correct" library. Previously this would have required that the library is dynamically loaded, with all the disadvantages like lacking symbol checking at build time. If I understood correctly, the main feature of function multi-versioning is that this is somehow supported now by the linker.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Gentoo Chat All times are GMT
Goto page 1, 2  Next
Page 1 of 2

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum