Multilib-overlay dead?

[xaviermiller]

Hello,

All is compiling fine, except atk which doesn't like introspection USE flag (some mismatch with 64 bit stuff while compiling for x86 ABI).

EDIT: what is the "nodeps" USE flag?

[ohnobinki]

EDIT: what is the "nodeps" USE flag?

There is no such useflag as far as I know... --nodeps is just a switch for emerge which asks it to not ensure that a package's dependencies are built before installing that package. When first moving to portage-multilib, gcc and glibc normally want to be rebuilt with the new multilib_abi_x86 useflag. But in the dependency graph, to build glibc with multilib_abi_x86 it wants to first have gcc compiled with multilib_abi_x86 which in turn requires a glibc built with multilib_abi_x86. This is actually portage-multilib being too picky about dependencies; the toolchain folks have already made sure that gcc and glibc are already multilib-ready so installing gcc with multilib_abi_x86 will not fail if glibc is missing the multilib_abi_x86 flag at the time. Using the --nodeps flag to emerge will ask it to ignore what it thinks is an impossible-to-escape dependency loop, like

Code: Select all

emerge -v1 --nodeps glibc gcc

I can't at the moment recall any exact scenarios this applies to.

[xaviermiller]

Hello,

This IS a USE flag in emul-linux-* packages, to not add contained packages as dependencies. In that case, we need to adapt the 32 bit / multilib application to need dependencies with multilib_abi_x86 USE flag enabled.

I know what --nodeps is, but I did'nt talk about that

[xaviermiller]

Hello,

I want to run crossdev, to generate a cross environment for mingw32 and i686.

But in both cases I get :

Code: Select all

>>> Emerging (1 of 1) cross-i686-gentoo-linux-gnu/binutils-2.21 from local
 * binutils-2.21.tar.bz2 RMD160 SHA1 SHA256 size ;-) ...                 [ ok ]
 * binutils-2.21-patches-1.0.tar.bz2 RMD160 SHA1 SHA256 size ;-) ...     [ ok ]
 * Package:    cross-i686-gentoo-linux-gnu/binutils-2.21
 * Repository: local
 * Maintainer: toolchain@gentoo.org
 * USE:        amd64 elibc_glibc kernel_linux multilib_abi_amd64 multilib_abi_x86 nls userland_GNU
 * FEATURES:   preserve-libs sandbox
 * QA Notice: USE Flag 'multilib_abi_default' not in IUSE for cross-i686-gentoo-linux-gnu/binutils-2.21
 * 
 * You disabled all ABIs
 * You should enable at least one ABI
 * Enabling the default ABI now
 * 
 * QA Notice: USE Flag 'multilib_abi_default' not in IUSE for cross-i686-gentoo-linux-gnu/binutils-2.21
 * ERROR: cross-i686-gentoo-linux-gnu/binutils-2.21 failed (setup phase):
 *   Could not determine your profile ABI(s).  Perhaps your USE flags or MULTILIB_ABIS are too restrictive for this package or your profile does not set DEFAULT_ABI.
 * 
 * Call stack:
 *          ebuild.sh, line 2594:  Called ebuild_main
 *          ebuild.sh, line 2515:  Called dyn_setup
 *          ebuild.sh, line  714:  Called get_abi_list
 *   auto-multilib.sh, line  133:  Called get_abi_order
 *   auto-multilib.sh, line  127:  Called die
 * The specific snippet of code:
 *              die "Could not determine your profile ABI(s).  Perhaps your USE flags or MULTILIB_ABIS are too restrictive for this package or your profile does not set DEFAULT_ABI."
 * 
 * If you need support, post the output of 'emerge --info =cross-i686-gentoo-linux-gnu/binutils-2.21',
 * the complete build log and the output of 'emerge -pqv =cross-i686-gentoo-linux-gnu/binutils-2.21'.
 * This ebuild is from an overlay named 'local': '/usr/local/portage/overlay/'
 * The complete build log is located at '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/temp/die.env'.
 * S: '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/work/binutils-2.21'

>>> Failed to emerge cross-i686-gentoo-linux-gnu/binutils-2.21, Log file:

>>>  '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/temp/build.log'

 * Messages for package cross-i686-gentoo-linux-gnu/binutils-2.21:
 * 
 * You disabled all ABIs
 * You should enable at least one ABI
 * Enabling the default ABI now
 * 
 * ERROR: cross-i686-gentoo-linux-gnu/binutils-2.21 failed (setup phase):
 *   Could not determine your profile ABI(s).  Perhaps your USE flags or MULTILIB_ABIS are too restrictive for this package or your profile does not set DEFAULT_ABI.
 * 
 * Call stack:
 *          ebuild.sh, line 2594:  Called ebuild_main
 *          ebuild.sh, line 2515:  Called dyn_setup
 *          ebuild.sh, line  714:  Called get_abi_list
 *   auto-multilib.sh, line  133:  Called get_abi_order
 *   auto-multilib.sh, line  127:  Called die
 * The specific snippet of code:
 *              die "Could not determine your profile ABI(s).  Perhaps your USE flags or MULTILIB_ABIS are too restrictive for this package or your profile does not set DEFAULT_ABI."
 * 
 * If you need support, post the output of 'emerge --info =cross-i686-gentoo-linux-gnu/binutils-2.21',
 * the complete build log and the output of 'emerge -pqv =cross-i686-gentoo-linux-gnu/binutils-2.21'.
 * This ebuild is from an overlay named 'local': '/usr/local/portage/overlay/'
 * The complete build log is located at '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/temp/die.env'.
 * S: '/var/tmp/portage/cross-i686-gentoo-linux-gnu/binutils-2.21/work/binutils-2.21'


 * binutils failed :(
 * If you file a bug, please attach the following logfiles:
 * /var/log/portage/cross-i686-gentoo-linux-gnu-info.log
 * /var/log/portage/cross-i686-gentoo-linux-gnu-binutils.log

What do I need to do ?

[hedmo]

hi all again

i have manage to get i work(it was /etc/make.profile) .now i am on to rebuilding my world.
i need some advice for the multilib_abi setup.when i recompile my world with MULTILIB_ABI="amd64 x86"
i get over 400 deps and over 100 fails like python,gcc,qt and kde.do i need this packages.

[hedmo]

i have manage to get to 70% of my world without any package.use.but now i need som help with some of the package
(x86 builds)


1.sys-apps/groff=Signal 11

2.sys-devel/llvm=/var/tmp/portage/sys-devel/llvm-2.9-r2/work/llvm-2.9/lib/VMCore/Release/Intrinsics.gen.tmp -gen-intrinsic
0 tblgen 0x08192808
1 tblgen 0x08192e72
2 0xf771f400 __kernel_sigreturn + 0
3 tblgen 0x0804e402
4 libc.so.6 0xf7436dc1
5 libc.so.6 0xf7436e4d
6 libc.so.6 0xf741f15b __libc_start_main + 251
7 tblgen 0x08054065

3.x11-libs/qt-core=usr/lib/gcc/x86_64-pc-linux-gnu/4.7.0-alpha20110611/../../../../x86_64-pc-linux-gnu/bin/ld: i386:x86-64 architecture of input file `endiantest.o' is incompatible with i386 output

thank/hedmo

[Carnildo]

Some packages try to use python-config directly as a python script, ( for instance sys-libs/talloc if you have the python use flag set) and fail because it's linked to the abi-wrapper which isn't written in python.

I was bored, so I fixed this. Behold, a version of abi-wrapper that is both a valid shell script and a valid python script:

Code: Select all

#!/bin/sh

# Please be careful to keep this file sh compatible

"""true"

hardcoded_abis="amd64  x86"
abi=
if [ "${ABI}" ]; then
        for abi in ${hardcoded_abis}; do
                [ "${abi}" = "${ABI}" ] && break
                abi=
        done
fi
if [ \( ! "${abi}" \) -a "${DEFAULT_ABI}" ]; then
        for abi in ${hardcoded_abis}; do
                [ "${abi}" = "${DEFAULT_ABI}" ] && break
                abi=
        done
fi
if [ ! "${abi}" ]; then
        # we're called from outside portage, so use the hardcoded abi list
        for abi in ${hardcoded_abis}; do
                [ -f "${0}-${abi}" ] && break
                abi=
        done
fi

if [ -f "${0}-${abi}" ]; then
        exec "${0}-${abi}" ${1+"$@"}
else
        if [ -L "${0}" ]; then
                LINK_TARGET="$(readlink "${0}")"
                exec "${LINK_TARGET}" ${1+"$@"}
        else
                echo "${0}: abi-wrapper couldn't find an executable for current abi ${abi}" >&2
                exit 1
        fi
fi
dummy='
"""#"

import os
import sys

hardcoded_abis = ["amd64", "x86"]
abi = "NOABI"

if ("ABI" in os.environ) and (os.environ["ABI"] in hardcoded_abis):
   abi = os.environ["ABI"]
elif ("DEFAULT_ABI" in os.environ) and (os.environ["DEFAULT_ABI"] in hardcoded_abis):
   abi = os.environ["DEFAULT_ABI"]
else:
   for test_abi in hardcoded_abis:
      if os.access(sys.argv[0] + "-" + test_abi, os.F_OK):
         abi = test_abi
         break

newcommand = sys.argv[0] + "-" + abi
if (abi != "NOABI") and os.access(newcommand, os.F_OK):
   sys.argv[0] = newcommand
   os.execvp(newcommand, sys.argv)
else:
   print(sys.argv[0] + ": abi-wrapper could not find an executable for current abi " + abi)
   sys.exit(1)

"""true"
'
"""#"

[Alexander E. Patrakov]

Some packages try to use python-config directly as a python script, ( for instance sys-libs/talloc if you have the python use flag set) and fail because it's linked to the abi-wrapper which isn't written in python.

I was bored, so I fixed this. Behold, a version of abi-wrapper that is both a valid shell script and a valid python script:

<snip>

IMHO this is a rather convoluted solution. I'd rather try to convince portage maintainers to stop abi-wrapping executables that are identical for x86 and amd64.

[asturm]

I'm right now at the task of migrating to multilib-portage. My special motivation is a mesa bug which I'm currently hunting, it should have actually been fixed in the 7.11.2 binary that is supplied by the emul-linux-x86-opengl-2012* package, but isn't for me.

Missing from the emul-linux-x86-*-99999999 ebuilds:
- emul-linux-x86-db (introduced 27 Jan 2012)

So... all packages pulling in emul-linux packages should be modified to depend on their respective {x86 deps}[multilib_abi_x86]? Because otherwise, this is going to pull in lots of useless bloat onto the system.

[Alexander E. Patrakov]

I'm right now at the task of migrating to multilib-portage. My special motivation is a mesa bug which I'm currently hunting, it should have actually been fixed in the 7.11.2 binary that is supplied by the emul-linux-x86-opengl-2012* package, but isn't for me.

Missing from the emul-linux-x86-*-99999999 ebuilds:
- emul-linux-x86-db (introduced 27 Jan 2012)

So... all packages pulling in emul-linux packages should be modified to depend on their respective {x86 deps}[multilib_abi_x86]? Because otherwise, this is going to pull in lots of useless bloat onto the system.

Ideally, yes. But the current dependencies of emul-linux-* ebuilds are something that you probably do want to install, and there is a "nodeps" USE flag anyway.

The source of bloat is different: the dependencies are specified incorrectly and incompletely in the main tree for the purposes of portage-multilib. Say you want to compile ffmpeg-9999 for both ABIs. Fine, it needs git to fetch and thus depends on it. Since ffmpeg depends on git, portage automatically adds multilib_abi_x86 USE flag to git and all its dependencies, and that's wrong. ffmpeg-9999[multilib_abi_x86] should depend on just "git", not on it[multilib_abi_x86], and portage cannot know that because both cases are possible and their syntax is the same. In other words, portage has no way to know if ffmpeg will just run the "git" binary or link to some git library (if it had any). IMHO, until this is fixed, this overlay is a non-starter - but this cannot be fixed in an overlay, it has to happen in the main tree, because every dependency has to be manually audited whether it is same-ABI or any-ABI (portage logically has no way to autoguess it correctly => someone has to write it explicitly in every ebuild => this has to go to the main tree).

Also this overlay forces you to have the same USE flags for the x86 and amd64 ABIs, and that's not always correct. E.g., this is incorrect if you don't care about 32-bit python modules - but because of this limitation you have to install a useless 32-bit version of python.

Finally, the developers ignored my e-mail concerning the "emerge -e world" part of the installation instructions.

[asturm]

Also this overlay forces you to have the same USE flags for the x86 and amd64 ABIs, and that's not always correct. E.g., this is incorrect if you don't care about 32-bit python modules - but because of this limitation you have to install a useless 32-bit version of python.

That is what has been itching me too, but I'm going to bite the bullet anyway, for now. Currently wading through a slough of circular dependencies it's a good chance to streamline my USE-flags a bit, since any useless one of them has now double the cost.

[Alexander E. Patrakov]

Also this overlay forces you to have the same USE flags for the x86 and amd64 ABIs, and that's not always correct. E.g., this is incorrect if you don't care about 32-bit python modules - but because of this limitation you have to install a useless 32-bit version of python.

That is what has been itching me too, but I'm going to bite the bullet anyway, for now. Currently wading through a slough of circular dependencies it's a good chance to streamline my USE-flags a bit, since any useless one of them has now double the cost.

I sent the following e-mail about the circular dependencies on January 21, 2012, but Thomas ignored it. Hope it helps you.

Hello.

I have tried to install an ~amd64 system with the multilib-portage
overlay today, following the "emerge -ave @world" method of adding
MULTILIB_ABI to the just-extracted stage3 snapshot. It required me to
resolve some circular dependencies in a non-trivial way. While some of
the dependencies are due to my USE flags and thus avoidable, I guess
that the following circular dependencies are inherent in this method.
They either have to be documented in the
"doc/portage-multilib-instructions" file, or "emerge -e @world"
instruction removed from there as too-far-from-working.

OK, so here are the steps that I did.

1. Unpacked the stage3 tarball, set up mounts, chrooted as per the
handbook.

2. Set up package.use so that layman USEs git. Set up some other flags
in my /etc/make.conf.

3. Installed layman: emerge -av layman

4. Added the overlay:

layman -L
layman --add multilib-portage
echo 'source /var/lib/layman/make.conf' >> /etc/make.conf

5. Followed the instructions in doc/portage-multilib-instructions until
the "emerge -e world" step, but not including it.

6. "emerge -ave @world", got circular dependencies. Some of them are
resolvable by changing the USE flags temporarily, these are not
reported below.

Note that I have no packages with multilib_abi_x86 so far, so the list
of the circular dependencies below may be incomplete. Also I understand
that my choice of the place where to break the loop might be suboptimal
in some cases.

1. sys-devel/gcc-4.5.3-r2::gentoo ->
sys-libs/glibc-2.14.1-r2::gentoo ->
sys-devel/gcc-4.5.3-r2::gentoo

Solved by "emerge -1 --nodeps sys-libs/glibc" (with no particular
reason why glibc and not gcc).

2. app-admin/eselect-1.2.18::gentoo -> sys-apps/file-5.10::gentoo ->
app-admin/eselect-python-20111108::gentoo->
app-admin/eselect-1.2.18::gentoo

Solved by "emerge -1 --nodeps app-admin/eselect" (again, with no
particular reason).

3. virtual/acl-0::gentoo -> sys-apps/acl-2.2.51::gentoo ->
sys-devel/gettext-0.18.1.1-r3::gentoo -> virtual/acl-0::gentoo

Solved by "emerge -1 --nodeps sys-devel/gettext". Reason: if one
postpones rebuilding gettext (as I did during the first try), it pops up
again and again in other circular dependencies.

4. dev-lang/perl-5.12.4-r1::gentoo -> sys-libs/db-4.8.30::gentoo ->
{sys-devel/automake-1.11.2-r1::gentoo, sys-devel/autoconf-2.68::gentoo}
-> dev-lang/perl-5.12.4-r1::gentoo

Solved by "emerge -1 --nodeps sys-devel/automake sys-devel/autoconf".

After that, portage was able to resolve the remaining circular
dependencies automatically, in the sense it doesn't complain anymore
if I build net-print/cups with temporary USE="-filters". When the build
finishes, I'll notify you.

So here is the proposed text to add before "emerge -e world":

"""
Forcefully re-emerge some packages that are known to be involved in
otherwise-unsolvable dependency loops:

emerge -1 --nodeps sys-libs/glibc app-admin/eselect \
sys-devel/gettext sys-devel/automake sys-devel/autoconf
"""


BTW, in all cases, unmerging of the old package generates the following
red warning, but the system seems to work anyway. Maybe the document
should warn the readers.

* ERROR: sys-devel/autoconf-2.68 failed (postrm phase):
* Could not determine your profile ABI(s). Perhaps your USE flags
or MULTILIB_ABIS are too restrictive for this package or your
profile does not set DEFAULT_ABI.
*
* Call stack:
* ebuild.sh, line 688: Called ebuild_main 'postrm'
* phase-functions.sh, line 1019: Called get_abi_order
* environment, line 462: Called die
* The specific snippet of code:
* die "Could not determine your profile ABI(s). Perhaps
your USE flags or MULTILIB_ABIS are too restrictive for this package
or your profile does not set DEFAULT_ABI.";
*
* If you need support, post the output of 'emerge --info
=sys-devel/autoconf-2.68',
* the complete build log and the output of 'emerge -pqv
=sys-devel/autoconf-2.68'.
* The complete build log is located at
'/home/build/portage/._unmerge_/sys-devel/autoconf-2.68/temp/build.log'.
* The ebuild environment file is located at
'/home/build/portage/._unmerge_/sys-devel/autoconf-2.68/temp/environment'.
* S:
'/home/build/portage/._unmerge_/sys-devel/autoconf-2.68/work/autoconf-2.68' !!!
FAILED postrm: 1
* The 'postrm' phase of the 'sys-devel/autoconf-2.68' package has
failed
* with exit value 1.

[Alexander E. Patrakov]

Currently wading through a slough of circular dependencies it's a good chance to streamline my USE-flags a bit, since any useless one of them has now double the cost.

Note that there is a NO_AUTO_FLAG variable that you can set in /etc/make.conf. Its purpose is to list those packages where portage should not automatically add multilib_abi_x86. Originally, it was intended as a hacky list of packages that don't cross-compile cleanly (i.e. fail to build for multilib_abi_x86). I used it also as a way to trim the dependency tree - e.g. it is a good idea to add git there as per the ffmpeg example above, as well as all other known packages that don't have any public shared libraries. So, if portage wants you to add the multilib_abi_x86 USE flag to package app-frob/foobarbaz, you can either agree or, if you know that all dependencies on foobarbaz can be satisfied by foobarbaz of the mismatched ABI, then add app-frob/foobarbaz to NO_AUTO_FLAG.

[asturm]

Also this overlay forces you to have the same USE flags for the x86 and amd64 ABIs, and that's not always correct. E.g., this is incorrect if you don't care about 32-bit python modules - but because of this limitation you have to install a useless 32-bit version of python.

That is what has been itching me too, but I'm going to bite the bullet anyway, for now. Currently wading through a slough of circular dependencies it's a good chance to streamline my USE-flags a bit, since any useless one of them has now double the cost.

I sent the following e-mail about the circular dependencies on January 21, 2012, but Thomas ignored it. Hope it helps you.

Thank you very much, meanwhile I'm almost through - no blockers left (thank you --autounmask=y), one or two perl packages have failed which I'm going to investigate later. Perhaps I should look into NO_AUTO_FLAG to trim down again a bit

I like how it is supposed to work after the initial mess, but the compile time/space cost for the automagic seems too high.

Anyway, I'm already thinking of a less invasive way of getting lib32 binaries, probably setting up a Gentoo Prefix, sourcing existing /etc/make.conf and readjusting it for 32-bit deployment, then putting some script together to create my own emul-linux packages to be served to the parent system. Bullshit?

[supernovae]

The source of bloat is different: the dependencies are specified incorrectly and incompletely in the main tree for the purposes of portage-multilib. Say you want to compile ffmpeg-9999 for both ABIs. Fine, it needs git to fetch and thus depends on it. Since ffmpeg depends on git, portage automatically adds multilib_abi_x86 USE flag to git and all its dependencies, and that's wrong. ffmpeg-9999[multilib_abi_x86] should depend on just "git", not on it[multilib_abi_x86], and portage cannot know that because both cases are possible and their syntax is the same. In other words, portage has no way to know if ffmpeg will just run the "git" binary or link to some git library (if it had any). IMHO, until this is fixed, this overlay is a non-starter - but this cannot be fixed in an overlay, it has to happen in the main tree, because every dependency has to be manually audited whether it is same-ABI or any-ABI (portage logically has no way to autoguess it correctly => someone has to write it explicitly in every ebuild => this has to go to the main tree).

Your git example is bad, and imho wrong. If it worked as you'd like you'd end up with a system with a confusing mix of 32-bit and 64-bit libraries. For example if you later emerged an application with the multilib_abi_x86 flag that required git libraries it would fail due to non-existant libraries even though you've already installed git.

[asturm]

For the record, I gave up on this again. What I really needed was up to date 32-bit opengl libs, I recommend anyone else wanting those to simply add the FireBurn overlay.

[Alexander E. Patrakov]

Your git example is bad, and imho wrong. If it worked as you'd like you'd end up with a system with a confusing mix of 32-bit and 64-bit libraries. For example if you later emerged an application with the multilib_abi_x86 flag that required git libraries it would fail due to non-existant libraries even though you've already installed git.

The example objection is based on misunderstanding. So let me rephrase my original point to avoid this misunderstanding.

Now portage-multilib implicitly adds the [multilib_abi_x86?] to all dependencies of all packages emerged with [multilib_abi_x86]. What I wanted is to make this dependency explicit where it is needed, i.e. to require (a tree-wide change!) that dependencies in ebuilds have one of the following forms: "package[multilib_abi_any]", "package[multilib_abi_same]", "package[multilib_abi_x86]" (or some other explicitly specified architecture), but to completely outlaw dependencies on just "package". So for example ffmpeg-9999 would depend on git[multilib_abi_any], and your example application that uses git libraries would depend on git[multilib_abi_same]. So in your example portage would notice that your application does not have all its required dependencies installed (because ffmpeg pulled only git[multilib_abi_amd64]), and ideally would suggest adding a multilib_abi_x86 USE flag to git.

So no confusing mix in the end - 32-bit libraries are installed only when they are absolutely needed, but a tree-wide change is required to audit and correct all dependencies.

OTOH I understand that Debian already had this discussion back in 2009: http://lists.debian.org/debian-devel/20 ... 00882.html and went on with a package-based solution that is approximately equivalent to NO_AUTO_FLAG. So here is their current solution (Multi-Arch: foreign/same), adapted for Gentoo:

In each ebuild, declare (by using either EROLE=archdep or EROLE=archindep) whether portage should add [multilib_abi_x86] to dependencies on this package. Outlaw ebuilds without EROLE. I.e., when examining dependencies, portage should add [multilib_abi_x86?] to dependencies on ebuilds with EROLE=archdep, but not with EROLE=archindep. If both kinds of dependencies on a package make sense (as it happens in your example with git), solve it with either split packages or virtual packages. E.g. dev-vcs/git would have EROLE=archdep and is supposed to be depended upon only by packages that link to its libraries, while virtual/git would be a dummy no-files ebuild that has EROLE=archindep, depends on dev-dcs/git of the same version, and is supposed to be depended upon by packages that only use the "git" executable.

I have no preference whether the original solution gets implemented, or the one proposed in this message after the "OTOH", because both will lead to correctly specified dependencies.

[FireBurn]

Out of interest how far are we from having a real multilib solution in upstream portage?

I only put together the FireBurn overlay to avoid a chroot after never quite getting graphics to render properly in Wine after using a functioning multilib overlay setup to work - I was fustrated that I'd compiled most of my system in both 64bit and 32bit for what turned out to be just a few libraries

[steveL]

I have no preference whether the original solution gets implemented, or the one proposed in this message after the "OTOH", because both will lead to correctly specified dependencies.

What about simply specifying the dependencies correctly in the first place? What you're discussing appears to be addressed by having LDEPEND which would be Library or required-Link dependencies, implicitly in DEPEND and RDEPEND. Their meaning would be that the package links to one or more libs from the LDEPEND (and thus that this linkage can and should be tracked especially wrt soname changes which cause problems like the old expat issue.)

It's always struck me as a major gap in the ebuild format that they're not specified (iirc they are in BSD ports, which portage was derived from), and leads to problems like this when trying to do new things (since the required data is not available), as well as difficulty implementing preserve-libs (whose checking I think is a good idea: just that it could be made more useful if there were some indication of which libraries are supposed to be linked-in, so it's easy to see which have been added by the build-system, information that can then be used as desired.)

(I no longer think they should be used for plugins as described in that mail fwiw; the plugin should LDEPEND on the main package, but not the other way round, though I suppose USE-conditioned circular-dep breaking logic could be used.)

[MalleRIM]

It's a pity that after all these years, where a multilib setup has almost become unnecessary on many systems, there is still so little development on this front.
The way I see it, the USE-Flag-approach sucks even more than what is currently in portage. It requires you to have the 64bit version of a package installed, even if you don't need it. For a GNOME-user installing skype or googleearth, that would mean a huge dependency like qt for two architechtures, although he only needs one. It also means you will have so set same use flags for all installed ABIs
In my naive mind, it should go something like this:
* There is a defined set of archetechtures, with one being the default one. This would typically be two architechtures (e.g. x86 and amd64), but could also be three (e.g. ppc, ppc64 and x86, running in an emulator for x86 only software) - I doubt, it has been attempted to run three architectures in one root.
* Portage can make slotted, seperate compiles of an ebuild for each supported architechture. Any ABI only requires the dependencies of that ABI installed, but no other ABI versions of the same package. Different ABI versions can have different USE Flags if required by the user (or USE dependencies, once we get them handled properly). This would be essential for keeping the system lean, very often packages for non-default ABIs only need to be installed with minimal USE flag settings (like skype does not need qt-core with qt3support).
* Duplicate files of different architechture compliles of the same ebuild are tolerated and recorded into both packages, only removed when all compiles of the package are removed. Most duplicate files should be architecture indedependent anyway. If colliding files differ (portage should run a diff on each of them), portage should keep the file of the system's main architechture, probably print a warning. These cases could be handled or explicitely ignored in the ebuild. Files in the bin directories could be renamed to $filename-$ABI automatically.
* Portage needs to make a difference between library dependencies and other dependencies, probably introduce a dependency class for the dependencies that don't need to be the same architechture, since that will be the minority. All other dependencies will be installed as the same architechture as the package pulling them in, unless explicitely stated otherwise (e.g. wine). Packages could be marked as being a certain architechture, instead of cluttering dependency lists. steveL's suggestion with LDEPEND is similar to what I had in mind, the only problem is, that for binary packages, library dependencies can be installed before or after, so they usually go in RDEPEND - that calls for two additional dependency classes.

Potential difficulties: A package doesn't work (well) on a non-supported architechture. Since most backwards compatible architechtures run software for their predecessors very well, this should not be a major issue. But: if a package version is stable for, let's say, amd64 but not for x86, should we allow the 32bit version to be installed on amd64 without unmasking? If not, install the latest stable that satisfies the requirements instead?

This doesn't look like a shitload of work to me, but then, I don't really understand how package management systems work under the hood. There must be a reason why no distribution actually has proper multilib handling.

[mv]

The way I see it, the USE-Flag-approach sucks even more than what is currently in portage. It requires you to have the 64bit version of a package installed, even if you don't need it.

This is not true: You can just disable the 64bit architecture in package.use if you do not want it.

Different ABI versions can have different USE Flags if required by the user

As far as I can see, this is the only drawback of the current implementation. However, I see no way how to change this. Using a new slotting mechanism as you suggested leads to all sort of problems: A package then suddenly provides more than one slot, there is no one-to-one correspondence between the metadata and /var/db/pkg anymore etc - essentially it means a ground-up change of the /var/db/pkg dataformat (the same directory must contain now useflags etc for each architecture) and probably of the metadata. All third-pary tools will break etc.
The other things you mentioned are unrelated (and even more problematic since many packages provide simultaneously libraries and programs/tools)

[MalleRIM]

This is not true: You can just disable the 64bit architecture in package.use if you do not want it.

Oh, I must have confused that with kanaka's overlay

Different ABI versions can have different USE Flags if required by the user

As far as I can see, this is the only drawback of the current implementation.

It's a major drawback. Also, all ABI versions being treated as one package means you have to recompile all ABIs whenever you add or remove one.

However, I see no way how to change this. Using a new slotting mechanism as you suggested leads to all sort of problems: A package then suddenly provides more than one slot, there is no one-to-one correspondence between the metadata and /var/db/pkg anymore etc - essentially it means a ground-up change of the /var/db/pkg dataformat (the same directory must contain now useflags etc for each architecture) and probably of the metadata. All third-pary tools will break etc.

True, that's one thing I haven't taken into account. It would probably mean you would need to create a subfolder for every ABI. That would break quite a few tools, but I don't think this kind of change would be hard to implement. As backwards compatibility, there could be links in the main folder to the corresponding files of the primary AbI, that way third party tools would keep working, though with only the primary ABI versions of packages. I think it may even be possible to partially achieve that functionality with a dirty wrapper around portage.

The other things you mentioned are unrelated (and even more problematic since many packages provide simultaneously libraries and programs/tools)

I think the current multilib-portage is facing the exact same problems, only that duplicate files are overwritten in the compile phase instead of dealing with them in the merge phase.

Again, to make sure I'm not misunderstood: I'm just trying to understand. I'm not claiming to know it better. I don't.

[ayvango]

I've updated portage tree and portage itself (multilib version of course) after a big pause. And one little thing brokes: x11-libs/pixman can not build into mutlib_abi_x86: is says "Missing IUSE: multilib_abi_x86". app-emulation/emul-linux-x86-xlibs strictly require pixman to be build for x86 target. So any packages that requires xlibs fail to build. Others build normally. The only configuration change I've done after upgrading and emerging latest version of portage-multilib is switching to new desktop profile.

How can it be that multilib overlay magic works fine for one packages adding "multilib_abi_x86" to their IUSE and failes to work witn anothers? What should I check in my installation?

update:
I've noticed some new feature ABI_x86. What sould I do with it?

[darklegion]

I've noticed some new feature ABI_x86. What sould I do with it?

I'm pretty sure you just need to add ABI_X86="64 32" to your make.conf. Works for me in any case. You'll need to unmask app-emulation/emul-linux-x86-xlibs-20130224-r1 as well otherwise you'll get a lot of blockers.

I also hope we get a 32bit version of Mesa soon as FireBurn's overlay doesn't work for anymore.