Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Multiarch binary server
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
SDubb
n00b
n00b


Joined: 06 Dec 2016
Posts: 7

PostPosted: Fri Jan 26, 2018 5:44 pm    Post subject: Multiarch binary server Reply with quote

I have several machine of various architectures for which I want to emerge binary packages. I can successfully use crossdev and distcc; but, this still puts more work on the other machines than I'd like. Also, I would prefer to make sure builds will succeed before deploying them to other machines. After much searching, I'm not sure both what to call this nor how to accomplish it.

Things I've considered:
crossdev/discc: Can't build everything remotely. Similar arches' packages not stored centrally.
chroot into NFS-exported /: While this gets very close, I can't guarantee network access for all machines.
prefix: This sounds like a way to run portage on other systems.

The ideal solution, in my mind, would be a local directory for each arch. I could fill in etc/portage/ and any other required path(s) either with files, an NFS mount, etc. for each arch. I could then run emerge such that only binary packages are built using the CFLAGS/etc. for the arch. Then I could expose the arch package directory via NFS/rsync/etc. so the other machines can install only binary packages.

Could someone help point me in the right direction? Pertinent variables, wikis, man page(s) etc.?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7466

PostPosted: Fri Jan 26, 2018 6:00 pm    Post subject: Reply with quote

Your point is that you will build binaries packages for each arches.
But the server only build them in /usr/portage/packages, which is its own directory for its binaries, and certainly not something for another arch.

But server side speaking, it's just setting PKGDIR="/usr/portage/packages/an_x86_host" emerge ...
And the server will build the binaries in that directory, allowing you to create structure with binaries per arch type.
This should be easy to add in the build server, as it certainly already use wrapper for each arch.

For client side, it mean using binaries and download them for their own arch
FEATURES="getbinpkg"
PORTAGE_BINHOST="http://.../an_x86_host"

Read this one https://wiki.gentoo.org/wiki/Binary_package_guide
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 47042
Location: 56N 3W

PostPosted: Fri Jan 26, 2018 6:05 pm    Post subject: Reply with quote

SDubb,

The multi arch sever bit is easy enough. Its a BINHOST per arch but they can all be on the same server.

The problem part is building for the different targets.
Solutions to that vary depending on the architectures involved and how determined you are to wring that the last ounce of performance out of the binary packages.

e.g. if you want to build arm packages on an amd64 host, you don't have much choice between cross compiling and a QEMU chroot.
If you have a mix of Intel and AMD (similar CPUs) are you prepared to find some less that ideal CFLAGS that will run everywhere, then those run everywhere CFLAGS are your way ahead.
Is there a half way house like an AMD BINHOST and an Intel BINHOST, so you build everything twice?

Don't rule out compiling a few packages locally either, where speed is a must have.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
SDubb
n00b
n00b


Joined: 06 Dec 2016
Posts: 7

PostPosted: Fri Jan 26, 2018 7:46 pm    Post subject: Reply with quote

krinn wrote:
Your point is that you will build binaries packages for each arches.
But the server only build them in /usr/portage/packages, which is its own directory for its binaries, and certainly not something for another arch.

But server side speaking, it's just setting PKGDIR="/usr/portage/packages/an_x86_host" emerge ...
And the server will build the binaries in that directory, allowing you to create structure with binaries per arch type.
This should be easy to add in the build server, as it certainly already use wrapper for each arch.

For client side, it mean using binaries and download them for their own arch
FEATURES="getbinpkg"
PORTAGE_BINHOST="http://.../an_x86_host"

Read this one https://wiki.gentoo.org/wiki/Binary_package_guide

Thank you. PKGDIR is definately one piece to my puzzle! I am familiar with the binary package guide; I use that for building binary packages on an lxd host to serve to the lxd images.
What is not clear, however, is how to tell emerge which make.conf/compiler to use.
man 5 make.conf makes it sound like Portage will be checking for these files in /. Should I set up a chroot for each arch and bind mount anything I need from the build server's /?


NeddySeagoon wrote:
SDubb,

The multi arch sever bit is easy enough. Its a BINHOST per arch but they can all be on the same server.

The problem part is building for the different targets.
Solutions to that vary depending on the architectures involved and how determined you are to wring that the last ounce of performance out of the binary packages.

e.g. if you want to build arm packages on an amd64 host, you don't have much choice between cross compiling and a QEMU chroot.
If you have a mix of Intel and AMD (similar CPUs) are you prepared to find some less that ideal CFLAGS that will run everywhere, then those run everywhere CFLAGS are your way ahead.
Is there a half way house like an AMD BINHOST and an Intel BINHOST, so you build everything twice?

Don't rule out compiling a few packages locally either, where speed is a must have.

Thanks for the reply. I wasn't clear about crossdev. I am _not_ trying to avoid crossdev. I _am_ trying to avoid distcc. That being said, what should/could be a strategy for invoking emerge for one arch vs. the other?
I'm thinking something like:
Code:
PKGDIR="/<arch_prefix>/usr/portage/packages/" UNKNOWN_VAR="/<arch_prefix>/etc/portage/" emerge ...

Assuming this is even possible, how many other variables need to be set or files/directories need to be in the /<arch_prefix> tree?
I suspect this is all spelled out in the portage, emerge, make.conf, etc. man pages; It's just a daunting task to sift through it all.

An ideal solution would avoid all local compilation. Maybe I'm crazy; but, it would be nice if this could work for some embedded situations where I don't even want to have a local compiler.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 47042
Location: 56N 3W

PostPosted: Fri Jan 26, 2018 8:05 pm    Post subject: Reply with quote

SDubb,

There are no ideal foreign arch solutions.

A QEMU chroot works but its slow and has unimplemented syscall limitations.
Pure cross compiling works sometimes. A lot of build systems are not cross compile friendly. A few are even cross compile hostile.
The least worst (that's not the same as the best) is probably cross distcc, which is a hybrid of the above.
You say you want to avoid that though.

Portage keeps all the bits separate.

For cross compiling and cross distcc, you select the compiler to be used with gcc-config.
Code:
$ gcc-config -l
 [1] aarch64-unknown-linux-gnu-5.4.0
 [2] aarch64-unknown-linux-gnu-6.3.0
 [3] aarch64-unknown-linux-gnu-6.4.0
 [4] aarch64-unknown-linux-gnu-7.1.0
 [5] aarch64-unknown-linux-gnu-7.2.0 *
 [6] aarch64-unknown-linux-gnu-7.3.0

 [7] aarch64-unknown-linux-musl-6.4.0 *
 [8] aarch64-unknown-linux-musl-7.2.0

That shows that aarch64-unknown-linux-gnu-7.2.0 * and aarch64-unknown-linux-musl-6.4.0 * are both active. The native compiler is [38] x86_64-pc-linux-gnu-7.2.0 *

Code:
crossdev -t <your-tuple-here>
Will give you a cross toolchain and an empty target root at /usr/<your-tuple-here>
Its empty in the sense that it has no binary packages ready for the target. The /usr/<your-tuple-here>/etc/portage is there and almost ready to go.
In theory you could do
Code:
<your-tuple-here>-emerge @system
and all the system set packages would cross compile and be installed in the target root.
There is some difference between theory and practice, as you might expect.

Heres a worked example for a Raspberry Pi 3 in 64 bit mode.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
SDubb
n00b
n00b


Joined: 06 Dec 2016
Posts: 7

PostPosted: Fri Jan 26, 2018 9:30 pm    Post subject: Reply with quote

NeddySeagoon wrote:
SDubb,

[snip]

Portage keeps all the bits separate.

For cross compiling and cross distcc, you select the compiler to be used with gcc-config.
Code:
$ gcc-config -l
 [1] aarch64-unknown-linux-gnu-5.4.0
 [2] aarch64-unknown-linux-gnu-6.3.0
 [3] aarch64-unknown-linux-gnu-6.4.0
 [4] aarch64-unknown-linux-gnu-7.1.0
 [5] aarch64-unknown-linux-gnu-7.2.0 *
 [6] aarch64-unknown-linux-gnu-7.3.0

 [7] aarch64-unknown-linux-musl-6.4.0 *
 [8] aarch64-unknown-linux-musl-7.2.0

That shows that aarch64-unknown-linux-gnu-7.2.0 * and aarch64-unknown-linux-musl-6.4.0 * are both active. The native compiler is [38] x86_64-pc-linux-gnu-7.2.0 *

Code:
crossdev -t <your-tuple-here>
Will give you a cross toolchain and an empty target root at /usr/<your-tuple-here>
Its empty in the sense that it has no binary packages ready for the target. The /usr/<your-tuple-here>/etc/portage is there and almost ready to go.
In theory you could do
Code:
<your-tuple-here>-emerge @system
and all the system set packages would cross compile and be installed in the target root.
There is some difference between theory and practice, as you might expect.

Heres a worked example for a Raspberry Pi 3 in 64 bit mode.

Thank you _so_ much! I have read all the cross building wikis so many times(I think even the page you linked); yet, I never grasped how Portage keeps it all separate for me. Re-reading, I don't know how I missed it.
I was just able to cross-compile busybox for armv7 and see where Portage puts everything. I'll start experimenting with using this for what I want.

NeddySeagoon wrote:
Pure cross compiling works sometimes. A lot of build systems are not cross compile friendly. A few are even cross compile hostile.

I realize that will be the case. I'm expecting a significant amount of pain for the cross compilations. I envision the machines lacking a compiler will have a substantially smaller set of packages to install. For machines with more packages, a few native builds are fine.

Looking at how crossdev/cross-emerge work, could I make, say, an x86_64-<name>-linux-gnu(where <name> represents various flavors of CFLAGS/etc. and use the /usr/x86_64-<name>-linux-gnu/ directory as a local copy for machines of "type" <name>? I realize the performance gain may be negligible compared to the increased storage/building/etc.; however, I think I just need to experience it before I retreat to a lowest common denominator. That aside, though, having a "central" place to store multiple machines' configurations is worth it to me.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 47042
Location: 56N 3W

PostPosted: Fri Jan 26, 2018 9:53 pm    Post subject: Reply with quote

SDubb,

SDubb wrote:
Looking at how crossdev/cross-emerge work, could I make, say, an x86_64-<name>-linux-gnu(where <name> represents various flavors of CFLAGS/etc. and use the /usr/x86_64-<name>-linux-gnu/ directory as a local copy for machines of "type" <name>? I realize the performance gain may be negligible compared to the increased storage/building/etc.; however, I think I just need to experience it before I retreat to a lowest common denominator. That aside, though, having a "central" place to store multiple machines' configurations is worth it to me.


I'm not sure how that would work. What you are asking is if your host is
Code:
x86_64-pc-linux-gnu
can you have
Code:
/usr/x86_64-intel-linux-gnu/
/usr/x86_64-AMD-linux-gnu/
cross compilers and targets.
I don't know.

Try it and tell us :)
You cannot have a cross environment for your native host.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
miket
Guru
Guru


Joined: 28 Apr 2007
Posts: 427
Location: Gainesville, FL, USA

PostPosted: Sat Jan 27, 2018 4:23 am    Post subject: Reply with quote

I'd been thinking about moving into building for different architectures, but I have too much going on right now to deal with that. I can tell you about some of my experience with binary build hosts.

It's come to the point that I don't do native building any more except for kernels--everything comes through the build host. I have the luxury of running the same Intel Core architecture all the way through, though some processors have more capabilities that others. In that way you could say I have something akin to multiple arches, but in a not-very-bothersome way.

I do find having commonality in setup to be very helpful. I share /usr/portage and /var/lib/overlays (the path I use for overlays) across all my machines. These are bind mounts within the build host and network mounts to the others. Recently I have starting sharing a third thing: custom profiles. I found out about those when reading a recent thread on a change the the make.conf format. I thought it was a pretty pointless exercise until I took notice of the grand benefit: I can share things like preferred USE flags, keywords, and package masks and never have to worry about synchronizing them across machines! At this point I can tell you it's a wonderful thing.

You could set up a baseline profile with preferences that include even GENTOO_MIRRORS and then set up separate derived profiles for each of your target arches. This makes life immensely simpler when it comes time to install packages on your separate hosts.

One happy aspect of custom profiles is that they are built up using the overlay system, meaning that if you already have your own overlays that you mount on all your machines, the custom profiles show up on the same shares.

If you have at least one local overlay set up, you would recall setting up the repo_name file in the profiles/ directory. Any local profiles go into the same directory. A profiles.desc file lists the overlays you define, and as many profile directories go into profiles.desc as profiles you define. The instructions at the Profile (Portage) page indicate how you make a profile inherit from other profiles. For me, the real attraction in going this way is that I get to populate that directory with files like make.defaults, package.accept_keywords, package.mask, and package.use.

So now when I go to the build root and type "eselect profile list", I see an entry like this one:
Code:
 [51]  local:localbase (stable) *
Now when I set that same profile on the respective target machines, I have a lot less setup bother on them.
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 7466

PostPosted: Sat Jan 27, 2018 12:10 pm    Post subject: Reply with quote

NeddySeagoon wrote:

I'm not sure how that would work. What you are asking is if your host is
Code:
x86_64-pc-linux-gnu
can you have
Code:
/usr/x86_64-intel-linux-gnu/
/usr/x86_64-AMD-linux-gnu/
cross compilers and targets.
I don't know.

Try it and tell us :)
You cannot have a cross environment for your native host.

You can again use your own wrapper for that.
crossdev use a wrapper that set portage ROOT= .... as per the cross-compiler to be use. the /usr/arch directory.
if you want multi-configuration for different hosts that use that same arch, you can wrap cross-dev in your own script but it wrap it to set PORTAGE_CONFIGROOT which define where portage read config files.

So instead of creating the x86_64-AMD-linux-gnu or x86_64-Intel-linux-gnu, you use the same one with different config files per hosts.
PORTAGE_CONFIGROOT=/usr/i686-pc-linux-gnu/amd i686-pc-linux-gnu-emerge ... (with /usr/i686-pc-linux-gnu/amd/etc/portage/)
PORTAGE_CONFIGROOT=/usr/i686-pc-linux-gnu/intel i686-pc-linux-gnu-emerge ...

The effect is that portage will use these directories the same way your build host use its /etc/portage directory (read make.conf...)
If it help you get what this does, your build host is setup this way:
ROOT=/ PORTAGE_CONFIGROOT=/

Because of this, it allow your build host to also build something specific for another host using its native compiler.
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 47042
Location: 56N 3W

PostPosted: Sat Jan 27, 2018 12:18 pm    Post subject: Reply with quote

krinn,

Every day is an education here.
Thank you.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum