Yet another profile/catalyst adventure

[2857]

Howdy all,

I'm trying to create a custom profile (let's call it zulu) which overrides some parts of @system set and so far I'm having pretty little success in it. My profile is based on musl one, so I've specified it inside of relevant parent file:

[2857]

Hmm, as it turns out, paths inside of portage_overlay setting are absolute for my install... so /root -> /var/db/repos
No more errors regarding profile...yet

[2857]

Ok after thorough digging, I came up to the conclusion that I need to make some changes to scripts/bootstrap.sh script; since I'm still using my custom profile I've thought it would be possible to just copy it over and alter what I need, but it seems that catalyst thinks otherwise...

Upon reading catalyst's sources, I come upon vars like $clst_repo_name (which resolves to gentoo) – but I cannot find where they're set and how I can override them? I mean of course I can copy whole catalyst to my profile and make a patch, but that seems too dumb – I feel like I'm missing something very obvious here.

Any advice is highly appreciated!!

[szatox]

Emerge can automagically patch the sources at build time for you. Check this out:
https://wiki.gentoo.org/wiki//etc/portage/patches#Adding_user_patches

However, since you're talking about setting variables, there really should be a better way to do that. Maybe you can find a hint in the eclasses from which that ebuild inherits.

[2857]

Thanks for your reply szatox... I can always go with hack-the-src way, but I still have the hope much of the stuff can be done via right configuration

Anyway, these are my findings so far:
- repo_basedir (/var/db/repos) and repo_name (gentoo) are set to these default values via pretty standard /etc/catalyst/catalyst.conf file (who would have thought it's mentioned in the manpage );
- catalyst will use full repo path to create stage snapshot as well (which's obvious), and it's quite useless for custom overlays unless they're copying half of ::gentoo one;
- it seems that $repo_name/scripts/bootstrap.sh file is somehow embedded into chrooting phase – or is it just being unpacked from the snapshot each time stage is set up for a build? I've come to this observation after trying to make local modifications to the script and not seeing their effect;
- while it seems that I can solve missing ebuilds problem via using something like OverlayFS and then combining my overlay with ::gentoo one, it seems there's no "official" way to deal with this problem... or there is?

One of the biggest confusions is also stage building behaviour: while stage1 is supposed to only bring toolchain, stage2 upon chrooting into it still get the very same toolchain packages queued and installed for some reason – but not the extra ones I've added to the stage1! I cannot understand where this set (binutils, gcc, etc) is coming from.

[pingtoo]

Hi 2857,

I am no expert in catalyst but just interesting in this topic. 5 month back I did bunch of catalyst build for musl/uclibc/glibc I run into many problems like you so I read catalyst code as well and I seems to remember the packages are somewhat hard code in script (don't remember where).

Anyway what part of @system you want to override?

Would it make sense to finish stage3 and use the stage3 build and for example use command

[2857]

OK it's time for a full disclosure, I think it doesn't need a separate topic. I've decided to go somehow public, clean the code and make it somehow readable.

So, I've decided to create a profile which allows you to go completely gccless and use LLVM/Clang as a main system toolchain. There are many similar efforts and overlays and I've personally used Clang and musl overlays for quite a long time without much trouble, so it's in pretty good shape now I would say. You can also compile kernel with it (and, FINALLY, elfutils). The only trick is, all the info I've gathered tell you how to make such a transition from within a working system, but there's no way to have a prebuilt system of such kind; no easy way to use Clang globally without few quite painful world rebuilds with circular deps.

You can grab the code here: https://github.com/2b57/toolchain-clang

Current approach:
GCC and binutils seem to be still deeply embedded in some parts of the system, so I've created two virtuals to encapsulate them: virtual/toolchain and virtual/binutils (that's where I had to edit scripts/bootstrap.sh script). My idea is that new profile will hardmask gcc and binutils, and LLVM/Clang will be pulled as the alternatives...however it doesn't work that way yet and I don't know why.

Build starts from with a special bootstrap profile, which only pulls LLVM/Clang stuff for the next stage. stage2, however, uses "proper" profile which contains masks for GCC/binutils, and that's where things go really south: it really doesn't like masked binutils and just dies there despite I pull virtual package for install and not sys-devel one

Modifications to boostrap.sh scripts are fairly minimal and I do have a hope it will be accepted upstream but that's quite far goal I guess

[pingtoo]

[2857]

Thanks for your reply pingtoo, it seems that order inside of profile is truly inversed. Just pushed a fix and started new build, hope it will work.

[NeddySeagoon]

2857,

Idle in #gentoo-releng on Libera IRC. The people that know catalyst are there.
You need to register your nick to be able to speak.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.

[2857]

Thanks for the tip! I will update the thread once I have some progress

[2857]

Good news, with the help from IRC I was able to produce working stage1!

However, stage2 doesn't seem so bright yet because there's some issue with libtool; that's what I'm getting trying to emerge gettext:

[2857]

Same for libarchive:

[pingtoo]

I wonder if this is cause by missing lib64 or usr/lib64?

I have some notes about run into problems during stage build for missing lib... but don't have exact detail of what the errors are. My notes said make symbolic link for /usr/<target-chost>/lib64 and /usr/<target-chost>/usr/lib64 to /lib64 and /usr/lib64 fix the problem. I am not sure about why create/replace /lib64 and /usr/lib64.

[2857]

I don't think so, because it's based on musl/17.0 profile, which's no-multilib. I've looked up other Clang-based distros (Adelie, OpenMandriva, Chimera) – all seem to switch to gettext-tiny. While it's a viable option, libarchive fails with exactly same error... I suppose it's some libtool bug? Also MacOS build of gettext succeeds, because there's gcc -> clang symlink, and configure script thinks it's native GCC

[2857]

Thanks to Sam, problem was resolved by using

[2857]

Good news everyone! Thanks to enormous amount of help from IRC I've managed to build a usable stage3!

Feel free to grab it here: https://github.com/2b57/toolchain-clang/releases

[Oleksa75]