Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Using portage as custom build system
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Portage & Programming
View previous topic :: View next topic  
Author Message
maxux
n00b
n00b


Joined: 25 Jul 2010
Posts: 10

PostPosted: Fri Feb 09, 2018 9:03 pm    Post subject: Using portage as custom build system Reply with quote

Hello,

I'm working on a project which builds a custom hand-made root filesystem by compiling needed software (like embedded things, but not for embedded devices).
For now, it a set of custom shell scripts which takes cares to build each parts/software (like ebuild), but in a really more easy/less extendable way.

I'm using Gentoo everyday, on each device I have (laptop, workstation, raspberry pi, ...), I think I'm experienced with emerge/portage.

I was wondering if it could be doable to use portage and emerge as build system for my project.
I imagine something like a clean repository, with couple of ebuilds for the different software to install, and use the power of portage to resolve dependencies and the compilation order.

At the end, it could end of a custom set which would install everything needed to a custom root prefix.

More specifically, is it possible to use emerge/portage by assuming compiler, etc. is already present and should not be handled by portage.
I would like to use emerge and portage to only works my small custom repository, not a full system and make an overlay next to it.

Is that a use case possible for emerge and portage ?

Thanks !
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 17455

PostPosted: Fri Feb 09, 2018 10:27 pm    Post subject: Reply with quote

Moved from Documentation, Tips & Tricks (since it was none of those things) to Portage & Programming. Question seems more related to the capabilities of Portage.
_________________
Slowly I turned. Step by step.
Back to top
View user's profile Send private message
ct85711
Veteran
Veteran


Joined: 27 Sep 2005
Posts: 1660

PostPosted: Sat Feb 10, 2018 12:24 am    Post subject: Reply with quote

I don't see why portage can't be used for what you are wanting to do; you'll have to create your own ebuilds for the packages; but a lot of that I'd link is covered with your build script. For the most part, everything in your @system, is your base toolkit to build/run the base system. After that, you can tell portage to install which ebuilds you want.

The ebuilds are designed to, to identify dependencies (so the packages are built in the correct order). USE flags, could possibly be ignored in your custom ebuilds if you are not interested in customizing specific capabilities.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1707

PostPosted: Sat Feb 10, 2018 12:28 am    Post subject: Reply with quote

Not quite sure if I get you right, but it seems that the thing you're looking for is called "portage overlay".
So, you create ebuilds for your software, put them in your overlay (either local one or public if you want to use it from several machines), and then you emerge them just like anything else.
Ebuilds contain information regarding sources, dependencies, use flags (both, supported by your program and required from dependencies - if any).
Since an overlay need some base, you make it relay on gentoo repository. All dependencies absent in your overlay will be pulled in from the regular portage tree instead.

On top of that, you may consider integrating it with catalyst to automate your builds. I don't like it (because reasons) but you may find it handy. This is where downloadable gentoo stage tarballs come from.
You can download standard recipes from catalyst project's homepage. Use stage1 and 2 directly, and either expand stage3 if you can make it use overlay or create stage 4 based on that (or inject your ebuilds to portage snapshot used by catalyst, this may be an option too)

Bonus: you can speed subsequent builds up by building and reusing binary packages. Some stuff, like gcc doesn't change very often, there is little point in trashing it and building from scratch every night.
Back to top
View user's profile Send private message
maxux
n00b
n00b


Joined: 25 Jul 2010
Posts: 10

PostPosted: Sun Feb 11, 2018 8:02 am    Post subject: Reply with quote

Thanks for your replies !

Quote:
you'll have to create your own ebuilds for the packages

Yep, that was obvious, I know.

Quote:
The ebuilds are designed to, to identify dependencies (so the packages are built in the correct order). USE flags, could possibly be ignored in your custom ebuilds if you are not interested in customizing specific capabilities.

I know, that's exactly why I want to use portage, and it's the main purpose of my post :p

Quote:
Not quite sure if I get you right, but it seems that the thing you're looking for is called "portage overlay" [...] Since an overlay need some base, you make it relay on gentoo repository.

Yes and no, I know what is an overlay and it's not exactly what I want, that's why I talked about clean repository. I don't want to overlay the default gentoo repository, I want mine, completly from scratch.
The main question/purpose of this post was: can portage works with a light repository, without anything related to the compiler, etc. (because this will be handled by a upper layer)

The idea behind is to have a local working system with toolchain installed, and a dedicated portage which will compiles and build the target full system. This target won't include any headers, compiles, etc. (I know this can be done using some --root tricks)

All the others tips, I was aware of it, all the bonus are, exactly, why I want to use portage for this system :)
Back to top
View user's profile Send private message
NeddySeagoon
Administrator
Administrator


Joined: 05 Jul 2003
Posts: 41562
Location: 56N 3W

PostPosted: Sun Feb 11, 2018 10:02 am    Post subject: Reply with quote

maxux,

You can set an alternate ROOT for portage to install into. There are several different *ROOT settings. you can play with.
crossdev does this when you install a cross compiler.

For arm64 I have /usr/aarch64-unknown-linux-gnu/ which is full of arm64 code, on my amd64 build system.
It has its own /etc/portage which applies when cross compiling. At the outset, there are no binary packages installed in /usr/aarch64-unknown-linux-gnu/
It sounds like that's close to what you want. The tool chain and all the repos are outside of /usr/aarch64-unknown-linux-gnu/

You cannot set up a cross environment for your host architecture but the *ROOT settings still work.
Portage uses the ebuild environment it finds on the build host to install things into your designated *ROOT.

I know it works as I'm a crossdev user but I don't know the detail.
_________________
Regards,

NeddySeagoon

Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail.
Back to top
View user's profile Send private message
szatox
Veteran
Veteran


Joined: 27 Aug 2013
Posts: 1707

PostPosted: Sun Feb 11, 2018 12:02 pm    Post subject: Reply with quote

Well, let's try going the other way, AKA define the actual goal first:

What software do you want to find in the final image?
Optionally; what software you don't want to find there?
Back to top
View user's profile Send private message
maxux
n00b
n00b


Joined: 25 Jul 2010
Posts: 10

PostPosted: Mon Feb 12, 2018 8:26 am    Post subject: Reply with quote

Thanks NeddySeagoon, I already used crossdev to build stuff for my raspberry pi, but I need to admit I couldn't handle it correctly to achieve the goal I wanted.
And it's true, what I needs is quite the same.

The resulting root should only contains binaries and needed files to makes software runs, without sources, includes, documentation, etc.
If the build-time dependencies can be stripped from the root, it's perfect.

szatox: half of the system can be used from official Gentoo repository (busybox, glibc, nftables, etc.), the other half is some custom projects in c/go. I'll just need to write theses ebuild.
An overlay could maybe do the trick, if the resulting root can be full build-time dependencies less.

I tried multiple things with the root settings, but I usualy hit some issue like mixing dependencies between the host and the target, I never found a way to have a fully distinct root for the same CHOST.

Thanks for you time :)
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Portage & Programming All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum