Making modifications to Gentoo packages

[9a3eedi]

I've used Gentoo a very long time ago (around 2006-2009) as one of my first Linux distributions, and have since moved on to other binary-based distros (right now I'm on Manjaro). But it has been so influential for me and I consider it the main distro that taught me how Linux works.

Back then I wasn't much of a programmer, but now I consider myself a very advanced C programmer, and can do a pretty good job with C++, Python, and other languages.

One thing that I find somewhat unsatisfying about binary-based distros like Arch or Ubuntu is that if I wanted to modify the package that was being distributed, right now this process is a little clunky. I want to be able to take full advantage of the free software nature of Linux, in that I can simply modify the package I installed from the package manager without having to jump through hoops. These modifications typically wouldn't be big ones, probably a diff of 10-100 lines. I

For example, say I downloaded an app that scans a PDF, and there was something in it that I don't like, say it was a bug or a missing feature. Does Gentoo give the user an easy way to inspect the code, modify it, and then keep the modification (perhaps in future updates)? What does the process of doing this look like? Also, to keep the spirit of free software, how can I submit my modifications as a patch later on if I find it might be useful for the community?

Since Gentoo is a source-based distro, I thought maybe it might be easier than with others, you would just maybe emerge a package with an option to not install it just yet, allowing you to modify it, and then continue with the compilation steps. Does Gentoo offer something like this?

Having said all this, I have barely contributed code to open-source as much as I want to. I just couldn't get myself to sit down and write code after a long hard day at work. But I want to get into it, and stuff like this might be more motivating for me.

[rab0171610]

These modifications typically wouldn't be big ones, probably a diff of 10-100 lines. I
For example, say I downloaded an app that scans a PDF, and there was something in it that I don't like, say it was a bug or a missing feature. Does Gentoo give the user an easy way to inspect the code, modify it, and then keep the modification (perhaps in future updates)? What does the process of doing this look like? Also, to keep the spirit of free software, how can I submit my modifications as a patch later on if I find it might be useful for the community?

For one program I use, I keep the ebuild in a local repository. When I install that program, the package manager uses instructions in this ebuild to download and compile the source code.
The default source code has two configuration options turned on by default that I do not need in Gentoo. Thanks to someone on here, I believe it was grknight, I was able to add a block to my ebuild to turn those options off at the configure stage.

Code: Select all

src_configure() {
	local emesonargs=(
        -Dxapp=false
        -Dman=false
)

I do not need xapp installed as a dependency nor do I need manual pages installed. This is just an example of one way that you can change configuration options. There are probably others depending on the build tools.

Most advanced users are going to refer to patches in the ebuild. Those patches are generally kept in the 'files' directory which is located inside the directory where the ebuild is located.
https://devmanual.gentoo.org/ebuild-wri ... s/patches/
https://wiki.gentoo.org/wiki/Creating_a_patch

Disclaimer: I am not recommending this . . . I added it for reference only.
For minor modifications, I personally like to use portage's bashrc file to create a hook to execute a bash script that uses the sed command to replace one line or one word. It is NOT recommended, the preferred method is using patches. I just point it out as it is possible to step through ebuild phases and do things during the various phases of the installation process that you cannot usually do with binary distro's package managers, such as Apt.
https://wiki.gentoo.org/wiki//etc/portage/bashrc
https://wiki.gentoo.org/wiki/Stepping_through_ebuilds

[b11n]

Also, take a look at portage patches for an approach that's a bit lighter than maintaining your own ebuild repository: https://wiki.gentoo.org/wiki//etc/portage/patches

tl;dr: You make your modifications to the source tarball, generate a patch from your changes, save it under /etc/portage/patches, and it's applied to subsequent builds (where possible, obviously). Very handy in cases where you just want to change a default, etc.

[Hu]

I second b11n's recommendation of /etc/portage/patches/, and want to add that if your patch ceases to apply due to upstream changing the code under you, then the build fails at the prepare stage. I consider this preferable to having Portage decide to just skip the patch and build an unpatched version of the program. This way, I can choose whether to disable my patch or to rebase it to work with the newer version. Sometimes, I will look at the package and realize that upstream addressed my preference in a different way, or that I no longer need the behavior my patch introduced. Other times, I decide to rebase the patch.

[9a3eedi]

Actually, the patches method sounds like exactly what I'm looking for. A custom ebuild might also work, but seems like too much work.

I assume that there's an easy way to retrieve the exact same source tarball that portage would then use to build so I can inspect and make modifications, right?

I might actually consider switching to Gentoo next time I decide to install a distro, if the inability to patch packages on my own annoys me enough thanks for the information

[NeddySeagoon]

9a3eedi,

If you use Gentoo, getting the sources is as easy as

Code: Select all

emerge --fetch ...

but that's not the whole story.
Gentoo may apply patches distributed in the ::gentoo repo. /etc/portage/patches/... are then applied on top of that

To get the sources as they will be built, run

Code: Select all

ebuild /path/to/ebuild <phase>

<phase> will be unpack or prepare, I forget which. That will also do the download if required.

On non gentoo distros, you need to read the ebuild and fetch everything in SRC_URI=
Applying patches is harder, since they can be conditional.
Patches can be in a tarball in SRC_URI, in ./files or both.

Just use Gentoo, its easier :)

[Hu]

<phase> will be unpack or prepare, I forget which.

prepare is probably a better choice. Each phase will imply the preceding if not disabled, so prepare will imply unpack, and unpack will imply fetch. The reason to prefer prepare is that Portage will prepare the sources before applying user patches, so if the user wants to write a patch that touches code close to a Gentoo patch, then the user needs to start from the Gentoo-modified sources, not from the raw upstream sources, and code the patch from there. If the user starts with the raw upstream sources, the user's local patch might collide with the Gentoo patch, resulting in a patch failure. If Gentoo happens not to patch close to where the user is working, then using either the Gentoo-modified sources or raw upstream sources will work.

[9a3eedi]

Just use Gentoo, its easier

This might get off-topic, but I got a few things stopping me from using Gentoo. Note that I am not criticising, I am just listing the reasons why I think Gentoo is not for me at this time.

• Source compilation: it's acceptable on a desktop with a modern CPU, but not on a ultrabook laptop (I use both). It already takes a crazy long time with AUR packages on Manjaro/Arch.. imagine building an entire system. If you're on battery, it'll drain it like crazy too. I tried using distcc on Manjaro to try to speed things up, but it doesn't really work.
• Excess configuration: Gentoo is best configured from a minimal environment. This implies customizing everything. I used to have the time and patience to customize my desktop to my liking, and to tolerate things breaking, but not anymore. Otherwise, I'd be using Arch instead of Manjaro, and that doesn't even need me to deal with USE flags.
• Unmasking up-to-date packages: last time I used gentoo (a very long time ago) I tended to want newer packages. In general, for my personal non-work computers, I want a rolling release distro in order to get the latest versions of things. On Gentoo, this often meant I needed to unmask packages, which tended to set off a chain of unmasking dependencies. It eventually becomes a mess of dependency hell and unstable software.
• Optimization not as important as it used to be: I really doubt it makes a difference anymore if you compile your average desktop application with -march=native. I also don't really care for enabling only the features I need. Today's hardware is so fast, that it feels like any optimization you do with compiling will have a unnoticeable effect on your day-to-day performance. At the same time, maybe it'll be noticeable on my 7th gen i7 laptop.. stuff has become slow lately.
• Desire to stick to one distro: I'd rather not distro-hop unless I have a very good reason. I've settled on Manjaro, it's not perfect, but it works for me.[/b][/u]

So if I decide to switch to Gentoo I would have to weigh out the tradeoffs between not being able to modify packages easily (among other problems in other distros), vs the above issues I have with Gentoo

Maybe my concerns are outdated or plain incorrect, but it's based on my experience with Gentoo, I'd love to hear your thoughts.

[sam_]

We already have binary packages: https://www.gentoo.org/news/2023/12/29/ ... inary.html

As for configuration (and indeed the optimisation, and unmasking point): no, that's up to you. Use the e.g. Plasma profile and meta packages and everything Just Works. If you want to meddle with things, that is your decision, but of course it's not then some inherent fault of Gentoo.

We're not trying to sell you on Gentoo if you're happy as you are! You came and asked a question about patching. We in particular excel at that. It's up to you if it's worth it.

[9a3eedi]

We already have binary packages: https://www.gentoo.org/news/2023/12/29/ ... inary.html

As for configuration (and indeed the optimisation, and unmasking point): no, that's up to you. Use the e.g. Plasma profile and meta packages and everything Just Works. If you want to meddle with things, that is your decision, but of course it's not then some inherent fault of Gentoo.

We're not trying to sell you on Gentoo if you're happy as you are! You came and asked a question about patching. We in particular excel at that. It's up to you if it's worth it.

Understood. I was under the impression that the binary packages are for a particular number of software, like web browsers or desktop environments, but I guess I was mistaken. I really do like the idea of compiling everything from source though I just wish it I can offload it from a laptop easily.. something like distcc but more seamless and works over the internet.

Thanks for clarifying about the configuration. I should perhaps give it another shot, perhaps when decide to get a new laptop Gentoo is always in the back of my mind despite me not using it currently

[NeddySeagoon]

9a3eedi,

To clarify a few points that sam_ made.
The binhost project is just over a year old. It provides stable packages only.
Previously, there were a few binary packages with names ending in -bin.
Some of these are still around.

distcc helps a bit but not as much as most people hope. The configure and linking steps are still done on the weak system, then not everything can use distcc either.

If you want lastest gentoo (almost latest) use all testing. It's mostly trouble free these days as all the rough edges are knocked off in overlays.
You can use the overlays or even live ebuilds if you want.

For your weaker systems, build your own binhost in a chroot on the desktop, then install your own binhost on the weak system(s). There is a wiki page.

[sam_]

Right, we provide binary packages for all the common DEs, lots of other packages, and we can add more on request if they take a while to build. There will be the odd thing you may need to build from source but those are usually very light packages indeed (certainly not "the entire system" if you don't want to do that).

To me, it sounds more like you're not sure what you want more than anything. There are ways to use Gentoo (and many people do) which avoid the problems you've listed. The question is if you actually want that, or if you really want to tinker with Gentoo. In any case, Gentoo will be here if or when you decide you want it, in whatever form.

[NeddySeagoon]

9a3eedi,

If you have the time, try Gentoo somewhere. A VM will let you do that.
It's the only way to see if it does what you need.

Should you not like it, we will refund your purchase price, in full. :)

[9a3eedi]

9a3eedi,

To clarify a few points that sam_ made.
The binhost project is just over a year old. It provides stable packages only.
Previously, there were a few binary packages with names ending in -bin.
Some of these are still around.

I see! good to know.

distcc helps a bit but not as much as most people hope. The configure and linking steps are still done on the weak system, then not everything can use distcc either.

In my experience with Manjaro, on a wifi connection, it did not respond fast enough to requests for compilation, so it just ended up compiling locally anyway. It did not also really take advantage of my desktop's 24 threads despite configuring for it. For C-based projects, the network latency was too slow to make it worth it (vs just compiling it locally). For other languages, it wasn't compatible.. Nowadays, lots of software don't use C/C++ anymore, lots of rust/go/C#/Python and other languages that distcc doesn't support.

Perhaps it was misconfiguration, but I was underwhelmed, and eventually disabled it.

[For your weaker systems, build your own binhost in a chroot on the desktop, then install your own binhost on the weak system(s). There is a wiki page.

This might work for the first time setup, but for regular updates, it becomes a hassle.

It would be amazing if I could just keep some daemon and port open on my desktop (or on a self-hosted server), have portage on my laptop request my desktop to download and build teh necessary sources (from A-Z, not per-file like distcc does), and then send the final binaries down to my laptop. But ahh I'm sure it's not easy to implement.

To me, it sounds more like you're not sure what you want more than anything. There are ways to use Gentoo (and many people do) which avoid the problems you've listed. The question is if you actually want that, or if you really want to tinker with Gentoo. In any case, Gentoo will be here if or when you decide you want it, in whatever form.

What I want is a stable everyday OS that is easy to set up and has sane defaults, and has updates with relatively recent packages, that I can tinker lightly with when I get the itch. I think that about summarizes it.

It's clear that the problems I've listed are either outdated or because I was just doing it wrong 15-20 years ago. Things have clearly changed since then and it's worth checking out when I get the chance.

[NeddySeagoon]

9a3eedi,

The chroot image works for updates too.
I keep an old AMD E35 from 2011 alive that way. It should really be replaced with a Raspberry Pi but that's another story.
Chroot in, do the upate. Export the repo and binhost over NFS (or any other way you like), ssh into the E35 and update using the binaries.
It's cut the update time from over a week to a few hours.

[Hu]

If the laptop and desktop have similar enough needs and have compatible processors, you may be able to skip the chroot step entirely. Make the desktop build for itself, and have it save binpkgs for the laptop to consume later. This does suppose that the laptop is mostly a (potentially stripped down) clone of the desktop: same init system, same Desktop Environment, etc. If there are any features that the laptop wants, and the desktop doesn't, then either you make the laptop build those for itself or you put them on the desktop despite not needing them.

[szatox]

Building packages on desktop and reusing them on laptop is one way.
Back in the days when I've been using a cheap (and very underpowered) laptop with Pentium 3 mobile CPU for travels and a _way_ more powerful desktop (64bit quad core from AMD), I'd just export laptop's root over NFS and then chroot into it from my desktop. Kinda 'borrowing' the strong CPU, which was still capable of running the code compiled for P3M
Obviously, it was highly insecure, but it's OK on an isolated network.