| View previous topic :: View next topic |
| Author |
Message |
abduct
Apprentice
Joined: 19 Mar 2015
Posts: 215
|
 Posted: Sun Feb 25, 2018 8:17 am Post subject: Theoretical use experience based on past experience |
 |
|
I have been using Gentoo for a while now, but always on heavily modified systems. Systems that have been abused and customized to a point where if I need a new feature or application in most cases it can be a struggle to get it added.
These systems have been custom built with hardened kernel sources with heavily modified kernels to improve security, have been badly maintained with package installations from users who mix stable and unstable packages to the max, partial upgrades galore, world updates never succeed and are never ran on a scheduled manor or at all, disabling of kernel modules so everything needs to be compiled into the kernel, and custom cflags (CFLAGS="-fPIE -Wl,-z,relro,-z,now -O3 -pipe -fstack-protector-all -falign-functions=64 -falign- loops=32 -fomit-frame-pointer -fforce-addr -march=native -Wformat-security -Wstack-protector" anyone?)
From these systems my personal laptop (with hardened kernel sources/config and mixed stable and unstable branches) is likely my most maintained system that only has a few issues on select packages, but still has some issues overall while installing packages for more advanced/large projects.
I am wondering if the experience would change if these constraints were made and how they would effect the usage of the computer gentoo is installed on:
-use an autogen config to support my hardware or some other config generator to specifically target my hardware
-leave cflag changes to a default
-use purely ~arch unstable repos rather than mixing repositories (I want to use ~arch because I would like the latest features rather than outdated software)
-upgrade world regularly (daily/weekly?)
-follow some sane system maintenance procedures?
-no hacky encryption for partitions or file volumes mounted over directories
-allow loading of kernel modules
If I follow some or all of these would the usage be a lot smoother? My main objectives are:
-Packages not likely to fail to build or install
-World updates complete in a timely manor and do not fail
-Applications wont need special tweaks or hacks to get installing
-more involved/advanced packages such as wine, qemu, x11 wont fial to build for cryptic reasons
-Can play games via wine, playonlinux, or qemu emulation
I have a suspicion that due to all the hardening, security, advanced cflags, lack of maintenance has caused gentoo to be cumbersome to use for me on the installations which have been abused. I am looking for a relativity pain free use for my next personal install and I am stuck between Arch and Gentoo for my next rolling release installation for my desktop that will house a DE or a well featured WM for getting work done (and playing video games) without the horse play/aggravation I seem to feel every time I need to get something done that isn't already installed. I am more after the "it just works" type of installation for 99% of the time, rather then my more often than not "it probably wont work easily" 99% of the time.
Also if anyone has any tips, tricks, or advice for ways to keep a system clean for longer without it breaking that would be great. For instance how to properly organize USE flags and the like to make upgrades go more smoothly.
Ps: Does autogen for kernel configs generate to your specific hardware or does it generate a well rounded config? If the later is there anything that can enable driver specific options for your hardware? I always used to hand build my kernel configs but for this desktop install I don't necessarily want the hassle of trying to find all the drivers myself by hand. Also how is wine/playonlinux support for games? Is it relatively easy to get things working with a functioning maintained installation of gentoo? Also which DE/WM works best for games, or does it matter? |
|
| Back to top |
|
 |
Irre
Guru
Joined: 09 Nov 2013
Posts: 434
Location: Stockholm
|
| Posted: Sun Feb 25, 2018 9:57 am Post subject: |
|
|
I let a cron job do these steps every morning:
emerge -Duvb --newuse --keep-going --exclude llvm --exclude gcc --with-bdeps=y @world
emerge --depclean
emerge -vb @preserved-rebuild
emerge --sync
Note that sync is the last step, not the first!
To see if anything went wrong run this command:
ls -ltr /var/tmp/portage/*/*/temp/build.log
In case of errors, rerun:
emerge -Duvb --newuse --keep-going --exclude llvm --exclude gcc --with-bdeps=y @world
and hope that they have been fixed last 24h! |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Sun Feb 25, 2018 10:07 am Post subject: |
|
|
abduct,
| abduct wrote: | | I have been using Gentoo for a while now, but always on heavily modified systems |
Isn't that why everyone uses Gentoo.
~arch mostly just works these days. Its not all plain sailing but there are a few things fou can do to get a 'get out of jail free' card'.
update monthy
don't update just before your system 'must work' (it might not)
keep binary packages so that a downgrade is only a minute with emerge -K
Do not use -O3 globally for several reasons.
It makes the code bigger in an attempt to make it faster.. This can actually make it slower.
It turns on -funsafe-math which can very bad for your floating point results.
The hardened kernel is dead but some features are appearing it the mainline kernel.
kernel module loading is only required for out of kernel modules, like nvidia-drivers.
I'm not a gamer, so I'll leave the games questions to others.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
abduct
Apprentice
Joined: 19 Mar 2015
Posts: 215
|
| Posted: Sun Feb 25, 2018 4:56 pm Post subject: |
|
|
| Irre wrote: | I let a cron job do these steps every morning:
...
and hope that they have been fixed last 24h! |
That seems reasonable. I don't think I have ever updated world besides during first install because I have always had problems with it on other machines for the first posts reasons.
Do you find with daily world updates that you hardly ever have to intervene manually in the process unless something extra ordinary happens?
| NeddySeagoon wrote: | abduct,
~arch mostly just works these days. Its not all plain sailing but there are a few things fou can do to get a 'get out of jail free' card'.
update monthy
don't update just before your system 'must work' (it might not)
keep binary packages so that a downgrade is only a minute with emerge -K
Do not use -O3 globally for several reasons.
It makes the code bigger in an attempt to make it faster.. This can actually make it slower.
It turns on -funsafe-math which can very bad for your floating point results.
The hardened kernel is dead but some features are appearing it the mainline kernel.
kernel module loading is only required for out of kernel modules, like nvidia-drivers.
I'm not a gamer, so I'll leave the games questions to others. |
I'm glad ~arch works nicely now a days, that gives me a bit of confidence. As for `emerge -K' how do you keep binary packages on the system for downgrading? I have never actually heard of this feature before. Steps to use/enable it would be awesome.
As for the cflags, they were set on the server over time by various users, the bad thing is each application is likely compiled with different clags and useflags that were not saved making unintended world upgrades basically not achievable without losing features or breaking some other application. I won't be making that mistake on the new install.
As for the hardened kernel features, do you think enabling them will impact the usability of the install or am I better staying with a vanilla kernel?
Thanks for the responses both of you. |
|
| Back to top |
|
|
Hu
Moderator
Joined: 06 Mar 2007
Posts: 21607
|
| Posted: Sun Feb 25, 2018 5:20 pm Post subject: Re: Theoretical use experience based on past experience |
|
|
| abduct wrote: | - use an autogen config to support my hardware or some other config generator to specifically target my hardware
|
Excluding unnecessary kernel support is generally a good idea, but it's unlikely to make a difference for your issues. Unnecessary kernel features generally just sit quietly until accessed.
| abduct wrote: | - leave cflag changes to a default
|
This is probably a good idea. Use the "safe" CFLAGS for your profile unless you have a specific reason to do otherwise.
| abduct wrote: | - use purely ~arch unstable repos rather than mixing repositories (I want to use ~arch because I would like the latest features rather than outdated software)
|
This is probably a good idea. In theory, the dependency resolver ought to catch all cases where this would be necessary, but since dependencies are maintained by people, sometimes they are incomplete.
| abduct wrote: | - follow some sane system maintenance procedures?
|
That sounds better than the implied alternative. Could you elaborate on what you mean?
| abduct wrote: | - no hacky encryption for partitions or file volumes mounted over directories
|
What does this mean? There are various clean ways to do encryption that should work well. However, consider also whether your threat model benefits from encryption. Full-disk encryption is only helpful against offline attackers. If an attacker gains code execution while the disk is unlocked for your use, it is unlocked for his use, too.
| abduct wrote: | - allow loading of kernel modules
|
This is usually unnecessary. Please describe why you think you need this.
| abduct wrote: | - Packages not likely to fail to build or install
- World updates complete in a timely manor and do not fail
|
Using only stable will help here, but you have a good reason above for why using stable is not acceptable to you.
| abduct wrote: | - Applications wont need special tweaks or hacks to get installing
|
This should be generally true. Please describe examples of where it was not true for you.
| abduct wrote: | - Can play games via wine, playonlinux, or qemu emulation
|
This depends heavily on the game and the versions involved. It also depends some on the kernel support for your hardware.
| abduct wrote: | | I always used to hand build my kernel configs but for this desktop install I don't necessarily want the hassle of trying to find all the drivers myself by hand. |
Once you get a working kernel configuration, it is usually easy to keep it working across upgrades. Finding that first minimal yet functional configuration may cost you a couple of hours if you have a lot of hardware requirements.
| abduct wrote: | | Also how is wine/playonlinux support for games? Is it relatively easy to get things working with a functioning maintained installation of gentoo? |
As above, it depends on the game. For games that work well in Wine, they often work with no changes, or with only changes that are recommended by the community. For games that do not work well in Wine, you likely will not get it working without substantial effort from Wine developers. Check the Wine AppDB before buying a game to determine whether it is well-supported.
| abduct wrote: | | Also which DE/WM works best for games, or does it matter? |
It should not matter. In some cases, some games do not play well with a compositor enabled. Other than that, they should be unaware of your desktop environment choice. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Sun Feb 25, 2018 5:21 pm Post subject: |
|
|
abduct,
Set FEATURES="buildpkg" to keep binaries of everything you build.
emerge -K tells portage to use binary package or fail.
If you already have a binary, emerge -K =<category>/<package>-<version> will install it and its dependencies.
Missing USE flags can be recovered.
will show everything installed and indicate changed USE flags in Green with a *.
You will need to do a bit of digging into the reasons why but eventually, you can set per package USE flags to keep things the way yo want them.
The GRE hardened kernel features have largely gone away with the hardened patch set. The Kernel Self Protection Project is worth reading.
I've switched from hardened-sources to gentoo-sources on servers. I've never used hardened-sources with Xorg. At one time it wasn't possible.
-fPIE has been free with the hardened profiles for a long time. With the /17.0/ its a default setting. It needs to fixed system wide, since changing it breaks static libraries.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Irre
Guru
Joined: 09 Nov 2013
Posts: 434
Location: Stockholm
|
| Posted: Sun Feb 25, 2018 6:21 pm Post subject: |
|
|
"Do you find with daily world updates that you hardly ever have to intervene manually in the process unless something extra ordinary happens?"
Yes, very little manual interventions today. But my systems are very simple and in the stable branch with a few exceptions (gcc, firefox, youtube-dl). I like the concept of incremental updates, in very small steps. And the emerge -b option makes it often very easy to go back a step in case of problems. |
|
| Back to top |
|
|
abduct
Apprentice
Joined: 19 Mar 2015
Posts: 215
|
| Posted: Sun Feb 25, 2018 7:31 pm Post subject: Re: Theoretical use experience based on past experience |
|
|
| Hu wrote: | | abduct wrote: | - follow some sane system maintenance procedures?
|
That sounds better than the implied alternative. Could you elaborate on what you mean? |
For most of my systems I do partial upgrades on specific applications that I need newer versions of. This is especially true for glsa-check advisories. Would upgrading world be better than partial upgrades for single applications?
| Hu wrote: | | abduct wrote: | - no hacky encryption for partitions or file volumes mounted over directories
|
What does this mean? There are various clean ways to do encryption that should work well. However, consider also whether your threat model benefits from encryption. Full-disk encryption is only helpful against offline attackers. If an attacker gains code execution while the disk is unlocked for your use, it is unlocked for his use, too. |
For my laptop I use file volume encryption with cryptsetup which unlocks from a keyfile on an encrypted sd card at boot. Followed by mounting the mapper over top of /home and /root. It's not that it's bad or hindered me in any way, but I was just curious if not encrypting the disk or any part of the file system would benefit myself any. Does full disk encryption effect IO performance at all when using for instance AES with hardware support? Or is support good enough that SSD read/write speeds wont be held back?
| Hu wrote: | | abduct wrote: | - allow loading of kernel modules
|
This is usually unnecessary. Please describe why you think you need this. |
For my hardened-sources machines I usually disable this to minimize attack surface. For this desktop machine I do not care much about this specific idea, plus I assume enabling it will remove some headaches in the future.
| Hu wrote: | | abduct wrote: | - Applications wont need special tweaks or hacks to get installing
|
This should be generally true. Please describe examples of where it was not true for you. |
There is too many to count, but it was mostly due to the abused systems where multiple dependencies would collide due to stable and unstable mixing (I would imagine). This should be resolved if I stick to one branch.
| NeddySeagoon wrote: | Set FEATURES="buildpkg" to keep binaries of everything you build.
emerge -K tells portage to use binary package or fail.
If you already have a binary, emerge -K =<category>/<package>-<version> will install it and its dependencies.
|
That's pretty neat! Is there a way to tell emerge to recover everything if a major failure has occurred or is it simply on a per package basis that needs to be ran for each package that was failed. Also does it automatically fix other packages which successfully updated but brought in newer dependencies that the failed package can not use?
What would be your steps for disaster recovery if I were to use Irre's cron job method of keeping my system up to date. Would I simply browse the log file and run emerge -K on each package that failed, or is there an easier automated method? Would a simple manual monthly update probably be more advised than an every morning update?
| NeddySeagoon wrote: | Missing USE flags can be recovered.
will show everything installed and indicate changed USE flags in Green with a *.
You will need to do a bit of digging into the reasons why but eventually, you can set per package USE flags to keep things the way yo want them. |
This is actually really useful. I might run it on one of my butchered machines later this week to see if I can fix the missing USE flags.
| NeddySeagoon wrote: | The GRE hardened kernel features have largely gone away with the hardened patch set. The Kernel Self Protection Project is worth reading.
I've switched from hardened-sources to gentoo-sources on servers. I've never used hardened-sources with Xorg. At one time it wasn't possible.
-fPIE has been free with the hardened profiles for a long time. With the /17.0/ its a default setting. It needs to fixed system wide, since changing it breaks static libraries. |
I've successfully used hardened-sources with Xorg, but it was a pain at some points. I might pull in the latest gentoo-sources and take a peak around the kernel config and see what features they have implemented from hardened-sources.
Do you think enabling these advanced hardened features would effect ease of use at all of the system? Are there any that can not be undone once the bulk of the system is installed? For instance if one kernel feature breaks some application for some reason can I simply recompile and reboot or will I need to recompile world and such.
| Irre wrote: | "Do you find with daily world updates that you hardly ever have to intervene manually in the process unless something extra ordinary happens?"
Yes, very little manual interventions today. But my systems are very simple and in the stable branch with a few exceptions (gcc, firefox, youtube-dl). I like the concept of incremental updates, in very small steps. And the emerge -b option makes it often very easy to go back a step in case of problems. |
How would one tell emerge to go back a step in case of a problem occurring? Also does your cron list of commands keep a lot of binary packages on disk? What would be your steps to manage and prune them? Or do they get deleted and replaced after a successful emerge every morning? |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Sun Feb 25, 2018 7:49 pm Post subject: |
|
|
abduct,
-K is just another switch to emerge.
If your system is up to date, does what you think it does.
You only need emerge -K for problem recovery when something builds and installs but won't work.
e.g. All the icons in libreoffice are black and you just spent 8 hours building it.
Going back to the old version with emerge -K only took 10 minutes. Yep, it happened to me.
For build failures, -K won't help. You don't have the binary anyway.
I'm using the Kernel Self Protection Project settings most places now and everything still works.
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Irre
Guru
Joined: 09 Nov 2013
Posts: 434
Location: Stockholm
|
| Posted: Sun Feb 25, 2018 7:53 pm Post subject: |
|
|
"How would one tell emerge to go back a step in case of a problem occurring? Also does your cron list of commands keep a lot of binary packages on disk? What would be your steps to manage and prune them? Or do they get deleted and replaced after a successful emerge every morning?"
I keep maps of binary packages on a server with a lot of space. To clean up simply run: eclean packages.
Where the are is specified in /etc/portage/make.conf. In my case:
PKGDIR="/sda2/transfer/bindist-cubieboard"
Lets say I want a previous version of "nano". | Code: | ls -l /sda2/transfer/bindist-cubieboard/*/nano*
-rw-r--r-- 1 root root 244924 Aug 29 21:15 /sda2/transfer/bindist-cubieboard/app-editors/nano-2.8.7.tbz2
-rw-r--r-- 1 root root 250498 Jan 4 09:53 /sda2/transfer/bindist-cubieboard/app-editors/nano-2.9.2.tbz2
-rw-r--r-- 1 root root 252182 Jan 31 09:15 /sda2/transfer/bindist-cubieboard/app-editors/nano-2.9.3.tbz2 |
run: | Code: | emerge -k =app-editors/nano-2.9.2
Calculating dependencies... done!
>>> Emerging binary (1 of 1) app-editors/nano-2.9.2::gentoo
* nano-2.9.2.tbz2 MD5 SHA1 size ;-) ... [ ok ]
>>> Extracting info
>>> Extracting app-editors/nano-2.9.2
>>> Installing (1 of 1) app-editors/nano-2.9.2::gentoo
* Removing /usr/share/info
* Removing /usr/share/doc
>>> Auto-cleaning packages...
>>> No outdated packages were found on your system. |
|
|
| Back to top |
|
|
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Mon Feb 26, 2018 1:22 pm Post subject: |
|
|
I love your cron job logic 
- in order to save few space you run depclean
- correctly handing depclean is running it with --pretend to see what it will do and handle it ; but this mean user need to handle that ; which mean it shouldn't be in any cron job... Not doing that you left depclean able to do mess (well, in real user, because user didn't handle it).
- in order to save yourself from depclean (user) mess as you don't handle it, you build binaries of package, which take lot of space.
That's good logic, i use it with my dog too: in order to keep him from getting sick, i shot him down: sure he is dead, but trust me, he is never sick! |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Mon Feb 26, 2018 3:01 pm Post subject: |
|
|
I've seen --depclean remove glibc. I must have been asleep at the console.
Still, that's why everyone has a statically linked busybox, isn't it?
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
Irre
Guru
Joined: 09 Nov 2013
Posts: 434
Location: Stockholm
|
| Posted: Mon Feb 26, 2018 8:47 pm Post subject: |
|
|
| --depclean never destroyed my systems so far, but I have binary packages and good backups if it happens. |
|
| Back to top |
|
|
NeddySeagoon
Administrator
Joined: 05 Jul 2003
Posts: 54220
Location: 56N 3W
|
| Posted: Mon Feb 26, 2018 8:54 pm Post subject: |
|
|
Irre,
Its got better over the years :)
_________________
Regards,
NeddySeagoon
Computer users fall into two groups:-
those that do backups
those that have never had a hard drive fail. |
|
| Back to top |
|
|
abduct
Apprentice
Joined: 19 Mar 2015
Posts: 215
|
| Posted: Sun Mar 04, 2018 1:52 am Post subject: |
|
|
What is the best way to keep the system up to date?
Would Irre's suggestion be the best way to handle it, all be it not in a cron job?
| Code: | emerge -Duvb --newuse --keep-going --exclude llvm --exclude gcc --with-bdeps=y @world
emerge --pretend --depclean
# If all looks good run emerge --depclean
emerge -vb @preserved-rebuild
emerge --sync |
Or is there a better order or other commands that should be ran for a better update process? What are some of your scripts, one liners, or command flags in general to update your systems?
Also what kind of update cycle is the best? Daily? Weekly? Monthly?
I am in the process of reinstalling my gentoo laptop in preparation/testing to migrate my desktop as I want to look at some of the newer technologies before I attempt to migrate my software over. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Gentoo Chat
