View previous topic :: View next topic |
Author |
Message |
denial n00b
Joined: 15 Nov 2007 Posts: 5
|
Posted: Thu Apr 19, 2012 10:51 pm Post subject: Conservative updating strategy |
|
|
tl/dr: can I be safe without ever running emerge -uND @world?
I run Gentoo on a Laptop that acts as a home workstation (never moves) and home server (remote ssh access). Due to its limited processing power, always updating everything takes quite some time. I also see no reason to update things that are working fine for me, unless I want particular features -- but I don't want the system to be vulnerable to known security issues.
So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features
Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed.
My process for this is a very hackish daily cron job that emails me GLSAs that affect the system. This report also pulls a list of security bugs active in the last 2 days and filters those that match installed packages (From what I gather GLSAs are not very reliable at the moment, and in any case can take quite some time, so this is my improvised alternative -- any other suggestions welcome! Especially since I'm not 100% sure that wget works as I think it does) This way I can check the GLSAs and bugs and decide what packages need to be updates or otherwise fixed (or removed) for security reasons.
Code: | #!/bin/bash -- Daily Securty Report
emerge --sync -q 1> /dev/null
eix-installed -a > /var/tmp/update-report/installed
wget -q "https://bugs.gentoo.org/buglist.cgi?f1=days_elapsed&known_name=sec-60&list_id=970687&o1=lessthan&product=Gentoo%20Security&query_format=advanced&v1=2&title=Bug%20List:%20sec-60&ctype=atom" -O /var/tmp/update-report/bug-rss
echo "Subject:security-report.sh" > /var/tmp/update-report/security-report
echo "The following GLSA(s) may affect the sytem:" >> /var/tmp/update-report/security-report
glsa-check -lnq affected >> /var/tmp/update-report/security-report
echo "The following security-related bugs that were active since yesterday may affect the system:" >> /var/tmp/update-report/security-report
for line in $(sed -e 's/-[0-9]\+.*$//' /var/tmp/update-report/installed); do grep -A3 "$line[^a-zA-Z]" /var/tmp/update-report/bug-rss ; done | grep "<id>https" | sed -e 's/.*<id>//' -e 's/<\/id>//' >> /var/tmp/update-report/security-report
date >> /var/tmp/update-report/security-report
sendmail root < /var/tmp/update-report/security-report |
The second part of the process is weekly cron-job that informs me of available updates and obsolete packages. I can then update these as needed. If I see portage or some other critical component I update it. Anything else I leave unless or look up whether it's useful (one exception: I always update flash).
Code: | #!/bin/bash -- Weekly Update Report
mkdir -p /var/tmp/update-report
mkdir -p /var/tmp/update-report
emerge -pu @world > /var/tmp/update-report/world-updates
eix-test-obsolete > /var/tmp/update-report/obsolete
sed -e 's/^These are/Subject:emerge @world available updates\nThese are/' /var/tmp/update-report/world-updates | grep -v "^$" | grep -v Calculating > /var/tmp/update-report/report
grep ^[[I] /var/tmp/update-report/obsolete | sed -e 's/^Installed/\neix-test-obsolete <root@shinybox>\nInstalled/' >> /var/tmp/update-report/report
date >> /var/tmp/update-report/report
sendmail root < /var/tmp/update-report/report |
The third part of the process is to look over some files to make sure everything is in order (I tend to enable use-flags per-package as needed).
Code: | /var/lib/portage/world make.conf package.use package.accept_keywords |
Also a look over emerge -puD @system emerge -puD @world to see if any big dependencies are looking very outdated.
Finally, every 6 months or so I plan to check whether a toolchain update is in order (I may also go for a emerge -uD @system at this point).
Code: | equery list gcc -p
equery list binutils -p
equery list glibc -p |
This is still mostly theoretical, so my question is: is this a good way to ensure a stable and secure system? Am I going to be in world of pain at some point? Am I leaving the system vulnerable in terms of security?
edit:cron script fix
Last edited by denial on Fri Apr 20, 2012 11:28 am; edited 1 time in total |
|
Back to top |
|
|
audiodef Watchman
Joined: 06 Jul 2005 Posts: 6639 Location: The soundosphere
|
Posted: Thu Apr 19, 2012 11:02 pm Post subject: |
|
|
I think the most important thing for you, since you want to keep things stable, is to, well, stay on stable and not ~arch.
I also think it's important, when using Gentoo, to update world on some kind of regular basis. I used to get frustrated with updates until I hit upon my current approach:
1. emerge --sync
2. layman -S (skip if you don't have overlays)
3. emerge -pvuDN world (read output)
4. cd ~/PortageLogs
5. emerge -pvuDN world > (month+year) - gives you something to review if something breaks
6. emerge -vuDN world
7. update any 9999 packages
8. emerge -p --depclean
I do this for all my Gentoo machines on the first of each month - no more or less often. All my machines are on ~arch, not stable, and things break very infrequently for me (and usually because someone threw a bomb down the pipe. libpng, udev, and such).
Think about incorporating any or all of this into your routine. I think if you do not update world on a regular basis, letting no more than a month or two go by between updates, some dam will break at some point and you'll be spending a lot of time fixing it. Gentoo is a rolling release distro and is meant to work best when updated regularly. Stuff has always broken for me in big ways whenever I wait too long to update world. _________________ decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN |
|
Back to top |
|
|
The Doctor Moderator
Joined: 27 Jul 2010 Posts: 2678
|
Posted: Fri Apr 20, 2012 2:44 am Post subject: |
|
|
For updating, any more than a month and you risk updates that cannot be done due to required packages simply not having an path that it can be upgraded on, most notably glibc, or other problems like massive blocks, circular dependencies, etc. I recently wiped my laptop's virtaulbox Gentoo install because it became too outdated.
I update every few days just to minimize the amount of time compiling. My system is about 5 years old with only two cores, and this usually takes about 30 minutes to an hour. This works for me. Keeping in mind the restrictions, you should play with the timing to figure out when you want to invest the time per month to do an update. You should do this at least once a month. As for security, I think you should be able to maintain a fairly secure system with monthly updates. _________________ First things first, but not necessarily in that order.
Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box. |
|
Back to top |
|
|
Hypnos Advocate
Joined: 18 Jul 2002 Posts: 2889 Location: Omnipresent
|
Posted: Fri Apr 20, 2012 3:37 am Post subject: Re: Conservative updating strategy |
|
|
denial wrote: | So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features
Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed. |
I've been doing exactly this for years without a major problem, though I did run into one interesting corner case:
(a) Some version of a library in slot :0 is removed from the ebuild tree
(b) It is replaced by a minor revision, but is no longer in slot :0 because there is a new major revision which now inhabits slot :0
(c) Because the library headers are poorly laid out, only the new slot :0 will install headers. The old slot :0 relies on the new slot :0 for the headers.
(d) If one replaces the old version by the minor revision, instead of the major revision (to avoid a big revdep-rebuild), dependent package updates will fail because there are no headers available.
While this situation seems rather esoteric, it illustrates the potential for weird failures because the developers assume you will do a deep upgrade without being selective about which packages or package versions get updated. _________________ Personal overlay | Simple backup scheme |
|
Back to top |
|
|
kimmie Guru
Joined: 08 Sep 2004 Posts: 531 Location: Australia
|
Posted: Fri Apr 20, 2012 4:38 am Post subject: |
|
|
TL;dr/ You're wasting your own time going against the gentoo update flow.
Is your system really too slow to stay on stable and fully update somewhere between once a week and once a month? Because rather than maintaining your update system and looking over dependencies, you could probably be doing something much more interesting. After all, time your box spends compiling doesn't really count... just leave it overnight and come back tomorrow. Use binary packages for the big buggers like browser or openoffice, and pare down your environment overall (eg. gnome-light instead of gnome, or an even lighter desktop environment), and disable USE flags for features you don't need that pull in dependencies.
After 8 years of gentoo, I've settled on weekly cronjob which syncs and emails me the emerge -puND results. I then update as I see fit, usually that means -uDN world but sometimes I'll manually --exclude or mask a big package for a while.
WRT toolchain, excluding bugfixes there's usually no compelling reason to update at all, especially if you have an old processor. You can easily go more than six months between toolchain updates. Mask major versions of gcc until you decide you want them, for whatever reason.
WRT kernel, linux-headers... stick with a long-term stable kernel that works on your hardware.
WRT security: you only really need to care about attack vectors that affect you. For you, that's probably openssh and your browser. So you could just regularly do a deep update on those. Polling bugzilla for any security-related report is casting a pretty wide net, you're going to put on weight eating all that seafood. If you're only running sshd and a browser, that's starting to get tinfoil paranoid... if you want that level of security your better off running your browser in a VM, and making sure not to visit http://please-root-me.com. Although, I have to admit, seafood does taste good cooked in tinfoil.
If you were to keep track of two systems for a couple of years, one doing weekly -uDNs and one using your proposed method, I'd put money on you spending far less time maintaining the former. So, my 20c is: don't bother, go with the flow. |
|
Back to top |
|
|
denial n00b
Joined: 15 Nov 2007 Posts: 5
|
Posted: Fri Apr 20, 2012 12:22 pm Post subject: |
|
|
Thanks for all the feedback advice!
audiodef wrote: | I think the most important thing for you, since you want to keep things stable, is to, well, stay on stable and not ~arch. |
Yes, I definitely avoid ~arch except where absolutely necessary.
audiodef wrote: | II think if you do not update world on a regular basis, letting no more than a month or two go by between updates, some dam will break at some point and you'll be spending a lot of time fixing it. Gentoo is a rolling release distro and is meant to work best when updated regularly. Stuff has always broken for me in big ways whenever I wait too long to update world. |
penguin swordmaster wrote: | For updating, any more than a month and you risk updates that cannot be done due to required packages simply not having an path that it can be upgraded on, most notably glibc, or other problems like massive blocks, circular dependencies, etc. |
Well I would update regularly, just not to the newest version. I would never have a package installed that is not in the current portage tree (eix-test-obsolete warns about that, right?). So I would be still be rolling, but at the back of the pack and skipping some minor versions/revisions in between updates.
penguin swordmaster wrote: | I update every few days just to minimize the amount of time compiling. |
I would argue that this means you actually spend more time compiling - individual sessions are shorter but overall you would be building more minor versions or revisions if they come out in quick sucession.
Hypnos wrote: | denial wrote: | So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features
Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed. |
I've been doing exactly this for years without a major problem, though I did run into one interesting corner case:
(a) Some version of a library in slot :0 is removed from the ebuild tree
(b) It is replaced by a minor revision, but is no longer in slot :0 because there is a new major revision which now inhabits slot :0
(c) Because the library headers are poorly laid out, only the new slot :0 will install headers. The old slot :0 relies on the new slot :0 for the headers.
(d) If one replaces the old version by the minor revision, instead of the major revision (to avoid a big revdep-rebuild), dependent package updates will fail because there are no headers available.
While this situation seems rather esoteric, it illustrates the potential for weird failures because the developers assume you will do a deep upgrade without being selective about which packages or package versions get updated. |
That's great to hear. What kind of system do you use this approach for? Are there any other suggestions you would have (i.e. rebuild @system, @world at some point? watch major libraries?).
Is there some sort of (official?) policy on how long old packages stay in portage?
In terms of the specific problem you point out, how can that be avoided or worked-around?
kimmie wrote: | TL;dr/ You're wasting your own time going against the gentoo update flow. |
Am I really going against it? Still rolling, just not as quickly. I accept that package testing is done on fully updated stable and so I miss out on some of that -- but I don't think the testing can take into account all the particularities of an install, so I don't feel I'm loosing too much there.
kimmie wrote: | Is your system really too slow to stay on stable and fully update somewhere between once a week and once a month? |
No, not really, I could do it if I really wanted to. I agree with your suggestions for a leaner system (I run awesome and disable most things by default).
kimmie wrote: | WRT toolchain, excluding bugfixes there's usually no compelling reason to update at all, especially if you have an old processor. You can easily go more than six months between toolchain updates. Mask major versions of gcc until you decide you want them, for whatever reason. |
To what extent does this apply to glibc and binutils? Will say the glibc-2.10.x branch continue to receive upstream bug and security fixes? Will something like this https://bugs.gentoo.org/show_bug.cgi?id=386329 ever be fixed for the older version?
kimmie wrote: | WRT security: you only really need to care about attack vectors that affect you. For you, that's probably openssh and your browser. |
Yes that's an excellent point. At the moment the only vector is probably the browser (well, flash mostly) as I am so secured behind an institutional firewall I can hardly get to my own machine. But I am in part looking to try this out to see if it would work for more extensive home-server and desktop systems I am likely to have quite soon -- which would be open more vectors and won't have that protection.
kimmie wrote: | Although, I have to admit, seafood does taste good cooked in tinfoil. |
Damnit, now I'm hungry! In need some sea-bass, lots of lemon and some good potatoes....
kimmie wrote: | If you were to keep track of two systems for a couple of years, one doing weekly -uDNs and one using your proposed method, I'd put money on you spending far less time maintaining the former. So, my 20c is: don't bother, go with the flow. |
If both can be secure I suppose this is what it boils down to. Does anyone disagree with this assessment? Has anyone tried both? |
|
Back to top |
|
|
audiodef Watchman
Joined: 06 Jul 2005 Posts: 6639 Location: The soundosphere
|
|
Back to top |
|
|
Hypnos Advocate
Joined: 18 Jul 2002 Posts: 2889 Location: Omnipresent
|
Posted: Fri Apr 20, 2012 1:38 pm Post subject: |
|
|
I do this on my laptop, which I use for both work and pleasure.
I don't have any special recommendations for particular packages or package sets to watch.
As a counterpoint to kimmie: the reason I follow this update strategy is that I find that deep upgrades can result in breakage besides that fixed by revdep-rebuild. By upgrading only when I have to, I greatly reduce my maintenance effort. Moreover, most deep upgrades don't affect system function, meaning that many upgrade+revdep-rebuild cycles (which require human monitoring at beginning) are entirely unnecessary. _________________ Personal overlay | Simple backup scheme |
|
Back to top |
|
|
kimmie Guru
Joined: 08 Sep 2004 Posts: 531 Location: Australia
|
Posted: Sat Apr 21, 2012 4:40 pm Post subject: |
|
|
Hypnos, I was thinking of the deep updates more as a security measure, as an alternative to cherry-picking security updates as denial was proposing. There's no need to deep-update the whole system, just the internet-facing packages.
denial, yeah, sorry, when I said toolchain, I was really thinking about gcc. I'd stick with a reasonably current glibc and binutils. Personally, what I do with glibc is to put off updating it until a new linux-headers arrives, or a GLSA tells me to.
For me, in the end it comes down to having updates in managable sized chunks (I'd much rather have 4 updates of 20 packages than 3 of 5 and one of 70), and reasonably regular full updates is the best way I've found to do this. Also, I like to keep my system reasonably similar to others out there, in the hope that if I have some weird package interaction I won't be the only one. Consider what happens when, say, you run into some silly desktop issue, like your removable drives refusing to mount automatically. But nobody else seems to be complaining of this. What's the first thing you're going to do? Update. So all the time you've saved by not updating is suddenly unsaved again, and you get bogged down for a couple of days in a huge update, and you end up cursing the stupid little dog next door which is always going Moon, Mooon, MOON, MOON, moon, tree, moon, cat CAT CAT MOON to the other stupid little dog down the road and forgetting which removable drive you were trying to mount in the first place way back when you were actually trying to get something done... (of course, this is a purely hypothetical scenario which has never happened to me, ). |
|
Back to top |
|
|
Hypnos Advocate
Joined: 18 Jul 2002 Posts: 2889 Location: Omnipresent
|
Posted: Sat Apr 21, 2012 4:56 pm Post subject: |
|
|
kimmie wrote: | Hypnos, I was thinking of the deep updates more as a security measure, as an alternative to cherry-picking security updates as denial was proposing. There's no need to deep-update the whole system, just the internet-facing packages. |
I simply update any installed packages marked as insecure. It is an interesting thought that one should go beyond that and do continuing deep updates on all Internet-facing packages. _________________ Personal overlay | Simple backup scheme |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|