Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Conservative updating strategy
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Networking & Security
View previous topic :: View next topic  
Author Message
denial
n00b
n00b


Joined: 15 Nov 2007
Posts: 5

PostPosted: Thu Apr 19, 2012 10:51 pm    Post subject: Conservative updating strategy Reply with quote

tl/dr: can I be safe without ever running emerge -uND @world?

I run Gentoo on a Laptop that acts as a home workstation (never moves) and home server (remote ssh access). Due to its limited processing power, always updating everything takes quite some time. I also see no reason to update things that are working fine for me, unless I want particular features -- but I don't want the system to be vulnerable to known security issues.

So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features

Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed.

My process for this is a very hackish daily cron job that emails me GLSAs that affect the system. This report also pulls a list of security bugs active in the last 2 days and filters those that match installed packages (From what I gather GLSAs are not very reliable at the moment, and in any case can take quite some time, so this is my improvised alternative -- any other suggestions welcome! Especially since I'm not 100% sure that wget works as I think it does) This way I can check the GLSAs and bugs and decide what packages need to be updates or otherwise fixed (or removed) for security reasons.
Code:
#!/bin/bash -- Daily Securty Report
emerge --sync -q 1> /dev/null
eix-installed -a > /var/tmp/update-report/installed

wget -q "https://bugs.gentoo.org/buglist.cgi?f1=days_elapsed&known_name=sec-60&list_id=970687&o1=lessthan&product=Gentoo%20Security&query_format=advanced&v1=2&title=Bug%20List:%20sec-60&ctype=atom" -O /var/tmp/update-report/bug-rss

echo "Subject:security-report.sh" > /var/tmp/update-report/security-report
echo "The following GLSA(s) may affect the sytem:" >> /var/tmp/update-report/security-report
glsa-check -lnq affected >> /var/tmp/update-report/security-report
echo "The following security-related bugs that were active since yesterday may affect the system:" >> /var/tmp/update-report/security-report
for line in $(sed -e 's/-[0-9]\+.*$//' /var/tmp/update-report/installed); do grep -A3 "$line[^a-zA-Z]" /var/tmp/update-report/bug-rss ;  done | grep "<id>https" | sed -e 's/.*<id>//' -e 's/<\/id>//' >> /var/tmp/update-report/security-report
date >> /var/tmp/update-report/security-report
sendmail root < /var/tmp/update-report/security-report

The second part of the process is weekly cron-job that informs me of available updates and obsolete packages. I can then update these as needed. If I see portage or some other critical component I update it. Anything else I leave unless or look up whether it's useful (one exception: I always update flash).
Code:
#!/bin/bash -- Weekly Update Report
mkdir -p /var/tmp/update-report
mkdir -p /var/tmp/update-report
emerge -pu @world > /var/tmp/update-report/world-updates
eix-test-obsolete > /var/tmp/update-report/obsolete
sed -e 's/^These are/Subject:emerge @world available updates\nThese are/' /var/tmp/update-report/world-updates | grep -v "^$" | grep -v Calculating > /var/tmp/update-report/report
grep ^[[I] /var/tmp/update-report/obsolete | sed -e 's/^Installed/\neix-test-obsolete <root@shinybox>\nInstalled/' >> /var/tmp/update-report/report
date >> /var/tmp/update-report/report
sendmail root < /var/tmp/update-report/report

The third part of the process is to look over some files to make sure everything is in order (I tend to enable use-flags per-package as needed).
Code:
 /var/lib/portage/world make.conf package.use package.accept_keywords

Also a look over emerge -puD @system emerge -puD @world to see if any big dependencies are looking very outdated.

Finally, every 6 months or so I plan to check whether a toolchain update is in order (I may also go for a emerge -uD @system at this point).
Code:
equery list gcc -p
equery list binutils -p
equery list glibc -p

This is still mostly theoretical, so my question is: is this a good way to ensure a stable and secure system? Am I going to be in world of pain at some point? Am I leaving the system vulnerable in terms of security?
edit:cron script fix


Last edited by denial on Fri Apr 20, 2012 11:28 am; edited 1 time in total
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Thu Apr 19, 2012 11:02 pm    Post subject: Reply with quote

I think the most important thing for you, since you want to keep things stable, is to, well, stay on stable and not ~arch.

I also think it's important, when using Gentoo, to update world on some kind of regular basis. I used to get frustrated with updates until I hit upon my current approach:

1. emerge --sync
2. layman -S (skip if you don't have overlays)
3. emerge -pvuDN world (read output)
4. cd ~/PortageLogs
5. emerge -pvuDN world > (month+year) - gives you something to review if something breaks
6. emerge -vuDN world
7. update any 9999 packages
8. emerge -p --depclean

I do this for all my Gentoo machines on the first of each month - no more or less often. All my machines are on ~arch, not stable, and things break very infrequently for me (and usually because someone threw a bomb down the pipe. libpng, udev, and such).

Think about incorporating any or all of this into your routine. I think if you do not update world on a regular basis, letting no more than a month or two go by between updates, some dam will break at some point and you'll be spending a lot of time fixing it. Gentoo is a rolling release distro and is meant to work best when updated regularly. Stuff has always broken for me in big ways whenever I wait too long to update world.
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
The Doctor
Moderator
Moderator


Joined: 27 Jul 2010
Posts: 2678

PostPosted: Fri Apr 20, 2012 2:44 am    Post subject: Reply with quote

For updating, any more than a month and you risk updates that cannot be done due to required packages simply not having an path that it can be upgraded on, most notably glibc, or other problems like massive blocks, circular dependencies, etc. I recently wiped my laptop's virtaulbox Gentoo install because it became too outdated.

I update every few days just to minimize the amount of time compiling. My system is about 5 years old with only two cores, and this usually takes about 30 minutes to an hour. This works for me. Keeping in mind the restrictions, you should play with the timing to figure out when you want to invest the time per month to do an update. You should do this at least once a month. As for security, I think you should be able to maintain a fairly secure system with monthly updates.
_________________
First things first, but not necessarily in that order.

Apologies if I take a while to respond. I'm currently working on the dematerialization circuit for my blue box.
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent

PostPosted: Fri Apr 20, 2012 3:37 am    Post subject: Re: Conservative updating strategy Reply with quote

denial wrote:
So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features

Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed.

I've been doing exactly this for years without a major problem, though I did run into one interesting corner case:

(a) Some version of a library in slot :0 is removed from the ebuild tree
(b) It is replaced by a minor revision, but is no longer in slot :0 because there is a new major revision which now inhabits slot :0
(c) Because the library headers are poorly laid out, only the new slot :0 will install headers. The old slot :0 relies on the new slot :0 for the headers.
(d) If one replaces the old version by the minor revision, instead of the major revision (to avoid a big revdep-rebuild), dependent package updates will fail because there are no headers available.

While this situation seems rather esoteric, it illustrates the potential for weird failures because the developers assume you will do a deep upgrade without being selective about which packages or package versions get updated.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
kimmie
Guru
Guru


Joined: 08 Sep 2004
Posts: 531
Location: Australia

PostPosted: Fri Apr 20, 2012 4:38 am    Post subject: Reply with quote

TL;dr/ You're wasting your own time going against the gentoo update flow.

Is your system really too slow to stay on stable and fully update somewhere between once a week and once a month? Because rather than maintaining your update system and looking over dependencies, you could probably be doing something much more interesting. After all, time your box spends compiling doesn't really count... just leave it overnight and come back tomorrow. Use binary packages for the big buggers like browser or openoffice, and pare down your environment overall (eg. gnome-light instead of gnome, or an even lighter desktop environment), and disable USE flags for features you don't need that pull in dependencies.

After 8 years of gentoo, I've settled on weekly cronjob which syncs and emails me the emerge -puND results. I then update as I see fit, usually that means -uDN world but sometimes I'll manually --exclude or mask a big package for a while.

WRT toolchain, excluding bugfixes there's usually no compelling reason to update at all, especially if you have an old processor. You can easily go more than six months between toolchain updates. Mask major versions of gcc until you decide you want them, for whatever reason.

WRT kernel, linux-headers... stick with a long-term stable kernel that works on your hardware.

WRT security: you only really need to care about attack vectors that affect you. For you, that's probably openssh and your browser. So you could just regularly do a deep update on those. Polling bugzilla for any security-related report is casting a pretty wide net, you're going to put on weight eating all that seafood. If you're only running sshd and a browser, that's starting to get tinfoil paranoid... if you want that level of security your better off running your browser in a VM, and making sure not to visit http://please-root-me.com. Although, I have to admit, seafood does taste good cooked in tinfoil.

If you were to keep track of two systems for a couple of years, one doing weekly -uDNs and one using your proposed method, I'd put money on you spending far less time maintaining the former. So, my 20c is: don't bother, go with the flow.
Back to top
View user's profile Send private message
denial
n00b
n00b


Joined: 15 Nov 2007
Posts: 5

PostPosted: Fri Apr 20, 2012 12:22 pm    Post subject: Reply with quote

Thanks for all the feedback advice!

audiodef wrote:
I think the most important thing for you, since you want to keep things stable, is to, well, stay on stable and not ~arch.

Yes, I definitely avoid ~arch except where absolutely necessary.
audiodef wrote:
II think if you do not update world on a regular basis, letting no more than a month or two go by between updates, some dam will break at some point and you'll be spending a lot of time fixing it. Gentoo is a rolling release distro and is meant to work best when updated regularly. Stuff has always broken for me in big ways whenever I wait too long to update world.

penguin swordmaster wrote:
For updating, any more than a month and you risk updates that cannot be done due to required packages simply not having an path that it can be upgraded on, most notably glibc, or other problems like massive blocks, circular dependencies, etc.

Well I would update regularly, just not to the newest version. I would never have a package installed that is not in the current portage tree (eix-test-obsolete warns about that, right?). So I would be still be rolling, but at the back of the pack and skipping some minor versions/revisions in between updates.

penguin swordmaster wrote:
I update every few days just to minimize the amount of time compiling.

I would argue that this means you actually spend more time compiling - individual sessions are shorter but overall you would be building more minor versions or revisions if they come out in quick sucession.

Hypnos wrote:
denial wrote:
So my updating strategy is:
(1) keep portage and related tools (eix etc) up to date
(2) update anything that is affected by a security bug
(3) update anything that is no longer in the portage tree
(4) update anything that is unstable or buggy
(5) update anything I want for shiny new features
Of course any update is followed by emerge --deplclean, revdep-rebuild, and dispatch-conf as needed.

I've been doing exactly this for years without a major problem, though I did run into one interesting corner case:

(a) Some version of a library in slot :0 is removed from the ebuild tree
(b) It is replaced by a minor revision, but is no longer in slot :0 because there is a new major revision which now inhabits slot :0
(c) Because the library headers are poorly laid out, only the new slot :0 will install headers. The old slot :0 relies on the new slot :0 for the headers.
(d) If one replaces the old version by the minor revision, instead of the major revision (to avoid a big revdep-rebuild), dependent package updates will fail because there are no headers available.

While this situation seems rather esoteric, it illustrates the potential for weird failures because the developers assume you will do a deep upgrade without being selective about which packages or package versions get updated.

That's great to hear. What kind of system do you use this approach for? Are there any other suggestions you would have (i.e. rebuild @system, @world at some point? watch major libraries?).
Is there some sort of (official?) policy on how long old packages stay in portage?
In terms of the specific problem you point out, how can that be avoided or worked-around?

kimmie wrote:
TL;dr/ You're wasting your own time going against the gentoo update flow.

Am I really going against it? Still rolling, just not as quickly. I accept that package testing is done on fully updated stable and so I miss out on some of that -- but I don't think the testing can take into account all the particularities of an install, so I don't feel I'm loosing too much there.
kimmie wrote:
Is your system really too slow to stay on stable and fully update somewhere between once a week and once a month?

No, not really, I could do it if I really wanted to. I agree with your suggestions for a leaner system (I run awesome and disable most things by default).
kimmie wrote:
WRT toolchain, excluding bugfixes there's usually no compelling reason to update at all, especially if you have an old processor. You can easily go more than six months between toolchain updates. Mask major versions of gcc until you decide you want them, for whatever reason.

To what extent does this apply to glibc and binutils? Will say the glibc-2.10.x branch continue to receive upstream bug and security fixes? Will something like this https://bugs.gentoo.org/show_bug.cgi?id=386329 ever be fixed for the older version?
kimmie wrote:
WRT security: you only really need to care about attack vectors that affect you. For you, that's probably openssh and your browser.

Yes that's an excellent point. At the moment the only vector is probably the browser (well, flash mostly) as I am so secured behind an institutional firewall I can hardly get to my own machine. But I am in part looking to try this out to see if it would work for more extensive home-server and desktop systems I am likely to have quite soon -- which would be open more vectors and won't have that protection.
kimmie wrote:
Although, I have to admit, seafood does taste good cooked in tinfoil.

Damnit, now I'm hungry! In need some sea-bass, lots of lemon and some good potatoes....
kimmie wrote:
If you were to keep track of two systems for a couple of years, one doing weekly -uDNs and one using your proposed method, I'd put money on you spending far less time maintaining the former. So, my 20c is: don't bother, go with the flow.

If both can be secure I suppose this is what it boils down to. Does anyone disagree with this assessment? Has anyone tried both?
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Fri Apr 20, 2012 12:49 pm    Post subject: Reply with quote

kimmie wrote:
TL;dr/


TL/DR :P :P :P
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent

PostPosted: Fri Apr 20, 2012 1:38 pm    Post subject: Reply with quote

I do this on my laptop, which I use for both work and pleasure.

I don't have any special recommendations for particular packages or package sets to watch.

As a counterpoint to kimmie: the reason I follow this update strategy is that I find that deep upgrades can result in breakage besides that fixed by revdep-rebuild. By upgrading only when I have to, I greatly reduce my maintenance effort. Moreover, most deep upgrades don't affect system function, meaning that many upgrade+revdep-rebuild cycles (which require human monitoring at beginning) are entirely unnecessary.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
kimmie
Guru
Guru


Joined: 08 Sep 2004
Posts: 531
Location: Australia

PostPosted: Sat Apr 21, 2012 4:40 pm    Post subject: Reply with quote

Hypnos, I was thinking of the deep updates more as a security measure, as an alternative to cherry-picking security updates as denial was proposing. There's no need to deep-update the whole system, just the internet-facing packages.

denial, yeah, sorry, when I said toolchain, I was really thinking about gcc. I'd stick with a reasonably current glibc and binutils. Personally, what I do with glibc is to put off updating it until a new linux-headers arrives, or a GLSA tells me to.

For me, in the end it comes down to having updates in managable sized chunks (I'd much rather have 4 updates of 20 packages than 3 of 5 and one of 70), and reasonably regular full updates is the best way I've found to do this. Also, I like to keep my system reasonably similar to others out there, in the hope that if I have some weird package interaction I won't be the only one. Consider what happens when, say, you run into some silly desktop issue, like your removable drives refusing to mount automatically. But nobody else seems to be complaining of this. What's the first thing you're going to do? Update. So all the time you've saved by not updating is suddenly unsaved again, and you get bogged down for a couple of days in a huge update, and you end up cursing the stupid little dog next door which is always going Moon, Mooon, MOON, MOON, moon, tree, moon, cat CAT CAT MOON to the other stupid little dog down the road and forgetting which removable drive you were trying to mount in the first place way back when you were actually trying to get something done... (of course, this is a purely hypothetical scenario which has never happened to me, :roll:).
Back to top
View user's profile Send private message
Hypnos
Advocate
Advocate


Joined: 18 Jul 2002
Posts: 2889
Location: Omnipresent

PostPosted: Sat Apr 21, 2012 4:56 pm    Post subject: Reply with quote

kimmie wrote:
Hypnos, I was thinking of the deep updates more as a security measure, as an alternative to cherry-picking security updates as denial was proposing. There's no need to deep-update the whole system, just the internet-facing packages.

I simply update any installed packages marked as insecure. It is an interesting thought that one should go beyond that and do continuing deep updates on all Internet-facing packages.
_________________
Personal overlay | Simple backup scheme
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Networking & Security All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum