emerge -U world - How often

Message

jchomarat · Post by **jchomarat** » Mon Apr 19, 2004 4:05 pm

Hello, I'm using gentoo since i'm registered to this forum, a few months then.
I have just done emerge sync, then update portage, then emerge -Upv world. I have more than 500 Megs to download, and to recompile. It will be probably another few days of hard working. How often do you update your packages. Do I have to upgrade everything. I am very happy with my box now, I can play, develop in mono etc ....

Thanks

Julien

Little Nemo · Post by **Little Nemo** » Mon Apr 19, 2004 4:20 pm

If you're happy with your system, don't upgrade world. I'd rather see whether "emerge -Upv system" lists anything you'd really want to update, and generally only upgrade individual packages. This makes recovery much easier if anything goes wrong (and there are numerous reports on "emerge -U world" breaking things), and you can deal with changes individually.

BiggJ · Post by **BiggJ** » Mon Apr 19, 2004 4:21 pm

It's really up to you (sort of)...

On my desktop machines, I cron an emerge sync once a week. And (if I remember) kick off an 'emerge -u world' about as often. On my server machines, I don't update everything. I only update packages which are needed to fix security issues. This is something you sould keep you eye on even if you don't want to update everything (which is why I say "sort of").

--J

miqorz · Post by **miqorz** » Mon Apr 19, 2004 4:21 pm

I haven't emerge -Ud world'd yet and i dont plan on it.

robmoss · Post by **robmoss** » Mon Apr 19, 2004 4:40 pm

You should definitely run the following command every day or so (note that it's still experimental, but it seems to work quite well):

Code: Select all

glsa-check -f all

I wouldn't ever, ever do an emerge -U world or an emerge -UD world. You should use -u, and for packages for which you want the unstable version, add the following line to /etc/portage/package.keywords (you may have to create /etc/portage first):

Code: Select all

sys-devel/gcc ~x86

or whatever package and keywords you want to use.

jchomarat · Post by **jchomarat** » Mon Apr 19, 2004 4:57 pm

Thanks guys, I think i'll stick with Little Nemo answer, emerge -Upv system.

robmoss2k, what is it for the command you mentionned?

robmoss · Post by **robmoss** » Tue Apr 20, 2004 2:25 am

jchomarat wrote:Thanks guys, I think i'll stick with Little Nemo answer, emerge -Upv system.

robmoss2k, what is it for the command you mentionned?

Okay. As you're probably aware, there are two different sorts of packages which you can use - those which are marked with the "x86" keyword, and those which are marked with the "~x86" keyword. Those which are marked with the "x86" keyword are considered stable and should work on any x86 system whatsoever without a hitch. Those marked with "~x86" are considered unstable or testing, and may break. Now, if you want to use the stable "x86" profile, but want to use certain unstable or testing packages, you SHOULD NEVER EVER EVER use the command:

Code: Select all

ACCEPT_KEYWORDS="~x86" emerge package

and then afterwards use:

Code: Select all

emerge -UDv world

to update your system. This will definitely, definitely break things. I promise.

The correct behaviour, which was introduced in portage-2.0.50 in an attempt to fix this, is to have a file located at /etc/portage/package.keywords, which tells portage which packages you'd like to use the unstable profile for. So an example package.keywords file might look like this:

Code: Select all

sys-devel/gcc ~x86
x11-base/xfree ~x86

This would tell portage that it should use the "~x86" keyword for gcc and xfree and the "x86" keyword for everything else.

Now, why is this the correct way to do things? Well, it's not unusual for a package to be released, only for it to later be discovered that the package is in fact broken. So that package is marked "-*" or removed entirely, and portage should downgrade it in order to ensure system stability. But if you use "emerge -UDv world" then you fundamentally break this process. Indeed, it's not unusual to have at least three or four broken packages floating around simply because you're not letting portage downgrade them.

So, if you want a stable system, you should NOT be using the -U flag. EVER. And if it breaks - well, you get to keep the pieces. The portage documentation tells you that this flag is deprecated for a very, very good reason, which I've described above. My system is rock stable, and I use this process to update it:

Code: Select all

emerge sync
emerge -uDpv world
emerge -uDv world
emerge -pv depclean
emerge -v depclean
revdep-rebuild -pv
revdep-rebuild -v
dispatch-conf

That basically updates the portage tree, tells me what's involved in updating everything, updates everything, tells me what packages are no longer necessary, removes them, tells me what packages have been broken by an upgrade or removal, and recompiles them. Then dispatch-conf updates my config files for me with minimal intervention.

The glsa-check programme is a new function of portage which performs security updates automatically. glsa-check is run with "-f all" in order for it to fix (-f) every security flaw (all) identified by a Gentoo Linux Security Advisory (GLSA). So if you want your system to be secure as well as stable, you want to run this regularly.

Personally, I run "glsa-check -f all" every hour, and the above set of commands for updating my system every day. I've not had a single breakage in over a year on my stable "x86" box, and only two or three on my unstable/testing "~x86" box.

gtaluvit · Post by **gtaluvit** » Tue Apr 20, 2004 2:35 am

Just a quick aside. You can take what rob said and replace -p with -a. This will ask you if you want to continue the emerge. This is nice for depclean and a -D emerge since you don't have to recalculate dependencies by just removing the -p flag.

robmoss · Post by **robmoss** » Tue Apr 20, 2004 2:36 am

Sorry, yes, you're right - interesting further aside... my flatmate wrote the code for the "-a" flag

gtaluvit · Post by **gtaluvit** » Tue Apr 20, 2004 2:44 am

btw, what is dispatch-conf? No man page, no help, and very little in the forums on its actual usage.

robmoss · Post by **robmoss** » Tue Apr 20, 2004 2:57 am

Yes, I know - that kinda sucks, as it's nothing short of perfect. Its configuration file explains things somewhat - it's located at /etc/dispatch-conf.conf. You'll need to do an "emerge rcs" to take full advantage of it, and also create the directory /etc/config-archive. It gives full revision control over config file updates, meaning you can easily drop back to older versions - but more importantly (to me anyway), you can automerge things which update config files with no more than whitespace, headers and comments, and files which the user has never edited. etc-update is horrible. Don't use it. Ever. Filthy!

Little Nemo · Post by **Little Nemo** » Tue Apr 20, 2004 6:59 am

Thanks for the comprehensive explanation, robmoss2k. This is a thread I'd like to see sticky. It seems vital for every Gentoo user.

jchomarat · Post by **jchomarat** » Tue Apr 20, 2004 8:42 am

Yeap, thanks a lot for this tutorial ... I was kind of scared of using depclean as i've seen so many posts where people broke their system. But, I guess by running revdep-rebuild right after you repair potential packages broken by the updates and the removals.

I'll definitely give a shot.

Chris-P · Post by **Chris-P** » Tue Apr 20, 2004 3:13 pm

Little Nemo wrote:Thanks for the comprehensive explanation, robmoss2k. This is a thread I'd like to see sticky. It seems vital for every Gentoo user.

Agreed! A very useful thread - it has taught me about the existence of dispatch-conf, which I never knew about! And I fully agree with robmoss2k on etc-update being filthy, especially if (like me) you came to the world of Linux via Gentoo. I wonder how many fresh nOOb Gentoo installs have been hosed by the use of this very package.

Chris.

lookitsme · Post by **lookitsme** » Tue Apr 20, 2004 3:54 pm

Chris-P wrote:[...]I wonder how many fresh nOOb Gentoo installs have been hosed by the use of this very package.
Chris.

I know of at least one

And yes, I agree. A usefull thread it is...

Carmine · Post by **Carmine** » Tue Apr 20, 2004 7:56 pm

Well well well, glad to know that, so far, I've messed up my system
with the "emerge -UD..." stuff and "etc-update" killer.

Great post, this is and the force flows powerful in robmoss2k.

Let's do a sync, -uD, dispatch-conf session...whooohoooo

F16PilotJumper · Post by **F16PilotJumper** » Tue Apr 20, 2004 9:58 pm

I let etc-update replace my fstab when i wasn't looking. messy messy to fix.

m0sk · Post by **m0sk** » Tue Apr 20, 2004 11:06 pm

robmoss2k wrote: The correct behaviour, which was introduced in portage-2.0.50 in an attempt to fix this, is to have a file located at /etc/portage/package.keywords, which tells portage which packages you'd like to use the unstable profile for. So an example package.keywords file might look like this:
Code: Select all
sys-devel/gcc ~x86
x11-base/xfree ~x86

I've read about this package.keywords file, but never bothered to look into it. Actually, according to man portage there are more files you can put in /etc/portage. Nice to (finally) discover the new features of portage 2.0.50

...

robmoss2k wrote: My system is rock stable, and I use this process to update it:
Code: Select all
emerge sync
emerge -uDpv world
emerge -uDv world
emerge -pv depclean
emerge -v depclean
revdep-rebuild -pv
revdep-rebuild -v
dispatch-conf

I'm trying this at the moment, the deplcean output seemed a little scary to me

Let's see what revdep-rebuild comes up with next.

Anyway, I wanted to throw in a quick thanks for the information posted here. Well done

Carmine · Post by **Carmine** » Wed Apr 21, 2004 2:50 pm

@m0sk:
Let me tell you what "a scary depclean" is!

Last night (error n.1 never ever attempt doing dangerous things
when there's no sun) I have discovered this thread's existence and went
to try to fix things...

First attempt with "emerge depclean" lead to:
- Removal of many many *.so library files so that:
- ls,mkdir & co didn't worked anymore.

Having realised that it would have been better if I did things in the morning,
I went to sleep...

This morning I booted from Live CD and:
error n.2: untarred stage3 over my system.
After booting I was able to emerge and do other stuff.
After emerge -vuD world came emerge depclean...
Final Error: emerge depclean choosed to bomb a lot of things
among which stuff like: shutdown emerge ....

Results: I'm stuck with a machine I can't use to do anything, and
perhaps this is the point of no return before a reinstallation.

This is _scary_

nevynxxx · Post by **nevynxxx** » Wed Apr 21, 2004 3:28 pm

Daily or when ever a log on, whichever is the longest timeframe.
I stick with

Code: Select all

emerge -uDav world

the a bit being the much nicer big brother to p. On all my systems except my server that becomes

Code: Select all

emerge sync && emerge -uDav world

since I sync to my internal server rather than a mirror. My server syncs to a mirror on a nightly basis at a quiet time, i figure if the server folks are bothered they will ban me

. But 1 hit per night at 3:30am from my ip aint gunna be a big strain.

nevynxxx · Post by **nevynxxx** » Wed Apr 21, 2004 3:30 pm

Carmine wrote:Results: I'm stuck with a machine I can't use to do anything, and
perhaps this is the point of no return before a reinstallation.

This is _scary_

Never ever *EVER* depclean without using -p or -a (if it supports it). If there are *ANY* packages you know you want, or don't know what they are, investigate further and just remove what you don't need. depclean can be very dangerous and I'm sure it warns of this.

<edit>
this is the message depclean gives

Code: Select all

*** WARNING *** : DEPCLEAN CAN  SERIOUSLY  IMPAIR YOUR SYSTEM. USE CAUTION.
*** WARNING *** : (Cancel: CONTROL-C) -- ALWAYS VERIFY ALL PACKAGES IN THE
*** WARNING *** : CANDIDATE LIST FOR  SANITY  BEFORE  ALLOWING DEPCLEAN TO
*** WARNING *** : UNMERGE ANY PACKAGES.
*** WARNING *** :
*** WARNING *** : USE FLAGS MAY HAVE AN EXTREME EFFECT ON THE OUTPUT.
*** WARNING *** : SOME LIBRARIES MAY BE USED BY PACKAGES BUT ARE NOT
*** WARNING *** : CONSIDERED TO BE A DEPEND DUE TO USE FLAG SETTINGS.
*** WARNING *** :
*** WARNING *** : Packages  in the list  that are  desired  may be added
*** WARNING *** : directly to the world file to cause them to be ignored
*** WARNING *** : by depclean and maintained in the future. BREAKAGES DUE
*** WARNING *** : TO UNMERGING AN  ==IN-USE LIBRARY==  MAY BE REPAIRED BY
*** WARNING *** : MERGING  *** THE PACKAGE THAT COMPLAINS ***  ABOUT THE
*** WARNING *** : MISSING LIBRARY.

scary enough for you?

Carmine · Post by **Carmine** » Wed Apr 21, 2004 4:02 pm

@nevynxxx:

Yes I was scared enough but not awake yet

I'm facing now a similar problem, where depclean wants to remove
'acl" and I know that a similar action led me to suffering on the other
machine...

Regards,

nevynxxx · Post by **nevynxxx** » Wed Apr 21, 2004 4:11 pm

Carmine wrote:@nevynxxx:

Yes I was scared enough but not awake yet

I'm facing now a similar problem, where depclean wants to remove
'acl" and I know that a similar action led me to suffering on the other
machine...

Regards,

then add acl to your world file! or forget about depclean, and don't ever use it.

m0sk · Post by **m0sk** » Wed Apr 21, 2004 4:41 pm

Carmine,

sorry to hear that

Carmine wrote: First attempt with "emerge depclean" lead to:
- Removal of many many *.so library files so that:
- ls,mkdir & co didn't worked anymore.

Well, one thing I noticed about depclean was that it wanted to remove fileutils (provides the ls, chmod, ... stuff). After checking on gentoo-portage, I learned that the fileutils package is merged into coreutils, which I had already installed. So I figured it'd be safe to continue. Indeed, apparently no problems. My only problem is that revdep-rebuild wants to rebuild openoffice-bin, which is a known "bug". Other than that, everything went fine.

But since you're talking about shared libs needed by ls et al, I'm guessing depclean was messing with glibc? That's very scary indeed

Although I didn't hose my install, I don't think I will perform the depclean/revdep rebuild routine on a regular basis, way too stressful imo

Anyway, good luck

robmoss · Post by **robmoss** » Wed Apr 21, 2004 10:22 pm

Carmine wrote:@m0sk:
Let me tell you what "a scary depclean" is!

Last night (error n.1 never ever attempt doing dangerous things
when there's no sun) I have discovered this thread's existence and went
to try to fix things...

First attempt with "emerge depclean" lead to:
- Removal of many many *.so library files so that:
- ls,mkdir & co didn't worked anymore.

Having realised that it would have been better if I did things in the morning,
I went to sleep...

This morning I booted from Live CD and:
error n.2: untarred stage3 over my system.
After booting I was able to emerge and do other stuff.
After emerge -vuD world came emerge depclean...
Final Error: emerge depclean choosed to bomb a lot of things
among which stuff like: shutdown emerge ....

Results: I'm stuck with a machine I can't use to do anything, and
perhaps this is the point of no return before a reinstallation.

This is _scary_

Yes, this is the acl problem. It's stupid. VERY stupid. On the Gentoo LiveCD, they've built ls, mkdir and the rest of coreutils with acl support. But they haven't put acl into anywhere that tells portage it should keep it. As a result, ls, mkdir et al link against a file (whose name I forget, I may have to investigate) which shouldn't really be there. And when it's removed by depclean... surprise, surprise! Everything breaks.

So. The way to fix this easily is to copy the missing files (there are two, I think they're called something like libacl.so.1 and libattr.so.1 - something like that anyway - and I think they live in /usr/lib) from a LiveCD (having booted from it) to /usr/lib (or wherever it is they're located on the LiveCD). Then, having done that, you should remerge coreutils with USE="-acl" set. This will take acl (Access Control Lists) support out of coreutils and it won't link against those two files. Once you've remerged coreutils, you can get rid of those two files again.

I think I should file a bug...