[HOWTO] Hassle-free emerge -uD world (update-world v1.8)

[mci_nano]

Try this version out, I made some changes which should help utf-8 problems. I can't test utf-8 problems myself, so you'll have to help.

It works

[iBormuth]

This is a nice one which I happen to use frequently. Thanks a lot.
Anyway, why do you sort the list before you write to emergelist? I always thought there was a logic behind the order portage handles packages.
Think you have an 'emerge -e world' that died on its way. If your script didn't sort, you could easily run 'update-world -p' and manually remove the first whatever lines from emergelist.
I'd remove the sort command or at least make it optional.

[count_zero]

This is a nice one which I happen to use frequently. Thanks a lot.
Anyway, why do you sort the list before you write to emergelist? I always thought there was a logic behind the order portage handles packages.
Think you have an 'emerge -e world' that died on its way. If your script didn't sort, you could easily run 'update-world -p' and manually remove the first whatever lines from emergelist.
I'd remove the sort command or at least make it optional.

Nah. The sort command is to make it easier to find packages that you want to edit out. When you run update-world --install, the entire list gets sent back to portage, which then reorders it again anyway. Check the sorted emergelist, and then check it again once the world starts compiling--you'll find that it's back in the order that portage wants it in. And you can still remove/change packages if the process dies for some reason without running update-world --prepare again. Just edit the emergelist how you like, and restart the update-world --install.

If you REALLY want things compiled in the "right" order, check out how to use Guenther's script to make your emerglist here.

I don't think that there should be any problem with the way things are now. If you can find a circumstance in which I'm wrong, let me know!

[ryker]

@count_zero, I notice your website with the update-world file has been down for a while. Did your server get hammered because of this script? Will it be back up soon?

[count_zero]

@count_zero, I notice your website with the update-world file has been down for a while. Did your server get hammered because of this script? Will it be back up soon?

Yeah, looks like my webhost is flaking out again.
I've put up a mirror on the front page in case the main website is down.
You can access it here: http://dixieflatline.homelinux.org/files/gentoo

[massysett]

Countzero: thanks. I saw your script in the GWN. This script is a great idea. I will try it next time I update my system (which I put off as long as possible.)

I was just emerging the latest GNOME, which is a ton of packages. One of them failed because a package it depended on needed to be emerged with a different USE flag. That got me thinking that it would have been neat if I could use your script to do any big emerge, not just world updates.

Maybe in the future it would be possible to use piped in output from emerge -p? Then any sort of emerge--just a single package with tons of dependencies, or a world update. For instance:

Code: Select all

# emerge -p gnome | update-world

Or

Code: Select all

# emerge -puDN world | update-world

Of course then you might not call the script update-world anymore This feature should be a part of emerge, or gentoolkit, or something, because it's very useful. Thanks for the script.

[count_zero]

Countzero: thanks. I saw your script in the GWN. This script is a great idea. I will try it next time I update my system (which I put off as long as possible.)

I was just emerging the latest GNOME, which is a ton of packages. One of them failed because a package it depended on needed to be emerged with a different USE flag. That got me thinking that it would have been neat if I could use your script to do any big emerge, not just world updates.

Maybe in the future it would be possible to use piped in output from emerge -p? Then any sort of emerge--just a single package with tons of dependencies, or a world update. For instance:

Code: Select all

# emerge -p gnome | update-world

Or

Code: Select all

# emerge -puDN world | update-world

Of course then you might not call the script update-world anymore This feature should be a part of emerge, or gentoolkit, or something, because it's very useful. Thanks for the script.

Good idea!
This is something that can already be done, with some sed filtering:

Code: Select all

emerge -p gnome | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*\]\ \([[:lower:][:upper:]0-9._+\/-]*\).*/=\1/p' > ~/.update-world/emergelist
update-world --install

Not as easy as

Code: Select all

emerge -p gnome | update-world

but it works.
I'll try to make it easier to do this in the future.
Happy updating!

[12345moon12345]

Love this script, thanks Countzero
I changed it a little bit so it doesn't use emerge on the final install but uses temerge instead.
Temerge is a script that mounts /var/tmp/portage as tmpfs which:
- greatly improves speed
- reduces noice of harddisk
- keeps my harddisk good for more years of reliable duty

it's a very simple change to update-world script:
cat $emergelist | xargs emerge -1
becomes: cat $emergelist | xargs temerge -1

off course the temerge script needs to be on your pc,
download or look for more info on temerge here

maybe more people would like my idea or the two scripts could also be combined (with a commandline option or variable in the script)

another idea: after everything has compiled, run prelink -a ??
another suggestion: at the beginning of the script: emerge --sync ??
just trying to think with you, do your own thing, I like it.

[STEDevil]

I personally think that revdep should be built into emerge as it seems silly to me to allow emerge to look as if it finished successfully while silently breaking lots of other things.

Amen to that. For years I have been amazed at the intuitiv stupidity in not automatically have emerge run a revdep-rebuild after finishing. I would think it goes without saying that 99,9% of users would like an updated WORKING system instead of an updated half BROKEN one.
So please, here is another plea for revdep-rebuild being auto-integrated into your excellent script. I just simply cant think of a reason why a normal user would like to stop with a halfbroken system when updating if the update script could just as well finish all the way to the end.

[rant]
I must say that the last 6 months have brought me several greate tools that for me has been missing badly in Gentoo since the very start. eix, cfg-update and now update-world is finally adding to emerge some of the huge pieces its missing to qualify as a modern useroriented packagemanager. I really like Gentoo in general, but sometimes the last years I have just wanted to bang my head against the desk in frustration of how aqward and userunfriendly some things are handled. These 3 new tools fixes a lot of those constantly reoccuring annoyances and enable me to use more of my computertime for work and entertainment instead of maintainace.
Hurray for a brighter, more userfriendly Gentoo future!
[/rant]

[count_zero]

@12345moon12345:
A variable is easy to set, I can include it with the next release so you can more easily choose which emerge-wrapper to use in this script. I've never tried (or heard of) temerge, but I'll have to check it out!

@STEDevil:
You're right, revdep-rebuild is something that should be run after updating a system. I'll have to see how I can make it work with update-world, it's probably not going to be a trivial thing.


Merry Christmas, all!
count_zero

[riaanho]

Hi count_zero

I used your script for the 1st time and ran into a problem where a KDE application KDAR was rebuild before KDELIBS.
I then used the script provided by Guenther Brunthaler and recompiled my entire system after upgrading to GCC 4.1.1

I also think that the generated file should be in the correct emerge sequence or did I do something wrong.

I think your script has great potential to be used whenever one wants to stay current with World.

Regards
Adriaan

[ryker]

@12345moon12345
Thanks for the temerge tip. It works nicely on my machine. Although, it's probably not very useful if your machine has <1G RAM.

@count_zero
I notice that the server and the mirror are both down. I have webspace for a mirror and I'd be happy to donate some of it for this script. Just pm or email me if you would like me to mirror the files.

[vinboy]

hi
sorry for being a noob.

so this script is used for normal updating? like the one we do every 2 weeks.
I assume this script is not used for upgrading GCC (having to compile everything).

[count_zero]

hi
sorry for being a noob.

so this script is used for normal updating? like the one we do every 2 weeks.
I assume this script is not used for upgrading GCC (having to compile everything).

This script can be used for compiling any number of packages, but is most useful when you're updating a large number of packages that you don't want to supervise. So yes, you can use it for both normal updates (emerge -uD world) and complete system updates (emerge -e world). If you're updating GCC, you should probably update that first and switch to the new profile (gcc-config) before using this script, or you might just be recompiling your world with your old GCC.

[vinboy]

why not include this in portage or in the portage repository with an ebuild?
this is something useful for many of us.

great job!

[KevinLarson]

I realized my problem was related to emerge, not this script.

[ryker]

why not include this in portage or in the portage repository with an ebuild?
this is something useful for many of us.

great job!

It would be nice to see this as part of the gentoolkit package. Having a seperate ebuild for it seems kind of silly since it's just a small script.

[ASID]

If it's possible, could you please add something like

Code: Select all

update-world --resume

to use in case of an unexpected shutdown?

Keep up the great job!

[count_zero]

If it's possible, could you please add something like

Code: Select all

update-world --resume

to use in case of an unexpected shutdown?

Keep up the great job!

This script has --resume built in. If for some reason the emerge gets halted (power failure, control-c, etc.), the 'emergelist' and 'failed list' are preserved. All you have to do to continue on is to run update-world --install, and the process will pick up right back where it left off.

[ASID]

This script has --resume built in. If for some reason the emerge gets halted (power failure, control-c, etc.), the 'emergelist' and 'failed list' are preserved. All you have to do to continue on is to run update-world --install, and the process will pick up right back where it left off.

Great! Thanx for the tip...

[drjimmy42]

If one of that packages that needs updating has an unsatisfied patch restriction, I get the message

Code: Select all

WARNING: The following packages have Fetch Restriction turned on:
app-emulation/crossover-office-bin-6.0.0

Please download the sources to /usr/portage/distfiles/ and rerun this script.

From the changelog at the top of the script it says that fetch restriction is handled properly in 1.5. I was wondering why this behavior is like this. IMHO it would be better for the script to go over unsatisfied fetch restrictions as any other failure and report it at the end. Is there a reason this was changed?

-------------------------------------------

Also, this brings up another point. I have noticed that if the last package in emergelist is the one to fail, update-world attempts to start up again and gets errors from emerge because it didn't pass it a package name. No harm done since its all done anyway, but a check here would make the output a lot prettier and easier to understand.

Thanks again for the fabulous script

[count_zero]

If one of that packages that needs updating has an unsatisfied patch restriction, I get the message

Code:

WARNING: The following packages have Fetch Restriction turned on:
app-emulation/crossover-office-bin-6.0.0

Please download the sources to /usr/portage/distfiles/ and rerun this script.



From the changelog at the top of the script it says that fetch restriction is handled properly in 1.5. I was wondering why this behavior is like this. IMHO it would be better for the script to go over unsatisfied fetch restrictions as any other failure and report it at the end. Is there a reason this was changed?

That's how it's supposed to work--it lets you know there is a fetch restriction so you can download the file before proceeding. Did you download the file and put it in your disfiles directory? If you did and it still gives the warning, then there's a problem.

[drjimmy42]

That's how it's supposed to work--it lets you know there is a fetch restriction so you can download the file before proceeding. Did you download the file and put it in your disfiles directory? If you did and it still gives the warning, then there's a problem.

Yeah, I saw the code where you check for it. It works as designed and if the file is there it works, its just that I use this script so that I can update unattended and deal with issues like breakages later. To me, a fetch restriction is as good as a break and I don't want to hold up the rest of the update for either one. I haven't gotten around to buying crossover-office yet and it just sits there glaring at me.

Its no big deal, I just commented the check out of the copy of the script I had, I was just wondering why this block to updating is different than others. Thanks again.

[count_zero]

That's how it's supposed to work--it lets you know there is a fetch restriction so you can download the file before proceeding. Did you download the file and put it in your disfiles directory? If you did and it still gives the warning, then there's a problem.

Yeah, I saw the code where you check for it. It works as designed and if the file is there it works, its just that I use this script so that I can update unattended and deal with issues like breakages later. To me, a fetch restriction is as good as a break and I don't want to hold up the rest of the update for either one. I haven't gotten around to buying crossover-office yet and it just sits there glaring at me.

Its no big deal, I just commented the check out of the copy of the script I had, I was just wondering why this block to updating is different than others. Thanks again.

You can always add '>=app-emulation/crossover-office-bin-6.0.0' (or greater than your installed version) to package.mask if you don't want to upgrade it. I saw this as a better solution than not reporting a fetch restriction, which can often be resolved using wget or downloading it from the website.

[TehZeppelin]

[ebuild FU ] dev-java/sun-jdk-1.4.2.13 [1.4.2.12-r2] USE="X alsa -doc -examples -jce -nsplugin" 35,509 kB
. . .
Fetch Restriction: 1 package (1 unsatisfied)

You'll need to download the sources for sun-jdk from the website and copy to /usr/portage/distfiles/ before running the script.
Fix these problems before re-running update-world.

I should be able to make some changes to the script so it handles these errors better.

Where can you get those sources?