| View previous topic :: View next topic |
| Author |
Message |
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
 Posted: Fri Mar 19, 2010 12:52 am Post subject: HOWTO: Major System Upgrades - Post II - Initial Upgrades ++ |
 |
|
Pre-Introduction -2) Huh?!?!?
This is the second of a series four posts describing and walking through the complete process of performing a Major Upgrade on a Gentoo based system. The other posts are:
Chapter 5) - Emerge GCC and glibc
Still don't bother running "emerge -puNDv world" just yet. Since we want to upgrade everything to the latest and greatest, we want to start with the tools which are going to do all the heavy lifting. For this reason, execute:
Compiling gcc is a chicken and the egg type issue. In general, the default is to compile gcc twice. As far as I currently understand, all distributions do this. The first time uses gcc-{current} version to compile gcc-{future} and then to have gcc-{future} compile gcc-{future}. This was and I believe still is the current behavior of ebuilds for gcc. The goal is to have a pristine gcc from which to compile all the other packages. For this reason, I want to force the issue and ensure gcc is compiled first. We do glibc as the same time because glibc is the companion package for gcc.
Now execute:
The listing returned by gcc-config will display at least the current and future versions of gcc. If you have previous versions of gcc installed but not yet unmerged, these will be listed as well.
Execute: | Code: | | # gcc-config # {where '#' equals the line number of version 4.4.3 in the list} |
or | Code: | | # gcc-config x86-pc-linux-gnu-4.4.3 {or x86_64-pc-linux-gnu-4.4.3 depending on the target PC} |
Chapter 6) - Emerge @System
Patience grasshopper, we're still not ready to run "emerge -puNDv world".
Instead, execute: | Code: | | # emerge -pev system |
The hope is that emerge doesn't issue any complaints. If it does, you'll need to resolve them. Things to look for include messages about e2fsprogs. Inspect these messages carefully. Because there are error messages and functions in e2fsprogs which are important to the system functionality, if you get emerge complaints here, you must resolve them. Generally, you need to do whatever steps are needed to emerge to the latest version of e2fsprogs. In my experience this has included unmerging one or two packages and then upgrading e2fsprogs. Depending on which rev of e2fsprogs and friends you may be upgrading from and to, you might need to upgrade e2fsprogs independently before upgrading the base system.
If there are no complaints from emerge at this point, we're going to upgrade the base system. Execute: | Code: | | # emerge -e system. |
Get some coffee. Have a bite to eat. Snuggle with the SO of your choice. Flame some ij10ts on Twitter. You know. Kick back and relax.
Note that gcc and glibc are going to be compiled twice ... again. What can I tell you? I'm conservative when it comes to clean programs and data preservation. Besides. It won't hurt. It just consumes more time. Take a siesta in fact. Relaxation is good for the soul.
If the target PC is running the unstable version of portage {2.2 series}, there will be completion messages at the end of 'emerge -e system' like this: | Code: |
... snipped for brevity ...
>>> package: net-nds/openldap-2.4.19-r1
* - /usr/lib/libldap-2.3.so.0
* - /usr/lib/libldap-2.3.so.0.2.31
* used by /usr/bin/ogg123 (media-sound/vorbis-tools-1.2.0-r2)
* used by /usr/bin/sudo (app-admin/sudo-1.7.0)
* used by /usr/bin/sudoedit (app-admin/sudo-1.7.0)
* used by /usr/lib/libnfsidmap_umich_ldap.so.0.0.0 (net-libs/libnfsidmap-0.21-r1)
* - /usr/lib/liblber-2.3.so.0
* - /usr/lib/liblber-2.3.so.0.2.31
* used by /usr/bin/ogg123 (media-sound/vorbis-tools-1.2.0-r2)
* used by /usr/bin/sudo (app-admin/sudo-1.7.0)
* used by /usr/bin/sudoedit (app-admin/sudo-1.7.0)
* used by /usr/lib/libnfsidmap_umich_ldap.so.0.0.0 (net-libs/libnfsidmap-0.21-r1)
Use emerge @preserved-rebuild to rebuild packages using these libraries |
Ignore these messages for now. Do not run "emerge @preserved-rebuild". Do not run revdep-rebuild. These processes should only be run after all the upgrade emerges have finished. This is true as a general rule. Some of the listed problems will be resolved when packages pulled in by 'emerge -uND world' complete. So, for now, not to worry. We'll worry about these at a much more appropriate time.
Chapter 7) - More Cleanups - General
At this point in the process, we now have a cleanly compiled base system with all needed configuration updates to emerge, upgrade and compile the rest of the world. World domination is at hand! Well ... not quite. Now we need to send out the hazmat crews and clean up the rest of the world first.
Now execute:
... wait for it ... | Code: | | # emerge -puNDv world |
There will most likely not be any packages listed for upgrading. Almost certainly, emerge will complain about packages for which no ebuilds exist, packages which can't be installed at the same time as other packages, packages masked as either removed or masked for removal soon, packages masked for licensing or masked for other reasons such as security issues.
This is going to be a re-iterative process where we will execute 'emerge -puNDv world'. Resolve whatever emerge complains about. Repeat.
Yes.
Rinse. Lather. Repeat. Rinse. Lather. Repeat.
This chapter will not be complete until emerge provides a listing without any complaints.
These are the 'rules of thumb' I follow regarding resolution of emerge complaints:- 'no ebuild exists' -- unmerge package
- 'packages can't be simultaneosly installed' -- determine older package and unmerge it.
- 'marked for removal' -- unmerge package.
- 'masked for ... ' -- read about reason and determine best course of action - usually unmerge package.
The whole topic of mitigating emerge complaints is fairly complex and the above suggested resolutions won't cover all instances. Unfortunately, such a discussion is complex enough to require a separate article with examples.
If you haven't yet started a list of packages to be manually re-installed later, get your note taking device(s) ready now. Personally, I still use pen(cil) and paper. At times, I've considered the blood of end users and a quill nib, but I've turned those times into massive exercises of self restraint.
There are some instances where it's required to remove a package in order to let emerge continue with it's thing, but you really need the package. In these cases, note the package down in your list of packages to be re-installed manually. You may also want to note down packages you need to replace. Taking notes is a good thing. Do it.
For our example laptop, it took more than 10 re-iterations of this process before 'emerge -puNDv world' was able to provide me with a list of packages to upgrade. And a few more re-iterations before the list provided stopped including complaints.
Chapter 8 ) - The Kernel Salutes!
At this point, I find it convenient to execute the following: | Code: | | # emerge -fuND world |
It's practical to pull down all the source packages which will be installed at one shot. If the target PC is a laptop as in my example, I have a functioning battery {good for about 2 hours of continuous emerging}. This means I can do goofy things like go to my local coffee shop with laptop in hand and allow emerging to continue taking place while sipping my fav beverage and gabbing with wannabe cronies.
Once all the needed packages have been downloaded, execute: | Code: | | # ls l /usr/portage/sys-kernel/vanilla-sources |
Inspect the listing and choose the kernel version you want to upgrade to: | Code: | total 356
-rw-r--r-- 1 root root 67545 Mar 15 16:37 ChangeLog
-rw-r--r-- 1 root root 27824 Mar 15 16:37 Manifest
-rw-r--r-- 1 root root 385 Aug 23 2006 metadata.xml
-rw-r--r-- 1 root root 509 Aug 25 2009 vanilla-sources-2.4.37.5.ebuild
-rw-r--r-- 1 root root 506 Jul 14 2009 vanilla-sources-2.4.37.ebuild
-rw-r--r-- 1 root root 508 Jul 14 2009 vanilla-sources-2.6.16.62.ebuild
... snipped for brevity ...
-rw-r--r-- 1 root root 546 Jan 7 12:42 vanilla-sources-2.6.31.9.ebuild
-rw-r--r-- 1 root root 544 Oct 5 11:36 vanilla-sources-2.6.31.ebuild
-rw-r--r-- 1 root root 546 Dec 14 15:42 vanilla-sources-2.6.32.1.ebuild
-rw-r--r-- 1 root root 547 Mar 15 16:16 vanilla-sources-2.6.32.10.ebuild
-rw-r--r-- 1 root root 546 Dec 19 14:20 vanilla-sources-2.6.32.2.ebuild
-rw-r--r-- 1 root root 546 Jan 7 12:42 vanilla-sources-2.6.32.3.ebuild
-rw-r--r-- 1 root root 547 Jan 18 16:30 vanilla-sources-2.6.32.4.ebuild
-rw-r--r-- 1 root root 546 Jan 23 06:50 vanilla-sources-2.6.32.5.ebuild
-rw-r--r-- 1 root root 546 Jan 25 14:21 vanilla-sources-2.6.32.6.ebuild
-rw-r--r-- 1 root root 546 Jan 29 02:15 vanilla-sources-2.6.32.7.ebuild
-rw-r--r-- 1 root root 546 Feb 9 14:23 vanilla-sources-2.6.32.8.ebuild
-rw-r--r-- 1 root root 546 Feb 24 14:18 vanilla-sources-2.6.32.9.ebuild
-rw-r--r-- 1 root root 544 Dec 3 04:13 vanilla-sources-2.6.32.ebuild
-rw-r--r-- 1 root root 546 Mar 15 16:16 vanilla-sources-2.6.33.1.ebuild
-rw-r--r-- 1 root root 545 Feb 24 15:23 vanilla-sources-2.6.33.ebuild
-rw-r--r-- 1 root root 548 Mar 13 13:12 vanilla-sources-2.6.34_rc1.ebuild |
What you're looking for is the last kernel prior to the newest release candidates. As per the example above, execute: | Code: | | # emerge =vanilla-sources-2.6.33.1 |
Since I usually use vanilla-source kernels, I choose vanilla-sources. Use the kernel sources and kernel build process you would normal use for the target PC. You want to perform the same type of service for the target PC as I show here. If you use gentoo-sources, then replace "vanilla" with "gentoo". If you use genkernel, then use that process instead. The goal is to upgrade to the last official release before getting into release candidate kernels. Unless you like bleeding all over your kernels. For those of you into such forays, by all means use the release candidate kernels. 
Update the kernel with the settings required for the target PC.
Installing the new, upgraded kernel is a separate topic. There are plenty of resources on the web to assist. Please consult them!
Once the new kernel is installed and grub.conf {or lilo} updated, reboot the target PC.
You want to at least be sure the new kernel works.
It's a good idea to be running the new, upgraded and properly installed kernel before proceding further. The rationale is a class of packages which build themselves against the currently running kernel. Therefore you need to have the kernel you're upgrading to as well as a properly formed /usr/src/linux/ symlink before these packages compile.
These packages, among others and not limited to them, require properly referenced kernel information:- ati-drivers
- ndiswrapper
- nvidia-drivers
- virtualbox-modules
- vmware-modules
pppoe related packages also fall in this group though their package names escape me at this time.
In general, packages involving proprietary drivers, virtual computing and network awareness {such as ppp and automatic network joining} will require access to the source code and configuration of the currently running kernel.
These packages either build against header files in the currently running kernel or they require specific kernel settings or both. There are no shortcuts. Install, configure and build your new kernel before continuing.
This ends Post II of Major System Upgrades.
As usual, suggestions for improvements can be entered here or you can PM me. I will be updating later with corrections for typos, grammar and to add several pertinent links as time permits.
Thank you and I hope you enjoyed the post! You may now continue to Post III - World Domination
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
 |
Znook
n00b
Joined: 09 Dec 2009
Posts: 3
|
| Posted: Tue Mar 30, 2010 12:32 pm Post subject: |
|
|
An excellent guide.
Just a query re kernels. I've just upgraded my box with a new kernel and the guide I followed said to do 'emerge gentoo-sources' which this gave me 2.6.31-gentoo-r10 as the highest version. A check on the Gentoo packages site also shows this to be the latest version.
However having read your guide and seeing your mention of going to /usr/portage/sys-kernel/vanilla-sources and seeing all those different kernels listed, I had a look on my box and sure enough it was there showing versions up to 2.6.33.1 You learn something everyday, thanks!
Following a look around I then discovered there's a 'gentoo-sources' directory showing 2.6.33 as its highest. This got me thinking - instead of my using 'emerge gentoo-sources' like I did before, I could at any time do an emerge --sync, find the latest non rc kernel listed in this directory and just issue 'emerge =gentoo-sources-2.6.[latest]' and continue with the guide's instructions and everything should still be okay?
Or am I missing something?
@Znook |
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Wed Mar 31, 2010 12:46 pm Post subject: |
|
|
| Znook wrote: | This got me thinking - instead of my using 'emerge gentoo-sources' like I did before, I could at any time do an emerge --sync, find the latest non rc kernel listed in this directory and just issue 'emerge =gentoo-sources-2.6.[latest]' and continue with the guide's instructions and everything should still be okay?
Or am I missing something?
@Znook |
You have it exactly right.
Kernels are one of the few areas where I consider portage to be a little weak. Portage always gives you the latest version of whatever is available with the caveat that people with stable {i.e. "x86" versus "~x86"} will not see release candidate kernels. If you're sensitive to mandatory coordination between 3rd party packages and kernel level, what you really want is the latest version of the kernel you're currently running under.
Personally, I want to start playing with the new btrfs file system. This means the latest vanilla kernels. I was very pleased when the respective ebuilds of the 3rd party packages I rely on were updated to deal with the 2.6.33/4 kernel series. {big thank you to the gentoo devs here!}
So yes, you understand precisely what is going on. 
And thank you very much for the kind words.
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
AchilleTalon
Guru
Joined: 11 Apr 2004
Posts: 368
Location: Montreal, Quebec, Canada
|
| Posted: Mon Apr 12, 2010 11:58 pm Post subject: |
|
|
Well, nice try, but the real problem when trying to upgrade a system, not
necessarily so old, since many changes took
place in the last few months, the real problems are
with dependencies and the fact the
emerge --pev system or whatever will not
succeed and you will be trapped with circular
dependencies. This is real pain in the ass and seems there is no receipe
to make a smooth update.
Gentoo is simply a load of shit to maintain and support, you never ever know
when a simple upgrade becomes a real nightmare.
_________________
Achille Talon Hop! |
|
| Back to top |
|
|
John R. Graham
Administrator
Joined: 08 Mar 2005
Posts: 10587
Location: Somewhere over Atlanta, Georgia
|
| Posted: Tue Apr 13, 2010 12:08 am Post subject: Re: HOWTO: Major System Upgrades - Post II - Initial Upgrade |
|
|
| dufeu wrote: | ...
For this reason, execute: |
If you're going to do that, it should be | Code: | | emerge -1 gcc glibc |
in order to not unnecessarily pollute the world set. It's also out-of-date information that you need to compile GCC twice: it does it itself each time.
- John
_________________
I can confirm that I have received between 0 and 499 National Security Letters. |
|
| Back to top |
|
|
dufeu
l33t
Joined: 30 Aug 2002
Posts: 924
Location: US-FL-EST
|
| Posted: Tue Apr 13, 2010 1:19 am Post subject: Re: HOWTO: Major System Upgrades - Post II - Initial Upgrade |
|
|
| John R. Graham wrote: | | dufeu wrote: | ...
For this reason, execute: |
If you're going to do that, it should be | Code: | | emerge -1 gcc glibc |
in order to not unnecessarily pollute the world set. It's also out-of-date information that you need to compile GCC twice: it does it itself each time.
- John |
I keep forgetting to add "1" on my emerge commands. You are correct about that.
I'll try to be clearer about my instructions. It's not that I want people to compile gcc twice. In fact, I explain that gcc is automatically compiled twice each time they see it happen and I explain why. What I'm uncomfortable about is that portage often will not move upgrades to gcc to be first in a group of upgrades. While this can be argued and, indeed, probably doesn't matter in most cases, if you're effectively re-compiling your entire system, I've yet to be convinced that it's OK it not have the most important part of the toolset compiled first.
In other words, I'm not telling people to compile gcc twice because of the old way of doing things. I'm telling them to compile gcc first before compiling anything else so that all the rest of their packages are compiled with a pristine and current gcc. I then point out each emerge of gcc is actually compiled twice and why. Then I explain that gcc will be compiled again (twice) in the course of emerge -e @system.
And, as you note that if you 'polute' your @world set, gcc will be compiled (twice) yet again during emerge -e @world. That makes a total of 6 times of which the last 4 are worthless.
It's my personal observation that systems where the packages are compiled in the same environment with the same set of USE flags and at the same gcc version result in the most stable systems. It's a very conservative approach but I firmly believe, especially when it comes to data integrity, being conservative works. The only harm is that the extra gcc compilations take time and cpu cycles.
Of course, YMMV and all that. There is something to be said for doing things the most efficient and elegant way possible. On the other hand, if you're recompiling virtually your entire system, then the extra pain of controlling the order of compilation has benefits too.
_________________
People whom think M$ is mediocre, don't know the half of it. |
|
| Back to top |
|
|
AchilleTalon
Guru
Joined: 11 Apr 2004
Posts: 368
Location: Montreal, Quebec, Canada
|
| Posted: Tue Apr 13, 2010 5:15 pm Post subject: |
|
|
And it is not to say the revdep-rebuild utility
is almost never able to figure-out the packages order when upgrading.
Serious work is required on the upgradability side of Gentoo.
I did switch to Gentoo a long time ago because specifically
upgrades where difficult and using latest stuff was something hard to do without
recompiling yourself packages from source.
_________________
Achille Talon Hop! |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
