What's your update process?

[stonespheres]

Relatively new to Gentoo and I'm just settling into the system after a few early mishaps, due to an unpracticed sense of patience!

I've been scouring the forums for an answer, and while this has been answered before, I'd like to see some fresh opinions on the matter.
What's your current update/upgrade process? I'm not just talking about

Code: Select all

emerge --sync

or your preferred alternative. I'd like to know what you do during what I call my "Sync Sundays".

I generally

Code: Select all

emerge --sync 

once a day, with Sundays set aside for the following process:

1. Sync portage tree
-----------------------

Code: Select all

# eix-sync

2. Upgrade whole system
-----------------------

Code: Select all

# emerge --ask --verbose --update --deep --with-bdeps=y --newuse @world

3. Bring configuration file(s) up to date
-----------------------

Code: Select all

# dispatch-conf	

4. Uninstall unused packages
-----------------------

Code: Select all

# emerge --depclean -pv

5. Clean up source code of the old packages
-----------------------

Code: Select all

# eclean -d distfiles

After installations or updates
-----------------------

Code: Select all

perl-cleaner --all

So, what's your process?

[skiwarz]

Here's the script I use. It runs every night while I'm asleep, and I just check the logs once a week or so to see if there have been any errors. And do a dispatch-conf whenever I remember to do so.

Code: Select all

#!/bin/bash
emerge --sync
emerge -qvu gentoo-sources
cur="linux-$(uname -r)"
fut=$(ls -l /usr/src/linux | cut -d'>' -f 2 | cut -d' ' -f 2)
    if [ $cur != $fut ]
    then
        cd /usr/src/linux
        cp /usr/src/$cur/.config /usr/src/config_archive/$cur
        cp /usr/src/$cur/.config /usr/src/linux/.config
        make olddefconfig
        make -j8
        make modules_install
        cp arch/x86/boot/bzImage /boot/kernel-${fut:6}
        cp .config /boot/.config
        emerge --depclean gentoo-sources
        emerge -qv gentoo-sources
        /usr/bin/eclean-kernel -d -n 2
        /usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg
        emerge -qv @module-rebuild
    fi
emerge -qvuDU --keep-going --with-bdeps=y world
emerge --depclean
eclean -d distfiles
if [ $cur != $fut ]
then
        /sbin/shutdown -r now
fi

It checks the kernel for a new version first, and restarts the machine at the end if it was updated.
I have eclean-kernel installed to mitigate having a ton of leftover kernel sources on my drive. And I have the "symlink" use flag enabled so the kernel symlink updates itself.
I also recommend using the "--keep-going" option in emerge. It helps get things done faster.
Try using the "U" (--changed-use) option instead of "N" (--newuse) as it won't re-build a package just because they added a use flag that you don't have enabled.

If you automate gentoo, it is so much less time-consuming.

[JustAnother]

Nice script. Now I'm motivated to write (ahem, grab) my own.

Concerning this line vs. just copying .config to a new home:

Code: Select all

make olddefconfig

Is this really necessary, or is it ok just to copy the current .config file and change the date suffix for the kernel within menuconfig?
I've found that it always makes sense to have the build date as a suffix on the kernel file.

[figueroa]

I believe automatic updates are a bad idea.

The following script runs in root's crontab in the wee hours of the morning on my portage sync server:

Code: Select all

#!/bin/sh
emerge --sync
echo "emerge -uDU @world -p" > /home/USERNAME/bin/emergeauto.txt
emerge -uDU @world -p >> /home/USERNAME/bin/emergeauto.txt 2>&1
#Put this line in your user crontab.
#/home/USERNAME/bin/mailx.scr

My user crontab does the following about an hour later (mailx.scr script):

Code: Select all

#!/bin/sh
mail -s "bethel emerge output" USERNAME@domain.us < /home/USERNAME/bin/emergeauto.txt

In the morning sometime, I see the email that tells me what updates are available. The standard output of the "emerge --sync" is also emailed to me by default separately for possible troubleshooting and portage messages.

[skiwarz]

Nice script. Now I'm motivated to write (ahem, grab) my own.

Concerning this line vs. just copying .config to a new home:

Code: Select all

make olddefconfig

Is this really necessary, or is it ok just to copy the current .config file and change the date suffix for the kernel within menuconfig?
I've found that it always makes sense to have the build date as a suffix on the kernel file.

That line takes the old config (that you copied over) and applies any added changes that came with the new kernel (added features, new code, etc). So while you could just use the same config, eventually after many updated kernels, something could break due to not having a required dependency, the wrong driver, etc.

[skiwarz]

I believe automatic updates are a bad idea.

The following script runs in root's crontab in the wee hours of the morning on my portage sync server:

Code: Select all

#!/bin/sh
emerge --sync
echo "emerge -uDU @world -p" > /home/USERNAME/bin/emergeauto.txt
emerge -uDU @world -p >> /home/USERNAME/bin/emergeauto.txt 2>&1
#Put this line in your user crontab.
#/home/USERNAME/bin/mailx.scr

My user crontab does the following about an hour later (mailx.scr script):

Code: Select all

#!/bin/sh
mail -s "bethel emerge output" USERNAME@domain.us < /home/USERNAME/bin/emergeauto.txt

In the morning sometime, I see the email that tells me what updates are available. The standard output of the "emerge --sync" is also emailed to me by default separately for possible troubleshooting and portage messages.

You're right, there are definitely disadvantages to automatic updates. I do run into issues sometimes, where things break right after an update and I need to go back and investigate. But for me this is far preferable to manually updating each day, or once a week and having to spend the time doing it.

[pietinger]

I believe automatic updates are a bad idea.

+1

I am doing this: viewtopic-p-8609355.html#8609355

[Goverp]

Better to do eclean BEFORE the update; that way you have stuff left to rebuild from if the update fails.

[stonespheres]

You're right, there are definitely disadvantages to automatic updates.

There's definitely advantages to both automatic and manual updates. To each, his own I say. Somethings bound to break sooner or later, it's just a matter of how we get there

[figueroa]

You're right, there are definitely disadvantages to automatic updates. I do run into issues sometimes, where things break right after an update and I need to go back and investigate. But for me this is far preferable to manually updating each day, or once a week and having to spend the time doing it.

Muscle memory takes over and updates take almost no time at all. I DON'T WATCH the compiling process. I move on to other things and attend to the update window eventually. It's a computer and it's happy to wait all day, or forever.

Even wise Debian (event Mint) users don't update automatically.

[Fitzcarraldo]

Below is the general procedure I use for system maintenance of my Gentoo installations, which I perform approximately weekly.

1. Update the ebuilds on the machine (see Gentoo Wiki – Project:Portage/Sync)

Code: Select all

# emaint sync -a

If I were using the deprecated Portage sync method I would instead have used the following commands:

Code: Select all

# emerge --sync # Update the ebuilds from the main Portage tree
# layman -S # Update the ebuilds from 3rd-party overlays

2. Upgrade the Portage package manager if the console output from Step 1 included a message telling me to upgrade portage

Code: Select all

# emerge -1v portage

3. As I use the eix and the mlocate utilities, I update their data files

Code: Select all

# eix-update && updatedb

4. Check if there are any News items I have not read yet

Code: Select all

# eselect news list

5. Read new News items and make necessary changes, if any

Code: Select all

# eselect news read <n>

6. Perform a dry run for the upgrade of any packages in the World file that have new versions

Code: Select all

# emerge -uvpDN --with-bdeps=y @world

(I could use the option 'U' instead of 'N' but I usually stick with 'N'.)

7. If no problems were flagged in Step 6, go to Step 9

8. Sort out any problem(s) flagged in Step 6 then go back to Step 6

9. Launch the upgrade of those packages in the World file that have new versions

Code: Select all

# emerge -uvDN --with-bdeps=y --keep-going @world

My decision on whether or not to include the option ‘--keep-going‘ will depend on the precise circumstances.

10. If Step 9 ran to completion successfully, go to Step 14

11. If Step 9 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go to Step 12, otherwise fix the problem(s)* and go back to Step 9

*Sometimes I find that one or more packages do not merge successfully during Step 9 but do merge successfully simply by repeating Step 9.

12. Resume the upgrade process

Code: Select all

# emerge --resume --skipfirst

13. If Step 12 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go back to Step 12, otherwise fix the problem(s) and go back to Step 9

14. Upgrade any packages that are still built against old versions of libraries if the console output from Step 9 or Step 12 includes a message telling me to do that

Code: Select all

# emerge @preserved-rebuild

15. If any problems remain, fix them and go back to Step 14

16. Scan libraries and binaries for missing shared library dependencies and re-merge any broken binaries and shared libraries

Code: Select all

# revdep-rebuild -i

Actually, I cannot recall the last time ‘revdep-rebuild‘ was needed, as Portage has improved so much over the years.

17. Remove outdated and unneeded packages

Code: Select all

# emerge --ask --depclean

18. Merge any configuration files

Code: Select all

# etc-update

I always check the differences between the listed existing and new configuration files before going ahead, and may edit the new configuration file if I deem it necessary.

19. As I use the mlocate utility I make sure its index file is bang up to date

Code: Select all

# updatedb

20. Optionally, I clear out any old source-code and binary packages

Code: Select all

# eclean-dist --deep

21. If I remember to do it, I check if there are any installed obsolete packages and then remove them

Code: Select all

# eix-test-obsolete

22. I make sure no temporary work files have been left around by any failed merges

Code: Select all

# rm -rf /usr/tmp/portage/*

Actually, I created a script in directory /etc/local.d/ to do this automatically when HDD free space gets low.

23. I wait for at least 24 hours (usually about a week) and then go to Step 1

Notes:

Actually, I have added the option ‘--with-bdeps=y‘ to EMERGE_DEFAULT_OPS in the file /etc/portage/make.conf so I do not need to type that option every time.

One of my laptops has a first-generation Intel Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine.

In order to prevent the number of log files increasing indefinitely, I added ‘clean-logs‘ to FEATURES in the file /etc/portage/make.conf so that log files older than seven days are deleted automatically.

[Goverp]

...
One of my laptops has a first-generation Intel Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine. ...

I find using the BFQ IO scheduler much improves interactive response when portage is busy compared with MQ-Deadline. But if I'm updating unattended, mq-deadline runs a little faster (these are subjective anecdotal timings...).

As I've lots of cores, have j=n/2, --jobs=n/3 and --loadaverage=n but I've not tried seriously evaluating the effects of twiddling these knobs. I've neither run out of memory nor locked up recently.

[Goverp]

My update process:

Code: Select all

# As root
pre-backup.sh     # Runs eclean-dist --deep and eclean-pkg --deep, and sets mq-deadline IO Scheduler
backup.sh <n>     # Runs dump with some sexy profiles.
backuprootfs.sh   # My root is on f2fs, which dump won't handle, so this creates a tarball.
emerge --sync
emerge --update --deep --changed-use @world     # --ask and --buildpkg amongst others in my make.conf
<sort out whatever portage wants to moan about this week>
emerge @preserved-rebuild     # If necessary
emerge --depclean
elogviewer    # Delete all the logs no higher than Info level, read and if necessary act upon the rest before deleting.

then if there's a kernel update, see (my wiki page on Kernel/Building from userspace)

Code: Select all

# as user
cd kernel-desktop
kmake source     # I have USE=symlink on gentoo-sources
kmake clean
kmake oldconfig
kmake -j25
su -
make modules_install
make install
# Repeat for kernel-laptop, building a kernel tarball into /var/cache/binpkgs

and finally on the laptop, as root

Code: Select all

emerge --sync    # Uses desktop as local server.
mount /var/cache/distfiles
mount /var/cache/binpkgs
emerge --update --deep --changed-use @world     # I have --usepkg in make.conf
emerge @preserved-rebuild
emerge --depclean
elogviewer
cd /
mount /boot
tar -xf /var/cache/binpkgs/<latest kernel>

Actually, I don't "emerge --sync", I sync the portage squashfs snapshot, which is much, much faster and consumes much less filesystem resources.

[stonespheres]

My update process:

Code: Select all

# as user
emerge @preserved-rebuild

I see a few mentions of using @preserved-rebuild here, is there any particular circumstance that requires its use, or is it something I should be running regularly?

[skiwarz]

My update process:

Code: Select all

# as user
emerge @preserved-rebuild

I see a few mentions of using @preserved-rebuild here, is there any particular circumstance that requires its use, or is it something I should be running regularly?

After you finish emerging something, occasionally you will see a message printed that says "existing preserved libs found". It means you upgraded a library, but some packages are still referencing old versions of it so they can't be deleted. Emerging @preserved-rebuild forces those packages to be rebuilt so the old library can finally be deleted. Pretty sure it's exactly the same as revdep-rebuild.
See https://wiki.gentoo.org/wiki/Preserved-rebuild

[figueroa]

revdep-rebuild is not exactly the same as @preserved-rebuild. @preserved-rebuild is needed from time-to-time and as a feature for portage lets the user decide when to rebuild packages that depend on a library that has been updated. This helps keep portage from borking a program you are currently working with during the update.

revdep-rebuild, not needed much these days because of the maturity of portage, is used to fix upgrades that have already broken something. I probably haven't needed to run revdep-rebuild for a year or so. I have done it on a trial basis with --pretend, but it hasn't found anything out of sorts.

[Goverp]

My update process:

Code: Select all

# as user
emerge @preserved-rebuild

...

Not sure how that got edited to show "'# as user"; it has to run as root.

[plp]

Sorry to rehash an old topic, but is this the most reliable way to keep system updated?

1: Sync your tree

Code: Select all

# emerge —sync

2: Perform a full world upgrade

Code: Select all

# emerge -a  -uvDU @world

3: once complete, depclean

Code: Select all

# emerge -a —depclean 

This morning during step two it complain about an awk conflict and stopped. How do y’all manage these issues?
Currently at work so I can’t post the relevant info.

[logrusx]

How do y’all manage these issues?

Open a new thread in Portage & Programming for your particular issue, resolve it and continue as usual.

Best Regards,
Georgi

[figueroa]

Sorry to rehash an old topic, but is this the most reliable way to keep system updated? ...

Yes it is. See my posts earlier in this thread for exactly what I do.