| View previous topic :: View next topic |
| Author |
Message |
stonespheres
n00b
Joined: 16 Apr 2021
Posts: 16
Location: Australia
|
 Posted: Sun May 23, 2021 2:39 am Post subject: What's your update process? |
 |
|
Relatively new to Gentoo and I'm just settling into the system after a few early mishaps, due to an unpracticed sense of patience!
I've been scouring the forums for an answer, and while this has been answered before, I'd like to see some fresh opinions on the matter.
What's your current update/upgrade process? I'm not just talking about or your preferred alternative. I'd like to know what you do during what I call my "Sync Sundays".
I generally once a day, with Sundays set aside for the following process:
1. Sync portage tree
-----------------------
2. Upgrade whole system
-----------------------
| Code: | | # emerge --ask --verbose --update --deep --with-bdeps=y --newuse @world |
3. Bring configuration file(s) up to date
-----------------------
4. Uninstall unused packages
-----------------------
| Code: | | # emerge --depclean -pv |
5. Clean up source code of the old packages
-----------------------
| Code: | | # eclean -d distfiles |
After installations or updates
-----------------------
So, what's your process?
_________________
Every emerge is an opportunity. |
|
| Back to top |
|
 |
skiwarz
Apprentice
Joined: 23 Feb 2014
Posts: 263
|
| Posted: Sun May 23, 2021 3:37 am Post subject: |
|
|
Here's the script I use. It runs every night while I'm asleep, and I just check the logs once a week or so to see if there have been any errors. And do a dispatch-conf whenever I remember to do so.
| Code: | #!/bin/bash
emerge --sync
emerge -qvu gentoo-sources
cur="linux-$(uname -r)"
fut=$(ls -l /usr/src/linux | cut -d'>' -f 2 | cut -d' ' -f 2)
if [ $cur != $fut ]
then
cd /usr/src/linux
cp /usr/src/$cur/.config /usr/src/config_archive/$cur
cp /usr/src/$cur/.config /usr/src/linux/.config
make olddefconfig
make -j8
make modules_install
cp arch/x86/boot/bzImage /boot/kernel-${fut:6}
cp .config /boot/.config
emerge --depclean gentoo-sources
emerge -qv gentoo-sources
/usr/bin/eclean-kernel -d -n 2
/usr/sbin/grub-mkconfig -o /boot/grub/grub.cfg
emerge -qv @module-rebuild
fi
emerge -qvuDU --keep-going --with-bdeps=y world
emerge --depclean
eclean -d distfiles
if [ $cur != $fut ]
then
/sbin/shutdown -r now
fi
|
It checks the kernel for a new version first, and restarts the machine at the end if it was updated.
I have eclean-kernel installed to mitigate having a ton of leftover kernel sources on my drive. And I have the "symlink" use flag enabled so the kernel symlink updates itself.
I also recommend using the "--keep-going" option in emerge. It helps get things done faster.
Try using the "U" (--changed-use) option instead of "N" (--newuse) as it won't re-build a package just because they added a use flag that you don't have enabled.
If you automate gentoo, it is so much less time-consuming. |
|
| Back to top |
|
|
JustAnother
Apprentice
Joined: 23 Sep 2016
Posts: 186
|
| Posted: Sun May 23, 2021 3:54 am Post subject: |
|
|
Nice script. Now I'm motivated to write (ahem, grab) my own.
Concerning this line vs. just copying .config to a new home:
Is this really necessary, or is it ok just to copy the current .config file and change the date suffix for the kernel within menuconfig?
I've found that it always makes sense to have the build date as a suffix on the kernel file. |
|
| Back to top |
|
|
figueroa
Advocate
Joined: 14 Aug 2005
Posts: 2963
Location: Edge of marsh USA
|
| Posted: Sun May 23, 2021 4:01 am Post subject: |
|
|
I believe automatic updates are a bad idea.
The following script runs in root's crontab in the wee hours of the morning on my portage sync server:
| Code: | #!/bin/sh
emerge --sync
echo "emerge -uDU @world -p" > /home/USERNAME/bin/emergeauto.txt
emerge -uDU @world -p >> /home/USERNAME/bin/emergeauto.txt 2>&1
#Put this line in your user crontab.
#/home/USERNAME/bin/mailx.scr |
My user crontab does the following about an hour later (mailx.scr script):
| Code: | #!/bin/sh
mail -s "bethel emerge output" USERNAME@domain.us < /home/USERNAME/bin/emergeauto.txt |
In the morning sometime, I see the email that tells me what updates are available. The standard output of the "emerge --sync" is also emailed to me by default separately for possible troubleshooting and portage messages.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
| Back to top |
|
|
skiwarz
Apprentice
Joined: 23 Feb 2014
Posts: 263
|
| Posted: Sun May 23, 2021 7:33 am Post subject: |
|
|
| JustAnother wrote: | Nice script. Now I'm motivated to write (ahem, grab) my own.
Concerning this line vs. just copying .config to a new home:
Is this really necessary, or is it ok just to copy the current .config file and change the date suffix for the kernel within menuconfig?
I've found that it always makes sense to have the build date as a suffix on the kernel file. |
That line takes the old config (that you copied over) and applies any added changes that came with the new kernel (added features, new code, etc). So while you could just use the same config, eventually after many updated kernels, something could break due to not having a required dependency, the wrong driver, etc. |
|
| Back to top |
|
|
skiwarz
Apprentice
Joined: 23 Feb 2014
Posts: 263
|
| Posted: Sun May 23, 2021 7:36 am Post subject: |
|
|
| figueroa wrote: | I believe automatic updates are a bad idea.
The following script runs in root's crontab in the wee hours of the morning on my portage sync server:
| Code: | #!/bin/sh
emerge --sync
echo "emerge -uDU @world -p" > /home/USERNAME/bin/emergeauto.txt
emerge -uDU @world -p >> /home/USERNAME/bin/emergeauto.txt 2>&1
#Put this line in your user crontab.
#/home/USERNAME/bin/mailx.scr |
My user crontab does the following about an hour later (mailx.scr script):
| Code: | #!/bin/sh
mail -s "bethel emerge output" USERNAME@domain.us < /home/USERNAME/bin/emergeauto.txt |
In the morning sometime, I see the email that tells me what updates are available. The standard output of the "emerge --sync" is also emailed to me by default separately for possible troubleshooting and portage messages. |
You're right, there are definitely disadvantages to automatic updates. I do run into issues sometimes, where things break right after an update and I need to go back and investigate. But for me this is far preferable to manually updating each day, or once a week and having to spend the time doing it. |
|
| Back to top |
|
|
pietinger
Moderator
Joined: 17 Oct 2006
Posts: 4169
Location: Bavaria
|
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2011
|
| Posted: Sun May 23, 2021 10:53 am Post subject: |
|
|
Better to do eclean BEFORE the update; that way you have stuff left to rebuild from if the update fails.
_________________
Greybeard |
|
| Back to top |
|
|
stonespheres
n00b
Joined: 16 Apr 2021
Posts: 16
Location: Australia
|
| Posted: Sun May 23, 2021 1:40 pm Post subject: |
|
|
| skiwarz wrote: |
You're right, there are definitely disadvantages to automatic updates. |
There's definitely advantages to both automatic and manual updates. To each, his own I say. Somethings bound to break sooner or later, it's just a matter of how we get there 
_________________
Every emerge is an opportunity. |
|
| Back to top |
|
|
figueroa
Advocate
Joined: 14 Aug 2005
Posts: 2963
Location: Edge of marsh USA
|
| Posted: Sun May 23, 2021 5:30 pm Post subject: |
|
|
| skiwarz wrote: | | You're right, there are definitely disadvantages to automatic updates. I do run into issues sometimes, where things break right after an update and I need to go back and investigate. But for me this is far preferable to manually updating each day, or once a week and having to spend the time doing it. |
Muscle memory takes over and updates take almost no time at all. I DON'T WATCH the compiling process. I move on to other things and attend to the update window eventually. It's a computer and it's happy to wait all day, or forever.
Even wise Debian (event Mint) users don't update automatically.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi
Last edited by figueroa on Fri Nov 12, 2021 9:53 pm; edited 1 time in total |
|
| Back to top |
|
|
Fitzcarraldo
Advocate
Joined: 30 Aug 2008
Posts: 2034
Location: United Kingdom
|
| Posted: Mon May 24, 2021 3:47 pm Post subject: |
|
|
Below is the general procedure I use for system maintenance of my Gentoo installations, which I perform approximately weekly.
1. Update the ebuilds on the machine (see Gentoo Wiki – Project:Portage/Sync)
If I were using the deprecated Portage sync method I would instead have used the following commands:
| Code: | # emerge --sync # Update the ebuilds from the main Portage tree
# layman -S # Update the ebuilds from 3rd-party overlays |
2. Upgrade the Portage package manager if the console output from Step 1 included a message telling me to upgrade portage
| Code: | | # emerge -1v portage |
3. As I use the eix and the mlocate utilities, I update their data files
| Code: | | # eix-update && updatedb |
4. Check if there are any News items I have not read yet
| Code: | | # eselect news list |
5. Read new News items and make necessary changes, if any
| Code: | | # eselect news read <n> |
6. Perform a dry run for the upgrade of any packages in the World file that have new versions
| Code: | | # emerge -uvpDN --with-bdeps=y @world |
(I could use the option 'U' instead of 'N' but I usually stick with 'N'.)
7. If no problems were flagged in Step 6, go to Step 9
8. Sort out any problem(s) flagged in Step 6 then go back to Step 6
9. Launch the upgrade of those packages in the World file that have new versions
| Code: | | # emerge -uvDN --with-bdeps=y --keep-going @world |
My decision on whether or not to include the option ‘--keep-going‘ will depend on the precise circumstances.
10. If Step 9 ran to completion successfully, go to Step 14
11. If Step 9 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go to Step 12, otherwise fix the problem(s)* and go back to Step 9
*Sometimes I find that one or more packages do not merge successfully during Step 9 but do merge successfully simply by repeating Step 9.
12. Resume the upgrade process
| Code: | | # emerge --resume --skipfirst |
13. If Step 12 did not run to completion successfully and it appears the package that failed to merge will not cause further problems, go back to Step 12, otherwise fix the problem(s) and go back to Step 9
14. Upgrade any packages that are still built against old versions of libraries if the console output from Step 9 or Step 12 includes a message telling me to do that
| Code: | | # emerge @preserved-rebuild |
15. If any problems remain, fix them and go back to Step 14
16. Scan libraries and binaries for missing shared library dependencies and re-merge any broken binaries and shared libraries
| Code: | | # revdep-rebuild -i |
Actually, I cannot recall the last time ‘revdep-rebuild‘ was needed, as Portage has improved so much over the years.
17. Remove outdated and unneeded packages
| Code: | | # emerge --ask --depclean |
18. Merge any configuration files
I always check the differences between the listed existing and new configuration files before going ahead, and may edit the new configuration file if I deem it necessary.
19. As I use the mlocate utility I make sure its index file is bang up to date
20. Optionally, I clear out any old source-code and binary packages
| Code: | | # eclean-dist --deep |
21. If I remember to do it, I check if there are any installed obsolete packages and then remove them
| Code: | | # eix-test-obsolete |
22. I make sure no temporary work files have been left around by any failed merges
| Code: | | # rm -rf /usr/tmp/portage/* |
Actually, I created a script in directory /etc/local.d/ to do this automatically when HDD free space gets low.
23. I wait for at least 24 hours (usually about a week) and then go to Step 1
Notes:
Actually, I have added the option ‘--with-bdeps=y‘ to EMERGE_DEFAULT_OPS in the file /etc/portage/make.conf so I do not need to type that option every time.
One of my laptops has a first-generation Intel Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine.
In order to prevent the number of log files increasing indefinitely, I added ‘clean-logs‘ to FEATURES in the file /etc/portage/make.conf so that log files older than seven days are deleted automatically.
_________________
Clevo W230SS: amd64, VIDEO_CARDS="intel modesetting nvidia".
Compal NBLB2: ~amd64, xf86-video-ati. Dual boot Win 7 Pro 64-bit.
OpenRC udev elogind & KDE on both.
Fitzcarraldo's blog |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2011
|
| Posted: Mon May 24, 2021 4:52 pm Post subject: |
|
|
| Fitzcarraldo wrote: | ...
One of my laptops has a first-generation Intel Core i7 CPU and I initially added ‘--jobs=8 --load-average=8‘ to EMERGE_DEFAULT_OPTS in order to merge packages in parallel, which speeds up upgrading. However, I found this slowed interactive use of the machine and therefore I changed the options to ‘--jobs=6 --load-average=6‘ which works a bit better on that machine. ... |
I find using the BFQ IO scheduler much improves interactive response when portage is busy compared with MQ-Deadline. But if I'm updating unattended, mq-deadline runs a little faster (these are subjective anecdotal timings...).
As I've lots of cores, have j=n/2, --jobs=n/3 and --loadaverage=n but I've not tried seriously evaluating the effects of twiddling these knobs. I've neither run out of memory nor locked up recently.
_________________
Greybeard |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2011
|
| Posted: Mon May 24, 2021 5:22 pm Post subject: |
|
|
My update process:
| Code: | # As root
pre-backup.sh # Runs eclean-dist --deep and eclean-pkg --deep, and sets mq-deadline IO Scheduler
backup.sh <n> # Runs dump with some sexy profiles.
backuprootfs.sh # My root is on f2fs, which dump won't handle, so this creates a tarball.
emerge --sync
emerge --update --deep --changed-use @world # --ask and --buildpkg amongst others in my make.conf
<sort out whatever portage wants to moan about this week>
emerge @preserved-rebuild # If necessary
emerge --depclean
elogviewer # Delete all the logs no higher than Info level, read and if necessary act upon the rest before deleting. |
then if there's a kernel update, see (my wiki page on Kernel/Building from userspace)
| Code: | # as user
cd kernel-desktop
kmake source # I have USE=symlink on gentoo-sources
kmake clean
kmake oldconfig
kmake -j25
su -
make modules_install
make install
# Repeat for kernel-laptop, building a kernel tarball into /var/cache/binpkgs |
and finally on the laptop, as root
| Code: | emerge --sync # Uses desktop as local server.
mount /var/cache/distfiles
mount /var/cache/binpkgs
emerge --update --deep --changed-use @world # I have --usepkg in make.conf
emerge @preserved-rebuild
emerge --depclean
elogviewer
cd /
mount /boot
tar -xf /var/cache/binpkgs/<latest kernel> |
Actually, I don't "emerge --sync", I sync the portage squashfs snapshot, which is much, much faster and consumes much less filesystem resources.
_________________
Greybeard |
|
| Back to top |
|
|
stonespheres
n00b
Joined: 16 Apr 2021
Posts: 16
Location: Australia
|
| Posted: Tue May 25, 2021 10:26 pm Post subject: |
|
|
| Goverp wrote: | My update process:
| Code: | # as user
emerge @preserved-rebuild
|
|
I see a few mentions of using @preserved-rebuild here, is there any particular circumstance that requires its use, or is it something I should be running regularly?
_________________
Every emerge is an opportunity. |
|
| Back to top |
|
|
skiwarz
Apprentice
Joined: 23 Feb 2014
Posts: 263
|
| Posted: Tue May 25, 2021 10:57 pm Post subject: |
|
|
| stonespheres wrote: | | Goverp wrote: | My update process:
| Code: | # as user
emerge @preserved-rebuild
|
|
I see a few mentions of using @preserved-rebuild here, is there any particular circumstance that requires its use, or is it something I should be running regularly? |
After you finish emerging something, occasionally you will see a message printed that says "existing preserved libs found". It means you upgraded a library, but some packages are still referencing old versions of it so they can't be deleted. Emerging @preserved-rebuild forces those packages to be rebuilt so the old library can finally be deleted. Pretty sure it's exactly the same as revdep-rebuild.
See https://wiki.gentoo.org/wiki/Preserved-rebuild |
|
| Back to top |
|
|
figueroa
Advocate
Joined: 14 Aug 2005
Posts: 2963
Location: Edge of marsh USA
|
| Posted: Wed May 26, 2021 4:30 am Post subject: |
|
|
revdep-rebuild is not exactly the same as @preserved-rebuild. @preserved-rebuild is needed from time-to-time and as a feature for portage lets the user decide when to rebuild packages that depend on a library that has been updated. This helps keep portage from borking a program you are currently working with during the update.
revdep-rebuild, not needed much these days because of the maturity of portage, is used to fix upgrades that have already broken something. I probably haven't needed to run revdep-rebuild for a year or so. I have done it on a trial basis with --pretend, but it hasn't found anything out of sorts.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
| Back to top |
|
|
Goverp
Advocate
Joined: 07 Mar 2007
Posts: 2011
|
| Posted: Wed May 26, 2021 10:06 am Post subject: |
|
|
| stonespheres wrote: | | Goverp wrote: | My update process:
| Code: | # as user
emerge @preserved-rebuild
|
|
... |
Not sure how that got edited to show "'# as user"; it has to run as root.
_________________
Greybeard |
|
| Back to top |
|
|
plp
n00b
Joined: 07 Dec 2021
Posts: 14
Location: Little Rock, AR, USA
|
| Posted: Mon Apr 15, 2024 12:52 pm Post subject: |
|
|
Sorry to rehash an old topic, but is this the most reliable way to keep system updated?
1: Sync your tree
2: Perform a full world upgrade
| Code: | | # emerge -a -uvDU @world |
3: once complete, depclean
| Code: | | # emerge -a —depclean |
This morning during step two it complain about an awk conflict and stopped. How do y’all manage these issues?
Currently at work so I can’t post the relevant info. |
|
| Back to top |
|
|
logrusx
Veteran
Joined: 22 Feb 2018
Posts: 1550
|
| Posted: Mon Apr 15, 2024 3:40 pm Post subject: |
|
|
| plp wrote: | How do y’all manage these issues?
|
Open a new thread in Portage & Programming for your particular issue, resolve it and continue as usual.
Best Regards,
Georgi |
|
| Back to top |
|
|
figueroa
Advocate
Joined: 14 Aug 2005
Posts: 2963
Location: Edge of marsh USA
|
| Posted: Mon Apr 15, 2024 4:24 pm Post subject: |
|
|
| plp wrote: | Sorry to rehash an old topic, but is this the most reliable way to keep system updated? ...
|
Yes it is. See my posts earlier in this thread for exactly what I do.
_________________
Andy Figueroa
hp pavilion hpe h8-1260t/2AB5; spinning rust x3
i7-2600 @ 3.40GHz; 16 gb; Radeon HD 7570
amd64/23.0/split-usr/desktop (stable), OpenRC, -systemd -pulseaudio -uefi |
|
| Back to top |
|
|
|
Portage & Programming
