| View previous topic :: View next topic |
| Author |
Message |
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
 Posted: Fri May 16, 2008 12:27 pm Post subject: World Update script running as a cron. |
 |
|
So I decided to write a script that can be run as a cron job that will do my world update every week and email me a report. I'm sure this has been done before but I couldn't find a nice simple bash script that did it and that did not require babysitting.
Feel free to tell me how lame my scripting skills are as this is the first bash script I've written with any meat to it.
Original Version:
| Code: |
#!/bin/bash
###################
#
# Script to bring Ziggy up to the latest and greatest
#
###################
#Ye Old log file.
log=log.bry
echo "Subject: World Update Report." > $log
#First we need to update our list.
emerge --sync
if [ $? -gt 0 ]; then
echo "Sync Failed, wierd." >> $log
#mail
sendmail -F someone@thismachine someone@somewhere < $log
exit
else
echo "Sync Successful, move along." >> $log
fi
#Now let's toss the list of updatables in there for emailage.
emerge -uDpv world >> $log
if [ $(grep -c blocking $log) -gt 0 ]; then
echo "There are package conflictions:" >> $log
grep -i blocking $log >> $log
#mail
sendmail -F someone@thismachine someone@somewhere < $log
exit
elif [ $(grep -c "0 packages" $log) -gt 0 ]; then
echo "There are no updates available" >> $log
#mail
sendmail -F someone@thismachine someone@somewhere < $log
exit
fi
emerge -uD world
if [ $? -gt 0 ]; then
echo "Emerge failed." >> $log
echo "" >> $log
echo "Emerge log:" >> $log
tail -20 /var/log/emerge.log >> $log
#mail
sendmail -F someone@thismachine someone@somewhere < $log
exit
else
echo "World update completed successfully." >> $log
date >> $log
fi
revlist=revlist.bry
revdep-rebuild -p > $revlist
while [ $(grep -c "Now you can remove" $revlist) -gt 0 ]
do
revdep-rebuild
if [ $? -gt 0 ]; then
echo "There was a problem with the rebuild" >> $log
echo "" >> $log
echo "Emerge.log:" >> $log
tail -20 /var/log/emerge.log >> $log
#mail
sendmail -F someone@thismachine someone@somewhere < $log
exit
fi
revdep-rebuild -p > $revlist
done
echo ""
echo "System completely updated, congratulations." >> $log
sendmail -F someone@thismachine someone@somewhere < $log
|
If you use this make sure to change someone@thismachine someone@somewhere to show what you want.
I have a cron job set up to run this script every Tuesday at 2:00 AM and it has been working great so far.
------------
edited to include updated script for comparison:
| Code: |
#!/bin/bash
###################
#
# Script to bring Ziggy up to the latest and greatest
#
###################
#Ye Old log file.
log=log.bry
worldtmp=worldtmp.bry
email="sendmail -F someplace@here someplace@somewhereelse"
blist=blist.bry
check=`find /etc -name "._cfg_*" -print|wc -l`
echo "Subject: World Update Report." > $log
if [ $1 == "--with-sync" || $2 == "--with-sync" ]; then
#First we need to update our list.
if emerge --sync; then
echo "Sync Successful, move along." >> $log
else
echo "Sync Failed, wierd." >> $log
#mail
$email < $log
exit
fi
else
echo "Skipping Sync..."
fi
#Now let's toss the list of updatables in there for emailage.
emerge -uDpv world >> $log
if grep -q blocking $log; then
echo "There are package conflictions:" >> $log
grep -i blocking $log >> $log
#mail
$email < $log
exit
elif grep -q "0 packages" $log; then
echo "There are no updates available" >> $log
#mail
$email < $log
exit
fi
#Watch out for kernel update attempt and remove if found.
if grep -q gentoo-sources $log; then
echo "" >> $log
echo "Found a kernel update trying to happen." >> $log
sed "/gentoo-sources/d" /var/lib/portage/world > $worldtmp
mv $worldtmp /var/lib/portage/world
echo "Removed kernel from world file, continuuing with emerge..." >> $log
echo "" >> $log
fi
if [ $1 == "--with-blist" || $2 == "--with-blist" ]; then
#Check world file for blacklisted items from predefined blacklist.
if grep -q -f $blist $log; then
echo "" >> $log
echo "There were updates available that you personally have blocked. Update cancelled. You should probably babysit this one." >> $log
echo "" >> $log
echo "Blacklisted update(s):" >> $log
grep -f $blist $log >> $log
$email < $log
exit
fi
fi
#Begin update.
if emerge -uD world; then
echo "World update completed successfully." >> $log
date >> $log
else
echo "World update completed successfully." >> $log
date >> $log
echo "Emerge failed." >> $log
echo "" >> $log
echo "Emerge log:" >> $log
tail -20 /var/log/emerge.log >> $log
#mail
$email < $log
exit
fi
#Rebuild until not necessary anymore.
revlist=revlist.bry
revdep-rebuild -p > $revlist
while grep -q "Now you can remove" $revlist; do
revdep-rebuild
if [ $? -gt 0 ]; then
echo "There was a problem with the rebuild" >> $log
echo "" >> $log
echo "Emerge.log:" >> $log
tail -20 /var/log/emerge.log >> $log
#mail
$email < $log
exit
fi
revdep-rebuild -p > $revlist
done
#Check whether or not configs need updating.
if [ $check -gt 0 ]; then
echo "There are configs that need updating." >> $log
echo "Be sure to log on and update them soon." >> $log
echo "" >> $log
else
echo "No configs need updating." >> $log
echo "" >> $log
echo ""
echo "System completely updated, congratulations." >> $log
#mail
$email < $log
|
Last edited by ziggysquatch on Tue May 27, 2008 6:57 pm; edited 4 times in total |
|
| Back to top |
|
 |
mamac
l33t
Joined: 29 Feb 2004
Posts: 890
|
| Posted: Fri May 16, 2008 1:38 pm Post subject: |
|
|
Hi,
I'd rather 'emerge -uDNv world' and after that 'emerge --depclean' or maybe at least add the output of 'emerge --depclean -p' in the log
That's a good idea but to my opinion that has to go together with a good management of /etc/portage/package.mask, .use and .keywords (i.e. it could update your gentoo-sources and change the link to /usr/src/linux without asking anything).
Good job though!
_________________
Powered by Gentoo Linux since 2003 |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Fri May 16, 2008 1:55 pm Post subject: |
|
|
Thanks for your input. I have had to manage my world file especially making sure that each time I update my kernel I remove it from the world file (although I have the build and symlink use flags shut off just in case).
That's a good idea to put the depclean in there I will have to add that. |
|
| Back to top |
|
|
timeBandit
Bodhisattva
Joined: 31 Dec 2004
Posts: 2719
Location: here, there or in transit
|
| Posted: Fri May 16, 2008 3:33 pm Post subject: Re: World Update script running as a cron. |
|
|
| ziggysquatch wrote: | | Feel free to tell me how lame my scripting skills are .... |
How about few friendly pointers instead? 
Many people (even experienced scripters sometimes) forget that you don't always need constructs like [ $? -ne 0 ] to test the return code of a process. The command can go right in the if statement. A zero exit code equates to true, nonzero to false. Examples in your script:
| Code: | if emerge --sync; then
echo "Sync Successful, move along." >> $log
else
echo "Sync Failed, wierd." >> $log
fi
...
if emerge -uD world >>$log; then
echo "World update completed successfully." >> $log
else
echo "Emerge failed." >> $log
fi |
The [ command is actually an external program so this can save many process forks in a large script. (I added the redirection to emerge just to show how it's done.)
Another trick: grep returns a zero status if it finds a match and its -q option suppresses all output. Thus:
| Code: | emerge -uDpv world >> $log
if grep -q blocking $log; then
echo "There are package conflicts:" >> $log
elif grep -q "0 packages" $log; then
echo "There are no updates available" >> $log
fi |
These techniques work with other conditional statements, too: | Code: | while grep -q "Now you can remove" $revlist; do
if ! revdep-rebuild; then
echo "There was a problem with the rebuild" >> $log
fi
revdep-rebuild -p > $revlist
done |
Finally, if you must do this, you might want to also mail the log to a machine other than the one you're updating. emerge world via cron is somewhat discouraged because sooner or later, it will break your system.
_________________
Plants are pithy, brooks tend to babble--I'm content to lie between them.
Super-short f.g.o checklist: Search first, strip comments, mark solved, help others. |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Fri May 16, 2008 4:01 pm Post subject: |
|
|
Thanks for the excellent pointers. I will be doing some code cleanup pretty soon.
I know my system will eventually break from this but at least it will email me right away (the email tip is good too but I don't have a separate machine for mail yet). Before I wrote this I had to bring this machine up to date and it hadn't had a world update/revdep-rebuild ever nor a reinstall and it has been running for 4 years so I've gotten accustomed to fixing the brokenness that may arrise.
I spent about 3 weeks cleaning up the mess and that's when I decided to write the script. My laptop on the other hand gets updated regularly because I use it more often than this machine. |
|
| Back to top |
|
|
AllenJB
Veteran
Joined: 02 Sep 2005
Posts: 1285
|
| Posted: Fri May 16, 2008 4:31 pm Post subject: |
|
|
TimeBandit already said it, but I'm going to say it again:
Running a world update as a cron job is a REALLY bad idea.
Things that can go wrong (off the top of my head, I'm sure there's more):
1) baselayout is updated, requiring some major config files to be updated and possibly other system critical files. For whatever reason, your system reboots... except it doesn't complete the reboot because the configuration is incorrect.
2) pam / shadow / some other login related package gets updated requiring configuration updates or perhaps other manual intervention for a major upgrade. In this case, for a remote system, you don't even need a reboot, all you need is for SSH to die for some reason. BAM! You can no longer log in to your system.
3) SSH / some other important service is upgraded on your system, requiring configuration updates to make the new version work. For whatever reason the server dies, but your server is set up to automatically restart it (using, for example, daemon tools, a cronjob or a simple inittab entry). BAM! The service no longer works as expected.
Also, when running world update cronjobs, you're going to have to ensure your backup script (you do run backups, right?) runs before the update. If it clashes, you could get a partially upgraded (and thus broken) backup. You could try to have it run after, but backing up a broken system (due to the above) is just plain silly. |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Fri May 16, 2008 4:44 pm Post subject: |
|
|
The backup does run before.
All of the things you mention are going to happen even if I am sitting right there doing the update myself. Also, all of that, including the baselayout problem (terrible thing) have happened to me before. That's the point of the log being mailed as soon as there is a problem or if there is a success. This way I can see what was updated and can know whether I should possibly check things out or attempt a reboot.
hmm. I should have it tell me if configs need updating. I will have to add that. |
|
| Back to top |
|
|
Section_8
l33t
Joined: 22 May 2004
Posts: 627
|
| Posted: Fri May 16, 2008 5:17 pm Post subject: |
|
|
Another thing to watch out for in automated updates like this: it emerges gentoo-sources with the symlink USE flag, then later emerges a package that installs a kernel module (such as nvidia-drivers). The kernel module builds with the wrong kernel version (the one that just emerged instead of the kernel you're actually running) and won't load.
I just have a simple weekly cron job the does emerge --sync and emerge --fetchonly -uDNv world. Like AllenJB, I won't actually emerge packages from a cron job. I also have -symlink on gentoo-sources. |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Fri May 16, 2008 5:22 pm Post subject: |
|
|
| Yeah, I've got -symlink and -build on gentoo-sources but I've been removing gentoo-sources from the world file. I think I'm going to add something that checks for gentoo-sources in the world file and either fails out or removes it for me if it finds it. |
|
| Back to top |
|
|
Section_8
l33t
Joined: 22 May 2004
Posts: 627
|
| Posted: Fri May 16, 2008 5:57 pm Post subject: |
|
|
| Another gotcha from automatic updates - a major enhancement to some package pulls in a truckload of dependencies you don't have installed. I like to see this coming and decide how to handle it. I'm a control freak with my system (which is why I have gentoo) and like to know what's getting installed. |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Tue May 20, 2008 7:04 pm Post subject: |
|
|
I posted the new updated script based on the suggestions by helpful people in this thread. I left the old one there for comparison so that the previous comments make sense and so that other people looking for script help might find an example that helps them.
I did add the if statement that looks for and, if it finds one, removes the gentoo-sources from the world file. Feel free to give me some new pointers because they are very helpful. You can even tell me how horrible I am for running this as a cron but I will still do it. It may be helpful for those who should think twice before setting up a cron that does this. |
|
| Back to top |
|
|
hielvc
Advocate
Joined: 19 Apr 2002
Posts: 2805
Location: Oceanside, Ca
|
|
| Back to top |
|
|
jcat
Veteran
Joined: 26 May 2006
Posts: 1337
|
| Posted: Thu May 22, 2008 7:01 am Post subject: |
|
|
While I agree that auto updates with portage can be troublesome, you could try and make your script a bit more bullet-proof to try avoid issues.
Add a check for some more keywords like "baselayout", "pam", "shadow" etc and abort the update if they are found in the list of updatable packages.
Cheers,
jcat |
|
| Back to top |
|
|
ziggysquatch
Apprentice
Joined: 16 Nov 2004
Posts: 172
Location: /USA/Minnesota
|
| Posted: Tue May 27, 2008 7:02 pm Post subject: |
|
|
Posted the new script with a black list option so you can have a file that lists packages that should stop an update from occurring (thanks jcat).
Also added the ability to use options to control whether a sync is required or if you want to look at the black list or not (mostly used while testing or if the update failed for some reason and you want/need to run it again).
I also fixed some painfully obvious flaws in the if statements... 
Thanks everyone for your comments. Let me know if you see anything else or if I can make it cooler. |
|
| Back to top |
|
|
|
Portage & Programming
