View previous topic :: View next topic |
Author |
Message |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Sun Sep 10, 2006 11:51 pm Post subject: [HOWTO] Hassle-free emerge -uD world (update-world v1.8) |
|
|
New Version 1.8 (2/16/07)
I've added a lot of new features to the newest version of update-world. It seems pretty stable (to me). Let me know if there are any problems.
Changelog v1.8:
-Cleaned up and modularized code. It should be a lot more readable now.
-Script now sources /etc/make.conf. Several portage variables are used by this script now.
-Added ability to 'emerge -p <package> | update-world. (realize that this is really only meant for updating existing packages...if you're installing a new one, you probably can't just skip a package failure, as it's likely a required dependency)
-Added interface with portage 2.1's elog capability. You can type 'update-world --log' to view build messages after updating (mesages stored in ~/update-world/<date>/notice/. Check out http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4. if you haven't set up elog yet.
-Added interface with revdep-rebuild. You can run it alone (update-world --revdep) to prepare a list of packages to install from revdep rebuild, or you can use it in conjunction with --install (update-world --revdep --install) to automatically run revdep-rebuild when the update is complete.
About update-world
If your experience is anything like mine, the process of updating a Gentoo system is no small endeavour. Typing 'emerge -uD world' is an almost guaranteed day or two periodically checking on the process to make sure it hasn't died because of a compile failure. I recently updated my system (680 packages !!!), which took literally 3 days because I would find that after I started the process and went to bed/work/etc., it would die within a few hours of starting it. 10 packages wouldn't compile properly, causing me to have to restart the emerge process 10 times. This is obviously not very efficient.
This nightmarish update process led me to come up with something that makes things easier. "update-world" is a bash script that controls the update process and forces portage to skip past failed builds and compile until the update is finished. Failed builds are added to a list for review once the update is finished.
The script does depend on the 'equery' command, part of app-portage/gentoolkit.
Here's how to do it:
First, make the script executable and put it somewhere in your $PATH.
Then, run the script.
Code: | update-world --prepare
or
update-world --empty
or
emerge <package> | update-world |
This begins the update process by creating a list of packages to be emerged, as produced by 'emerge -uD world'. (Alternatively, you can use 'update-world --empty' to create a list of all packages on the system, akin to 'emerge -e world'). This command creates a list called 'emergelist' in ~/.update-world/. Now, you can use your favorite editor to modify this list how you like--remove packages, change versions, etc. When you have the emergelist how you like it, you're ready to move onto the next step:
Code: | update-world --install |
This command actually starts the build process, just as if you had typed emerge -uD world. The difference is, if a package fails for some reason or another, that package is added to a list 'failedlist' in ~/.update-world/. Also, a link to the portage log file for that package (usually located in /var/log/portage) is created in ~/.update-world/<date>/log/. Once the process is done, you'll be presented with a list of failed packages to take care of all at once, saving much time.
Now you can automatically run revdep-rebuild whenever you update with update-world. You can use it alone or in conjunction with --install.
Code: | update-world --revdep
or
update-world --revdep --install |
Take advantage of portage 2.1's elog feature by running 'update-world --log' after an update to get those important messages that whiz by when doing a large update.
Look at http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4. for more information on setting this up. These notices are stored in ~/.update-world/notice.
If you ever need to stop the process, just hit 'Control-C' to abort the process. The update can then be restarted at any time, just where it left off, by retyping update-world --install.
I've only tested this on my machine, but it worked flawlessly, taking only 16 hours to compile all 680 packages. And the ten packages that wouldn't compile were waiting for me to fix at my leisure.
So, you can download the latest stable version of the script here: http://dixieflatline.homelinux.org/files/gentoo/update-world
Mirror 1: http://dixieflatline.homelinux.org/files/gentoo/
Mirror 2 (thanks ryker!): http://www.mindlesstechie.net/gentoo/update-world/
Featured in Gentoo Weekly Newsletter 12/04/06
http://www.gentoo.org/news/en/gwn/20061204-newsletter.xml
Here's the code:
Code: |
#!/bin/bash
# update-world 1.8
# count_zero 2007
# Licensed under the GPLv2
#
# update-world is a bash script that forces an emerge -uD world to continue, even
# in the face of (inevitable) compile failures. Failed builds are added to a list,
# and the script jumps to the next package in the queue.
#
# just run update-world --prepare (or --empty) followed by update-world --install.
#
# Release Notes:
# v1.8: Cleaned and modularized code. Added ability to utilize revdep-rebuild if wished. Can now pipe other
# emerges into update-world. Sources /etc/make.conf. Added interfacing with elog. Some bugfixes.
# v1.7 broken.
# v1.6.1: Bugfixes, added utf-8 support
# v1.6: Script preserves colored output. Fixed version numbering in help output.
# v1.5: now handles 'fetch restriction' properly
# v1.4: uses '--newuse' for emerges to respect new use flags
# v1.3: uses '--oneshot' for emerges so they don't get added to the world file.
# v1.2: Script now detects log directly instead of accepting only /var/log/portage
# v1.1: minor bugfix to allow package without version to be added to 'emergelist'
# v1.0: Release
#
#
#### Help ####
# Help information
displayhelp ()
{
echo "update-world 1.8
count_zero 2007
licensed under the GPLv2
http://www.fsf.org/licensing/licenses/gpl.txt
Description:
This is a bash script to automate the 'emerge -uD world' process.
The default program directory is ~/.update-world.
Error logs are filed in ~/.update-world/<date>/log.
Ebuild notices (elogs) are filed in ~/.update-world/<date>/notice.
Usage:
update-world [ --help | -h ]
Display this help file and exit.
update-world [ --prepare | -p ]
Prepares a list of packages produced by 'emerge -puvD world'
and outputs them to the file 'emergelist' in the current directory.
Add, remove, or change the version of any of the packages listed
in this file with your favorite editor before continuing.
update-world [ --empty | -e ]
Same as --prepare except it performs 'emerge -ev world' to recompile
all packages on a system.
update-world [ --install | -i ]
Begins the emerge process based upon the 'emergelist' file created
from running 'update-world --prepare'. Once finished, failed builds
will be saved in the file 'failedlist' in the program directory.
update-world [ --revdep | -r ]
Run revdep-rebuild to prepare the emergelist. Can be used alone
or in conjunction with --install.
emerge -p <package> | update-world
Let portage produce an emergelist determined by a package. This can be
useful for large packages with lots of dependencies (kde, gnome, xorg, etc.).
Once complete, run update-world --install.
update-world [ --log | -l ]
View the ebuild notices (elogs) of packages installed by update-world,
sorted by date.
"
}
#### end help ####
# Set the program directory and important variables
source /etc/make.conf
export EMERGE_DEFAULT_OPTS=""
homedir=~/.update-world
workdir="$homedir/`date +%F`"
logdir="$workdir/log"
noticedir="$workdir/notice"
mkdir -p $logdir
mkdir -p $noticedir
touch $homedir/failedlist
emergelist=$homedir/emergelist
emergetemp=$workdir/emergetemp
failedlist=$homedir/failedlist
parameter="-puvD --newuse"
#### error-check ####
# make sure there are no errors in the emerge process.
errorcheck ()
{
cat $emergelist | xargs emerge -p > $emergetemp
if [[ -n `cat $emergetemp | grep '\[blocks'` ]]
then echo "WARNING: You have a blocking package. Fix this before continuing."; exit 1
exit 1
elif grep -q "masked by:" $emergetemp
then echo "WARNING: You may need to unmask some packages before continuing."; exit 1
elif grep -q "emerge: there are no ebuilds to satisfy" $emergetemp
then echo "WARNING: One of the specified packages doesn't exist"; exit 1
elif [[ -n `cat $emergetemp | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*F[[:upper:][:lower:] ]*\].*/&/p'` ]]
then fetchrestricted=`cat $emergetemp | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*F[[:upper:][:lower:] ]*\].*/&/p' | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*\]\ \([[:lower:][:upper:]0-9._+\/-]*\).*/\1/p'`
echo -e "\nWARNING: The following packages have Fetch Restriction turned on:\n$fetchrestricted"
for each in $fetchrestricted
do emerge =$each
done
exit 1
else :
fi
}
#### end error-check ####
#### empty ####
# parameter for '--empty' to emerge -e world
empty ()
{
parameter='-pev'
}
#### end empty
#### prepare ####
# Prepare the list of packages to emerge into emerglist
prepare ()
{
if [[ -e "$emergelist" ]]
then echo -en "The file 'emergelist' already exists and will be overwritten.\nContinue anyway? (Y/n) "
read continue
if [[ "$continue" = 'y' || "$continue" = 'yes' || "$continue" = 'Y' || "$continue" = 'Yes' || "$continue" = 'YES' || "$continue" = '' ]]
then :
else echo exiting.
exit 0
fi
else :
fi
emerge $parameter world > $emergelist
errorcheck
emerge $parameter world
echo -e "\nThese are the packages that will be installed. If you want to alter this list,\njust edit the file 'emergelist' in $homedir. When finished,\nrun 'update-world --install'"
cat $emergelist | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*\]\ \([[:lower:][:upper:]0-9._+\/-]*\).*/\1/p' | sed 's/.*/=&/' | sort -d > $emergetemp
cp $emergetemp $emergelist
rm -f $failedlist
}
#### end prepare ####
#### installworld ####
# Install the packages listed in emergelist
installworld ()
{
# Make sure there's an emergelist
if [[ -s "$emergelist" ]]
then :
else echo "WARNING: no 'emergelist' exists. Did you run 'update-world --prepare' first?"; exit 1
fi
errorcheck
condition=false
# set begin time
before_time="${before_time:=`date +%s`}"
# Until loop to make portage continue until finished
until [[ $condition = true ]]
do
cat $emergelist | xargs emerge -1
# Detect which packages have been successfully emerged and remove from the queue
installed=`tac /var/log/emerge.log | sed '/Started\ emerge/,$d' | grep "completed emerge" | sed 's/^.*)\ //' | sed 's/\ to.*//'`
for each in `cat "$emergelist" | sed 's/^=//'`
do
if [[ -n `echo "$installed" | grep "$each"` ]]
then
# Checking emerge logs for any ebuild notices
einfo=""
notice="$noticedir/${each/\//:}-notice.log"
elog="${PORT_LOGDIR}/elog/`ls -lt $PORT_LOGDIR/elog | grep ${each/\//:} | sed '2,$ d' | sed 's/.*\ //'`"
if [[ -n "$PORTAGE_ELOG_SYSTEM" ]] && [[ -n "${einfo:=`cat $elog | sed '/^$/d' | sed '/^[ ]*$/d' | sed 's/.*/ * &/'`}" ]]
then
echo ------------------------ > $notice
echo -e "$each \n" >> $notice
echo "$einfo" >> $notice
echo ------------------------ >> $notice
else :
fi
# Removing successful emerges
each=`echo ${each//\//\\/} | sed 's/\-[0-9].*//'`
cat $emergelist | sed "/${each//\//\\/}/d" > $emergetemp
mv $emergetemp $emergelist
fi
done
# See if all packages have been emerged. If so, exit function "installworld"
if [[ -z `cat "$emergelist"` ]]
then condition=true
break
else :
fi
## If there are remaining packages, find why the emerge stopped
errorlog=`ls -t $PORT_LOGDIR | sed '2,$d'`
# if user-initiated "control-c" just exit.
if [[ -n `cat $PORT_LOGDIR/$errorlog | grep 'signal 2'` ]]
then echo -e "\n*** User hit 'Control-C' ... exiting.\n"
exit 1
else :
fi
# if package failed, give user chance to stop update-world
sleep 5
echo
echo "*** Hit Control-C to exit, or just wait to continue with emerge."
echo
sleep 10
echo "*** Continuing with emerge."
# Detect failed emerge and add to failedlist, remove failed package from emergelist
failedpkg=`tac /var/log/emerge.log | sed '1d' | sed '/terminating/,$d' | sed '/completed\ emerge/,$d' | grep ">>>" | sed 's/.*)\ //' | sed 's/\ to.*//'`
#### uh-oh, stop if something's really wrong to prevent infinite emerge (tried to emerge same package again)
if [[ -n `tail -n 1 $failedlist | grep $failedpkg` ]]
then echo "The failed package $failedpkg could not be merged and must be successfully installed before continuing."
exit 1
else :
fi
echo $failedpkg >> $failedlist
# if package added to emergelist manually doesn't have a version, make sure it's still removed if it fails
if [[ -z `cat "$emergelist" | grep "$failedpkg"` ]]
then failedpkg2=`equery which =$failedpkg | sed 's/\/[^/]*$//' | sed -n 's/.*\/\([^/]*\/[^/]*\)/\1/p'`
cat $emergelist | sed "/^${failedpkg2//\//\\/}/d" > $emergetemp
else cat $emergelist | sed "/${failedpkg//\//\\/}/d" > $emergetemp
fi
mv $emergetemp $emergelist
# link to the log file of the failed package
ln -s $PORT_LOGDIR/$errorlog $logdir/${errorlog}_error-log
echo
echo
echo "*** $failedpkg compile failed, skipping."
# Is there a package in 'emergelist' that REQUIRES the failed package as a dep? If so, remove it too.
failedpkgdeps=`equery which =$failedpkg | sed 's/\/[^/]*$//' | sed -n 's/.*\/\([^/]*\/[^/]*\)/\1/p'`
deps=`equery depends $failedpkgdeps | sed '/^\[/d' | sed 's/.*/=&/'`
for each in `echo $deps`
do
if [[ -n `cat $emergelist | grep "$each"` ]]
then
if [[ -n `emerge -p $each | grep "$failedpkg"` ]]
then
each2=`cat $emergelist | grep $each | sed 's/\=//'`
echo "$each2 (depends on $failedpkg)" >> $failedlist
cat $emergelist | sed "/${each//\//\\/}/d" > $emergetemp
mv $emergetemp $emergelist
echo "*** $each depends on $failedpkg, skipping."
else :
fi
else :
fi
done
if [[ -z `cat "$emergelist"` ]]
then condition=true
break
else :
fi
echo "*** Continuing with emerge world"
done
}
#### end installworld ####
#### revdep-rebuild ####
# this script can call revdep-rebuild and get it's emergelist from there.
# useful to run in conjunction with update-world --install, it will automatically
# run revdep-rebuild after the emerge -uD world is complete
revdep ()
{
rm /root/.revdep-rebuild*.?_*
revdep-rebuild --package-names --pretend | tee $emergelist
cat $emergelist | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*\]\ \([[:lower:][:upper:]0-9._+\/-]*\).*/\1/p' | sed 's/.*/=&/' > $emergetemp
mv $emergetemp $emergelist
}
#### end revdep-rebuild ####
#### piped ####
# emergelist is piped from another emerge (i.e., emerge kde | update-world)
piped ()
{
echo "$piped" > $emergelist
errorcheck
cat $emergelist | sed -n 's/^\[ebuild[[:upper:][:lower:] ]*\]\ \([[:lower:][:upper:]0-9._+\/-]*\).*/\1/p' | sed 's/.*/=&/' > $emergetemp
mv $emergetemp $emergelist
cat $emergelist | xargs emerge -p
echo -e "\nThese are the packages that will be installed. If you want to alter this list,\njust edit the file 'emergelist' in $homedir. When finished, run\n'update-world --install'"
}
#### end piped ####
#### finished ####
# Display closing messages
finished ()
{
echo -e "\n\nCongratulations! 'update-world' complete."
elapsed_time
echo
# All done! display the failed packages, if any
if [[ -z `cat $failedlist` ]]
then echo "All packages emerged successfully"
else echo "These packages couldn't be merged due to compile errors:"
echo
cat $failedlist
echo
echo -e "Look in $logdir/ for the portage log files of the failed builds.\nCheck bugs.gentoo.org or the Gentoo Forums for help."
fi
if [[ -n "$PORTAGE_ELOG_SYSTEM" ]]
then echo -e "\n * To review portage elog notices, type \"update-world --log\"\n"
else echo " * To enable portage notice viewing with elog, check out http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=3&chap=1#doc_chap4."
fi
}
#### end finished ####
#### elapsed_time ####
# just for fun let's see how long the process takes...
elapsed_time ()
{
after_time=`date +%s`
time=$(( after_time - before_time ))
days=$(( time / 86400 ))
time=$(( time - ( days * 86400 ) ))
hours=$(( time / 3600 ))
time=$(( time - ( hours * 3600 ) ))
minutes=$(( time / 60 ))
time=$(( time - ( minutes * 60 ) ))
seconds=$time
if [[ "$days" -eq 0 ]]
then days=""
elif [[ "$days" -eq 1 ]]
then days="$days day, "
elif [[ "$days" -gt 1 ]]
then days="$days days, "
fi
if [[ "$hours" -eq 0 ]]
then hours=""
elif [[ "$hours" -eq 1 ]]
then hours="$hours hour, "
elif [[ "$hours" -gt 1 ]]
then hours="$hours hours, "
fi
if [[ "$minutes" -eq 0 ]]
then minutes=""
elif [[ "$minutes" -eq 1 ]]
then minutes="$minutes minute, and "
elif [[ "$minutes" -gt 1 ]]
then minutes="$minutes minutes, and "
fi
seconds="$seconds seconds"
echo -e "Total compiling time: $days$hours$minutes$seconds.\n"
}
#### end elapsed_time ####
#### What should the script do when invoked? ####
# I have nine functions: displayhelp, empty, prepare, errorcheck, installworld, revdep, piped, elapsed_time, and finished
arguments="$@"
# update-world --help
if [[ -n `echo "$arguments" | sed -n 's/-h/&/p'` || -n `echo "$arguments" | sed -n 's/--help/&/p'` ]]
then
displayhelp
exit 0
else :
fi
# update-world --empty
if [[ -n `echo -e \\t "$arguments" | sed -n 's/-e/&/p'` || -n `echo "$arguments" | sed -n 's/--empty/&/p'` ]]
then echo Preparing list of packages...
empty
prepare
exit 0
else :
fi
# update-world --prepare
if [[ -n `echo "$arguments" | sed -n 's/-p/&/p'` || -n `echo "$arguments" | sed -n 's/--prepare/&/p'` ]]
then echo Preparing list of packages...
prepare
exit 0
else :
fi
# update-world --install
if [[ -n `echo "$arguments" | sed -n 's/-[[:lower:]]*i/&/p'` || -n `echo "$arguments" | sed -n 's/--install/&/p'` ]]
then
installworld
check=true
else :
fi
# update-world --revdep
if [[ -n `echo "$arguments" | sed -n 's/-[[:lower:]]*r/&/p'` || -n `echo "$arguments" | sed -n 's/--revdep/&/p'` ]]
then
revdep
if [[ -s "$emergelist" ]]
then installworld
else :
fi
check=true
else :
fi
# emerge -p <package> | update-world
# piped command MUST include --pretend !!!
if [[ "$arguments" = "" ]]
then echo "No arguments supplied. If you're piping from the emerge command (emerge -p <package> | update-world), this is fine. Otherwise, cancel and type 'update-world --help' for usage information."
until [[ `echo "$piped" | wc -l` -eq 10 ]] && [[ -z `echo "$piped" | grep 'Calculating dependencies'` ]] || [[ -z `echo "${piped:-1}" | tail -n 4` ]]
do read line
piped="${piped}
$line"
done
else :
fi
if [[ -n `echo "$piped" | sed -n 's/Calculating\ dependencies/&/p'` ]]
then
piped
exit 0
else :
fi
# update-world --log
if [[ -n `echo "$arguments" | sed -n 's/-l/&/p'` || -n `echo "$arguments" | sed -n 's/--log/&/p'` ]]
then echo "Choose a date: "
select directory in `ls -Al | grep ^d | sed 's/.*\ //'`
do
if [[ -n "$directory" ]]
then date=$directory
break
else echo "invalid selection"
fi
done
rm -f $homedir/$date/notice/log
for each in `ls $homedir/$date/notice/`
do cat $homedir/$date/notice/$each >> $homedir/$date/notice/log
done
less $homedir/$date/notice/log
check=true
exit 0
else :
fi
# Check if arguments were valid
if [[ ${check:-false} == false ]]
then echo -e "Error: Invalid Command.\nRun 'update-world --help' for usage."
exit 1
else :
fi
# done! Display closing text
finished
exit 0
|
Let me know if you find any bugs or have feedback. Good luck!
[Update] 9/13/06 (version 1.1)
Fixed script to prevent errors when adding a package without specifying a version number to 'emergelist'.
[Update] 10/27/06 (version 1.2)
Script now automatically detects the portage log directory instead of just assuming /var/log/portage.
[Update] 11/22/06 (version 1.4) I guess my edit didn't take yesterday for v1.3...
Added --newuse and --oneshot to emerge variables.
[Update] 11/22/06 (version 1.5)
Now handles 'fetch restriction' properly, prompting user to download sources before continuing.
[Update] 12/06/06 (version 1.6)
Script preserves colored emerge output. Fixed version numbering in help output.
[Update] 12/09/06 (version 1.7)
Cleaned up the code a little, fixed some bugs. Check out the new feature for managing USE flags!
Edit: seems to be broken
[Update] 12/10/06 (version 1.6.1)
bugfixes and utf-8 support.
[Update] 2/16/07 (version 1.
Cleaned and modularized code. Added ability to utilize revdep-rebuild if wished. Can now pipe other
emerges into update-world. Sources /etc/make.conf. Added interfacing with elog. Some bugfixes. _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin
Last edited by count_zero on Fri Feb 16, 2007 1:38 pm; edited 16 times in total |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
therealkurisu n00b
![n00b n00b](/images/ranks/rank_rect_0.gif)
![](images/avatars/13017794744505644f1145a.jpg)
Joined: 11 Sep 2006 Posts: 2 Location: Mesa, AZ
|
Posted: Mon Sep 11, 2006 1:51 pm Post subject: A wonderful solution to a common problem! |
|
|
I'm fairly new to Gentoo (and linux in general ^_^) but I find this script to be VERY useful. I've ran it 5 or so times so far and haven't had any problems at all. It saves SO much time from compiling errors and general problems (and allows for skipping updates on known problematic areas!).
Automated skipping, ftw. I reccomend that everyone try it out.
Good job count_zero.
(Also, <3 Crono and therefore, through association, you ) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
Doogman Apprentice
![Apprentice Apprentice](/images/ranks/rank_rect_2.gif)
![](images/avatars/1519254887455733d092f65.jpg)
Joined: 24 Sep 2004 Posts: 242
|
Posted: Mon Sep 11, 2006 8:05 pm Post subject: |
|
|
This looks like a good idea.
While my dual-core system can compile packages at a good clip, rebuilding world always stalls out at some point requiring user intervention. Instead of grinding away at the build list while I'm at work, the computer spends a good deal of time at idle until I get home again. With this script, most of the work should be done while I'm away.
Maybe I'll finally tackle that gcc upgrade I've been putting off. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
devsk Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
![](images/avatars/gallery/BaldursGate/baldursgate2_minsc.gif)
Joined: 24 Oct 2003 Posts: 2995 Location: Bay Area, CA
|
Posted: Tue Sep 12, 2006 8:47 pm Post subject: |
|
|
Removing deps of the failed pkgs is definitely better idea than portage's 'emerge --resume --skipfirst' which can break your system in no time. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Sun Sep 17, 2006 2:21 am Post subject: |
|
|
It was discussed in another thread about combining this script with emerging packages in the correct order, a la this thread by Guenther Brunthaler (thanks for a job well done, Guenther ).
If you run Guenther's script up until the point at which you will start rebuilding your system (gcc already emerged), you'll end up with a file '/root/recompile-remaining-packages'. This file contains all of the packages to be emerged, in the order to best rebuild the toolchain and the rest of the system. It's pretty easy to interface this with update-world:
Code: | grep "item " /root/recompile-remaining-packages | sed 's/item\ .*\ //' > ~/.update-world/emergelist |
and then
Code: | update-world --install |
Have fun with interruption-free emerging of your system in the "correct" order. ![Cool 8)](images/smiles/icon_cool.gif) _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
Bynds n00b
![n00b n00b](/images/ranks/rank_rect_0.gif)
Joined: 29 Sep 2006 Posts: 1
|
Posted: Fri Sep 29, 2006 12:53 pm Post subject: My Aspirin |
|
|
This script has just cured every headache I had with my Gentoo box (I'm not a heavy user )
It took me weeks to update my old compaq server when I dusted it off after about 2 years in the cupboard, this would have saved me about half my time.
Fantastic! |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
bender02 Tux's lil' helper
![Tux's lil' helper Tux's lil' helper](/images/ranks/rank_rect_1.gif)
![](images/avatars/12083739844625523ec4ca.gif)
Joined: 05 May 2006 Posts: 82 Location: Nashville, TN, USA
|
Posted: Thu Oct 26, 2006 10:19 pm Post subject: |
|
|
Great script, thanks!
I have one thing though, while running "update-world --install", I get an error like
Code: | cat: /var/log/portage/elog: Is a directory |
and the dir inside .update-world which is supposed to contain the logs for failed packages contains only a link to /var/log/portage/elog.
My logs structure for elog is that /var/log/portage/elog is a directory containing logs for various emerges. Is there an easy way to fix this? Thanks. _________________ (I love deadlines. I like the whooshing sound they make as they fly by. [DA]) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Fri Oct 27, 2006 8:03 pm Post subject: |
|
|
The problem was that the script assumed that /var/log/portage was the portage log directory. I've updated the script so that it detects the setting of PORT_LOGDIR in /etc/make.conf. I've updated the code in the first post, but my (free) webhost is giving me some problems so I can't update the download yet ![Sad :(](images/smiles/icon_sad.gif) _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
bender02 Tux's lil' helper
![Tux's lil' helper Tux's lil' helper](/images/ranks/rank_rect_1.gif)
![](images/avatars/12083739844625523ec4ca.gif)
Joined: 05 May 2006 Posts: 82 Location: Nashville, TN, USA
|
Posted: Sat Oct 28, 2006 3:57 am Post subject: |
|
|
Great! Thanks! (But I'm gonna get to test it only next time I sync...) _________________ (I love deadlines. I like the whooshing sound they make as they fly by. [DA]) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Sat Oct 28, 2006 5:42 pm Post subject: |
|
|
Download link fixed! _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
ryker Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/1584949304407aaec0793da.jpg)
Joined: 28 May 2003 Posts: 412 Location: Portage, IN
|
Posted: Sat Nov 11, 2006 3:56 pm Post subject: |
|
|
This script needs more exposure! This is the most useful emerge wrapper that I have seen yet. They should feature it on GWN sometime.
Thank you for your time and effort on this. _________________ Athlon 64 3200+, 80G WD sata hd + 200G IDE, 1G Geil DDR400, MSI K8T Neo
IntelCore2Duo 2.0Ghz MSI laptop,100G SATA hd, 2G RAM |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
richfish Apprentice
![Apprentice Apprentice](/images/ranks/rank_rect_2.gif)
Joined: 03 Mar 2006 Posts: 202 Location: Phoenix, AZ
|
Posted: Mon Nov 20, 2006 8:59 pm Post subject: |
|
|
Shouldn't this script use --oneshot for merging packages so it doesn't pollute the world file with all the packages it updates? |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Tue Nov 21, 2006 3:19 am Post subject: |
|
|
Good idea!
Easy to implement too. I just added '-1' to the end of line 101 in the script. _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
RageOfOrder Tux's lil' helper
![Tux's lil' helper Tux's lil' helper](/images/ranks/rank_rect_1.gif)
![](images/avatars/12538416834ce879939ea81.png)
Joined: 06 Aug 2006 Posts: 99 Location: EH?!?!
|
Posted: Tue Nov 21, 2006 9:29 am Post subject: |
|
|
Wow, this is just awesome. I've been putting this off for weeks. I think I'll finally update everything on wednesday night.
Thanks for all your hard work! _________________ ...And then stuff happened. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
dkure n00b
![n00b n00b](/images/ranks/rank_rect_0.gif)
Joined: 20 Apr 2004 Posts: 51 Location: Sydney, Australia
|
Posted: Wed Nov 22, 2006 8:07 am Post subject: |
|
|
It would be great if there was two more features.
This script should include --newuse, to make sure use flags are respected.
Second is more of a request, and is a package fetch option. This would fetch the packages selected in the .update_world file. Allowing packages to be downloaded in the background overnight before packages are selected if need be.
I agree that this script should certainly get featured on GNN, makes updating simpler and more specialized to peoples needs. |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Wed Nov 22, 2006 2:54 pm Post subject: |
|
|
Quote: | This script should include --newuse, to make sure use flags are respected. |
I guess the meaning of "--newuse" has changed since I last read the man page...I thought that (when used in conjunction with --update world) it updated ONLY those packages that would be affected by recently changed USE flags.
From the Gentoo Wiki (http://gentoo-wiki.com/MAN_emerge):
Quote: | --newuse (-N)
Tells emerge to include installed packages where USE flags have changed since compilation. An asterisk marks when a USE flag has changed since the package was compiled. |
However, the gentoo handbook suggests that it might be useful in better updating an entire system.
From the installation handbook (http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2&chap=1):
Quote: | Code Listing 14: Updating your entire system
# emerge --update --deep world
Since security updates also happen in packages you have not explicitly installed on your system (but that are pulled in as dependencies of other programs), it is recommended to run this command once in a while.
If you have altered any of your USE flags lately you might want to add --newuse as well. Portage will then verify if the change requires the installation of new packages or recompilation of existing ones:
|
So i guess I'm a little confused about whether --newuse would be helpful in this situation, where all packages will be updated anyway. Doesn't portage look a new use flags anyway? Maybe someone could clarify this? Implementing this in the script is pretty trivial.
Quote: | Second is more of a request, and is a package fetch option. This would fetch the packages selected in the .update_world file. Allowing packages to be downloaded in the background overnight before packages are selected if need be. |
I'll look into this, parallel fetching would be a useful feature. I believe it is going to be built into portage sometime in the future, but it could probably be implemented here. I'll let you know.
Quote: | I agree that this script should certainly get featured on GNN, makes updating simpler and more specialized to peoples needs. |
Thanks for all the helpful feedback everyone, it makes doing stuff like this very enjoyable. ![Very Happy :D](images/smiles/icon_biggrin.gif) _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
ryker Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/1584949304407aaec0793da.jpg)
Joined: 28 May 2003 Posts: 412 Location: Portage, IN
|
Posted: Wed Nov 22, 2006 4:05 pm Post subject: |
|
|
I believe --newuse also implies --update. Of course, you can always add additional parameters such as -D to catch more packages. I added --newuse to the update-world script on my system and it works fine.
I just changed Code: |
# parameter for '--empty' to emerge -e world
if [[ $1 == '--empty' || $1 == '-e' ]]
then parameter='-pev'
else parameter='-puvD' |
to Code: |
# parameter for '--empty' to emerge -e world
if [[ $1 == '--empty' || $1 == '-e' ]]
then parameter='-pev'
else parameter='-puvD --newuse' |
I love this script so much, I was planning on writing and submitting a tip to the GWN this weekend. _________________ Athlon 64 3200+, 80G WD sata hd + 200G IDE, 1G Geil DDR400, MSI K8T Neo
IntelCore2Duo 2.0Ghz MSI laptop,100G SATA hd, 2G RAM |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
padoor Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
![](images/avatars/gallery/The Jetsons/cartoon_the_jetsons_spacely.gif)
Joined: 30 Dec 2005 Posts: 4185 Location: india
|
Posted: Wed Nov 22, 2006 4:53 pm Post subject: |
|
|
this is looking good for me too.
please instruct how do i install and run this .
which script do i download and how i can use it?
i need codes for full proceedure.
i am still newbee for these operations. _________________ reach out a little bit more to catch it (DON'T BELIEVE the advocate part under my user name) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
ryker Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/1584949304407aaec0793da.jpg)
Joined: 28 May 2003 Posts: 412 Location: Portage, IN
|
Posted: Wed Nov 22, 2006 8:55 pm Post subject: |
|
|
It's pretty straight forward and the instructions are clearly given in the first post.
Basically, all you have to do is:
1. Download the latest version of the script from here. and save it as /usr/bin/update-world.
2. chmod +x /usr/bin/update-world
That's it. Now just type update-world and it tells you what to do. _________________ Athlon 64 3200+, 80G WD sata hd + 200G IDE, 1G Geil DDR400, MSI K8T Neo
IntelCore2Duo 2.0Ghz MSI laptop,100G SATA hd, 2G RAM |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
padoor Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
![](images/avatars/gallery/The Jetsons/cartoon_the_jetsons_spacely.gif)
Joined: 30 Dec 2005 Posts: 4185 Location: india
|
Posted: Thu Nov 23, 2006 1:22 am Post subject: |
|
|
that was pretty easy
and it works noww.
i am due for world update in 2 days.
i will post the result , thanks for direction ryker ![Smile :)](images/smiles/icon_smile.gif) _________________ reach out a little bit more to catch it (DON'T BELIEVE the advocate part under my user name) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
padoor Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
![](images/avatars/gallery/The Jetsons/cartoon_the_jetsons_spacely.gif)
Joined: 30 Dec 2005 Posts: 4185 Location: india
|
Posted: Thu Nov 23, 2006 2:28 am Post subject: |
|
|
i did an emerge --sync
then emerge -Du systaem.
update-world -p made a file list as follows
localhost ramaswamy # update-world -p
The file 'emergelist' already exists and will be overwritten.
Continue anyway? (y/n) y
These are the packages that would be merged, in order:
Calculating world dependencies ..... ..... ..
!!! Ebuilds for the following packages are either all
!!! masked or don't exist:
net-im/ymessenger
... done!
[ebuild U ] dev-java/sun-jdk-1.5.0.09-r1 [1.5.0.08] USE="X alsa -doc -examples -jce -nsplugin" 32,469 kB
[ebuild N ] virtual/jdk-1.5.0 0 kB
[ebuild U ] x11-themes/gnome-backgrounds-2.16.2 [2.16.1] USE="-debug" 1,172 kB
[ebuild U ] dev-java/sun-jre-bin-1.5.0.09-r1 [1.5.0.08-r1] USE="X alsa -nsplugin" 0 kB
[ebuild U ] dev-cpp/gtkmm-2.10.2-r1 [2.8.3] USE="-accessibility% -debug -doc% -examples%" 0 kB
[ebuild FU ] dev-java/sun-jdk-1.4.2.13 [1.4.2.12-r2] USE="X alsa -doc -examples -jce -nsplugin" 35,509 kB
[ebuild U ] www-client/mozilla-firefox-bin-2.0 [1.5.0.7] LINGUAS="-ar -bg% -ca -cs -da -de -el -en_GB -es -es_AR -es_ES -eu% -fi -fr -fy% -fy_NL% -ga -ga_IE -gu% -gu_IN% -hu -it -ja -ko -lt% -mk -mn% -nb -nb_NO -nl -nn% -nn_NO% -pl -pt% -pt_BR -pt_PT% -ru -sk -sl -sv -sv_SE -tr -zh% -zh_CN -zh_TW (-he%) (-ro%)" 0 kB
[ebuild U ] gnome-base/gnome-vfs-2.16.3 [2.16.2] USE="hal ipv6 ssl -avahi -debug -doc -gnutls -samba" 1,855 kB
[ebuild U ] gnome-base/gnome-desktop-2.16.2 [2.16.1] USE="-debug -doc" 1,202 kB
[ebuild U ] gnome-base/eel-2.16.3 [2.16.1] USE="X -debug" 657 kB
[ebuild U ] gnome-base/nautilus-2.16.3 [2.16.2-r1] USE="X gnome -beagle -debug" 4,168 kB
[ebuild U ] gnome-extra/evolution-data-server-1.8.2 [1.8.1-r1] USE="ipv6 ldap ssl -debug -doc -kerberos -keyring -krb4 -nntp" 6,800 kB
[ebuild U ] gnome-extra/nautilus-cd-burner-2.16.2 [2.16.1-r1] USE="cdr dvdr -debug" 698 kB
[ebuild U ] media-libs/xine-lib-1.1.3_pre20061119 [1.1.3_pre20061112] USE="X aac aalib alsa arts dvd esd gnome gtk ipv6 mad nls opengl oss sdl truetype vorbis win32codecs xv -a52 (-altivec) -debug -directfb -dts -dxr3 -fbcon -flac -imagemagick -libcaca -mmap -mng -modplug -pulseaudio -samba -speex -theora -v4l -vcd -vidix -xinerama -xvmc" 6,814 kB
[ebuild U ] gnome-base/control-center-2.16.2 [2.16.1] USE="alsa eds -debug" 1,975 kB
[ebuild U ] gnome-extra/gnome-games-2.16.2 [2.16.1.1] USE="-artworkextra -debug -guile (-avahi%)" 5,134 kB
[ebuild U ] www-client/epiphany-2.16.2 [2.16.1] USE="python -debug -doc" 4,130 kB
[ebuild U ] media-sound/sound-juicer-2.16.2 [2.16.1] USE="ogg -debug -flac" 1,236 kB
[ebuild U ] app-arch/file-roller-2.16.2 [2.16.1] USE="gnome%* -debug" 1,105 kB
[ebuild U ] gnome-extra/gtkhtml-3.12.2 [3.12.1] USE="-debug -static" 1,255 kB
[ebuild U ] mail-client/evolution-2.8.2.1 [2.8.1.1] USE="crypt dbus hal ipv6 ldap spell ssl -bogofilter -debug -doc -kerberos -krb4 -mono -nntp -pda -profile" 12,618 kB
[ebuild U ] net-misc/knemo-0.4.6 [0.4.5] USE="arts -debug -wifi -xinerama" 589 kB
[ebuild U ] kde-base/nsplugins-3.5.5-r1 [3.5.5] USE="arts -debug -kdeenablefinal -kdehiddenvisibility -xinerama" 0 kB
Total: 23 packages (22 upgrades, 1 new), Size of downloads: 119,393 kB
Fetch Restriction: 1 package (1 unsatisfied)
These are the packages that will be installed. If you want to alter this list,
just edit the file 'emergelist' in /root/.update-world. When finished, run
'update-world --install'
localhost ramaswamy #
localhost ramaswamy # update-world --install
Calculating dependencies
emerge: there are no ebuilds to satisfy "=Fetch".
cat: /wget-log.3: No such file or directory
*** Hit Control-C to exit, or just wait to continue with emerge.
*** Continuing with emerge.
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
sed: -e expression #1, char 0: no previous regular expression
ln: creating symbolic link `/root/.update-world/2006-11-23/log/wget-log.3_error-log': File exists
*** compile failed, skipping.
grep: invalid option -- ,
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
grep: unrecognized option `--all-packages'
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
grep: unknown directories method
grep: option `--direct' requires an argument
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
grep: unknown devices method
grep: unrecognized option `--indirect'
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
*** Continuing with emerge world
Usage:
emerge [ options ] [ action ] [ ebuildfile | tbz2file | dependency ] [ ... ]
emerge [ options ] [ action ] < system | world >
emerge < --sync | --metadata | --info >
emerge --resume [ --pretend | --ask | --skipfirst ]
emerge --help [ system | world | config | --sync ]
Options: -[abBcCdDefgGhikKlnNoOpqPsStuvV] [--oneshot] [--newuse] [--noconfmem]
[ --color < y | n > ] [ --columns ]
[--nospinner]
[ --deep ] [--with-bdeps < y | n > ]
Actions: [ --clean | --depclean | --prune | --regen | --search | --unmerge ]
For more help try 'emerge --help' or consult the man page.
Congratulations! 'emerge world' complete.
All packages emerged successfully
localhost ramaswamy #
is this my options error or the fetch has some problem?
regular emerge -Du world does fetch the files and started with jdk
what mistake i am doing?? _________________ reach out a little bit more to catch it (DON'T BELIEVE the advocate part under my user name) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Thu Nov 23, 2006 4:33 am Post subject: |
|
|
ryker wrote: | I believe --newuse also implies --update. Of course, you can always add additional parameters such as -D to catch more packages. I added --newuse to the update-world script on my system and it works fine.
I just changed Code: |
# parameter for '--empty' to emerge -e world
if [[ $1 == '--empty' || $1 == '-e' ]]
then parameter='-pev'
else parameter='-puvD' |
to Code: |
# parameter for '--empty' to emerge -e world
if [[ $1 == '--empty' || $1 == '-e' ]]
then parameter='-pev'
else parameter='-puvD --newuse' |
|
Done!
I tested it also, and seems to work well.
Version 1.4 now reflects these changes. The script above and the download link are updated. _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Thu Nov 23, 2006 4:48 am Post subject: |
|
|
@padoor:
you've uncovered a couple of problems, I think they are pretty easily fixed.
Quote: | Calculating world dependencies ..... ..... ..
!!! Ebuilds for the following packages are either all
!!! masked or don't exist:
net-im/ymessenger |
Portage isn't finding this package for your system. It's probably masked on your system, and *could* (though maybe it shouldn't, find out why it's masked) be unmasked by adding 'net-im/ymessenger' to /etc/portage/package.unmask.
Quote: |
[ebuild FU ] dev-java/sun-jdk-1.4.2.13 [1.4.2.12-r2] USE="X alsa -doc -examples -jce -nsplugin" 35,509 kB
. . .
Fetch Restriction: 1 package (1 unsatisfied)
|
You'll need to download the sources for sun-jdk from the website and copy to /usr/portage/distfiles/ before running the script.
Fix these problems before re-running update-world.
I should be able to make some changes to the script so it handles these errors better. _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
padoor Advocate
![Advocate Advocate](/images/ranks/rank-G-1-advocate.gif)
![](images/avatars/gallery/The Jetsons/cartoon_the_jetsons_spacely.gif)
Joined: 30 Dec 2005 Posts: 4185 Location: india
|
Posted: Thu Nov 23, 2006 5:07 am Post subject: |
|
|
i will finish emerge of jdk and stop the emerge -Du world and try again.
ymessenger is full masked and i have the latest version installed. so it can skip this file
lets see after i finish this jdk emerge, _________________ reach out a little bit more to catch it (DON'T BELIEVE the advocate part under my user name) |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
count_zero Guru
![Guru Guru](/images/ranks/rank_rect_3.gif)
![](images/avatars/156825843140a94e9a1f577.gif)
Joined: 17 May 2004 Posts: 460 Location: Little Rock, Arkansas, USA
|
Posted: Thu Nov 23, 2006 6:22 am Post subject: |
|
|
The script version 1.5 (second one tonight ) properly handles "Fetch Restriction" now, prompting you to download the sources before continuing. _________________ "We must all hang together, or assuredly we shall all hang separately."
-Ben Franklin |
|
Back to top |
|
![](templates/gentoo/images/spacer.gif) |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|