| View previous topic :: View next topic |
| Author |
Message |
dagurasu
n00b
Joined: 29 Apr 2005
Posts: 74
|
 Posted: Sun Jul 17, 2005 1:02 am Post subject: Script: Protects all installed ebuilds from emerge --sync |
 |
|
It's all in the comments:
Hopefully this is MUCH better than the first version.
| Code: |
#!/bin/bash
# savebuilds1.2
# 1.2 replaced "class" with category for correct terminology
# 1.1... fundamental fixes, first working version
# some minor fixes, and now only overlays .ebuilds which are no longer in portage
# dagurasu July 16, 2005
## This program is distributed under the terms of the GPL version 2
# USE AT YOUR OWN RISK. NO WARRANTY. This script may wreck your OS, smoke your hard
# drive, drown your cat, or make you pregnant. Read it first and decide for
# yourself if it looks safe.
# Script to create an overlay and copy installed ebuilds which are no longer in portage to it, because the ones in
# /usr/portage may be destroyed by emerge --sync
#
# Background and purpose:
#
# When installed packages are dropped from portage they will be removed from the local
# tree when emerge --sync is run. The packages will then be forced to be updated if
# any pakcage emerged with "emerge -u" depends on it. This is true even if -D is not specified
# and even if the package being updated is perfectly happy with current version of the
# package in question. Copying installed packages to an overlay prevents this from
# happening. This allows package.mask to be used to freeze packages for much
# longer times and allows "emerge -u" to really only update things that need to be
# updated... potentially saving ALOT of compiling time. The best thing is you don't
# need to manually deal with overlays or exclude lists other than package.mask
#
#Who wants this?:
# Anyone who wants to maintain their own kernel patches, or just keep a few system critical packages
# untouched while still being able to blindly upgrade lots of other stuff... just in case the latest tux-racer
# is really cool. Also anyone who wants to keep their world packages up2date without unecesarily
# updating their dependancies just because they are no longer in portage.
# Note:
# The overlay is deleted fist every time the script is run, so old unmerged packages
# will NOT build up in your overlay, AND ANYTHING ELSE YOU PUT IN THIS OVERLAY WILL BE LOST!
# Instructions:
# Just put somewhere handy and run it. It sets up and turns on the overlay in
# /usr/local/portage_installed for you. DO NOT USE THIS OVERLAY FOR ANY OTHER
# PURPOSE. DATA IN THIS OVERLAY WILL BE DESTROYED. USE A SEPARATE OVERLAY FOR OTHER
# PURPOSES.
# In version 1.1 the default is to run emerge --sync before creating the overlay
# calling savebuilds --nosync will overide this behavior
#
# Plans:
# Maybe future versions will copy portage directories of all installed packages to the overlay
# , then run emerge sync, and then delete all directories where the installed version is still in portage
# I think this should allow the old packages to be rebuilt if ever desired without updating them,
# but it would require that the script be run BEFORE emerge --sync removes the installed version from portage.
# The current version grabs the ebuild only from /var/db/pkg, where it will always be available so long as the
# package is installed
#
overlay=/usr/local/portage_installed
echo
if [ -d $overlay ]; then
echo Deleting previous copy of $overlay
rm -rf $overlay
fi
echo "Creating portage overlay in ${overlay}"
mkdir $overlay
if [ ! -d $overlay ]; then
echo "error making portage overlay directory"
exit
fi
echo "Checking if overlay ${overlay} is enabled:"
if [ ! `grep -e "$overlay" /etc/make.conf | grep -e "PORTDIR_OVERLAY"|cut -c 1-1 | grep -e "P"` ]; then
echo " .... no"
echo "Enabling overlay directory ${overlay} in /etc/make.conf ."
echo "PORTDIR_OVERLAY=\"\${PORTDIR_OVERLAY} ${overlay}\"" >> /etc/make.conf
else
echo " .... yes"
fi
if [ `echo $@|grep -wc -e"--nosync"` == 1 ]; then
echo Skipping portage update
else
echo Now performing "emerge --sync"
/usr/bin/emerge --sync
fi
echo
echo "Creating list of installed packages. "
echo
ls -1 --color=none /var/db/pkg/*/*/*.ebuild > /tmp/portage_installed
echo
echo "Building overlay directory structure and"
echo "copying locally installed ebuilds which are no longer in /usr/portage/."
echo "Please wait....."
echo
for buildfile in $(< /tmp/portage_installed) ;do
# echo $buildfile
cvsfile=`grep -e "\$Header:" $buildfile |awk -F ".ebuild" '{print $1}'`
category=`echo $cvsfile | awk -F "/" '{print $5}'`
# echo category: $category
base_pkg=`echo $cvsfile | awk -F "/" '{print $6}'`
# echo base package: $base_pkg
build_name=`echo $buildfile | awk -F "/" '{print $6".ebuild"}'`
# echo file name: $build_name
#
# only copy the build to the overlay IF it's not in the current portage tree
if [ ! -f /usr/portage/$category/$base_pkg/$build_name ]; then
echo
echo $category/$base_pkg/$build_name not found in portage
#make the overlay category directory if it doesn't exist
if [ ! -d $overlay/$category ]; then
echo creating directory $overlay/$category
mkdir $overlay/$category
fi
if [ ! -d $overlay/$category ]; then
echo "error creating directory ${overlay}${category}"
exit
fi
#
# make the overlay package directory if it doesn't exist
if [ ! -d $overlay/$category/$base_pkg ]; then
echo creating directory $overlay/$category/$base_pkg
mkdir $overlay/$category/$base_pkg
fi
if [ ! -d $overlay/$category/$base_pkg ]; then
echo "error creating directory ${overlay}/${category}/${base_pkg}"
exit
fi
echo copying ebuild to $overlay/$category/$base_pkg/$build_name
cp $buildfile $overlay/$category/$base_pkg
if [ ! -f $overlay/$category/$base_pkg/$build_name ]; then
echo "error copying ${buildfile} to overlay"
exit
fi
fi
grep -v "${buildfile}" installed>tmp;cat tmp>installed
done
rm /tmp/portage_installed
echo
echo Creation of installed package overlay is complete.
echo
|
Let me know if there are any gaping holes in this scheme. I have tested it a little and it seems to work well for me so far. I will be travelling but I'll check back once and awhile for suggestions.
BTW this script would probably be a little prettier if I was still allowed to use qpkg. Equery seems to be lacking... specifically it seems to only generated package names WITH the versions numbers... very hard to figure out which "-" to grep on to get the base name correct. That's why I went to the trouble of parsing the .ebuild file.
My now specific and I think workable outline of how to copy the source too... just haven't got around to doing it, somewhere on page one here:
http://forums.gentoo.org/viewtopic-t-372840-highlight-.html
Last edited by dagurasu on Tue Aug 23, 2005 2:39 am; edited 9 times in total |
|
| Back to top |
|
 |
dagurasu
n00b
Joined: 29 Apr 2005
Posts: 74
|
| Posted: Sun Jul 17, 2005 3:54 am Post subject: |
|
|
I just really think it's a bummer that to simply freeze a kernel version you have to mes with a package.mask AND also setup an overlay and copy the ebuild out to it. And if you don't do it until you already know about the problem then you have to know to go digging in /var/db/pkg. Not a user friendly procedure for a routine need.
Edit: Delete some ramblings.. since determined to be dead ends.
Last edited by dagurasu on Wed Aug 24, 2005 2:41 am; edited 4 times in total |
|
| Back to top |
|
|
dagurasu
n00b
Joined: 29 Apr 2005
Posts: 74
|
| Posted: Sun Jul 17, 2005 7:50 pm Post subject: |
|
|
New version 1.1 drastically helps with problems I discovered in last post.
Now it only copies .ebuild 's to the overlay which are installed but NOT found in the portage tree.
This means emerge package or emerge package (without a -u) will only fail for packages which you really have held back, which is not necesarily so bad (possibly even good)... and after all even if the rest of the files are kept, the real source probably wont be available from cvs, and we don't want to keep the source for the whole system laying around just in case part of it becomes unavailable. The point is just to avoid recompiling tons of binaries unecesarily every week. Once your binary no longer does it for you, you'll still need to upgrade(except of course for the kernel or something like that where the installed files really are the source). If you REALLY know you want to keep some source around, you'll still have to plan ahead for that. Complete toolchain and up rebuilds and such where you really want to rebuild everything will still force you to upgrade to a version where the source is available (or to keep a possibly out-of-sync binary installed).
Also, now it runs emerge --sync by default before creating the overlay. To overide run savebuilds --nosync
Edit: To my surprise I'm finding emerge --sync doesn't clear out the distfiles so it's still around(at leat for one example I'm looking at). I didn't realize all the source files were kept on the system(presumeably just for installed or fetched packages)... no wonder it all takes so much space, but I guess it makes sense. Anyway, it's just the ebuilds and digest files(actually patch files, the Manifest can be remade) that get wiped by emerge --sync. Now I've solved the .ebuld issue with this script and I've found the source files, I've also found that "ebuild package digest" after searching through a ton of mirrors can eventually find a stale or archinved mirror and recover the patches...(at least for some amount of time after it's lost) maybe there's a simpler way to do that to. There may be more improvements to this script coming in the future which make it easier to even do rebuilds of packages you've held back. I'm still learning... stay tuned. |
|
| Back to top |
|
|
barefootcoder
Tux's lil' helper
Joined: 09 Jan 2004
Posts: 93
|
| Posted: Fri Jan 13, 2006 7:00 pm Post subject: |
|
|
Hey dagurasu ...
Check out my new script here if you get a chance. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Unsupported Software
