View previous topic :: View next topic |
Author |
Message |
puddpunk l33t
Joined: 20 Jul 2002 Posts: 681 Location: New Zealand
|
Posted: Sun Jul 21, 2002 1:00 am Post subject: Free 100Mb+ of space!! |
|
|
when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...
So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!
so, quickly that was out of there! and i had found another Gb of space!
All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.
Anybody else have some space-saving tips? |
|
Back to top |
|
|
Naan Yaar Bodhisattva
Joined: 27 Jun 2002 Posts: 1549
|
Posted: Sun Jul 21, 2002 2:29 am Post subject: Re: Free 100Mb+ of space!! |
|
|
/var/tmp/portage also gets filled up if you abort ebuilds. I killed a mozilla build and it left 14M worth of stuff in that dir alone!
puddpunk wrote: | when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...
So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!
so, quickly that was out of there! and i had found another Gb of space!
All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.
Anybody else have some space-saving tips? |
|
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20103
|
Posted: Tue Jul 23, 2002 11:01 pm Post subject: Re: Free 100Mb+ of space!! |
|
|
puddpunk wrote: | OVER 1Gb OF FILES!!!
so, quickly that was out of there! and i had found another Gb of space!
| Though not critical, doing this does require you to download the stuff again if you reinstall it.
I emerge using -b so I have the binaries available. If I run out of diskspace, I'll just start backing
up that directory to a CD.
Also... 1G is definately more than 100Mb+ (as suggested in the post title). I was expecting to
find something freeing up 100M or so _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
puddpunk l33t
Joined: 20 Jul 2002 Posts: 681 Location: New Zealand
|
Posted: Wed Jul 24, 2002 8:47 am Post subject: ! |
|
|
Yea, i know. i just didnt want to say "FREE 1GB of space!" and then get flamed because ppl only had 100mb of crud in their distfiles dir |
|
Back to top |
|
|
puddpunk l33t
Joined: 20 Jul 2002 Posts: 681 Location: New Zealand
|
Posted: Wed Jul 24, 2002 8:48 am Post subject: ? |
|
|
is it safe to kill all the stuff in /var/tmp/portage? |
|
Back to top |
|
|
Naan Yaar Bodhisattva
Joined: 27 Jun 2002 Posts: 1549
|
Posted: Wed Jul 24, 2002 11:34 am Post subject: |
|
|
Yes. I believe of less consequence than /usr/portage/distfiles. |
|
Back to top |
|
|
Swishy Guru
Joined: 06 Jun 2002 Posts: 491 Location: NZ
|
Posted: Wed Jul 24, 2002 3:13 pm Post subject: Re: Free 100Mb+ of space!! |
|
|
puddpunk wrote: | when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...
So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!
so, quickly that was out of there! and i had found another Gb of space!
All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.
Anybody else have some space-saving tips? |
Did you just del , the zipped source or blow away the dir and create a new one?????
Cheers
Dale. _________________ Theres no substitute for C.I. |
|
Back to top |
|
|
debian n00b
Joined: 24 Jun 2002 Posts: 22
|
Posted: Thu Jul 25, 2002 1:09 am Post subject: Re: Free 100Mb+ of space!! |
|
|
Swishy wrote: | puddpunk wrote: | when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...
So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!
so, quickly that was out of there! and i had found another Gb of space!
All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.
Anybody else have some space-saving tips? |
Did you just del , the zipped source or blow away the dir and create a new one?????
Cheers
Dale. |
FYI, blow away the directory works for me, no problem encountered so far.
Code: | rm /usr/portage/distfiles/* && rm -r /var/tmp/portage/* |
Of course, you'd like to backup some 'unemergable' files like realplayer's rpm and sun-jdks, which required you to download manually. |
|
Back to top |
|
|
J0rd n00b
Joined: 09 Jul 2002 Posts: 19 Location: @home
|
Posted: Fri Jul 26, 2002 2:17 am Post subject: |
|
|
i think the emerge flag '--clean' does something like this doesnt it? _________________ /me out |
|
Back to top |
|
|
delta407 Bodhisattva
Joined: 23 Apr 2002 Posts: 2876 Location: Chicago, IL
|
Posted: Fri Jul 26, 2002 2:23 am Post subject: |
|
|
You're thinking of having "clean" in the FEATURES variable specified in /etc/make.conf. "emerge clean" is completely different. _________________ I don't believe in witty sigs. |
|
Back to top |
|
|
Kosmo n00b
Joined: 24 May 2002 Posts: 74
|
Posted: Sat Aug 03, 2002 9:41 pm Post subject: |
|
|
I have once thought about writing a script that would check the size of the distfiles directory, and if it found it was too big, deleting the files which were accessed the longest time ago, until the directory wasn't too big anymore.
This way, most of the files deleted will be the sources of packages who have totally new versions, not just new -r number.
Since i am currently installing Gentoo on a pc with a 4GB HD, i will probably write this script one of the next days, and post it here. |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20103
|
Posted: Sat Aug 03, 2002 9:46 pm Post subject: |
|
|
Good idea, then it can be put into a cron job. _________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
Kosmo n00b
Joined: 24 May 2002 Posts: 74
|
Posted: Sat Aug 03, 2002 10:02 pm Post subject: |
|
|
oh yeah, i forgot to mention it would be made for a cron job, good of you to remind me |
|
Back to top |
|
|
Kosmo n00b
Joined: 24 May 2002 Posts: 74
|
Posted: Sun Aug 04, 2002 12:08 am Post subject: |
|
|
I thought it was more difficult to code, but this seems to work.
I have put this script in /etc/cron.daily, you may want to put it in weekly or something, hourly seems overkill to me
Comments are welcome
Code: |
#!/bin/sh
# Directory to check
dir=/usr/portage/distfiles/
# Maximum size directory should be in kilobytes
# DONT SET THIS TOO LOW
maxsize=500000
#-------------------------------------------------------
size=`du $dir | cut -f 1`
cd $dir
while [ "$size" -gt "$maxsize" ]
do
echo "about to delete something"
rm -R `ls --time=access --sort=time $dir | tail -1`
size=`du $dir | cut -f 1`
done
|
|
|
Back to top |
|
|
cat5 n00b
Joined: 10 Apr 2002 Posts: 5 Location: Ottawa, On, Canada
|
Posted: Sat Aug 17, 2002 3:23 pm Post subject: |
|
|
I had the same problem of download the same source tarball at least 3 times. (3 gentoo boxes, live on the net)
Solution: Built another box, slapped it on the internal network and copy /usr/portage and nfs mount the partition.
There. I now have 1 computer keeping all the source tarballs (I WANT to keep them) and 3 computers to share them.
Why delete /usr/portage/distfiles especially when you have more than one box. When building new boxes, during the install, I just emerge portmap and run it, and move /usr/portage somewhere else, mount the drive and create the symlink needed to point to the right area on my nfs server (which is gentoo.. and has a TON of HD space) _________________ cat5@catfive.org |
|
Back to top |
|
|
Kosmo n00b
Joined: 24 May 2002 Posts: 74
|
Posted: Sat Aug 17, 2002 3:33 pm Post subject: |
|
|
If you keep your distfiles for a very long time, you will have tarballs of old packages. If you use the script i wrote, it will delete the least used packages, which presumably will be the ones that aren't used anymore. This needs a correct setting of the maxsize var for your enviroment, so it keeps all needed, but deletes unneeded. |
|
Back to top |
|
|
Oizoken n00b
Joined: 17 Apr 2002 Posts: 18
|
Posted: Wed Aug 21, 2002 8:30 am Post subject: |
|
|
had the same problem
made a quick fix (not perfect, but it works wel):
Code: |
#!/bin/bash
# cleans the old distfiles out
# wherever your distfile dir is, this is the default
FILEDIR="/usr/portage/distfiles"
# needed to filter out some files after cutting of version
# the cutting wasn't always correct
function filter
{
for ENTRY in $1
do
if echo $ENTRY | grep -vi $2 &> /dev/null
then
echo $ENTRY
fi
done | sort | uniq
}
# how many files with the same name exist?
function number
{
PACKAGE=$1
cd $FILEDIR
ls ${PACKAGE}* | wc -l
}
# sort the packages according to date, it's quite safe to assume
# that the oldest files can be removed in favor of the new ones
function datesort
{
PACKAGE=$1
typeset -i nr=$2
((nr=nr-1))
ls --sort=time ${PACKAGE}* | tail -n $nr
}
#get a list of different packages
cd $FILEDIR
LISTING=$(
for FILENAME in *
do
echo ${FILENAME%%-[0-9]*}
done
)
# filtering of incorrect version-cuts (note that .gz and .bz2 should have
# been cut of by the above command to cut of the version nrs)
# add any u like to keep eg multiple versions of let's say freetype
# or remove the packages from Xfree (Xsrc-1 -2 & -3 who gets
# cut incorrect)
# eg: LISTING=$(filter "$LISTING" "freetype")
# Note: Filtering is case-insensitive and regexp-ready
LISTING=$(filter "$LISTING" "patch")
LISTING=$(filter "$LISTING" ".gz")
LISTING=$(filter "$LISTING" ".bz2")
LISTING=$(filter "$LISTING" ".zip")
# actually see if there are multiple files for the same package and
# remove the oldest ones
for PACKAGE in $LISTING
do
typeset -i nr=$(number "$PACKAGE")
if (($nr>1))
then
REMOVABLES=$(datesort "$PACKAGE" "$nr")
for FILE in $REMOVABLES
do
echo "REMOVING "${PACKAGE}" : "${FILE}
rm -rf $FILE
done
fi
done
|
|
|
Back to top |
|
|
dingo n00b
Joined: 18 Aug 2002 Posts: 58
|
Posted: Mon Aug 26, 2002 10:43 pm Post subject: |
|
|
Kosmo wrote: | oh yeah, i forgot to mention it would be made for a cron job, good of you to remind me |
I actually put emere -rsync and emerge -u world in a weekly cron job, works nicely. |
|
Back to top |
|
|
Decibel n00b
Joined: 28 Aug 2002 Posts: 7 Location: Austin, TX
|
Posted: Wed Aug 28, 2002 7:04 pm Post subject: Can you cross-reference to the versions that are still valid |
|
|
FreeBSD has a similar issue with /usr/ports/distfiles, but they supply a utility (/usr/ports/Tools/scripts/distclean.sh) that will remove any distfiles that are no longer referenced by any ports. This is typically used to get rid of old versions, since ports doesn't support multiple versions like emerge does.
Maybe a good alternative for gentoo would be to remove if it's not currently installed? Or perhaps if it's more than X versions behind what's considered current.
Of course, if you don't care about nuking something you might have do eventually download, the scripts that have been presented work peachy.
BTW, another trick for those with multiple gentoo machines is to share the distfiles directory with multiple machines. In fact, if you use any other OS that grabs source files (such as FreeBSD ports), they should be able to share with gentoo just fine. (I'm sharing between FreeBSD and gentoo right now) |
|
Back to top |
|
|
wHAcKer Apprentice
Joined: 18 Oct 2002 Posts: 228 Location: Grimbergen, Belgium
|
Posted: Fri Nov 15, 2002 11:59 am Post subject: |
|
|
I kinda like to remove the whole directories, it gives me the feeling of getting rid of old crap (just like i did when i removed my windows partition). The fact that I have to download it again isn't really a problem because usually my cable connection gets impressive speeds |
|
Back to top |
|
|
pjp Administrator
Joined: 16 Apr 2002 Posts: 20103
|
Posted: Wed Nov 20, 2002 6:16 pm Post subject: |
|
|
disk full wrote: | DArtagnan wrote: | Under /var/tmp I have:
Code: |
$ pwd
/var/tmp
voicu@starshooter tmp $ ls
ccache portage portage-pkg rmerge2
|
Only this place takes 1Gb from my HD.
Is it safe to remove all these?
Thanks. |
puddpunk wrote: | You can blow away your portage tmp dir, i know that.
Run:
and it cleans out your ccache
I dont know what portage-pkg is, but im guessing you can nail your rmerge2 dir as well (only helps when your resuming world updates if i remember) |
sisyphus wrote: | I had exactly the same problem as I created too small a var partition. If you're in the same sit (and haven't got a monolithic / partition) you can take the same approach I did:
Code: | mkdir /[dir with space]/portage
mv /var/tmp/portage/* /[dir with space]/portage/
rm -rf /var/tmp/portage
ln -s /[dir with space]/portage /var/tmp/portage
|
There may be a better way of doing it (is there an environment variable you can set temporarily when emerging hungry apps?) so I'm quite prepared to be shot down for not RingTFM . That said, it worked for me and let me re-emerge my fried glibc.[/code] |
|
_________________ Quis separabit? Quo animo? |
|
Back to top |
|
|
ghetto Guru
Joined: 10 Jul 2002 Posts: 369 Location: BC, Canada
|
Posted: Wed Nov 20, 2002 6:38 pm Post subject: problem with non monolithic partitions |
|
|
I had a similar problem except mine was that my '/' partition was not large enough to support '/opt' so i moved '/opt' to my '/home' partiton and then linked it to where it should be. Then later i found out that if i put '/opt' on it own partition that i wont loose any of its data if i have to reinstall because the installation process doesnt need to write to '/opt' _________________ Blizzard you suck. |
|
Back to top |
|
|
S_aIN_t Guru
Joined: 11 May 2002 Posts: 488 Location: Ottawa
|
Posted: Sun Dec 08, 2002 11:18 pm Post subject: |
|
|
the funny thing is that i did this earlier today. someone in the thread pointer out that there is a similar system in freebsd. that is precisely why i remembred.
but anyway, i just to the deletion by hand. once a month or so.. why else have a big hdd? |
|
Back to top |
|
|
wyrickre n00b
Joined: 10 Jul 2002 Posts: 27 Location: Colorado Springs, CO
|
Posted: Wed Mar 05, 2003 1:53 am Post subject: |
|
|
Here's the solution I use... it's a perl script that attempts to only keep the latest version of a package's source in the distfiles directory. I call it cleandistfiles.pl
Code: |
#!/usr/bin/perl
use strict;
use warnings;
my $distdir = "/usr/portage/distfiles";
chdir($distdir) or die "Cannot cd to '$distdir'";
my @files;
while(<*>)
{
next if -d $_;
push @files, $_;
}
@files = reverse(sort(@files));
my $hash = {};
foreach my $fname (@files)
{
my $key = $fname;
$key =~ s/\.zip|\.tar\.(gz|bz2)|\.t(gz|bz2)$//;
$key =~ s/[-_ \d\.]//g;
if(exists $hash->{$key})
{
print "deleting $fname";
if(unlink($fname))
{
print "\n";
}
else
{
print " - cannot delete!\n";
}
}
else
{
$hash->{$key} = $fname;
}
}
|
_________________ Geek used to be a 4 letter word; Now it's a 6 figure one. |
|
Back to top |
|
|
Ari Rahikkala Guru
Joined: 02 Oct 2002 Posts: 370 Location: Finland
|
|
Back to top |
|
|
|