Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Free 100Mb+ of space!!
View unanswered posts
View posts from last 24 hours

Goto page 1, 2, 3  Next  
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks
View previous topic :: View next topic  
Author Message
puddpunk
l33t
l33t


Joined: 20 Jul 2002
Posts: 681
Location: New Zealand

PostPosted: Sun Jul 21, 2002 1:00 am    Post subject: Free 100Mb+ of space!! Reply with quote

when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...

So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!

so, quickly that was out of there! and i had found another Gb of space!

All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.

Anybody else have some space-saving tips?
Back to top
View user's profile Send private message
Naan Yaar
Bodhisattva
Bodhisattva


Joined: 27 Jun 2002
Posts: 1549

PostPosted: Sun Jul 21, 2002 2:29 am    Post subject: Re: Free 100Mb+ of space!! Reply with quote

/var/tmp/portage also gets filled up if you abort ebuilds. I killed a mozilla build and it left 14M worth of stuff in that dir alone!
puddpunk wrote:
when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...

So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!

so, quickly that was out of there! and i had found another Gb of space!

All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.

Anybody else have some space-saving tips?
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 20048

PostPosted: Tue Jul 23, 2002 11:01 pm    Post subject: Re: Free 100Mb+ of space!! Reply with quote

puddpunk wrote:
OVER 1Gb OF FILES!!!

so, quickly that was out of there! and i had found another Gb of space!
Though not critical, doing this does require you to download the stuff again if you reinstall it.
I emerge using -b so I have the binaries available. If I run out of diskspace, I'll just start backing
up that directory to a CD.


Also... 1G is definately more than 100Mb+ (as suggested in the post title). I was expecting to
find something freeing up 100M or so :D
_________________
Quis separabit? Quo animo?
Back to top
View user's profile Send private message
puddpunk
l33t
l33t


Joined: 20 Jul 2002
Posts: 681
Location: New Zealand

PostPosted: Wed Jul 24, 2002 8:47 am    Post subject: ! Reply with quote

Yea, i know. i just didnt want to say "FREE 1GB of space!" and then get flamed because ppl only had 100mb of crud in their distfiles dir ;)
Back to top
View user's profile Send private message
puddpunk
l33t
l33t


Joined: 20 Jul 2002
Posts: 681
Location: New Zealand

PostPosted: Wed Jul 24, 2002 8:48 am    Post subject: ? Reply with quote

is it safe to kill all the stuff in /var/tmp/portage?
Back to top
View user's profile Send private message
Naan Yaar
Bodhisattva
Bodhisattva


Joined: 27 Jun 2002
Posts: 1549

PostPosted: Wed Jul 24, 2002 11:34 am    Post subject: Reply with quote

Yes. I believe of less consequence than /usr/portage/distfiles.
Back to top
View user's profile Send private message
Swishy
Guru
Guru


Joined: 06 Jun 2002
Posts: 491
Location: NZ

PostPosted: Wed Jul 24, 2002 3:13 pm    Post subject: Re: Free 100Mb+ of space!! Reply with quote

puddpunk wrote:
when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...

So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!

so, quickly that was out of there! and i had found another Gb of space!

All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.

Anybody else have some space-saving tips?

Did you just del , the zipped source or blow away the dir and create a new one?????

Cheers
Dale.
_________________
Theres no substitute for C.I.
Back to top
View user's profile Send private message
debian
n00b
n00b


Joined: 24 Jun 2002
Posts: 22

PostPosted: Thu Jul 25, 2002 1:09 am    Post subject: Re: Free 100Mb+ of space!! Reply with quote

Swishy wrote:
puddpunk wrote:
when i noticed that my /usr partition was filling up rather quickly, i got to thinking: Even redhat didnt use up that much space...

So i sniffed around, then I found the directory /usr/portage/distfiles full with OVER 1Gb OF FILES!!!

so, quickly that was out of there! and i had found another Gb of space!

All the files in there are the files that portage has downloaded, but it doesnt get rid of it after you compiled in case you want to do it again, it will pluck it out of /usr/portage/distfiles instead of downloading it again.

Anybody else have some space-saving tips?

Did you just del , the zipped source or blow away the dir and create a new one?????

Cheers
Dale.


FYI, blow away the directory works for me, no problem encountered so far.
Code:
rm /usr/portage/distfiles/* && rm -r /var/tmp/portage/*

Of course, you'd like to backup some 'unemergable' files like realplayer's rpm and sun-jdks, which required you to download manually. :)
Back to top
View user's profile Send private message
J0rd
n00b
n00b


Joined: 09 Jul 2002
Posts: 19
Location: @home

PostPosted: Fri Jul 26, 2002 2:17 am    Post subject: Reply with quote

i think the emerge flag '--clean' does something like this doesnt it?
_________________
/me out
Back to top
View user's profile Send private message
delta407
Bodhisattva
Bodhisattva


Joined: 23 Apr 2002
Posts: 2876
Location: Chicago, IL

PostPosted: Fri Jul 26, 2002 2:23 am    Post subject: Reply with quote

You're thinking of having "clean" in the FEATURES variable specified in /etc/make.conf. "emerge clean" is completely different.
_________________
I don't believe in witty sigs.
Back to top
View user's profile Send private message
Kosmo
n00b
n00b


Joined: 24 May 2002
Posts: 74

PostPosted: Sat Aug 03, 2002 9:41 pm    Post subject: Reply with quote

I have once thought about writing a script that would check the size of the distfiles directory, and if it found it was too big, deleting the files which were accessed the longest time ago, until the directory wasn't too big anymore.

This way, most of the files deleted will be the sources of packages who have totally new versions, not just new -r number.

Since i am currently installing Gentoo on a pc with a 4GB HD, i will probably write this script one of the next days, and post it here.
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 20048

PostPosted: Sat Aug 03, 2002 9:46 pm    Post subject: Reply with quote

Good idea, then it can be put into a cron job.
_________________
Quis separabit? Quo animo?
Back to top
View user's profile Send private message
Kosmo
n00b
n00b


Joined: 24 May 2002
Posts: 74

PostPosted: Sat Aug 03, 2002 10:02 pm    Post subject: Reply with quote

oh yeah, i forgot to mention it would be made for a cron job, good of you to remind me :-)
Back to top
View user's profile Send private message
Kosmo
n00b
n00b


Joined: 24 May 2002
Posts: 74

PostPosted: Sun Aug 04, 2002 12:08 am    Post subject: Reply with quote

I thought it was more difficult to code, but this seems to work. :D
I have put this script in /etc/cron.daily, you may want to put it in weekly or something, hourly seems overkill to me :wink:

Comments are welcome

Code:

#!/bin/sh

# Directory to check
dir=/usr/portage/distfiles/

# Maximum size directory should be in kilobytes
# DONT SET THIS TOO LOW
maxsize=500000

#-------------------------------------------------------

size=`du $dir | cut -f 1`
cd $dir

while [ "$size" -gt "$maxsize" ]
do

        echo "about to delete something"
        rm -R `ls --time=access --sort=time $dir | tail -1`
        size=`du $dir | cut -f 1`

done
Back to top
View user's profile Send private message
cat5
n00b
n00b


Joined: 10 Apr 2002
Posts: 5
Location: Ottawa, On, Canada

PostPosted: Sat Aug 17, 2002 3:23 pm    Post subject: Reply with quote

I had the same problem of download the same source tarball at least 3 times. (3 gentoo boxes, live on the net)

Solution: Built another box, slapped it on the internal network and copy /usr/portage and nfs mount the partition.

There. I now have 1 computer keeping all the source tarballs (I WANT to keep them) and 3 computers to share them.

Why delete /usr/portage/distfiles especially when you have more than one box. When building new boxes, during the install, I just emerge portmap and run it, and move /usr/portage somewhere else, mount the drive and create the symlink needed to point to the right area on my nfs server (which is gentoo.. and has a TON of HD space)
_________________
cat5@catfive.org
Back to top
View user's profile Send private message
Kosmo
n00b
n00b


Joined: 24 May 2002
Posts: 74

PostPosted: Sat Aug 17, 2002 3:33 pm    Post subject: Reply with quote

If you keep your distfiles for a very long time, you will have tarballs of old packages. If you use the script i wrote, it will delete the least used packages, which presumably will be the ones that aren't used anymore. This needs a correct setting of the maxsize var for your enviroment, so it keeps all needed, but deletes unneeded.
Back to top
View user's profile Send private message
Oizoken
n00b
n00b


Joined: 17 Apr 2002
Posts: 18

PostPosted: Wed Aug 21, 2002 8:30 am    Post subject: Reply with quote

had the same problem
made a quick fix (not perfect, but it works wel):

Code:

#!/bin/bash

# cleans the old distfiles out

# wherever your distfile dir is, this is the default
FILEDIR="/usr/portage/distfiles"

# needed to filter out some files after cutting of version
# the cutting wasn't always correct

function filter
{
        for ENTRY in $1
        do
                if echo $ENTRY | grep -vi $2 &> /dev/null
                then
                        echo $ENTRY
                fi
        done | sort | uniq
}

# how many files with the same name exist?
function number
{
        PACKAGE=$1
        cd $FILEDIR
        ls ${PACKAGE}* | wc -l
}

# sort the packages according to date, it's quite safe to assume
# that the oldest files can be removed in favor of the new ones

function datesort
{
        PACKAGE=$1
        typeset -i nr=$2
        ((nr=nr-1))
        ls --sort=time ${PACKAGE}* | tail -n $nr
}

#get a list of different packages

cd $FILEDIR

LISTING=$(
for FILENAME in *
do
        echo ${FILENAME%%-[0-9]*}
done
)

# filtering of incorrect version-cuts (note that .gz and .bz2 should have
# been cut of by the above command to cut of the version nrs)
# add any u like to keep eg multiple versions of let's say freetype
# or remove the packages from Xfree (Xsrc-1 -2 & -3 who gets
# cut incorrect)
# eg: LISTING=$(filter "$LISTING" "freetype")
# Note: Filtering is case-insensitive and regexp-ready

LISTING=$(filter "$LISTING" "patch")
LISTING=$(filter "$LISTING" ".gz")
LISTING=$(filter "$LISTING" ".bz2")
LISTING=$(filter "$LISTING" ".zip")

# actually see if there are multiple files for the same package and
# remove the oldest ones

for PACKAGE in $LISTING
do
        typeset -i nr=$(number "$PACKAGE")
        if (($nr>1))
        then
                REMOVABLES=$(datesort "$PACKAGE" "$nr")
                for FILE in $REMOVABLES
                do
                        echo "REMOVING "${PACKAGE}" : "${FILE}
                        rm -rf $FILE
                done
        fi
done
Back to top
View user's profile Send private message
dingo
n00b
n00b


Joined: 18 Aug 2002
Posts: 58

PostPosted: Mon Aug 26, 2002 10:43 pm    Post subject: Reply with quote

Kosmo wrote:
oh yeah, i forgot to mention it would be made for a cron job, good of you to remind me :-)


I actually put emere -rsync and emerge -u world in a weekly cron job, works nicely.
Back to top
View user's profile Send private message
Decibel
n00b
n00b


Joined: 28 Aug 2002
Posts: 7
Location: Austin, TX

PostPosted: Wed Aug 28, 2002 7:04 pm    Post subject: Can you cross-reference to the versions that are still valid Reply with quote

FreeBSD has a similar issue with /usr/ports/distfiles, but they supply a utility (/usr/ports/Tools/scripts/distclean.sh) that will remove any distfiles that are no longer referenced by any ports. This is typically used to get rid of old versions, since ports doesn't support multiple versions like emerge does.

Maybe a good alternative for gentoo would be to remove if it's not currently installed? Or perhaps if it's more than X versions behind what's considered current.

Of course, if you don't care about nuking something you might have do eventually download, the scripts that have been presented work peachy. :)

BTW, another trick for those with multiple gentoo machines is to share the distfiles directory with multiple machines. In fact, if you use any other OS that grabs source files (such as FreeBSD ports), they should be able to share with gentoo just fine. (I'm sharing between FreeBSD and gentoo right now)
Back to top
View user's profile Send private message
wHAcKer
Apprentice
Apprentice


Joined: 18 Oct 2002
Posts: 228
Location: Grimbergen, Belgium

PostPosted: Fri Nov 15, 2002 11:59 am    Post subject: Reply with quote

I kinda like to remove the whole directories, it gives me the feeling of getting rid of old crap (just like i did when i removed my windows partition). The fact that I have to download it again isn't really a problem because usually my cable connection gets impressive speeds ;)
Back to top
View user's profile Send private message
pjp
Administrator
Administrator


Joined: 16 Apr 2002
Posts: 20048

PostPosted: Wed Nov 20, 2002 6:16 pm    Post subject: Reply with quote

disk full wrote:
DArtagnan wrote:
Under /var/tmp I have:
Code:

$ pwd
/var/tmp
voicu@starshooter tmp $ ls
ccache  portage  portage-pkg  rmerge2


Only this place takes 1Gb from my HD.
Is it safe to remove all these?
Thanks.
puddpunk wrote:
You can blow away your portage tmp dir, i know that.

Run:
Code:
# ccache -c

and it cleans out your ccache

I dont know what portage-pkg is, but im guessing you can nail your rmerge2 dir as well (only helps when your resuming world updates if i remember)
sisyphus wrote:
I had exactly the same problem as I created too small a var partition. If you're in the same sit (and haven't got a monolithic / partition) you can take the same approach I did:

Code:
mkdir /[dir with space]/portage
mv /var/tmp/portage/* /[dir with space]/portage/
rm -rf /var/tmp/portage
ln -s /[dir with space]/portage /var/tmp/portage


There may be a better way of doing it (is there an environment variable you can set temporarily when emerging hungry apps?) so I'm quite prepared to be shot down for not RingTFM :-). That said, it worked for me and let me re-emerge my fried glibc.[/code]

_________________
Quis separabit? Quo animo?
Back to top
View user's profile Send private message
ghetto
Guru
Guru


Joined: 10 Jul 2002
Posts: 369
Location: BC, Canada

PostPosted: Wed Nov 20, 2002 6:38 pm    Post subject: problem with non monolithic partitions Reply with quote

I had a similar problem except mine was that my '/' partition was not large enough to support '/opt' so i moved '/opt' to my '/home' partiton and then linked it to where it should be. Then later i found out that if i put '/opt' on it own partition that i wont loose any of its data if i have to reinstall because the installation process doesnt need to write to '/opt'
_________________
Blizzard you suck.
Back to top
View user's profile Send private message
S_aIN_t
Guru
Guru


Joined: 11 May 2002
Posts: 488
Location: Ottawa

PostPosted: Sun Dec 08, 2002 11:18 pm    Post subject: Reply with quote

the funny thing is that i did this earlier today. someone in the thread pointer out that there is a similar system in freebsd. that is precisely why i remembred.

but anyway, i just to the deletion by hand. once a month or so.. why else have a big hdd? :D
Back to top
View user's profile Send private message
wyrickre
n00b
n00b


Joined: 10 Jul 2002
Posts: 27
Location: Colorado Springs, CO

PostPosted: Wed Mar 05, 2003 1:53 am    Post subject: Reply with quote

Here's the solution I use... it's a perl script that attempts to only keep the latest version of a package's source in the distfiles directory. I call it cleandistfiles.pl
Code:

#!/usr/bin/perl

use strict;
use warnings;

my $distdir = "/usr/portage/distfiles";

chdir($distdir) or die "Cannot cd to '$distdir'";

my @files;
while(<*>)
{
    next if -d $_;
    push @files, $_;
}

@files = reverse(sort(@files));
my $hash = {};

foreach my $fname (@files)
{
    my $key = $fname;
    $key =~ s/\.zip|\.tar\.(gz|bz2)|\.t(gz|bz2)$//;
    $key =~ s/[-_ \d\.]//g;
    if(exists $hash->{$key})
    {
        print "deleting $fname";
        if(unlink($fname))
        {
            print "\n";
        }
        else
        {
            print " - cannot delete!\n";
        }
    }
    else
    {
        $hash->{$key} = $fname;
    }
}

_________________
Geek used to be a 4 letter word; Now it's a 6 figure one.
Back to top
View user's profile Send private message
Ari Rahikkala
Guru
Guru


Joined: 02 Oct 2002
Posts: 370
Location: Finland

PostPosted: Wed Mar 05, 2003 11:45 am    Post subject: Reply with quote

https://forums.gentoo.org/viewtopic.php?t=3011
_________________
<laurentius> gentoo linux?
<ari> Yesh.
<laurentius> they look horny
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Documentation, Tips & Tricks All times are GMT
Goto page 1, 2, 3  Next
Page 1 of 3

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum