Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Cleaning out stale distfiles
View unanswered posts
View posts from last 24 hours

Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next  
Reply to topic    Gentoo Forums Forum Index Unsupported Software
View previous topic :: View next topic  
Author Message
mlybarger
Guru
Guru


Joined: 04 Sep 2002
Posts: 473

PostPosted: Tue Feb 04, 2003 1:25 am    Post subject: wiped out /usr/portage/distfiles Reply with quote

well jefftang, i tried the perl script, and my /usr/portage/distfiles is completely empty.
Back to top
View user's profile Send private message
arkane
l33t
l33t


Joined: 30 Apr 2002
Posts: 918
Location: Phoenix, AZ

PostPosted: Tue Feb 04, 2003 3:42 am    Post subject: Reply with quote

nice script, I just tried it and it works as advertised.
Back to top
View user's profile Send private message
jefftang
n00b
n00b


Joined: 04 Jan 2003
Posts: 30

PostPosted: Tue Feb 04, 2003 5:32 pm    Post subject: Re: wiped out /usr/portage/distfiles Reply with quote

Ooh. Sorry about that. I guess I should have posted a disclaimer with it. Better late than never.

"This script has was developed run and tested on exactly one gentoo machine by someone new to gentoo. This script erases files. It could potentially erase almost anything on your system. Run it with the -p flag first."

That said, did you run my version or robm's. I'm running robm's as it seems to have better error checking and handles different systems better. Happily /usr/portage/distfiles is not really a terribly vital directory. I think (see disclaimer) that

Code:

emerge -f -e world


will repopulate it with the correct files. And it's interruptible unlike trying to rebuild everything.
Let me know if you want to figure out what went wrong with the script.


mark_lybarger wrote:
well jefftang, i tried the perl script, and my /usr/portage/distfiles is completely empty.
Back to top
View user's profile Send private message
needlern1
Guru
Guru


Joined: 16 Jul 2002
Posts: 376
Location: Marietta, Ga 30068

PostPosted: Wed Feb 05, 2003 1:38 pm    Post subject: Reply with quote

Have just read through this thread. Would like to clean up my distfiles. Is there a consensus as to what script I should use? Is kerframil's now in its end stage and useable? Thanks, Bill
Back to top
View user's profile Send private message
jefftang
n00b
n00b


Joined: 04 Jan 2003
Posts: 30

PostPosted: Wed Feb 05, 2003 5:20 pm    Post subject: re: Which script? Reply with quote

Well, obviously I had some concerns with kerframil's script or I wouldn't have written my own. These might be wrong since I'm still somewhat new to gentoo, but kerfamil's script:


    Only works on tarballs: There are a number of files in distfiles that aren't tar files that will never get cleaned out.

    Always erases older version: Sometimes there's more than one version of a package installed and you might need both sets of files

    Doesn't clean out removed files: Even if you remove a package the tar files will stick around.

    Uses special cases: There's a list of files to be ignored. Always a bad idea to have something like that if you can help it.


My script fixes all these problems by going about it differently. Don't use mine; use robm's version as it handles some things much better.

Also there seem to be some patches that reference /usr/portage/packages/All which doesn't appear to exist on my system and that just confuses me.

If I'm wrong about any of this please let me know.

needlern1 wrote:
Have just read through this thread. Would like to clean up my distfiles. Is there a consensus as to what script I should use? Is kerframil's now in its end stage and useable? Thanks, Bill
Back to top
View user's profile Send private message
needlern1
Guru
Guru


Joined: 16 Jul 2002
Posts: 376
Location: Marietta, Ga 30068

PostPosted: Thu Feb 06, 2003 1:17 am    Post subject: Reply with quote

Thanks jefftang. After I clear up another problem, I'll use robm's. Bill :)
Back to top
View user's profile Send private message
Penulci
Tux's lil' helper
Tux's lil' helper


Joined: 12 Jun 2002
Posts: 119
Location: Vienna, Austria

PostPosted: Fri Mar 07, 2003 10:38 pm    Post subject: Cleaning out stale distfiles (Kerframil's Script) Reply with quote

I used your script which really works fine. I guess I found a little "error" in your script. It uses to delete some newer files and keeps the older ones. The problem seems to occur if the version number contains two position numbers. I will post an example:


Example:

...
Deleted: linux-2.4.19 in favour of linux-2.4.20
Deleted: micq-0.4.10.1 in favour of micq-0.4.9.4 (it seems the script "thinks" 10 is smaller than 9)
Deleted: mrtg-2.9.21 in favour of mrtg-2.9.22
...

The problem is, I am not really a Perl Expert otherwise I would have changed the line and posted it.

greetings Penulci
Back to top
View user's profile Send private message
far
Guru
Guru


Joined: 10 Mar 2003
Posts: 394
Location: Stockholm, Sweden

PostPosted: Mon Mar 10, 2003 12:57 am    Post subject: Reply with quote

At the risk of starting a Python vs. Perl flame-war :) I am submitting a Python version of robm's script. It should do just about the same thing, but the ususal disclaimers apply. Use at your own peril!
BTW, I'm not trying to make a statement here, I just like Python, thats all.

Code:
#!/usr/bin/env python
import sys
import re
import string
import os
import os.path
import commands
import getopt

make_conf = {}
make_conf['PORTDIR'] = '/usr/portage'
make_conf['DISTDIR'] = '/usr/portage/distfiles'

def read_make_conf(make_conf):
    file = open('/etc/make.conf', 'r')
    while 1:
        line = file.readline()
        if line == '':
            break
        match = re.search('^\s*(\w+)=(.+)$', line)
        if match != None:
            make_conf[match.group(1)] = match.group(2)
    file.close()

    for var, value in make_conf.items():
        while 1:
            match = re.search('"?\$\{(\w+)\}"?', make_conf[var])
            if match == None:
                break
            make_conf[var] = re.sub('"?\$\{' + match.group(1) + '\}"?',
                                    make_conf[match.group(1)],
                                    value)
    return make_conf

opt_p = 0
try:
    if len(sys.argv) > 1:
        opt_p = getopt.getopt(sys.argv[1:], "p")[0][0][0] == '-p'
except getopt.GetoptError:
    print "oj!"
    pass

make_conf = read_make_conf(make_conf)
           
print 'PORTDIR =', make_conf['PORTDIR']
print 'DISTDIR =', make_conf['DISTDIR']
if make_conf.has_key('PORTDIR_OVERLAY'):
    print 'PORTDIR_OVERLAY =', make_conf['PORTDIR_OVERLAY']
else:
    print 'PORTDIR_OVERLAY ='
   
files = {}
status, packages = commands.getstatusoutput('qpkg -v -I -nc')
if status != 0:
    sys.exit("Can't list packages\n")
for package in string.split(packages):
    match = re.search('(.*)/(.*)-(\d.*)', package)
    category, program, version = match.group(1, 2, 3)
    digest_suffix = ('/' + category
                     + '/' + program + '/files/digest-' + program
                     + '-' + version)
    digest = make_conf['PORTDIR'] + digest_suffix
    if (make_conf.has_key('PORTDIR_OVERLAY')
        and (not os.path.isfile(digest))):
        digest = make_conf['PORTDIR_OVERLAY'] + digest_suffix
    if os.path.isfile(digest):
        file = open(digest, 'r')
        while 1:
            line = file.readline()
            if line == '':
                break
            fields = string.split(line)
            if len(fields) == 4:
                [hashtype, hash, filename, size] = fields
                files[filename] = 1
        file.close()

if len(files.keys()) == 0:
    sys.exit("sanity check: no package files found.  This can't be right.\n")

try:
    list = os.listdir(make_conf['DISTDIR'])
except os.OSError:
    sys.exit("can't open " + make_conf['DISTDIR'])

for file in list:
    abs_file = make_conf['DISTDIR'] + '/' + file
    if (os.path.isfile(abs_file) and (not os.path.islink(abs_file))
        and (not (files.has_key(file) and files[file]))):
        if opt_p:
            print 'Would erase', abs_file
        else:
            os.remove(abs_file)
Back to top
View user's profile Send private message
drizzt
Guru
Guru


Joined: 21 Jul 2002
Posts: 428

PostPosted: Mon Mar 10, 2003 5:42 am    Post subject: Why not ??? Reply with quote

...implement the feature of deleting old unused sourcefiles in portage directly like this:

emerge -up world

bla bla (emerges newer package)

package old-bla slated for removal...
package old-bla unmerged safely...
sourcefile bla deleted safely...

this would keep the tree up-to-date in a safe way, wouldn't it ???
_________________
People don't have to earn my respect. I offer my respect to them, but be careful to lose my respect...
Back to top
View user's profile Send private message
GurliGebis
Retired Dev
Retired Dev


Joined: 08 Aug 2002
Posts: 509

PostPosted: Mon Mar 10, 2003 1:08 pm    Post subject: Reply with quote

hmm, it might be at bit difficult since I'm using NFS on 3 computers, so they have /usr/portage/distfiles on the same computer.
There might be some problems with this then.
_________________
Queen Rocks.
Back to top
View user's profile Send private message
dreamer3
Guru
Guru


Joined: 24 Sep 2002
Posts: 553

PostPosted: Wed Mar 12, 2003 3:43 am    Post subject: Re: Why not ??? Reply with quote

drizzt wrote:
...implement the feature of deleting old unused sourcefiles in portage directly like this:

package old-bla slated for removal...
package old-bla unmerged safely...
sourcefile bla deleted safely...

this would keep the tree up-to-date in a safe way, wouldn't it ???

Unless ones likes to keep X copies of old source around just in case... X for me = 1 or 2. I'm dialup so I don't delete things that take hours to download all that often...
Back to top
View user's profile Send private message
Doctor
n00b
n00b


Joined: 11 Feb 2003
Posts: 26

PostPosted: Tue Mar 18, 2003 2:42 am    Post subject: Reply with quote

Feel free to diss my bad code, but I decided to do this a different way. This one only deletes files that aren't used by any packages currently in portage. Personally I call this script distfiles...
Code:
#!/usr/bin/perl -w
use Getopt::Std;
use File::Basename;
getopts('pvhld');

# Step 1: create a list of ALL files in portage, dupecheck
# Step 2: list files in /usr/portage/distfiles that aren't on the above list
# Step 3: optionally delete those files

$distfilesdir = "/usr/portage/distfiles";
$portagecachedir = "/var/cache/edb/dep";

if($opt_h || !($opt_p || $opt_d || $opt_l)) {
  $opt_d=$opt_h; # I'm too lazy to make this not warn about $opt_h being
  $opt_p=$opt_h; # used once the "proper" way...
  print STDERR "Usage: $0 [OPTION]...\n";
  print STDERR "What does this program do?\n\n";
  print STDERR "  -d Delete old files that are no longer needed in $distfilesdir\n";
  print STDERR "  -p List old files that would be deleted if -d was used\n";
  print STDERR "  -l List all of the possible files in $distfilesdir\n";
  print STDERR "  -v Verbose Mode\n";
  print STDERR "  -h Display this help and exit\n";
  print STDERR "\nReport bugs to <mage\@shadowmage.org>.\n";
  exit;
}

if ((!-d $portagecachedir) || (!-d $distfilesdir)) {
print "I can\'t find either $distfilesdir or $portagecachedir\n";
print "I\'m designed for gentoo's portage ;)\n";
exit;
}

opendir(DIR, $portagecachedir);
my @sections=sort(readdir(DIR));
closedir(DIR);

%philes = ();

foreach (@sections) {
  if(!/^\./) {
    my $section=$_;
    if ($opt_v) { print "$section\n"; }
    opendir(DIR, "$portagecachedir/$section");
    my @ebuilds=sort(readdir(DIR));
    closedir(DIR);
    foreach (@ebuilds) {
      if(!/^\./) {
        my $ebuild=$_;
        if ($opt_v) { print "$section/$ebuild\n"; }
        open(FILE, "$portagecachedir/$section/$ebuild");
        my @mom = <FILE>;
        close(FILE);
        $temp=join("", @mom);
        @mom = split(/\n/,$temp);
        #chomp(@mom);
        #print $mom[3];
        @files = split(/ /,$mom[3]);
        foreach(@files) {
          # dont do things that are (, ) or end with a ? because thats part of
          # portages logic
          if (!/^\(/ && !/^\)/ && !/\?$/) {
            $temp=basename($_);
            $philes{$temp} = "$ebuild";
          }
        }
      }
    }
  }
}
if ($opt_l) {
  @keys = sort keys %philes;
  foreach (@keys) {
    print $philes{$_} . ": $_\n";
  }
  $opt_l=$opt_l;
}
elsif ($opt_p || $opt_d) {
  opendir(DIR, $distfilesdir);
  my @files = readdir(DIR);
  closedir(DIR);
  foreach(@files) {
    if (!/^\./ && !$philes{$_} && !-d "$distfilesdir/$_") {
      $yayflag=1;
      if ($opt_d) { unlink($distfilesdir."/".$_); }
      else { print "$_\n"; }
    }
  }
  if (!$yayflag) {
    print "Nothing found to remove\n";
  }
}
Back to top
View user's profile Send private message
markyd
n00b
n00b


Joined: 16 Apr 2002
Posts: 44

PostPosted: Tue Mar 18, 2003 4:10 am    Post subject: A neater alternative Reply with quote

Hey guys,
This is the code I use and i find that it works perfectly, at least on my system. :D

It will even download the newer/missing sources if they were not stored in your dist directory.

Code:

DIST_TMP="/tmp/olddist"
DIST_DIR="/usr/portage/distfiles"
OTHER_PKG="lftp emu10k1"

mkdir $DIST_TMP
mv $DIST_DIR/* $DIST_TMP

export FETCHCOMMAND="/bin/mv $DIST_TMP/\`/bin/echo \${URI}|/bin/sed -e 's/^.*\/\([^/]*\)$/\1/'\` \${DISTDIR} || /usr/bin/wget -t 5 --passive-ftp \${URI} -P \${DISTDIR}"
export RESUMECOMMAND="/bin/mv  $DIST_TMP/\`/bin/echo \${URI}|/bin/sed -e 's/^.*\/\([^/]*\)$/\1/'\` \${DISTDIR} && /usr/bin/wget -t 5 --passive-ftp \${URI} -P \${DISTDIR}"

emerge -e --update --deep --fetchonly world
emerge -e --update --deep --fetchonly $OTHER_PKG

unset RESUMECOMMAND FETCHCOMMAND
echo "$DIST_DIR is now clean of stale files"
echo "Please review then delete $DIST_TMP by typing rm -rf $DIST_TMP"
echo "Packages like dev-java/sun-jdk do not correctly list source files needed"


Just set the variables and away you go.

DIST_DIR is where your dist files are stored.
DIST_TMP is a temp directory for storing the dist files (bust not be subdir of DIST_DIR).
OTHER_PKG is other sources that are not part of world that you want to keep.

Enjoy,
Mark
Back to top
View user's profile Send private message
stimpy
n00b
n00b


Joined: 18 Apr 2002
Posts: 49
Location: NZ

PostPosted: Thu Apr 10, 2003 2:43 am    Post subject: slight improvement(?) to python version Reply with quote

Far writes previous about his python version. To me using python makes sense as you can inherit all the portage stuff - you get a whole lot for free.
Which leads me to my suggestion: Instead of parsing the make.conf yourself, why not import the portage python package, and use the portage.settings array? I.e. replace the make_conf and read_make_conf "stuff" with something like
Code:
import portage
make_conf = {}
make_conf['PORTDIR'] = portage.settings["PORTDIR"]
make_conf['DISTDIR'] = portage.settings["DISTDIR"]
make_conf['PORTDIR_OVERLAY'] = portage.settings["PORTDIR_OVERLAY"]

Of course it may slow things down somewhat (inheriting a lot more than it needs in the portage package) but gains a whole lot in that you don't need to know anything about where your portage setting are made. (And for me, far's code kept barfing as it had problems with my FETCHCOMMAND make.conf setting.)
BTW I'm no Python expert, so I hope I'm using the right terms etc.
Back to top
View user's profile Send private message
far
Guru
Guru


Joined: 10 Mar 2003
Posts: 394
Location: Stockholm, Sweden

PostPosted: Thu Apr 24, 2003 9:10 am    Post subject: Reply with quote

Quote:
Which leads me to my suggestion: Instead of parsing the make.conf yourself, why not import the portage python package,


You're quite right. Here is a shorter version:

Code:

#!/usr/bin/env python
import sys, re, os, os.path, getopt, portage

def visit(dummy, dirname, files):
    global packages
    for file in files:
        (base, ext) = os.path.splitext(file)
        if (ext == ".ebuild"):
            packages.append(re.sub('^\/([^\/]+\/){3,3}|\/[^\/]+$', '', dirname)
                            + '/' + base)

opt_p = 0
try:
    if len(sys.argv) > 1:
        opt_p = getopt.getopt(sys.argv[1:], "p")[0][0][0] == '-p'
except getopt.GetoptError:
    pass

conf = portage.config()
portdir = conf.environ()['PORTDIR']
distdir = conf.environ()['DISTDIR']
portdir_overlay = ''
if conf.environ().has_key('PORTDIR_OVERLAY'):
    portdir_overlay =  conf.environ()['PORTDIR_OVERLAY']
print 'PORTDIR =', portdir
print 'DISTDIR =', distdir
print 'PORTDIR_OVERLAY =', portdir_overlay
   
files = {}

installRoot = '/var/db/pkg'
packages = []
dummy = ""
os.path.walk(installRoot, visit, dummy)

for package in packages:
    match = re.search('(.*)/(.*)-(\d.*)', package)
    category, program, version = match.group(1, 2, 3)
    digest_suffix = ('/' + category
                     + '/' + program + '/files/digest-' + program
                     + '-' + version)
    digest = portdir + digest_suffix
    if portdir_overlay and (not os.path.isfile(digest)):
        digest = portdir_overlay + digest_suffix
    if os.path.isfile(digest):
        file = open(digest, 'r')
        while 1:
            line = file.readline()
            if not line:
                break
            fields = line.split()
            if len(fields) == 4:
                [hashtype, hash, filename, size] = fields
                files[filename] = 1
        file.close()

if len(files.keys()) == 0:
    sys.exit("sanity check: no package files found.  This can't be right.\n")

try:
    list = os.listdir(distdir)
except os.OSError:
    sys.exit("can't open " + distdir)

for file in list:
    abs_file = distdir + '/' + file
    if (os.path.isfile(abs_file) and (not os.path.islink(abs_file))
        and (not (files.has_key(file) and files[file]))):
        if opt_p:
            print 'Would erase', abs_file
        else:
            os.remove(abs_file)
Back to top
View user's profile Send private message
compu-tom
Guru
Guru


Joined: 09 Jan 2003
Posts: 415
Location: Berlin, Germany

PostPosted: Fri Apr 25, 2003 5:50 pm    Post subject: Reply with quote

far wrote:
Here is a shorter version:

Great work! :)
Back to top
View user's profile Send private message
juparave
n00b
n00b


Joined: 22 Jan 2003
Posts: 9

PostPosted: Sat Apr 26, 2003 1:17 am    Post subject: Java Reply with quote

Just wondering, what happends with the sun j2sdk files? large zip files that are not auto-dowloaded?

Greetings
Back to top
View user's profile Send private message
RoadRunner
Bodhisattva
Bodhisattva


Joined: 12 Jun 2002
Posts: 490
Location: Coimbra, Portugal

PostPosted: Sun Apr 27, 2003 1:21 am    Post subject: Reply with quote

I've gone through some of those scripts and there seem to be different aproaches, all of them work for a single machine. But I think for something like this to be applied into Portage, it needs to work in any possible situation. Including mine =) Here's my point of view:

- Making a tmp and copying files there it's not a good idea, too much extra space needed.

- Imagine you have several machines and a central machine where you have the distfiles, shared to the other machines using NFS (or samba, whatever). We beed to make sure we don't delete files that are installed on one system but not on another.

- Now, let's make things harder and suppose we have different machines, like one using stable Gentoo and another using testing. We need to have several versions of the same software, otherwise, if a new -r version comes out, it will have to download again. The case is also possible if you have different arch machines.

- Also, there are situations (specially on ~arch) where a version get's downgraded due to problems with the new one. Again, we'd have to download the old version again.

There may be even more possible situations, where this scripts don't work as we hope.

Now, some seconds thinking about this brought the obvious to me. Why don't we delete only the versions that are not on the portage tree? we know that the rsync at the end deletes obsolete ebuilds, why doesn't it check for the distfiles also? This wat we had our problem solved. It's not perfect, but it would minimize the probability of deleting a package that may be needed.

I don't know how this could be implemented, but I posted my idea anyway so you can comment it.
Back to top
View user's profile Send private message
erik_swanson
Retired Dev
Retired Dev


Joined: 02 Feb 2003
Posts: 123
Location: Corvallis, OR USA

PostPosted: Sun May 11, 2003 11:01 am    Post subject: Reply with quote

This is a "one-liner" I wrote to safely clean out obsolete distfiles that are no longer referenced by any ebuild in portage.

Code:

bash <(comm -23 <(find /usr/portage/distfiles/ -maxdepth 1 -type f -printf %f\\n | sort) <(find /usr/portage/ /usr/local/portage/ -path '*/*/files/digest-*' -print0 | xargs -0 awk '{print $3}' | sort -u) | sed -e 's|^|rm -i /usr/portage/distfiles/|')



If your portage tree, distfiles, or portdir_overlay are in locations other than the norm (/usr/portage, /usr/portage/distfiles, /usr/local/portage, respectively), you will need to edit the one-liner accordingly.

If it wants to delete any distfiles that are actually used by an ebuild, there is a problem with said ebuild. Most likely, the ebuild requires a distfile to be retrieved manually and hasn't been updated to use RESTRICT="fetch".
Back to top
View user's profile Send private message
alec
Apprentice
Apprentice


Joined: 19 Apr 2002
Posts: 270
Location: Here

PostPosted: Sun May 11, 2003 3:34 pm    Post subject: Reply with quote

erik.swanson wrote:
This is a "one-liner" I wrote to safely clean out obsolete distfiles that are no longer referenced by any ebuild in portage.


Thanks - this is awesome. I just ran it and it didn't clean out anything I wanted to keep.
Back to top
View user's profile Send private message
elmie
Tux's lil' helper
Tux's lil' helper


Joined: 12 Sep 2002
Posts: 124
Location: Brisbane - Australia

PostPosted: Tue May 20, 2003 9:45 am    Post subject: Reply with quote

excellent script mate... I had over 3.5gb of distfiles, now I recovered 1gb back..
thanks!
Back to top
View user's profile Send private message
dma
Guru
Guru


Joined: 31 Jan 2003
Posts: 437
Location: Charlotte, NC, USA

PostPosted: Tue May 20, 2003 10:10 pm    Post subject: Reply with quote

Unfortunately, existing ebuilds and the "files" listings are updated all the time:

Code:
root@laureate:~/bin# diff /var/db/pkg/media-gfx/imagemagick-5.5.6/imagemagick-5.5.6.ebuild /usr/portage/media-gfx/imagemagick/imagemagick-5.5.
6.ebuild
3c3
< # $Header: /home/cvsroot/gentoo-x86/media-gfx/imagemagick/imagemagick-5.5.6.ebuild,v 1.1 2003/04/24 15:24:18 darkspecter Exp $
---
> # $Header: /home/cvsroot/gentoo-x86/media-gfx/imagemagick/imagemagick-5.5.6.ebuild,v 1.3 2003/05/15 00:51:49 darkspecter Exp $
5,6c5
< inherit libtool
< inherit flag-o-matic
---
> inherit libtool flag-o-matic
23c22
< SRC_URI="mirror://sourceforge/imagemagick/${MY_P}.tar.gz"
---
> SRC_URI="mirror://sourceforge/${PN}/${MY_P}.tar.bz2"
33d31
<       >=media-libs/freetype-2.0


<BIG SIGH...>

I guess they should be purged since they won't be used anyway if I do a re-merge.

I was wondering about it when I tried this code:
Code:
import os,sys,portage,xpak,string,re,commands,traceback
from stat import *
from output import *

root="/"
vt=portage.db[root]["vartree"]
big_hash={}

for node in vt.getallnodes():
        x=vt.resolve_key(node)
        matches=vt.dbapi.match(x)
        for x in matches:
                try:
                        src_uri=string.split(portage.portdb.aux_get(x, ["SRC_URI"])[0]," ")
                except Exception, e:
                        pass

                for entry in src_uri:
                        entry2=os.path.basename(entry)
                        big_hash[entry2]=1

for entry in big_hash.keys():
        print "*** ", entry


distlist = os.listdir("/usr/portage/distfiles")
for entry in distlist:
        if entry not in big_hash:
                print entry


...
Code:
!!! aux_get(): ebuild for 'media-gfx/gimp-print-4.2.4' does not exist at:
!!!            /usr/portage/media-gfx/gimp-print/gimp-print-4.2.4.ebuild
Back to top
View user's profile Send private message
Wylie
n00b
n00b


Joined: 05 May 2002
Posts: 5
Location: Sydney Australia

PostPosted: Fri Jun 20, 2003 2:25 pm    Post subject: Reply with quote

Fantastic scripts, Ive been looking for something like this for a while now, however I still dont think they are quite 'there' yet. (but they certainly do help).

Personally, I think theres a lot of merit in RoadRunner's post above (you wouldnt believe this by my userid, but no... I dont know him). I too am one who mounts my portage directory on multiple machines via NFS.

Unfortunatly I dont 'do' Python, but its definately the way to go, considering that you can simply import portage and get access to a lot of the pre-existing internals.

I dont know if anyone is continuing to work on these (I sure hope so), but what I would like to see is something based on Far's revised python script above, which only deletes files that are no longer referenced in the portage tree. This would solve the problem of trying not to delete files that may be required on other systems that share the same portage dir. Everything drops out of the tree eventually.

I would also like to see is the same script clean out both the DISTFILES dir, and the PACKAGES dir - including symlinks etc.

I build binary packages for different architectures - I have multiple i686 and i586 machines, and this helps speed things up. I first install to the faster machine of a given architecture, then update the slower ones with the just built binary package, saving a compile. Due to the multiple architectures, I have modified make.conf so that the packages are stored by architecture (ie: in make.conf on each system I have PKGDIR=${PORTDIR}/packages/${CHOST}, so the script should use the config info from make.conf as a base, (it will only see the config for one arch - the running system) and not simply default to /usr/poratge/packages etc.

If I knew python, I'd probably try to add these kind of features myself, but failing that, if anyone is interested, I for one think that this is the kind of functionality needed (ie: deleting whats not in the portage tree, and handling the packages dir as well - in a smart way).

Anyway, thats my 10c worth. Keep up the great work, all of you. I mainly just posted so I get updates via email, but hey... while Im here... :)
_________________
Deedle Deedle Dee, Wubba Wubba Wubba
Back to top
View user's profile Send private message
Tiganu
Tux's lil' helper
Tux's lil' helper


Joined: 02 Oct 2002
Posts: 105
Location: Montreal

PostPosted: Fri Jun 20, 2003 3:35 pm    Post subject: Reply with quote

To add another 10c to the discussion ...

1. quite nice scripts here :)

2. I really like RoadRunner's suggestion of deleting the sources/binaries once the resspective .ebuild is removed from the portage tree.
Right now emerge rsync cleans out the old .ebuilds; I think that this step could be extended (optionally w/ a flag in make.conf) to clean out the sources for the deleted .ebuilds if they (src) are not refferenced by other existing .ebuilds still (to handle the -r1, -r2 ... of the same version still around)
Back to top
View user's profile Send private message
RoadRunner
Bodhisattva
Bodhisattva


Joined: 12 Jun 2002
Posts: 490
Location: Coimbra, Portugal

PostPosted: Sat Jun 21, 2003 4:54 am    Post subject: Reply with quote

Wylie wrote:
(you wouldnt believe this by my userid, but no... I dont know him).

A poor RoadRunner just can't have a bit of peace.. Now I got to run for my feathers =)

It's just amazing how almost on every forum i'm in, there's a While E. Coyote nick =)

About the topic, as anybody submitted a bug about this, either with a script or just the idea as an improvement? Might be nice to get the dev's attention on this matter.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Unsupported Software All times are GMT
Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8  Next
Page 2 of 8

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum