View previous topic :: View next topic |
Author |
Message |
me22 n00b
Joined: 30 Jun 2002 Posts: 45 Location: Canada
|
Posted: Fri Jan 01, 2010 8:13 pm Post subject: Helper script: distfiles cleanup |
|
|
After a number of years, my /usr/portage/distfiles has gotten up to about 18 GB of stuff, so I figured that for the new year I'd clean it out a bit. Since it's almost 10k files, though I knew I'd need a script if I didn't just want to wipe it. Here's what I came up with.
It attempts to guess outdated packages by regex and modification date, so without arguments it just shows the most recent file and the size of the older ones, ordered by the file groups taking the most space:
Code: |
$ old_versions.pl
linux-*.tar.bz2: linux-2.6.30.tar.bz2
50355835 linux-2.6.27.tar.bz2
49441874 linux-2.6.26.tar.bz2
48601689 linux-2.6.25.tar.bz2
46737783 linux-2.6.24.tar.bz2
45488158 linux-2.6.23.tar.bz2
45119878 linux-2.6.22.tar.bz2
43375937 linux-2.6.20.tar.bz2
41272919 linux-2.6.17.tar.bz2
42733268 linux-2.6.19.tar.bz2
41863580 linux-2.6.18.tar.bz2
454990921 total
gcc-*.tar.bz2: gcc-4.4.2.tar.bz2
59368714 gcc-4.3.4.tar.bz2
62869928 gcc-4.4.1.tar.bz2
58929447 gcc-4.3.2.tar.bz2
59369954 gcc-4.3.3.tar.bz2
58964610 gcc-4.3.1.tar.bz2
39707720 gcc-4.1.2.tar.bz2
28193401 gcc-3.4.6.tar.bz2
23972413 gcc-3.3.6.tar.bz2
391376187 total
|
With a --rm argument, it will rm -i the ones it thinks are no longer used:
Code: |
distfiles # old_versions.pl --rm
boost_*.tar.bz2: boost_1_41_0.tar.bz2
rm: remove regular file `boost_1_39_0.tar.bz2'?
rm: remove regular file `boost_1_35_0.tar.bz2'?
rm: remove regular file `boost_1_34_1.tar.bz2'?
rm: remove regular file `boost_1_34_0.tar.bz2'?
rm: remove regular file `boost_1_34_pre20061214.tar.bz2'? y
rm: remove regular file `boost_1_33_1.tar.bz2'?
rm: remove regular file `boost_1_33_0.tar.bz2'?
fgfs-base-*.tar.bz2: Only have fgfs-base-0.9.10.tar.bz2
mplayer-*.tar.bz2: mplayer-1.0_rc4_p20090919.tar.bz2
rm: remove regular file `mplayer-1.0_rc2_p20090731.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p20090322.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p28450.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p28348.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p28288.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p28058.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p27725.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p27458.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p27120.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p26753.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p26454.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p26450.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p26300.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p26258.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p25993.tar.bz2'? y
rm: remove regular file `mplayer-1.0_rc2_p24929.tar.bz2'? y
|
Here's the perl code:
Code: |
#!/usr/bin/perl
use strict;
my %seen = ();
my %keep = ();
my %gone = ();
my %size = ();
foreach my $file (split(/\n/, `ls -t /usr/portage/distfiles`)) {
my $pkg = $file;
$pkg =~ s{\d+([._-]\d+)*([a-z]\d?)?([_-]r\d+(.\d+)*)?([_-]rc\d+)?([_-]pre\d+)?(?=.)}{*}g;
$pkg =~ tr/A-Z/a-z/;
if ($pkg =~ m{^\*\.(zip|tar\.(gz|bz2))$}) { next; }
$size{$pkg} += (-s "$file");
if ($seen{$pkg}) {
$gone{$pkg} .= "$file\n";
} else {
$seen{$pkg} = $file;
$keep{$pkg} .= "$file\n";
}
}
foreach my $pkg (reverse sort {$size{$a} <=> $size{$b}} keys %seen) {
if ($gone{$pkg}) {
print "$pkg: $keep{$pkg}";
if ($ARGV[0] eq "--rm") {
system("rm", "-i", split(/\n/, $gone{$pkg}));
} else {
system("wc", "-c", split(/\n/, $gone{$pkg}));
}
} else {
print "$pkg: Only have $keep{$pkg}";
}
}
|
Of course, since it's distfiles, killing extra stuff is safe, but I prefer not to do that.
I hope someone else finds this useful too. _________________ ~ Scott "me22" McMurray |
|
Back to top |
|
|
sera Retired Dev
Joined: 29 Feb 2008 Posts: 1017 Location: CET
|
Posted: Fri Jan 01, 2010 9:01 pm Post subject: |
|
|
eclean, contained in app-portage/gentoolkit, could have been used instead. |
|
Back to top |
|
|
me22 n00b
Joined: 30 Jun 2002 Posts: 45 Location: Canada
|
Posted: Fri Jan 01, 2010 9:30 pm Post subject: |
|
|
sera wrote: | eclean, contained in app-portage/gentoolkit, could have been used instead. |
Ah, thanks. Guess I didn't look hard enough for the "right" way.
That said, eclean -p distfiles seems to take a few minutes to run, so I'm still happy to have my quick approximation. (Especially since I get most of the advantage just from cleaning up NVidia drivers and a handful of other packages.) _________________ ~ Scott "me22" McMurray |
|
Back to top |
|
|
sera Retired Dev
Joined: 29 Feb 2008 Posts: 1017 Location: CET
|
Posted: Sat Jan 02, 2010 8:58 am Post subject: |
|
|
There is no "right" way. The difference is eclean is well tested and maintained for you.
Yours is certainly easier to expand if the need should arise. For instance if you want to mv instead of rm. |
|
Back to top |
|
|
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|