| View previous topic :: View next topic |
| Author |
Message |
TheCoop
Veteran
Joined: 15 Jun 2002
Posts: 1814
Location: Where you least expect it
|
 Posted: Sat Dec 20, 2003 9:19 pm Post subject: Checking for stale /etc files |
 |
|
Due to the portage /etc protection, and package that installs files in /etc and is then unmerged leaves the files there. Over time /etc can get very cluttered. Therefore here is a very simple one-liner that will print out the paths of all the files in /etc that do not belong to any package. However, beware of deleting them, as deleting some files in /etc may seriously damage your health as you try to find a livecd to boot from
| Code: | for n in /etc/*; do if [[ `qpkg -f $n` = "" ]]; then echo $n; fi; done
|
You can also change it to run under /etc/init.d or /etc/conf.d, cos atm it doesn't do recursive checks
ps. I know most of you probably would have got this by now, but there are a fair few, including me as of 10 mins ago, dont know the first thing about bash scripting and therefore would not be able to do this. Just bear with me, will you? 
_________________
95% of all computer errors occur between chair and keyboard (TM)
"One World, One web, One program" - Microsoft Promo ad.
"Ein Volk, Ein Reich, Ein Führer" - Adolf Hitler
Change the world - move a rock
Last edited by TheCoop on Sat Dec 20, 2003 11:38 pm; edited 1 time in total |
|
| Back to top |
|
 |
frekiR
n00b
Joined: 15 Dec 2003
Posts: 42
Location: Sweden
|
| Posted: Sat Dec 20, 2003 11:26 pm Post subject: |
|
|
Very nice, good job  |
|
| Back to top |
|
|
neenee
Veteran
Joined: 20 Jul 2003
Posts: 1786
|
| Posted: Sat Dec 20, 2003 11:42 pm Post subject: |
|
|
thanks for sharing this. i too do not know enough
about many things in linux myself  |
|
| Back to top |
|
|
Tazmanian
Apprentice
Joined: 01 Jul 2003
Posts: 222
|
| Posted: Sun Dec 28, 2003 12:12 am Post subject: Re: Checking for stale /etc files |
|
|
| TheCoop wrote: | | Code: | | for n in /etc/*; do if [[ `qpkg -f $n` = "" ]]; then echo $n; fi; done |
You can also change it to run under /etc/init.d or /etc/conf.d, cos atm it doesn't do recursive checks |
A modified version that checks /etc recursively and only queries packages that you have installed:
| Code: | for n in `find /etc -type f` ; do
if [[ `qpkg -I -f $n` = "" ]] ; then
echo $n
fi
done |
|
|
| Back to top |
|
|
BlindSpy
Apprentice
Joined: 20 Mar 2004
Posts: 263
|
| Posted: Tue Apr 26, 2005 5:05 pm Post subject: |
|
|
Awesome work guys! I've been looking for something like this since ecatmur's cruft script has stopped working for me =).
_________________
Symlinks to:
xorg.conf |
|
| Back to top |
|
|
Hack_Benjamin
Apprentice
Joined: 12 Nov 2004
Posts: 158
|
| Posted: Wed Jul 13, 2005 11:36 am Post subject: |
|
|
| if i wanted to print what the output was to a file instead of just showing it to me in the term how would i do it? |
|
| Back to top |
|
|
extragedy
n00b
Joined: 19 May 2005
Posts: 10
|
| Posted: Thu Jul 14, 2005 12:43 am Post subject: |
|
|
| you can always put ">> your file" without the quotes at the end of the echo file |
|
| Back to top |
|
|
Hack_Benjamin
Apprentice
Joined: 12 Nov 2004
Posts: 158
|
| Posted: Thu Jul 14, 2005 11:16 am Post subject: |
|
|
sorry, i thought it would be obvious 
my plan is to hav that script, then save its output and then open an editor to remove what you wish to keep and then do something like:
su -c
for i in cat `file` rm $i
ace.
great script |
|
| Back to top |
|
|
Grilo
Tux's lil' helper
Joined: 22 Apr 2003
Posts: 114
Location: Canada
|
| Posted: Fri Jul 15, 2005 5:15 pm Post subject: |
|
|
Thanks for the script. I thought you should be able to tell it how to skip the files you knew for sure you shouldnt delete. So Luke MacGregor (dpl) modified the script. here it is
| Code: |
#!/bin/bash
# this is the list of files that you don't want to trash, make sure they're
# in quotations.
KEEP_FILES=("/etc/make.conf" "/etc/X11/Sessions/e17" "/etc/mtab")
NUM_FILES=${#KEEP_FILES[*]}
BOOL=1
OUTPUT="etc-check.output"
# first remove the output file if it is there.
if [ -e $OUTPUT ]; then
rm $OUTPUT
fi
for n in `find /etc -type f` ; do
BOOL=1
if [[ `qpkg -I -f $n` = "" ]] ; then
for i in ${KEEP_FILES[@]} ; do
if [[ $n == ${i} ]] ; then
BOOL=0
echo not inserting $n
fi
done
if [[ $BOOL == "1" ]] ; then
echo $n >> $OUTPUT
fi
fi
done
|
in the keep files just list the ones to skip.
this is new and improved. it will check to see if you have an output file then delete it so that you have a new clean list after being run
good luck
Grilo
_________________
Knowledge is power but the drive to learn is harnessing wizdom |
|
| Back to top |
|
|
rickj
Guru
Joined: 06 Feb 2003
Posts: 427
Location: Calgary, Alberta, Canada
|
| Posted: Sun Apr 30, 2006 6:40 pm Post subject: |
|
|
| Could some knowledgeable person please update these scripts, in the light of the fact that qpkg has gone the way of the KSR33 Teletype? |
|
| Back to top |
|
|
TheCoop
Veteran
Joined: 15 Jun 2002
Posts: 1814
Location: Where you least expect it
|
| Posted: Sun Apr 30, 2006 10:53 pm Post subject: |
|
|
| Code: | #!/bin/bash
# this is the list of files that you don't want to trash, make sure they're
# in quotations.
KEEP_FILES=("/etc/make.conf" "/etc/X11/Sessions/e17" "/etc/mtab")
NUM_FILES=${#KEEP_FILES[*]}
BOOL=1
OUTPUT="/tmp/etc-check.output"
# first remove the output file if it is there.
if [ -e $OUTPUT ]; then
rm $OUTPUT
fi
for n in `find /etc -type f` ; do
BOOL=1
if [[ $(equery -q -C b $n) = "" ]] ; then
for i in ${KEEP_FILES[@]} ; do
if [[ $n == ${i} ]] ; then
BOOL=0
echo not inserting $n
fi
done
if [[ $BOOL == "1" ]] ; then
echo $n >> $OUTPUT
fi
fi
done |
It takes quite a while though. Best to leave it running in a detached screen or something (sh etc-check.sh && beep)
_________________
95% of all computer errors occur between chair and keyboard (TM)
"One World, One web, One program" - Microsoft Promo ad.
"Ein Volk, Ein Reich, Ein Führer" - Adolf Hitler
Change the world - move a rock |
|
| Back to top |
|
|
think4urs11
Bodhisattva
Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud
|
| Posted: Mon May 01, 2006 12:45 pm Post subject: |
|
|
Tested it on two machines and must say it is slow even when comparing to trees getting older
times for TheCoop-script
a) P4-2400: 120:11 min
b) Via Eden 600: stopped after 656 minutes (approx. 50% done)
my script
a) P4-2400: 4:11 minutes
b) Via Eden 600: 37:57 minutes
all whats needed to speed it up is
replace
| Code: | | if [[ $(equery -q -C b $n) = "" ]] ; then |
with | Code: | | if [[ $(qfile $n) = "" ]] ; then |
qfile is part of app-portage/portage-utils
HTH
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself |
|
| Back to top |
|
|
dmartinsca
Guru
Joined: 09 Dec 2005
Posts: 303
Location: Ontario, Canada
|
| Posted: Mon May 01, 2006 5:28 pm Post subject: |
|
|
| Think4UrS11 wrote: | Tested it on two machines and must say it is slow even when comparing to trees getting older
times for TheCoop-script
a) P4-2400: 120:11 min
b) Via Eden 600: stopped after 656 minutes (approx. 50% done)
my script
a) P4-2400: 4:11 minutes
b) Via Eden 600: 37:57 minutes
all whats needed to speed it up is
replace
| Code: | | if [[ $(equery -q -C b $n) = "" ]] ; then |
with | Code: | | if [[ $(qfile $n) = "" ]] ; then |
qfile is part of app-portage/portage-utils
HTH |
How many files in /etc and how many packages do you have installed?
find /etc -type f | wc -l
find /var/db/pkg -name "CONTENTS" | wc -l
I'm working on a script that just uses grep on the CONTENTS files in /var/db/pkg, doesn't look like it's going to be much faster though. |
|
| Back to top |
|
|
think4urs11
Bodhisattva
Joined: 25 Jun 2003
Posts: 6659
Location: above the cloud
|
| Posted: Mon May 01, 2006 5:48 pm Post subject: |
|
|
| dmartinsca wrote: | | How many files in /etc and how many packages do you have installed? |
(values for Via Eden)
find /etc -type f | wc -l
1097
find /var/db/pkg -name "CONTENTS" | wc -l
298
output of my script:
337
_________________
Nothing is secure / Security is always a trade-off with usability / Do not assume anything / Trust no-one, nothing / Paranoia is your friend / Think for yourself |
|
| Back to top |
|
|
BitJam
Advocate
Joined: 12 Aug 2003
Posts: 2508
Location: Silver City, NM
|
| Posted: Mon May 01, 2006 10:44 pm Post subject: |
|
|
Here is a very fast Perl program that will find cruft in /etc/ and its subdirectories. It is very fast because it reads the CONTENTS files just once and puts possible target files (from /etc) in a hash.
It works fine as is, but it could use a bit more work. The list of IGNORE files could probably be larger and I want to add a feature so it can read in additional ignore files both from the command line and from a file so each person can customize which files to ignore on their own system without modifying the code.
It has two limitations, both of which are easy to fix. The first is that it will fail for filenames with spaces in them in the CONTENTS files. Second, it only searches for beginning matches for files from the IGNORE list which makes it very easy to exclude directories. There should probably be an IGNORE_DIR list that still behaves this way and an IGNORE_FILE list that only ignores exact matches.
I am posting it now because it works and is very fast but I'm tired now and may not get around to adding those other features right away.
| Code: | #!/usr/bin/perl -w
use strict;
use Getopt::Long;
my $TARGET = "/etc";
my $DB_DIR = "/var/db/pkg";
my @IGNORE = ("/etc/OLD_ETC",
"/etc/splash",
"/etc/runlevels",
"/etc/ssl/certs",
"/etc/X11/xorg.conf",
"/etc/group",
"/etc/shadow",
"/etc/gshadow-",
"/etc/fstab",
"/etc/localtime",
"/etc/make.conf",
"/etc/passwd",
"/etc/resolv.conf",
"/etc/hosts",
);
my $ME = $0; $ME =~ s{.*/}{};
my $USAGE = <<USAGE;
Usage: $ME [options]
Options:
--target DIR Look for cruft in DIR (default: /etc)
--db_dir DIR Look for package files in DIR (default: /var/db/package)
--pretend Does nothing yet.
--verbose Prints a few summary lines.
--help Print this usage.
USAGE
my $PRETEND;
my $VERBOSE;
GetOptions(
"help" => sub { print $USAGE; exit},
"pretend" => \$PRETEND,
"verbose" => \$VERBOSE,
"target" => \$TARGET,
"db_dir" => \$DB_DIR,
) or die $USAGE;
#--- Grab valid files from all contents files ---
my %VALID;
for my $dir (<$DB_DIR/*>) {
for my $package (<$dir/*>) {
open(CONTENTS, "$package/CONTENTS") or next;
while (<CONTENTS>) {
m/^(?:obj|dir) ($TARGET\S*)/o and $VALID{$1}++;
}
close CONTENTS;
}
}
$VERBOSE and print "Found @{[scalar keys %VALID]} valid file names.\n";
my @FILES = `find $TARGET -print`;
chomp(@FILES);
$VERBOSE and print "Found @{[scalar @FILES]} target files.\n";
my @CRUFT;
FILE:
for my $file (@FILES) {
$VALID{$file} and next FILE;
for my $re (@IGNORE) {
$file =~ /^$re/ and next FILE;
}
push @CRUFT, $file;
}
print join "\n", @CRUFT, "\n";
$VERBOSE and print "Found @{[scalar @CRUFT]} cruft files.\n"; |
|
|
| Back to top |
|
|
mikemcquaid
Tux's lil' helper
Joined: 11 May 2004
Posts: 108
Location: Edinburgh, Scotland
|
| Posted: Sun May 07, 2006 11:35 pm Post subject: |
|
|
BitJam:
Can't find string terminator "USAGE" anywhere before EOF at ./etccheck line 25. |
|
| Back to top |
|
|
BitJam
Advocate
Joined: 12 Aug 2003
Posts: 2508
Location: Silver City, NM
|
| Posted: Mon May 08, 2006 12:05 am Post subject: |
|
|
| Make sure that line 34 (that just contains the string "USAGE") has no trailing whitespace. |
|
| Back to top |
|
|
electrofreak
l33t
Joined: 30 Jun 2004
Posts: 713
Location: Ohio, USA
|
| Posted: Sat Jun 10, 2006 9:07 pm Post subject: |
|
|
| Code: | | bash: qpkg: command not found |
The perl script works just fine, but seems to spit out WAY to many files.
_________________
Desktop: ABit AN8, Athlon64 X2 4400+ 939 2.75GHz, 2x1GB Corsair XMS DDR400, 2x160GB SATA RAID-0, 2x20"W, Vista Ultimate x64
Laptop: 15.4" MacBook Pro 2.4Ghz, 2x1GB RAM, 160GB, Mac OS X 10.5.1
Server: PIII 550Mhz, 3x128MB RAM, 160GB, Ubuntu Server 7.10 |
|
| Back to top |
|
|
ttuttle
Tux's lil' helper
Joined: 03 Oct 2004
Posts: 131
|
| Posted: Tue Dec 26, 2006 5:32 pm Post subject: |
|
|
Try this:
| Code: | find /etc | sort > ~/etc-actual
find /var/db/pkg/ -name CONTENTS | xargs -n 1 cat | cut -d" " -f 2 | grep "^/etc" | sort | uniq > ~/etc-intended
diff ~/etc-actual ~/etc-intended | less |
Files marked "< file" are files you have but aren't listed in any package; files marked "> file" are files that are listed in a package but no longer on your system (perhaps example config files you renamed or removed).
It will, like most of these scripts, flag files like /etc/X11/xorg.conf that are obviously important but don't actually come with any package. As the script does not remove anything, it's up to you to remove what you don't want and keep what you do.
_________________
Visit my website. |
|
| Back to top |
|
|
|
Documentation, Tips & Tricks
