Checking for stale /etc files

[TheCoop]

Due to the portage /etc protection, and package that installs files in /etc and is then unmerged leaves the files there. Over time /etc can get very cluttered. Therefore here is a very simple one-liner that will print out the paths of all the files in /etc that do not belong to any package. However, beware of deleting them, as deleting some files in /etc may seriously damage your health as you try to find a livecd to boot from

Code: Select all

for n in /etc/*; do if [[ `qpkg -f $n` = "" ]]; then echo $n; fi; done

You can also change it to run under /etc/init.d or /etc/conf.d, cos atm it doesn't do recursive checks

ps. I know most of you probably would have got this by now, but there are a fair few, including me as of 10 mins ago, dont know the first thing about bash scripting and therefore would not be able to do this. Just bear with me, will you?

[frekiR]

Very nice, good job

[neenee]

thanks for sharing this. i too do not know enough
about many things in linux myself

[Tazmanian]

Code: Select all

for n in /etc/*; do if [[ `qpkg -f $n` = "" ]]; then echo $n; fi; done

You can also change it to run under /etc/init.d or /etc/conf.d, cos atm it doesn't do recursive checks

A modified version that checks /etc recursively and only queries packages that you have installed:

Code: Select all

for n in `find /etc -type f` ; do
    if [[ `qpkg -I -f $n` = "" ]] ; then
        echo $n
    fi
done

[BlindSpy]

Awesome work guys! I've been looking for something like this since ecatmur's cruft script has stopped working for me =).

[Hack_Benjamin]

if i wanted to print what the output was to a file instead of just showing it to me in the term how would i do it?

[extragedy]

you can always put ">> your file" without the quotes at the end of the echo file

[Hack_Benjamin]

sorry, i thought it would be obvious

my plan is to hav that script, then save its output and then open an editor to remove what you wish to keep and then do something like:

su -c
for i in cat `file` rm $i


ace.


great script

[Grilo]

Thanks for the script. I thought you should be able to tell it how to skip the files you knew for sure you shouldnt delete. So Luke MacGregor (dpl) modified the script. here it is

Code: Select all

#!/bin/bash
# this is the list of files that you don't want to trash, make sure they're
# in quotations.

KEEP_FILES=("/etc/make.conf" "/etc/X11/Sessions/e17" "/etc/mtab")

NUM_FILES=${#KEEP_FILES[*]}

BOOL=1

OUTPUT="etc-check.output"

# first remove the output file if it is there.
if [ -e $OUTPUT ]; then
       rm $OUTPUT
fi

for n in `find /etc -type f` ; do
   BOOL=1
   if [[ `qpkg -I -f $n` = "" ]] ; then
       for i in ${KEEP_FILES[@]} ; do
               if [[ $n == ${i} ]] ; then
                       BOOL=0
                       echo not inserting $n
               fi
       done
       if [[ $BOOL == "1" ]] ; then
               echo $n >> $OUTPUT
       fi
   fi
done

in the keep files just list the ones to skip.

this is new and improved. it will check to see if you have an output file then delete it so that you have a new clean list after being run


good luck
Grilo

[rickj]

Could some knowledgeable person please update these scripts, in the light of the fact that qpkg has gone the way of the KSR33 Teletype?

[TheCoop]

Code: Select all

#!/bin/bash
# this is the list of files that you don't want to trash, make sure they're
# in quotations.

KEEP_FILES=("/etc/make.conf" "/etc/X11/Sessions/e17" "/etc/mtab")

NUM_FILES=${#KEEP_FILES[*]}

BOOL=1

OUTPUT="/tmp/etc-check.output"

# first remove the output file if it is there.
if [ -e $OUTPUT ]; then
       rm $OUTPUT
fi

for n in `find /etc -type f` ; do
   BOOL=1
   if [[ $(equery -q -C b $n) = "" ]] ; then
       for i in ${KEEP_FILES[@]} ; do
               if [[ $n == ${i} ]] ; then
                       BOOL=0
                       echo not inserting $n
               fi
       done
       if [[ $BOOL == "1" ]] ; then
               echo $n >> $OUTPUT
       fi
   fi
done

It takes quite a while though. Best to leave it running in a detached screen or something (sh etc-check.sh && beep)

[think4urs11]

Tested it on two machines and must say it is slow even when comparing to trees getting older

times for TheCoop-script
a) P4-2400: 120:11 min
b) Via Eden 600: stopped after 656 minutes (approx. 50% done)

my script
a) P4-2400: 4:11 minutes
b) Via Eden 600: 37:57 minutes

all whats needed to speed it up is
replace

Code: Select all

   if [[ $(equery -q -C b $n) = "" ]] ; then

with

Code: Select all

if [[ $(qfile $n) = "" ]] ; then

qfile is part of app-portage/portage-utils

HTH

[dmartinsca]

Tested it on two machines and must say it is slow even when comparing to trees getting older

times for TheCoop-script
a) P4-2400: 120:11 min
b) Via Eden 600: stopped after 656 minutes (approx. 50% done)

my script
a) P4-2400: 4:11 minutes
b) Via Eden 600: 37:57 minutes

all whats needed to speed it up is
replace

Code: Select all

   if [[ $(equery -q -C b $n) = "" ]] ; then

with

Code: Select all

if [[ $(qfile $n) = "" ]] ; then

qfile is part of app-portage/portage-utils

HTH

How many files in /etc and how many packages do you have installed?
find /etc -type f | wc -l
find /var/db/pkg -name "CONTENTS" | wc -l

I'm working on a script that just uses grep on the CONTENTS files in /var/db/pkg, doesn't look like it's going to be much faster though.

[think4urs11]

How many files in /etc and how many packages do you have installed?

(values for Via Eden)

find /etc -type f | wc -l
1097
find /var/db/pkg -name "CONTENTS" | wc -l
298
output of my script:
337

[BitJam]

Here is a very fast Perl program that will find cruft in /etc/ and its subdirectories. It is very fast because it reads the CONTENTS files just once and puts possible target files (from /etc) in a hash.

It works fine as is, but it could use a bit more work. The list of IGNORE files could probably be larger and I want to add a feature so it can read in additional ignore files both from the command line and from a file so each person can customize which files to ignore on their own system without modifying the code.

It has two limitations, both of which are easy to fix. The first is that it will fail for filenames with spaces in them in the CONTENTS files. Second, it only searches for beginning matches for files from the IGNORE list which makes it very easy to exclude directories. There should probably be an IGNORE_DIR list that still behaves this way and an IGNORE_FILE list that only ignores exact matches.

I am posting it now because it works and is very fast but I'm tired now and may not get around to adding those other features right away.

Code: Select all

#!/usr/bin/perl -w 

use strict;
use Getopt::Long;

my $TARGET = "/etc";
my $DB_DIR = "/var/db/pkg";
my @IGNORE = ("/etc/OLD_ETC", 
              "/etc/splash", 
              "/etc/runlevels",
              "/etc/ssl/certs",
              "/etc/X11/xorg.conf",
              "/etc/group",
              "/etc/shadow",
              "/etc/gshadow-",
              "/etc/fstab",
              "/etc/localtime",
              "/etc/make.conf",
              "/etc/passwd",
              "/etc/resolv.conf",
              "/etc/hosts",
              );

my $ME = $0; $ME =~ s{.*/}{};
my $USAGE = <<USAGE;
Usage: $ME [options]
Options:
  --target DIR   Look for cruft in DIR (default: /etc)
  --db_dir DIR   Look for package files in DIR (default: /var/db/package)
  --pretend      Does nothing yet.
  --verbose      Prints a few summary lines.
  --help         Print this usage.
USAGE

my $PRETEND;
my $VERBOSE;

GetOptions(
    "help"    => sub { print $USAGE; exit},
    "pretend" => \$PRETEND,
    "verbose" => \$VERBOSE,
    "target"  => \$TARGET,
    "db_dir"  => \$DB_DIR,
) or die $USAGE;

#--- Grab valid files from all contents files ---
my %VALID;

for my $dir (<$DB_DIR/*>) {
    for my $package (<$dir/*>) {
        open(CONTENTS, "$package/CONTENTS") or next;
        while (<CONTENTS>) {
            m/^(?:obj|dir) ($TARGET\S*)/o and $VALID{$1}++;
        }
        close CONTENTS;
    }
}

$VERBOSE and print "Found @{[scalar keys %VALID]} valid file names.\n";

my @FILES = `find $TARGET -print`;
chomp(@FILES);

$VERBOSE and print "Found @{[scalar @FILES]} target files.\n";

my @CRUFT;
FILE:
for my $file (@FILES) {
    $VALID{$file} and next FILE;
    for my $re (@IGNORE) {
        $file =~ /^$re/ and next FILE;
    }
    push @CRUFT, $file;
}

print join "\n", @CRUFT, "\n";

$VERBOSE and print "Found @{[scalar @CRUFT]} cruft files.\n";

[mikemcquaid]

BitJam:
Can't find string terminator "USAGE" anywhere before EOF at ./etccheck line 25.

[BitJam]

Make sure that line 34 (that just contains the string "USAGE") has no trailing whitespace.

[electrofreak]

Code: Select all

bash: qpkg: command not found

The perl script works just fine, but seems to spit out WAY to many files.

[ttuttle]

Try this:

Code: Select all

find /etc | sort > ~/etc-actual
find /var/db/pkg/ -name CONTENTS | xargs -n 1 cat | cut -d" " -f 2 | grep "^/etc" | sort | uniq > ~/etc-intended
diff ~/etc-actual ~/etc-intended | less

Files marked "< file" are files you have but aren't listed in any package; files marked "> file" are files that are listed in a package but no longer on your system (perhaps example config files you renamed or removed).

It will, like most of these scripts, flag files like /etc/X11/xorg.conf that are obviously important but don't actually come with any package. As the script does not remove anything, it's up to you to remove what you don't want and keep what you do.