Filesystem cruft script: clean your system, save disk space!

[Keffin]

battle for wesnoth (the only thing I have that uses clanlib) still seemed to work fine.

Battle for Wesnoth does not use Clanlib. Only SDL.

David

Sorry, my mistake (predictably considering your sig ), clanlib is on my system as a dependency for pingus, not battle for wesnoth.

Lol, I posted this then deleted it when it appeared on a new page and I wondered why on earth I would be posting this in this thread .

[djm]

I also tried running cruft-blame.py on a couple of entries but kept getting:
Code:

File "./cruft-blame.py", line 128
log = LogFile("/var/log/emerge.log", True)
^
IndentationError: expected an indented block

Indentation matters in python, which is what the error is about - so if you copied and pased it, try downloading the file (use wget if your browser insists on just opening it)

[jtally]

I also tried running cruft-blame.py on a couple of entries but kept getting:
Code:

File "./cruft-blame.py", line 128
log = LogFile("/var/log/emerge.log", True)

^
IndentationError: expected an indented block

Indentation matters in python, which is what the error is about - so if you copied and pased it, try downloading the file (use wget if your browser insists on just opening it)

I also get this same error when using the cruft-blame found at http://home.jesus.ox.ac.uk/~ecatmur/my- ... t-blame.py even if I use wget to download it. The version of cruft-blame found at http://home.jesus.ox.ac.uk/~rmoss/porta ... t-blame.py however works just fine.

[hepta_sean]

Hello again!

ecatmur: Hope, you will be back!

Until then, FYI, I have made some patches to the script treating problems, that occured lately:

* The thing with the urw-fonts, already reported earlier in this thread:

Code: Select all

-$(has_version media-fonts/urw-fonts     && fontdir urw-fonts)
+$(has_version "<=media-fonts/urw-fonts-2.0-r1" && fontdir default/Type1)
+$(has_version ">=media-fonts/urw-fonts-2.1" && fontdir urw-fonts)

* I switched from gdm to entrance. -- New false positives:

Code: Select all

+has_version x11-misc/entrance \
+       && PRUNE="${PRUNE}
+       $(echo /var/lib/{:*.Xauth,entrance_ipc_`cat /var/run/entranced.pid`\|*})
+       /var/run/entranced.pid"

* alsa-lib creates the following symlink:

Code: Select all

+$(has_version media-libs/alsa-lib      && echo "/usr/include/sound")

* The new version of mozilla-firefox has some new false positives, I added to XPIApp():

Code: Select all

-       echo "$path"/{chrome/{chrome.rdf,overlayinfo},components/{compreg.dat,xpti.dat},chrome,install.log,searchplugins}
+       echo "$path"/{chrome/{chrome.rdf,overlayinfo},components/{compreg.dat,xpti.dat},components.ini,chrome,extensions,install.log,searchplugins}

* The /usr/lib/eclipse directory is only used for the unslotted versions. I added a 0, so it should get reported once there are only slotted versions left:

Code: Select all

-has_version dev-util/eclipse-sdk \
+has_version dev-util/eclipse-sdk 0 \
        && PRUNE="${PRUNE}
        $(echo /usr/lib/eclipse/{Uninstaller,features,plugins})"

* To solve the issues with the man-pages of samba, svgalib and possibly other packages, which install them into /usr/man, I replace all occurrences of /usr/man/ in the prune list with /usr/share/man/ (The trailing slash is important to let the symlink itself in the prune list):

Code: Select all

 | sed 's/.*\.py$/\0\n\0c\n\0o/' \
+| sed 's/^\/usr\/man\//\/usr\/share\/man\//' \

* With the same kind of hack, the problem with the GL thingies installing into /usr/include/GL, which really is /usr/X11R6/include/GL, can be solved:

Code: Select all

+| sed 's/^\/usr\/include\/GL\//\/usr\/X11R6\/include\/GL\//' \

Hence, these can go:

Code: Select all

-$(has_version media-libs/gle            && echo "/usr/X11R6/include/GL/gle.h")
-$(has_version media-libs/glut          && echo /usr/X11R6/include/GL/glut{,f90}.h)
-$(has_version media-video/ati-drivers   && echo "/usr/X11R6/include/GL/glxATI.h")

* And I also use this hack for my /opt symlink pointing to /usr/opt (Attention: You probably wouldn't want to add this, if /opt is on the root or its own partition):

Code: Select all

+| sed 's/^\/opt\//\/usr\/opt\//' \
 | sort -u \
 >/tmp/cruft-portagefiles

* Prepared with this I gave cruft a try on /opt:

Code: Select all

-       /etc/opt
-       /opt
-       /var/opt

It works very well! Except for tomcat with a lot of false positives (WEB-INF things and others), which I unmerged, since I didn't use it, anyway. But you probably would want to add it sometime, ...
Personally, I think /opt should be considered portage administrated and treated by cruft, and other things should go to /usr/local, but you disagreed earlier in this thread, ...

If you're interested the latest version of my modified cruft can be found here (Again: For "normal" systems the line regarding /usr/opt should be removed.) and a patch against the last "official" version is here.

Regards,
sean

[ecatmur]

Here for a moment, although I'm still a little busy...

I've made all the changes hepta_sean suggested, and I'm working on making it more flexible (ick!)
Great work everyone, hopefully I can reward your patience...

[riksta]

i don't know if this has been said before, but if the script is outputting each file that should be removed on a single line, then you can use the 'xargs' command like so

./script | xargs rm -rf

[djm]

it has been said before, but that's far too dangerous a way to do it

do this instead

./script | tee cruft-list

then edit cruft-list if there's any thing that shouldn't be being removed

and then you can do rm -r `cat cruftlist` (or use xargs if you want)

at least replace the rm -rf in what you said to rm -ri

[stillman]

cruft lists
/etc/lircd.conf
/etc/lircmd.conf

which are needed for LIRC

/etc/samba/credentials

which i use in fstab to mount smbfs(well, kind of individual but i'm not the only one)

/etc/wvdial.conf

needed for dialup

/etc/xcdroast.conf


hope that helps

[BlindSpy]

ed im just wondering if you've updated the version number with the latest updates you've done? (since the hepta_sean updates)

[hotplainrice]

Now I see one advantage of using gobolinux filesystem.. No cruft

[kamagurka]

Now I see one advantage of using gobolinux filesystem.. No cruft

explain please.

[hotplainrice]

Everything that belongs to a program is self-contained in a directory (e.g. /Programs/Foo) which has links pointing to the files inside. This way if you want to delete a program. Just delete the whole directory or version you don't want anymore.

the directories would look like this under /Programs/GCC

ls -l /Programs/GCC
total 1
drwxr-xr-x 9 gobo gobo 248 2003-12-08 13:35 3.3.2
lrwxrwxrwx 1 gobo gobo 5 2004-07-17 18:26 Current -> 3.3.2

ls -l /Programs/GCC/3.3.2
total 5
drwxr-xr-x 2 gobo gobo 384 2003-12-08 13:35 bin
drwxr-xr-x 3 gobo gobo 72 2003-11-30 23:04 include
drwxr-xr-x 2 gobo gobo 168 2003-11-30 23:04 info
drwxr-xr-x 3 gobo gobo 552 2003-12-03 10:39 lib
drwxr-xr-x 4 gobo gobo 96 2003-11-30 23:04 man
drwxr-xr-x 2 gobo gobo 112 2003-12-02 00:52 Resources
lrwxrwxrwx 1 gobo gobo 20 2004-07-17 18:26 share -> /System/Links/Shared
drwxr-xr-x 3 gobo gobo 72 2003-11-30 23:04 Shared

This is on my fileserver which I used to test GoboLinux. I don't really like it (its a desktop distro and I love portage ) but the idea of filesystem as a package manager is great. Read the docs as I can't explain things well.
http://www.gobolinux.org

[BlindSpy]

I added this to http://gentoo-wiki.com here

[imon fyre]

ran cruft, and here is some of the stuff that i think doesnt need to go

Code: Select all

/etc/distcc
/etc/distcc/hosts
/etc/domainname
/etc/timezone/usr/lib/distcc/bin/c++
/usr/lib/distcc/bin/cc
/usr/lib/distcc/bin/g++
/usr/lib/distcc/bin/gcc
/usr/lib/distcc/bin/i586-pc-linux-gnu-c++
/usr/lib/distcc/bin/i586-pc-linux-gnu-g++
/usr/lib/distcc/bin/i586-pc-linux-gnu-gcc
/usr/lib/opengl/nvidia/lib/libnvidia-tls.so.1
/usr/lib/opengl/xfree/include/glext.h
/usr/qt/3/etc/settings/.qtrc.lock
/usr/qt/3/etc/settings/qtrc
/usr/sbin/gcc-config
/usr/share/gcc-data/i386-pc-linux-gnu
/usr/share/gcc-data/i386-pc-linux-gnu/3.3
/usr/share/gcc-data/i386-pc-linux-gnu/3.3/info
/usr/share/gcc-data/i386-pc-linux-gnu/3.3/info/dir.old
/usr/share/gcc-data/i586-pc-linux-gnu/3.3/info/dir
/usr/share/gcc-data/i586-pc-linux-gnu/3.3/info/dir.old
/var/cache/apache-mm/mm.23709.sem

ps, nice script dude... really shows some ingenuity

[marienZ]

Nice script, it caught heaps of leftover .ph files from long-gone perl versions, a bunch of files in /etc, and some other stuff.

I've made some changes to reduce the number of false positives. patch available.

/etc/udev/rules.d/ and /etc/logrotate.d/ are directories where both portage and the user can place config files. I marked the entire directory as "not cruft", because the name(s) of user-created files are not predictable. Advantage: user-created config files are not marked as cruft. Disadvantage: if another package installs something to /etc/logrotate.d/ and that package is then removed, the leftover file is not marked as cruft (it should).

I assume that files matching /var/log/*.[0-9].gz are rotated logs. It would probably be better to mark e.g. /var/log/foo.3.gz as cruft if /var/log/foo.gz doesn't exist, but I'm too lazy (and don't know enough bash) to do that.

/var/lib/postgresql is gentoo's default database location. I'm not sure if that's the "right" place to put a database.

The rest should be obvious (just adds support for some packages and updates some things that were moved around in newer ebuilds).

Remaining (possible) problems:

I've deleted some gimp-related symlinks (/usr/bin/gimp to /usr/bin/gimp-2 and similar). I'm not sure if those should exist or not, and currently too lazy to figure it out.

/usr/lib/opengl/xorg-x11/include/glext.h is marked as cruft. The latest opengl-update asked me to move this file around. I'm not sure how to handle this. I hope newer xorg-x11 and opengl-update ebuilds take care of this.

[BlindSpy]

I've deleted some gimp-related symlinks (/usr/bin/gimp to /usr/bin/gimp-2 and similar). I'm not sure if those should exist or not, and currently too lazy to figure it out.

That should not exist but some people creat them to start gimp with the normal command (gimp) instead of gimp-2.

I've also got another one that nags me. VSFTPD requires the log file /var/log/vsftpd/vsftp.log to be there to run, yet it is detected as cruft. I think the user has to create the log file for vsftpd manually no matter where it goes but is there some way to just manually ignore that name?

[webkiller71]

it says says I should delete my /selinux folder and my selinux stuff in /etc/security/selinux
I also run dhcp in a chroot and it says I should delete /chroot/dhcp
also uptimed should be deleted

[marienZ]

I've deleted some gimp-related symlinks (/usr/bin/gimp to /usr/bin/gimp-2 and similar).

That should not exist but some people creat them to start gimp with the normal command (gimp) instead of gimp-2.

If I understand /usr/portage/media-gfx/gimp/ChangeLog correctly those were also created by some older ebuilds. Not sure why they weren't deleted by portage though.

I've also got another one that nags me. VSFTPD requires the log file /var/log/vsftpd/vsftp.log to be there to run, yet it is detected as cruft. I think the user has to create the log file for vsftpd manually no matter where it goes but is there some way to just manually ignore that name?

You probably want to change

Code: Select all

has_version net-ftp/vsftpd \
        && PRUNE="${PRUNE}
        /etc/ftpusers
        /etc/vsftpd/vsftpd.conf
        /usr/share/vsftpd"

to

Code: Select all

has_version net-ftp/vsftpd \
        && PRUNE="${PRUNE}
        /etc/ftpusers
        /etc/vsftpd/vsftpd.conf
        /usr/share/vsftpd
        /var/log/vsftp/vsftpd.log"

in the script.

[59729]

These should probably not be removed...

****** app-antivirus/clamav : [ I] 0.75 (0) ******

/etc/clamav.conf
/etc/freshclam.conf


****** net-im/bitlbee : [M~I] 0.90-r1 (0)e ******
/var/lib/bitlbee/lappen.accounts
/var/lib/bitlbee/lappen.nicks

not only lappen.accounts but all *.accounts and *.nicks files are used by bitlbee



And a little questions are these safe to delete?
**** dev-lang/python : [ I] 2.3.3-r1 (2.3) ****

Code: Select all

/etc/xml
/etc/xml/catalog
/lib/udev-state/devices.tar.bz2
/usr/bin/i386-pc-linux-gnu-c++
/usr/bin/i386-pc-linux-gnu-g++
/usr/bin/i386-pc-linux-gnu-gccd /
/usr/sbin/gcc-config
/var/lib/iptraf/iptraf.cfg
/var/lib/iptraf/ports.dat
/var/lib/iptraf/savedfilters.dat
/var/run/cgisock
/var/run/iptraf/iptraf-itrafmoncount.dat
/var/run/iptraf/iptraf-lanmoncount.dat
/var/run/iptraf/iptraf-processcount.dat

/usr/lib/python2.3/lib-tk
/usr/lib/python2.3/lib-tk/Canvas.py
/usr/lib/python2.3/lib-tk/Canvas.pyc
/usr/lib/python2.3/lib-tk/Canvas.pyo
/usr/lib/python2.3/lib-tk/Dialog.py
/usr/lib/python2.3/lib-tk/Dialog.pyc
/usr/lib/python2.3/lib-tk/Dialog.pyo
/usr/lib/python2.3/lib-tk/FileDialog.py
/usr/lib/python2.3/lib-tk/FileDialog.pyc
/usr/lib/python2.3/lib-tk/FileDialog.pyo
/usr/lib/python2.3/lib-tk/FixTk.py
/usr/lib/python2.3/lib-tk/FixTk.pyc
/usr/lib/python2.3/lib-tk/FixTk.pyo
/usr/lib/python2.3/lib-tk/ScrolledText.py
/usr/lib/python2.3/lib-tk/ScrolledText.pyc
/usr/lib/python2.3/lib-tk/ScrolledText.pyo
/usr/lib/python2.3/lib-tk/SimpleDialog.py
/usr/lib/python2.3/lib-tk/SimpleDialog.pyc
/usr/lib/python2.3/lib-tk/SimpleDialog.pyo
/usr/lib/python2.3/lib-tk/Tix.py
/usr/lib/python2.3/lib-tk/Tix.pyc
/usr/lib/python2.3/lib-tk/Tix.pyo
/usr/lib/python2.3/lib-tk/Tkconstants.py
/usr/lib/python2.3/lib-tk/Tkconstants.pyc
/usr/lib/python2.3/lib-tk/Tkconstants.pyo
/usr/lib/python2.3/lib-tk/Tkdnd.py
/usr/lib/python2.3/lib-tk/Tkdnd.pyc
/usr/lib/python2.3/lib-tk/Tkdnd.pyo
/usr/lib/python2.3/lib-tk/Tkinter.py
/usr/lib/python2.3/lib-tk/Tkinter.pyc
/usr/lib/python2.3/lib-tk/Tkinter.pyo
/usr/lib/python2.3/lib-tk/tkColorChooser.py
/usr/lib/python2.3/lib-tk/tkColorChooser.pyc
/usr/lib/python2.3/lib-tk/tkColorChooser.pyo
/usr/lib/python2.3/lib-tk/tkCommonDialog.py
/usr/lib/python2.3/lib-tk/tkCommonDialog.pyc
/usr/lib/python2.3/lib-tk/tkCommonDialog.pyo
/usr/lib/python2.3/lib-tk/tkFileDialog.py
/usr/lib/python2.3/lib-tk/tkFileDialog.pyc
/usr/lib/python2.3/lib-tk/tkFileDialog.pyo
/usr/lib/python2.3/lib-tk/tkFont.py
/usr/lib/python2.3/lib-tk/tkFont.pyc
/usr/lib/python2.3/lib-tk/tkFont.pyo
/usr/lib/python2.3/lib-tk/tkMessageBox.py
/usr/lib/python2.3/lib-tk/tkMessageBox.pyc
/usr/lib/python2.3/lib-tk/tkMessageBox.pyo
/usr/lib/python2.3/lib-tk/tkSimpleDialog.py
/usr/lib/python2.3/lib-tk/tkSimpleDialog.pyc
/usr/lib/python2.3/lib-tk/tkSimpleDialog.pyo
/usr/lib/python2.3/lib-tk/turtle.py
/usr/lib/python2.3/lib-tk/turtle.pyc
/usr/lib/python2.3/lib-tk/turtle.pyo

[ScribeOfTheNile]

ran cruft, and here is some of the stuff that i think doesnt need to go

Code: Select all

[snip]
/etc/timezone
[/snip]

I'd like to second that; it's created by KDE when setting a timezone.

BTW: Your script is ub3r.

[nsahoo]

Although cruft cleaning can be really useful .. I am too scared to use your script to delete any file. it seems safer than only suicide.

[blscreen]

Great script, thank you and keep on the good work!

I have some false positives here, for example:

Code: Select all

# cruft-blame.py /etc/snort/snort.conf
/etc/snort/snort.conf: 04/18/04 12:58:25
Cannot find any possible owners. Possible reasons are that the owner
was merged using ebuild instead of emerge, that it is generated or
cached data, that it is a config file, or that you created it. If you
believe this is reproducible, please report it.
# qpkg -f /etc/snort/snort.conf
net-analyzer/snort *
# qpkg -c /etc/snort/snort.conf
# 

Reemerging snort does not help.

[BlindSpy]

I've deleted some gimp-related symlinks (/usr/bin/gimp to /usr/bin/gimp-2 and similar).

That should not exist but some people creat them to start gimp with the normal command (gimp) instead of gimp-2.

If I understand /usr/portage/media-gfx/gimp/ChangeLog correctly those were also created by some older ebuilds. Not sure why they weren't deleted by portage though.

I've also got another one that nags me. VSFTPD requires the log file /var/log/vsftpd/vsftp.log to be there to run, yet it is detected as cruft. I think the user has to create the log file for vsftpd manually no matter where it goes but is there some way to just manually ignore that name?

You probably want to change

Code: Select all

has_version net-ftp/vsftpd \
        && PRUNE="${PRUNE}
        /etc/ftpusers
        /etc/vsftpd/vsftpd.conf
        /usr/share/vsftpd"

to

Code: Select all

has_version net-ftp/vsftpd \
        && PRUNE="${PRUNE}
        /etc/ftpusers
        /etc/vsftpd/vsftpd.conf
        /usr/share/vsftpd
        /var/log/vsftp/vsftpd.log"

in the script.

hey thanks man =)

[dashnu]

This script rocks, If you use it right from the start of a install your system stays clean.

I run it after every emerge -C if I decied I dont like the app or what-not. If i get scared to delete something I just put a .DELETE after the file name and test for a while and if nothing breaks I wipe it.

Anyhoot Good Work I think It should be put in portage under

sys-USEATOWNRISK/cruft

[fernandotcl]

I don't know if this was already mentioned before (I didn't read the whole thread ), but the following files are marked to be deleted:

Code: Select all

/var/log
/var/log/news/news.crit
/var/log/news/news.err
/var/log/news/news.notice
/var/run/ppp0.pid

The files in /log are probably being marked due to sysklogd. And files in /var/run shouldn't ever be marked, imo.

Btw, it's the best Portage script I've seen so far.