| View previous topic :: View next topic |
| Author |
Message |
curmudgeon
Veteran
Joined: 08 Aug 2003
Posts: 1741
|
 Posted: Thu Apr 26, 2018 8:59 pm Post subject: clearing cruft |
 |
|
As I look around my now eight year old system, I occasionally notice orphan files (/usr/share/config/akonadi/mysql-global.conf, /usr/share/config/kdm/kdmrc, and /etc/env.d/44qt4-graphicssystem - qt4 is no longer installed on the system, to name a few).
Is there any method (perhaps a program that searches for files and directories not owned by any package) for locating and removing these? |
|
| Back to top |
|
 |
krinn
Watchman
Joined: 02 May 2003
Posts: 7470
|
| Posted: Fri Apr 27, 2018 6:34 pm Post subject: |
|
|
you can use equery b to check if a file is own by a package ; just keep in mind a file not own by a package doesn't mean the file is not use by the package!
so if packageA install files "AA" and "BB", equery b AA or equery b BB will answer -> packageA
but if packageA create file "CC", equery b CC will say -> nobody ; still packageA may need file "CC"
i suppose this could be automated (will take time on each file to run equery b on them) to point only files not own. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sat Apr 28, 2018 1:37 am Post subject: |
|
|
qfile in portage-utils, is much quicker than equery b.
While q is a much better fit for automation, I feel sure someone has done this before, perhaps in eix? |
|
| Back to top |
|
|
curmudgeon
Veteran
Joined: 08 Aug 2003
Posts: 1741
|
| Posted: Sat Apr 28, 2018 2:02 am Post subject: |
|
|
Probably going to start first by finding old files. For example:
| Code: |
ls -l /usr/share/locale/fi/LC_MESSAGES/
total 108
-rw-r--r-- 1 root root 106001 2011-10-24 10:09:33 libc.mo
|
No idea why that was installed in the first place (never asked for Finnish), and no idea why it wasn't removed (it is obviously from ver many versions of libc ago).
| krinn wrote: |
so if packageA install files "AA" and "BB", equery b AA or equery b BB will answer -> packageA
but if packageA create file "CC", equery b CC will say -> nobody ; still packageA may need file "CC" |
Yes, but as far as I know, packages can't create files in the system tree (they can of course under /etc/ or /var/ or /run/, but not under say /usr/). Please correct me if I am wrong about this (I could be :) ). Those are really what I am targeting. |
|
| Back to top |
|
|
Proinsias
Tux's lil' helper
Joined: 06 Oct 2014
Posts: 133
Location: Scotland
|
|
| Back to top |
|
|
desultory
Bodhisattva
Joined: 04 Nov 2005
Posts: 9410
|
| Posted: Sat Apr 28, 2018 3:48 am Post subject: |
|
|
| steveL wrote: | | While q is a much better fit for automation, I feel sure someone has done this before, perhaps in eix? |
I simply cannot imagine why.
| curmudgeon wrote: | Yes, but as far as I know, packages can't create files in the system tree (they can of course under /etc/ or /var/ or /run/, but not under say /usr/). Please correct me if I am wrong about this (I could be  ). Those are really what I am targeting. |
As you surmised, you are. The real fun comes in when you have files that are managed via ebuilds but not tracked by the package manager, as can be done using the pkg_{pre,post}{inst,rm} functions and files added at run time, which (spoiler alert) you probably have. |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Sat Apr 28, 2018 10:25 pm Post subject: |
|
|
I used to have a self-written perl script to do this; deleted it and didn't keep a backup (argh) because I found out Paludis had a built in command for it that ran faster. And I've long since dumped that…
The /usr/share/locale thing is easy though - just install localepurge. |
|
| Back to top |
|
|
curmudgeon
Veteran
Joined: 08 Aug 2003
Posts: 1741
|
| Posted: Sat Apr 28, 2018 11:35 pm Post subject: |
|
|
| Proinsias wrote: | | https://github.com/vaeth/find_cruft |
That looks reasonable. While it isn't always obviously that a file is cruft, it often is. Like most things, it will take some effort to get the best results, but having a list of candidates to examine is a big head start. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Sun Apr 29, 2018 3:25 pm Post subject: |
|
|
| desultory wrote: | | The real fun comes in when you have files that are managed via ebuilds but not tracked by the package manager, as can be done using the pkg_{pre,post}{inst,rm} functions and files added at run time, which (spoiler alert) you probably have. |
Hmm, it bothers me that those are not tracked, when they so easily could be (with a permissive sandbox, for example.) |
|
| Back to top |
|
|
desultory
Bodhisattva
Joined: 04 Nov 2005
Posts: 9410
|
| Posted: Wed May 02, 2018 4:00 am Post subject: |
|
|
| steveL wrote: | | Hmm, it bothers me that those are not tracked, when they so easily could be (with a permissive sandbox, for example.) |
I was pointing out where there be dragons, not suggesting that keeping them as pets is good for your insurance rates. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Wed May 02, 2018 2:40 pm Post subject: |
|
|
| desultory wrote: | | I was pointing out where there be dragons, not suggesting that keeping them as pets is good for your insurance rates. |
Lul, and I appreciate it.
Just seems odd it's not been corrected, after so many years, when it could be so easily. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Thu May 03, 2018 1:58 pm Post subject: |
|
|
| steveL wrote: | | it bothers me that those are not tracked, when they so easily could be |
Files installed in pkg_* phases are not tracked, because they cannot be tracked for an individual package (otherwise they were installed in the src_install phase).
Typical examples are font-caches which are generated by the ebuild installing the program which is able to generate these caches and also by the ebuilds of the individual fonts: These files belong to the collection of all these packages. If you want to track them properly, you need new concepts which are currently not part of the package manager.
Other examples of a similar type are the ccache symlinks. |
|
| Back to top |
|
|
steveL
Watchman
Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery
|
| Posted: Thu May 03, 2018 2:52 pm Post subject: |
|
|
| mv wrote: | Files installed in pkg_* phases are not tracked, because they cannot be tracked for an individual package (otherwise they were installed in the src_install phase).
Typical examples are font-caches which are generated by the ebuild installing the program which is able to generate these caches and also by the ebuilds of the individual fonts: These files belong to the collection of all these packages. If you want to track them properly, you need new concepts which are currently not part of the package manager.
Other examples of a similar type are the ccache symlinks. |
Thanks for the informative example.
I am guessing that there must be programs which clean the fontcache, etc, for you? Perhaps simply by standard generation across the directories.
Let me muddle through the example of a fontcache being built by a program and also on installation.
I'd expect the font files to be installed, and a standard helper to be run in both cases. If the standard helper is what's being built, then ofc it must run on installation. (although, like a service, I'd expect it to be manual, by default. After all, we have a running system, and the new program could be flawed.)
And yes, you're right that the cached files are effectively /var, but the installed basis is not.
I'm pretty sure that there are other types of file copies going on, or there isn't really an issue (since cached files are system-specific /var, afaic: they're not meant to be tracked as part of package installation, by definition: all that's really happening is that the helper is being run automatically.) Or at least, not one I can see any sysadmin complaining about.
I'd have the same perspective on python or perl modules.
So they can certainly be tracked, but it's not what we do with cache files, since they belong to the system, not the package.
If all pkg_* file copies and removals fit into this category, then fair enough: nothing to bother about. (I'd allow /etc and /var in pkg_*, perhaps restricted to directories the package installed, and then explicit whitelist for others, all of which admin can veto, ie: redirect to a holding area for review, if desired.)
ccache usage during the build process makes it somewhat trickier, but IDK anything about it, and the overall view of cache files still seems apposite. |
|
| Back to top |
|
|
mv
Watchman
Joined: 20 Apr 2005
Posts: 6747
|
| Posted: Thu May 03, 2018 4:03 pm Post subject: |
|
|
| steveL wrote: | | If all pkg_* file copies and removals fit into this category |
If they don't fall into this category, I would consider it a bug.
| Quote: | | ccache usage during the build process makes it somewhat trickier |
I suppose that I did not express correctly what I meant: I meant the /usr/lib/ccache/bin directory which consists of symlinks with the system's compilers names (and thus depends on which compilers you have installed). This directory is not only updated when you install/uninstall/update ccache but also whenever you install/uninstall/update a compiler.
Some more (random) examples:
| Code: | /usr/share/info/dir
/usr/lib64/graphviz/config
/etc/xml/docbook |
The content of all these files depends on what is on your system (which docbook versions, graphviz plugins, ...) so it cannot be stored in the database (or if it is stored then only with the wrong checksum/date). |
|
| Back to top |
|
|
Ant P.
Watchman
Joined: 18 Apr 2009
Posts: 6920
|
| Posted: Fri May 04, 2018 3:17 am Post subject: |
|
|
| steveL wrote: | | I am guessing that there must be programs which clean the fontcache, etc, for you? Perhaps simply by standard generation across the directories. |
You're in luck, you might find the contents of /var/cache/fontconfig/CACHEDIR.TAG enlightening... |
|
| Back to top |
|
|
|
Other Things Gentoo
