Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
clearing cruft
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
curmudgeon
Veteran
Veteran


Joined: 08 Aug 2003
Posts: 1503

PostPosted: Thu Apr 26, 2018 8:59 pm    Post subject: clearing cruft Reply with quote

As I look around my now eight year old system, I occasionally notice orphan files (/usr/share/config/akonadi/mysql-global.conf, /usr/share/config/kdm/kdmrc, and /etc/env.d/44qt4-graphicssystem - qt4 is no longer installed on the system, to name a few).

Is there any method (perhaps a program that searches for files and directories not owned by any package) for locating and removing these?
Back to top
View user's profile Send private message
krinn
Watchman
Watchman


Joined: 02 May 2003
Posts: 6855

PostPosted: Fri Apr 27, 2018 6:34 pm    Post subject: Reply with quote

you can use equery b to check if a file is own by a package ; just keep in mind a file not own by a package doesn't mean the file is not use by the package!
so if packageA install files "AA" and "BB", equery b AA or equery b BB will answer -> packageA
but if packageA create file "CC", equery b CC will say -> nobody ; still packageA may need file "CC"

i suppose this could be automated (will take time on each file to run equery b on them) to point only files not own.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sat Apr 28, 2018 1:37 am    Post subject: Reply with quote

qfile in portage-utils, is much quicker than equery b.

While q is a much better fit for automation, I feel sure someone has done this before, perhaps in eix?
Back to top
View user's profile Send private message
curmudgeon
Veteran
Veteran


Joined: 08 Aug 2003
Posts: 1503

PostPosted: Sat Apr 28, 2018 2:02 am    Post subject: Reply with quote

Probably going to start first by finding old files. For example:

Code:

ls -l /usr/share/locale/fi/LC_MESSAGES/
total 108
-rw-r--r-- 1 root root 106001 2011-10-24 10:09:33 libc.mo


No idea why that was installed in the first place (never asked for Finnish), and no idea why it wasn't removed (it is obviously from ver many versions of libc ago).

krinn wrote:

so if packageA install files "AA" and "BB", equery b AA or equery b BB will answer -> packageA
but if packageA create file "CC", equery b CC will say -> nobody ; still packageA may need file "CC"


Yes, but as far as I know, packages can't create files in the system tree (they can of course under /etc/ or /var/ or /run/, but not under say /usr/). Please correct me if I am wrong about this (I could be :) ). Those are really what I am targeting.
Back to top
View user's profile Send private message
Proinsias
Tux's lil' helper
Tux's lil' helper


Joined: 06 Oct 2014
Posts: 85
Location: UK

PostPosted: Sat Apr 28, 2018 3:38 am    Post subject: Reply with quote

https://github.com/vaeth/find_cruft
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 9070

PostPosted: Sat Apr 28, 2018 3:48 am    Post subject: Reply with quote

steveL wrote:
While q is a much better fit for automation, I feel sure someone has done this before, perhaps in eix?
I simply cannot imagine why.

curmudgeon wrote:
Yes, but as far as I know, packages can't create files in the system tree (they can of course under /etc/ or /var/ or /run/, but not under say /usr/). Please correct me if I am wrong about this (I could be :) ). Those are really what I am targeting.
As you surmised, you are. The real fun comes in when you have files that are managed via ebuilds but not tracked by the package manager, as can be done using the pkg_{pre,post}{inst,rm} functions and files added at run time, which (spoiler alert) you probably have.
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5291

PostPosted: Sat Apr 28, 2018 10:25 pm    Post subject: Reply with quote

I used to have a self-written perl script to do this; deleted it and didn't keep a backup (argh) because I found out Paludis had a built in command for it that ran faster. And I've long since dumped that…

The /usr/share/locale thing is easy though - just install localepurge.
Back to top
View user's profile Send private message
curmudgeon
Veteran
Veteran


Joined: 08 Aug 2003
Posts: 1503

PostPosted: Sat Apr 28, 2018 11:35 pm    Post subject: Reply with quote

Proinsias wrote:
https://github.com/vaeth/find_cruft


That looks reasonable. While it isn't always obviously that a file is cruft, it often is. Like most things, it will take some effort to get the best results, but having a list of candidates to examine is a big head start.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Sun Apr 29, 2018 3:25 pm    Post subject: Reply with quote

desultory wrote:
The real fun comes in when you have files that are managed via ebuilds but not tracked by the package manager, as can be done using the pkg_{pre,post}{inst,rm} functions and files added at run time, which (spoiler alert) you probably have.
Hmm, it bothers me that those are not tracked, when they so easily could be (with a permissive sandbox, for example.)
Back to top
View user's profile Send private message
desultory
Administrator
Administrator


Joined: 04 Nov 2005
Posts: 9070

PostPosted: Wed May 02, 2018 4:00 am    Post subject: Reply with quote

steveL wrote:
Hmm, it bothers me that those are not tracked, when they so easily could be (with a permissive sandbox, for example.)
I was pointing out where there be dragons, not suggesting that keeping them as pets is good for your insurance rates.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Wed May 02, 2018 2:40 pm    Post subject: Reply with quote

desultory wrote:
I was pointing out where there be dragons, not suggesting that keeping them as pets is good for your insurance rates.
Lul, and I appreciate it.

Just seems odd it's not been corrected, after so many years, when it could be so easily.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6241

PostPosted: Thu May 03, 2018 1:58 pm    Post subject: Reply with quote

steveL wrote:
it bothers me that those are not tracked, when they so easily could be

Files installed in pkg_* phases are not tracked, because they cannot be tracked for an individual package (otherwise they were installed in the src_install phase).
Typical examples are font-caches which are generated by the ebuild installing the program which is able to generate these caches and also by the ebuilds of the individual fonts: These files belong to the collection of all these packages. If you want to track them properly, you need new concepts which are currently not part of the package manager.
Other examples of a similar type are the ccache symlinks.
Back to top
View user's profile Send private message
steveL
Watchman
Watchman


Joined: 13 Sep 2006
Posts: 5153
Location: The Peanut Gallery

PostPosted: Thu May 03, 2018 2:52 pm    Post subject: Reply with quote

mv wrote:
Files installed in pkg_* phases are not tracked, because they cannot be tracked for an individual package (otherwise they were installed in the src_install phase).
Typical examples are font-caches which are generated by the ebuild installing the program which is able to generate these caches and also by the ebuilds of the individual fonts: These files belong to the collection of all these packages. If you want to track them properly, you need new concepts which are currently not part of the package manager.
Other examples of a similar type are the ccache symlinks.
Thanks for the informative example.
I am guessing that there must be programs which clean the fontcache, etc, for you? Perhaps simply by standard generation across the directories.

Let me muddle through the example of a fontcache being built by a program and also on installation.
I'd expect the font files to be installed, and a standard helper to be run in both cases. If the standard helper is what's being built, then ofc it must run on installation. (although, like a service, I'd expect it to be manual, by default. After all, we have a running system, and the new program could be flawed.)
And yes, you're right that the cached files are effectively /var, but the installed basis is not.

I'm pretty sure that there are other types of file copies going on, or there isn't really an issue (since cached files are system-specific /var, afaic: they're not meant to be tracked as part of package installation, by definition: all that's really happening is that the helper is being run automatically.) Or at least, not one I can see any sysadmin complaining about.
I'd have the same perspective on python or perl modules.

So they can certainly be tracked, but it's not what we do with cache files, since they belong to the system, not the package.
If all pkg_* file copies and removals fit into this category, then fair enough: nothing to bother about. (I'd allow /etc and /var in pkg_*, perhaps restricted to directories the package installed, and then explicit whitelist for others, all of which admin can veto, ie: redirect to a holding area for review, if desired.)

ccache usage during the build process makes it somewhat trickier, but IDK anything about it, and the overall view of cache files still seems apposite.
Back to top
View user's profile Send private message
mv
Watchman
Watchman


Joined: 20 Apr 2005
Posts: 6241

PostPosted: Thu May 03, 2018 4:03 pm    Post subject: Reply with quote

steveL wrote:
If all pkg_* file copies and removals fit into this category

If they don't fall into this category, I would consider it a bug.
Quote:
ccache usage during the build process makes it somewhat trickier

I suppose that I did not express correctly what I meant: I meant the /usr/lib/ccache/bin directory which consists of symlinks with the system's compilers names (and thus depends on which compilers you have installed). This directory is not only updated when you install/uninstall/update ccache but also whenever you install/uninstall/update a compiler.
Some more (random) examples:
Code:
/usr/share/info/dir
/usr/lib64/graphviz/config
/etc/xml/docbook

The content of all these files depends on what is on your system (which docbook versions, graphviz plugins, ...) so it cannot be stored in the database (or if it is stored then only with the wrong checksum/date).
Back to top
View user's profile Send private message
Ant P.
Watchman
Watchman


Joined: 18 Apr 2009
Posts: 5291

PostPosted: Fri May 04, 2018 3:17 am    Post subject: Reply with quote

steveL wrote:
I am guessing that there must be programs which clean the fontcache, etc, for you? Perhaps simply by standard generation across the directories.

You're in luck, you might find the contents of /var/cache/fontconfig/CACHEDIR.TAG enlightening...
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum