Can't delete ".Trash-1000"

blandoon · Posted: Tue Aug 24, 2010 9:23 pm Post subject: Can't delete ".Trash-1000"

I have a partition on my Gentoo server that is exported as a samba share, and it has a folder called ".Trash-1000" which is taking up a lot of space. I assume this got there when people who had the drive mounted via samba deleted files into the trash - but now I just want to get rid of them because they are causing problems with using dar to back up the whole partition.

If I try to delete the folder from the command line using rm, the rm command runs away with all of my memory and swap, and just hangs until I kill it. Granted, this server is not a powerful box, because it is in a very small and quiet enclosure (1GHz C7 processor and 512 MB RAM).

This isn't a disk hardware problem, because I have relocated this partition to a new drive (using lvm), and fsck says the file system is perfectly clean... do I just have so many files in here that my lightweight server can't handle it? If that's the case, do I have any other options to get rid of this thing?
_________________
"Give a man a fire and he's warm for one night, but set fire to him and he's warm for the rest of his life..."

rh1 · Guru Joined: 10 Apr 2010 Posts: 501

What if you go into the directory and try deleteing smaller sections instead of all at once? Does it still hang?

blandoon · Posted: Tue Aug 24, 2010 9:57 pm Post subject:

There seem to be three subfolders - two smaller ones which I was able to delete, and a third one which is pretty big, and which always hangs when I try to delete it (or even to do an ls on it).
_________________
"Give a man a fire and he's warm for one night, but set fire to him and he's warm for the rest of his life..."

Hu · Administrator Joined: 06 Mar 2007 Posts: 23381

Does it run away even if you do ls -f |head? By disabling sorting, you might reduce memory requirements for the listing. Also, it would be interesting to see the output of ls -ld on the offending directory, to give us an idea of how many files might be in it.

If ls -f works, try find -print |wc -l to get a count of its contents, including subdirectories.

blandoon · Posted: Wed Aug 25, 2010 10:41 pm Post subject:

Thanks very much for that. ls -f worked, so I tried the following (took quite a while to come back):

krinn · Watchman Joined: 02 May 2003 Posts: 7471

well if find can handle it but rm couldn't, xargs the rm
find -print0 | xargs --null rm

Hu · Administrator Joined: 06 Mar 2007 Posts: 23381

cwr · Veteran Joined: 17 Dec 2005 Posts: 1969

Or as a very last resort, brute-force it; if ls can list the files, write a script
to remove them individually. There are peverse ways of getting filenames
via a binary editor and the directory itself, but on the whole, don't go there.

Will

blandoon · Posted: Sat Aug 28, 2010 7:11 pm Post subject:

Thanks for all the suggestions so far - this has been an interesting exercise to say the least.

It seems like find can't handle listing the files (at least not in a reasonable amount of time)... the only thing that can so far is ls -f, and even trying to do that directly into an xargs seems to take up an excessive amount of memory. I started playing around with a script to delete the files in small blocks at a time so as not to make the server unusable (while it runs in the background for days and days).

Here's what I got so far - I added a bunch of safety catches and tweakable parameters so that I can tune the load on the server to a reasonable level (and because I'm a lousy programmer and I want to keep from shooting myself in the foot):

Hu · Administrator Joined: 06 Mar 2007 Posts: 23381

Since xargs is designed to batch up the input and kick off a large command, it could require substantial memory in some cases. You might be able to influence this with the --max-lines and/or --max-args options.

Use nice and/or ionice to reduce the priority of the cleanup process, so that it has less impact on the server.

You should add a set -e near the top of the script, so that any unhandled errors cause the script to exit instead of proceeding on in a potentially dangerous fashion.

krinn · Watchman Joined: 02 May 2003 Posts: 7471

you could also lower files number in that directory without remove them.

using mv from a partition to the same partition won't really move the file but just the location of the file -> faster than anything else.
so as example
mkdir -p /todelete/a
mkdir /todelete/b ... you got the point
then mv .Trash-1000/a* /todelete/a

you will endup with lighter directories instead of a big one at a lighting fast speed as the files aren't access
look :

Hu · Administrator Joined: 06 Mar 2007 Posts: 23381

blandoon · Posted: Mon Aug 30, 2010 5:20 pm Post subject:

I did try a few times to find a wildcard string that would match enough files to be useful, but few enough not to hang - I never had any luck. I also tried using xargs with the --max-args argument and couldn't get that to work either, but I may not have been doing it correctly... I'll look into that some more.

There are two major inefficiencies with the script above: One is that ls -f also returns directory entries which rm cannot delete, so I added a bunch of extra logic to try to deal with those. The other (related) problem is trying to determine when you are done, taking into account that there may be subdirectories, which is the clunkiest part of the script.

I think trying to move the files a few thousand at a time is a good idea; I'm going to give that a shot too.

Thanks, all of this is very useful.
_________________
"Give a man a fire and he's warm for one night, but set fire to him and he's warm for the rest of his life..."