Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Filesystems go missing
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware
View previous topic :: View next topic  
Author Message
Carnildo
Guru
Guru


Joined: 17 Jun 2004
Posts: 594

PostPosted: Fri Jun 21, 2013 5:05 am    Post subject: Filesystems go missing Reply with quote

My desktop computer recently developed a habit of losing access to its filesystems (physical, network, and virtual) a few hours after startup. The root filesystem is still there, but everything else (/dev, /proc, /home, and so on) goes missing: the mountpoints show up as empty directories, or in the case of /dev, the static device nodes on the root filesystem. The filesystems still show up as mounted, and any terminal with a working directory in one of the filesystems can still see its contents (but only if accessed using relative paths).

There are no relevant entries in dmesg or /var/log/messages, and the disappearance doesn't seem to be related to anything I do. Nothing changed around the time the problem developed, but since I don't reboot my computer very often, this could be long-delayed fallout from an upgrade as much as three months ago.

I'm running kernel 3.6.11; other version info is available on request.

Any ideas?
Back to top
View user's profile Send private message
audiodef
Watchman
Watchman


Joined: 06 Jul 2005
Posts: 6639
Location: The soundosphere

PostPosted: Fri Jun 21, 2013 3:00 pm    Post subject: Reply with quote

How often do you update?
_________________
decibel Linux: https://decibellinux.org
Github: https://github.com/Gentoo-Music-and-Audio-Technology
Facebook: https://www.facebook.com/decibellinux
Discord: https://discord.gg/73XV24dNPN
Back to top
View user's profile Send private message
Carnildo
Guru
Guru


Joined: 17 Jun 2004
Posts: 594

PostPosted: Fri Jun 21, 2013 11:03 pm    Post subject: Reply with quote

I usually update once a week, but I'll often put off things like X or running applications.
Back to top
View user's profile Send private message
wcg
Guru
Guru


Joined: 06 Jan 2009
Posts: 588

PostPosted: Sat Jun 22, 2013 12:03 am    Post subject: Reply with quote

At a guess, some corrupted data structure in the kernel's
dcache (directory cache) or inode cache. You can try to drop
caches when it happens and see if that fixes it.
Code:

echo 3 > /proc/sys/vm/drop_caches


(See /usr/src/linux/Documentation/sysctl/vm.txt.)

If that works, you can change the cache flush to
Code:

echo 2 > /proc/sys/vm/drop_caches

to skip freeing pagecache and only free the dcache and
inode cache and see if that works.
_________________
TIA
Back to top
View user's profile Send private message
Carnildo
Guru
Guru


Joined: 17 Jun 2004
Posts: 594

PostPosted: Sat Jun 22, 2013 2:10 am    Post subject: Reply with quote

wcg wrote:
At a guess, some corrupted data structure in the kernel's
dcache (directory cache) or inode cache. You can try to drop
caches when it happens and see if that fixes it.
Code:

echo 3 > /proc/sys/vm/drop_caches

Didn't fix it.
Back to top
View user's profile Send private message
wcg
Guru
Guru


Joined: 06 Jan 2009
Posts: 588

PostPosted: Sat Jun 22, 2013 2:55 am    Post subject: Reply with quote

Well, the shell has a concept of "current working directory", but that
part seems to be working, because listing the contents of directories
relative to the current directory works. You can check it of course:
Code:

pwd
env | grep `pwd`


But I expect that the results will be what you expect them to be.

The problem seems to be absolute paths (beginning with '/').
I do not really see how that could be other than an inode or
dcache problem.

(Is the shell mangling the first character of the pathname as
a command parameter? Then it seems like it would mangle
the first character of relative pathnames, too, and for commands
with more command line options, the shell would not know
which one was the pathname.)

What about something like emacs' dired (directory editing
mode), which will list the contents of a directory in an
editing window? (You may not have emacs. I expect that
vim has some command that will do the same thing, like
Code:

vim /var

) nano apparently does not have that functionality by default.
"nano /tmp" simply reports that /tmp is a directory (meaning
not an editable text file).

Do you see the directory with an editor's directory listing
command when passing both absolute and relative paths
to the editor?

I guess it could be some filesystem bug. What kind of filesystem
is this?
_________________
TIA
Back to top
View user's profile Send private message
Carnildo
Guru
Guru


Joined: 17 Jun 2004
Posts: 594

PostPosted: Sat Jun 22, 2013 3:31 am    Post subject: Reply with quote

wcg wrote:
I guess it could be some filesystem bug. What kind of filesystem
is this?

All of 'em. "/" is NFSv2 remounted as NFSv3, /home is NFSv3, "/dev" is a devtmpfs, "/proc" is procfs, "/run" and "/tmp" are tmpfs, "/sys" is a sysfs, "/mnt/cdrom" is an ISO9660 filesystem, "/mnt/memcard" is FAT32, and so on.
Back to top
View user's profile Send private message
wcg
Guru
Guru


Joined: 06 Jan 2009
Posts: 588

PostPosted: Sat Jun 22, 2013 12:36 pm    Post subject: Reply with quote

So probably not a filesystem bug. VFS at most.

This is not a problem that I see reported a lot. I suspect that
having root mounted over nfs has something to do with it,
although I do not know what exactly. Maybe nfs users will
have more insight or be able to suggest diagnostics.
_________________
TIA


Last edited by wcg on Mon Jun 24, 2013 1:56 pm; edited 1 time in total
Back to top
View user's profile Send private message
TomWij
Retired Dev
Retired Dev


Joined: 04 Jul 2012
Posts: 1553

PostPosted: Sat Jun 22, 2013 8:11 pm    Post subject: Re: Filesystems go missing Reply with quote

Carnildo wrote:
The root filesystem is still there, but everything else (/dev, /proc, /home, and so on) goes missing: the mountpoints show up as empty directories, or in the case of /dev, the static device nodes on the root filesystem. The filesystems still show up as mounted, and any terminal with a working directory in one of the filesystems can still see its contents (but only if accessed using relative paths).


Interesting, haven't heard about this, though it sounds like an udev or kernel bug.

Carnildo wrote:
I'm running kernel 3.6.11; other version info is available on request.


Could you try a more recent kernel? The stable 3.8.13, the unstable 3.9.7 or the development 3.10-rc6?
Back to top
View user's profile Send private message
Carnildo
Guru
Guru


Joined: 17 Jun 2004
Posts: 594

PostPosted: Sun Jun 23, 2013 9:13 pm    Post subject: Re: Filesystems go missing Reply with quote

TomWij wrote:
Carnildo wrote:
The root filesystem is still there, but everything else (/dev, /proc, /home, and so on) goes missing: the mountpoints show up as empty directories, or in the case of /dev, the static device nodes on the root filesystem. The filesystems still show up as mounted, and any terminal with a working directory in one of the filesystems can still see its contents (but only if accessed using relative paths).


Interesting, haven't heard about this, though it sounds like an udev or kernel bug.

That's the conclusion I came to as well, after having the problem with ever-more-minimal systems (I'd narrowed it down to one of udev, bash, init, md5sum, or the kernel, and I'm pretty sure that bash and md5sum running as an unprivileged user can't cause it).

Quote:
Carnildo wrote:
I'm running kernel 3.6.11; other version info is available on request.


Could you try a more recent kernel? The stable 3.8.13, the unstable 3.9.7 or the development 3.10-rc6?

3.8.13 has been running without trouble for a little over 24 hours, which is longer than 3.6.11 ever managed once things started going wrong.
Back to top
View user's profile Send private message
augury
l33t
l33t


Joined: 22 May 2004
Posts: 722
Location: philadelphia

PostPosted: Tue Jun 25, 2013 7:28 am    Post subject: Reply with quote

Your disk may be going bad. It's a physical thing. Overheating, dirt.

What sort of spindles are you running? Any noise? Difficulty starting up? You can *usually* save the data but the reliability will decrease. The performance hit isn't to bad.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Kernel & Hardware All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum