Gentoo Forums
Gentoo Forums
Gentoo Forums
Quick Search: in
Server crashes, can't figure out why?
View unanswered posts
View posts from last 24 hours

 
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo
View previous topic :: View next topic  
Author Message
Karsten1973
Apprentice
Apprentice


Joined: 17 Dec 2004
Posts: 298
Location: Berlin, Germany

PostPosted: Thu Aug 07, 2014 10:27 am    Post subject: Server crashes, can't figure out why? Reply with quote

Hey guys,

I'm running Gentoo on my server - its an old box without a graphic card or a keyboard. Everything I do I do via ssh.

But lately my server just stops running. I usually notice because mpd stops playing. It runs samba, a backup routine, mpd... not much more. I try to use popfile to sort my imap files, and thats new - might be a reason.

When the server crashes, it still reacts to pings, but not to ssh. That keeps me from figuring out what happend.

How would you proceed? Still a noob here...
_________________
I could be arguing on my own time.
Back to top
View user's profile Send private message
bbgermany
Veteran
Veteran


Joined: 21 Feb 2005
Posts: 1490
Location: Oranienburg/Germany

PostPosted: Thu Aug 07, 2014 10:38 am    Post subject: Reply with quote

Hi,

since you dont have a graphic card installed, you could to run you tty0 on a serial connection. Maybe this can help you with debugging a bit.

bb
_________________
1st: i5-4440, 16GB, 1,75TB
2nd: Celeron E3400, 2GB, 500GB
3rd: Athlon II X2 215, 8GB, 7TB
4th: Samung N350, 2GB, 250GB
5th: Rasperry Pi, 512MB, 4GB USB + NFS
6th: Cubietruck, 2GB, 8GB NAND + NFS
Back to top
View user's profile Send private message
Karsten1973
Apprentice
Apprentice


Joined: 17 Dec 2004
Posts: 298
Location: Berlin, Germany

PostPosted: Thu Aug 07, 2014 10:54 am    Post subject: Reply with quote

I was hoping for some way to use log files to figure out what is happening. Maybe its the memory thats full, or something like that?
_________________
I could be arguing on my own time.
Back to top
View user's profile Send private message
wraeth
n00b
n00b


Joined: 08 May 2007
Posts: 64
Location: Australia

PostPosted: Thu Aug 07, 2014 11:23 am    Post subject: Reply with quote

You could try running `vmstat 5 > /path/to/file.log`. This will capture various system statistics every 5 seconds and, with that redirect, save it to a log file for you to review when you can access it again.

If you don't have it, it's from the sys-process/procps package.

Edit:
It is worth noting that this will have to be run either with a nohup or in a session that you leave connected:
Code:
nohup vmstat 5 > /root/vmstat.log &
Back to top
View user's profile Send private message
Karsten1973
Apprentice
Apprentice


Joined: 17 Dec 2004
Posts: 298
Location: Berlin, Germany

PostPosted: Thu Aug 07, 2014 11:28 am    Post subject: Reply with quote

Thanks, I will try that first!
_________________
I could be arguing on my own time.
Back to top
View user's profile Send private message
dataking
Apprentice
Apprentice


Joined: 20 Apr 2005
Posts: 196

PostPosted: Thu Aug 07, 2014 8:52 pm    Post subject: Reply with quote

wraeth wrote:
You could try running `vmstat 5 > /path/to/file.log`. This will capture various system statistics every 5 seconds and, with that redirect, save it to a log file for you to review when you can access it again.

If you don't have it, it's from the sys-process/procps package.

Edit:
It is worth noting that this will have to be run either with a nohup or in a session that you leave connected:
Code:
nohup vmstat 5 > /root/vmstat.log &


screen, for the win.

man screen
_________________
-= the D@7@k|n& =-
Back to top
View user's profile Send private message
Aiken
Tux's lil' helper
Tux's lil' helper


Joined: 22 Jan 2003
Posts: 148
Location: Toowoomba/Australia

PostPosted: Fri Aug 08, 2014 12:43 am    Post subject: Reply with quote

I have sysklogd set up on a machine I can use as a central logger. At times on a headless machine when I have a problem I'll make use of netconsole.

https://www.kernel.org/doc/Documentation/networking/netconsole.txt

It may provide some clues to what is happening. If you don't want to set up a machine for logging that page also describes how to do this with netcat on the receiving machine.
_________________
Beware the grue.
Back to top
View user's profile Send private message
Karsten1973
Apprentice
Apprentice


Joined: 17 Dec 2004
Posts: 298
Location: Berlin, Germany

PostPosted: Mon Aug 11, 2014 3:25 pm    Post subject: Reply with quote

@Aiken: I hope for a easier solution... :)

@all:

Server crashed while I was away. Popfile no longer works on it, so that can no longer be the problem. Here ist what vmstat gave:
Code:
vanity ~ # head vmstat.log
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 844388 198548 304376    0    0    56    16  235  508  2  1 94  3  0
 0  0      0 844288 198548 304392    0    0     0     0  391  967  2  1 97  0  0
 0  0      0 844288 198556 304392    0    0     0     6  429 1007  2  1 97  0  0
 1  0      0 843760 198568 304792    0    0    82     0  607 1362  2  2 96  1  0
 0  0      0 844008 198576 304916    0    0    26     2  614 1413  2  2 95  1  0
 0  0      0 844008 198576 304928    0    0     0     0  326  792  1  1 98  0  0
 0  0      0 843264 198584 305672    0    0   154     2  614 1387  2  1 96  0  0
 0  0      0 843604 198592 306232    0    0   113    14  555 1229  2  2 95  1  0
vanity ~ # tail vmstat.log
 4  0   4376  80052  19944   8016    0    0     0     2 2003  838  0 51 49  0  0
 4  0   4376  80052  19944   8016    0    0     0     0 2002  837  0 51 49  0  0
 4  0   4376  80052  19952   8016    0    0     0     2 2003  839  0 51 49  0  0
 4  0   4376  80052  19952   8016    0    0     0     0 2003  836  0 51 49  0  0
��H
_��H
�H
�1�H
�U�H
��H


Looks to me as if memory ran out eventually. But what was the cause for that? Any ideas how to find out what happend?

/
_________________
I could be arguing on my own time.
Back to top
View user's profile Send private message
Display posts from previous:   
Reply to topic    Gentoo Forums Forum Index Other Things Gentoo All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum