| View previous topic :: View next topic |
| Author |
Message |
Karsten1973
Guru
Joined: 17 Dec 2004
Posts: 303
Location: Berlin, Germany
|
 Posted: Thu Aug 07, 2014 10:27 am Post subject: Server crashes, can't figure out why? |
 |
|
Hey guys,
I'm running Gentoo on my server - its an old box without a graphic card or a keyboard. Everything I do I do via ssh.
But lately my server just stops running. I usually notice because mpd stops playing. It runs samba, a backup routine, mpd... not much more. I try to use popfile to sort my imap files, and thats new - might be a reason.
When the server crashes, it still reacts to pings, but not to ssh. That keeps me from figuring out what happend.
How would you proceed? Still a noob here...
_________________
I could be arguing on my own time. |
|
| Back to top |
|
 |
bbgermany
Veteran
Joined: 21 Feb 2005
Posts: 1844
Location: Oranienburg/Germany
|
| Posted: Thu Aug 07, 2014 10:38 am Post subject: |
|
|
Hi,
since you dont have a graphic card installed, you could to run you tty0 on a serial connection. Maybe this can help you with debugging a bit.
bb
_________________
Desktop: Ryzen 5 5600G, 32GB, 2TB, RX7600
Notebook: Dell XPS 13 9370, 16GB, 1TB
Server #1: Ryzen 5 Pro 4650G, 64GB, 16.5TB
Server #2: Ryzen 4800H, 32GB, 22TB |
|
| Back to top |
|
|
Karsten1973
Guru
Joined: 17 Dec 2004
Posts: 303
Location: Berlin, Germany
|
| Posted: Thu Aug 07, 2014 10:54 am Post subject: |
|
|
I was hoping for some way to use log files to figure out what is happening. Maybe its the memory thats full, or something like that?
_________________
I could be arguing on my own time. |
|
| Back to top |
|
|
wraeth
Developer
Joined: 08 May 2007
Posts: 72
Location: Australia
|
| Posted: Thu Aug 07, 2014 11:23 am Post subject: |
|
|
You could try running `vmstat 5 > /path/to/file.log`. This will capture various system statistics every 5 seconds and, with that redirect, save it to a log file for you to review when you can access it again.
If you don't have it, it's from the sys-process/procps package.
Edit:
It is worth noting that this will have to be run either with a nohup or in a session that you leave connected:
| Code: | | nohup vmstat 5 > /root/vmstat.log & |
|
|
| Back to top |
|
|
Karsten1973
Guru
Joined: 17 Dec 2004
Posts: 303
Location: Berlin, Germany
|
|
| Back to top |
|
|
dataking
Apprentice
Joined: 20 Apr 2005
Posts: 251
|
| Posted: Thu Aug 07, 2014 8:52 pm Post subject: |
|
|
| wraeth wrote: | You could try running `vmstat 5 > /path/to/file.log`. This will capture various system statistics every 5 seconds and, with that redirect, save it to a log file for you to review when you can access it again.
If you don't have it, it's from the sys-process/procps package.
Edit:
It is worth noting that this will have to be run either with a nohup or in a session that you leave connected:
| Code: | | nohup vmstat 5 > /root/vmstat.log & |
|
screen, for the win.
man screen
_________________
-= the D@7@k|n& =- |
|
| Back to top |
|
|
Aiken
Apprentice
Joined: 22 Jan 2003
Posts: 239
Location: Toowoomba/Australia
|
| Posted: Fri Aug 08, 2014 12:43 am Post subject: |
|
|
I have sysklogd set up on a machine I can use as a central logger. At times on a headless machine when I have a problem I'll make use of netconsole.
https://www.kernel.org/doc/Documentation/networking/netconsole.txt
It may provide some clues to what is happening. If you don't want to set up a machine for logging that page also describes how to do this with netcat on the receiving machine.
_________________
Beware the grue. |
|
| Back to top |
|
|
Karsten1973
Guru
Joined: 17 Dec 2004
Posts: 303
Location: Berlin, Germany
|
| Posted: Mon Aug 11, 2014 3:25 pm Post subject: |
|
|
@Aiken: I hope for a easier solution... 
@all:
Server crashed while I was away. Popfile no longer works on it, so that can no longer be the problem. Here ist what vmstat gave:
| Code: | vanity ~ # head vmstat.log
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 0 844388 198548 304376 0 0 56 16 235 508 2 1 94 3 0
0 0 0 844288 198548 304392 0 0 0 0 391 967 2 1 97 0 0
0 0 0 844288 198556 304392 0 0 0 6 429 1007 2 1 97 0 0
1 0 0 843760 198568 304792 0 0 82 0 607 1362 2 2 96 1 0
0 0 0 844008 198576 304916 0 0 26 2 614 1413 2 2 95 1 0
0 0 0 844008 198576 304928 0 0 0 0 326 792 1 1 98 0 0
0 0 0 843264 198584 305672 0 0 154 2 614 1387 2 1 96 0 0
0 0 0 843604 198592 306232 0 0 113 14 555 1229 2 2 95 1 0
vanity ~ # tail vmstat.log
4 0 4376 80052 19944 8016 0 0 0 2 2003 838 0 51 49 0 0
4 0 4376 80052 19944 8016 0 0 0 0 2002 837 0 51 49 0 0
4 0 4376 80052 19952 8016 0 0 0 2 2003 839 0 51 49 0 0
4 0 4376 80052 19952 8016 0 0 0 0 2003 836 0 51 49 0 0
��H
_��H
�H
�1�H
�U�H
��H
|
Looks to me as if memory ran out eventually. But what was the cause for that? Any ideas how to find out what happend?
/
_________________
I could be arguing on my own time. |
|
| Back to top |
|
|
|
Other Things Gentoo
