| View previous topic :: View next topic |
| Author |
Message |
bernjuer
n00b
Joined: 20 Nov 2002
Posts: 22
|
 Posted: Fri Jun 06, 2003 3:11 pm Post subject: Graphical System Statistics with rrd and some scripts |
 |
|
Requirements
Apache, rrd, perl, bash, lmsensors, metalog,snmp
Optional: SpamAssassin and Amavis
Download
http://www.weihenstephan.org/~bernjuer/rrd/
Remarks
I wanted to monitor some new systems and decided
to use rrd instead of mrtg. Soon I found out that
the setup of rrd ist quite complex, so I decided
to hack some scripts together. Oh, I did this
while having a bad flue.
You have been warned.
Description
A set of scripts to set up a system monitor.
Tracked informations are:
* system load
* trafic on two net devices
* system temperature
* mail trafic
Data is shown in 4 time frames:
* actual (last 3 hours)
* daily (last 24 hours)
* montly (last 5 weeks)
* yearls (last 15 months)
Data is sample every 5 minutes for the net devices and
for load, temperature and email are gathered every 15 minutes.
The graphics are generated on the fly by using rrd's
cgi functionality.
Installation
Create a directory somewhere, I used /home/httpd/rrd.
Create a User called rrd and set his home directory to
the freshly created directory.
Add this user to the group "cron" by writing rrd after
the line starting with "cron" in /etc/group. The line
should look like "cron:x:16:cron,bernd,rrd".
Allow apache to enter and this directory
"chown .apache /home/httpd/rrd"
"chmod g+rwx /home/httpd/rrd"
Allow Apache to run CGIs in the chosen directory
by adding something like
| Code: | <Directory /home/httpd/htdocs/rrd>
AllowOverride All
Options ExecCGI
Order allow,deny
Allow from all
</Directory> |
to Apache's config file (/etc/apache/conf/commonapache.conf).
Restart Apache by calling "/etc/init.d/apache restart".
If you want to analyze mail logs, the user rrd must have
read access to /var/log/mail/current. To do this,
create a group called logger with the user rrd by adding a
line like"logger:x:82:rrd" to /etc/group.
Allow read access to the directory by calling
"chown .logger /var/log/mail"
"chmod g+rx /var/log/mail"
"su" to the new user ("su rrd").
Create the rrd databases by running createrrds.sh
createrrds.sh is:
| Code: |
#!/bin/bash
#
# bernds basic aproach to data gathering
# V0.1 GPL, no warranty, yadadada...
#
# data for eth0, interval 5 min (--step 300)
# UNKNOWN after 10min w/o data (600)
#
# Store every sample (1) for ten days
# 10*24*60*60 /300
#
# 2880 samples* 300s per sample / 60 / 60 / 24 = 10
#
#
# Store average of every half hour (6) for 50 days
# 100*24*60*60 / 300 / 6
#
# 2400 samples* 30min per avg / 60 / 24 = 50
#
# Store average of every 3 hours (36) for 500 days (1.5 years)
#
# 4000 samples* 3h per avg / 24 = 50
rrdtool create eth0.rrd --step 300 \
DS:input:COUNTER:600:0:U \
DS:output:COUNTER:600:0:U \
RRA:AVERAGE:0.5:1:2880 \
RRA:AVERAGE:0.5:6:2400 \
RRA:AVERAGE:0.5:36:4000
#data for eth1, 5 min (300 s)
rrdtool create eth1.rrd --step 300 \
DS:input:COUNTER:600:0:U \
DS:output:COUNTER:600:0:U \
RRA:AVERAGE:0.5:1:2880 \
RRA:AVERAGE:0.5:6:2400 \
RRA:AVERAGE:0.5:36:4000
#data for load, 5 min
rrdtool create load.rrd --step 300 \
DS:val:COUNTER:600:0:U \
RRA:AVERAGE:0.5:1:2880 \
RRA:AVERAGE:0.5:6:2400 \
RRA:AVERAGE:0.5:36:4000
#data for temp, 15 min (900s )
rrdtool create temp.rrd --step 900 \
DS:temp1:GAUGE:1800:0:U \
DS:temp2:GAUGE:1800:0:U \
RRA:AVERAGE:0.5:1:2880 \
RRA:AVERAGE:0.5:3:2400 \
RRA:AVERAGE:0.5:12:4000
#data for mail, 15 min
rrdtool create mail.rrd --step 900 \
DS:in:COUNTER:1800:0:U \
DS:out:COUNTER:1800:0:U \
DS:spam:COUNTER:1800:0:U \
DS:virus:COUNTER:1800:0:U \
RRA:AVERAGE:0.5:1:2880 \
RRA:AVERAGE:0.5:3:2400 \
RRA:AVERAGE:0.5:12:4000
|
Create the CGI scripts by running createCGIs.sh
createCGIs.sh is:
| Code: | #!/bin/bash
./createCGI.pl 3 > index.cgi
./createCGI.pl 0 > day.cgi
./createCGI.pl 1 > month.cgi
./createCGI.pl 2 > year.cgi
chmod +x *cgi
chown apache.apache *cgi |
createCGI.pl is:
| Code: | #!/usr/bin/perl
use strict;
my @temps =("Temperatur","temp.rrd","temp1","temp2");
my @load =("Last","load.rrd","val");
my @mail =("EMail","mail.rrd","in","out","spam","virus");
my @eth0 =("eth0","eth0.rrd","input","output");
my @eth1 =("eth1","eth1.rrd","input","output");
my @sysinf=(\@temps,\@load,\@mail,\@eth0,\@eth1);
my @colors=("#00c000","#0000c0","#404040","#700000");
# <RRD::GRAPH tempa.gif --start -10800 -A --lazy --title="Temperatur" -h 80 -w 150
# DEF:cel1=temp.rrd:temp1:AVERAGE
# DEF:cel2=temp.rrd:temp2:AVERAGE
# LINE1:cel1#00a000:"CPU"
# LINE1:cel2#0000a0:"Board">
my $OPTS="-A -i "; # --lazy
my $SIZELARGE="-h 80 -w 610";
my $SIZESMALL="-h 80 -w 140";
my $BACKSMALL=-1;
my $BACKLARGE=-1;
my $PAGENAME="";
my $par = $ARGV[0];
if ($par == "0") {
$BACKSMALL=60*60*3; #3h
$BACKLARGE=60*60*24*1.5; #36h
$PAGENAME ="Aktuell";
}
if ($par == "1") {
$BACKSMALL=60*60*24*6; #1w
$BACKLARGE=60*60*24*7*5; #1m
$PAGENAME ="Monat";
}
if ($par == "2") {
$BACKSMALL=60*60*24*40; #1m
$BACKLARGE=60*60*24*500; #1y
$PAGENAME ="Jahr";
}
if ($par == "3") {
$BACKSMALL=60*60*6; #6h
$PAGENAME ="Systeminfo";
$SIZESMALL="-h 200 -w 300";
}
if ($BACKSMALL<0) { die "no para found, use {0,1,2,3}";}
printf("#!/usr/bin/rrdcgi\n");
printf("<HTML>\n".
"<!-- page generated by createCGI by bj 2003-6-6 (para ".$ARGV[0].") -->\n".
"<meta http-equiv=REFRESH content=120>\n".
"<HEAD><TITLE>$PAGENAME</TITLE></HEAD>\n".
"<BODY bgcolor=\"#a0a0a0\">\n<CENTER>\n");
if ($par != "3") {
print(" <TABLE border=0>");
for (my $i=0;$i<5;$i++) {
print(" <TR>\n");
print(" <TD>\n");
printOnePic($i,$OPTS,$SIZELARGE,$BACKLARGE);
print(" </TD>\n");
print(" <TD>\n");
printOnePic($i,$OPTS,$SIZESMALL,$BACKSMALL);
print(" </TD>\n");
print(" </TR>\n");
}
print(" </TABLE>\n");
print("<p>\n");
print("<a href=\"index.cgi\">Zurück</a><br>\n");
} else {
my $sysname = `uname -n`;
print(" <h2> ".$sysname." </h2>");
print(" <TABLE border=0>");
print(" <TR>\n");
print(" <TD>\n");
printOnePic(0,$OPTS,$SIZESMALL,$BACKSMALL);
print(" </TD>\n");
print(" <TD>\n");
printOnePic(2,$OPTS,$SIZESMALL,$BACKSMALL);
print(" </TD>\n");
print(" </TR><TR>\n");
print(" <TD>\n");
printOnePic(3,$OPTS,$SIZESMALL,$BACKSMALL);
print(" </TD>\n");
print(" <TD>\n");
printOnePic(4,$OPTS,$SIZESMALL,$BACKSMALL);
print(" </TD>\n");
print(" </TR>\n");
print(" </TABLE>\n");
print("<p>\n");
print("<a href=\"day.cgi\">Heute</a><br>\n");
print("<a href=\"month.cgi\">Monat</a><br>\n");
print("<a href=\"year.cgi\">Jahr</a><br>\n");
}
printf("</CENTER>\n</BODY>\n</HTML>\n");
sub printOnePic {
my $i = @_[0];
my $opts= @_[1];
my $size= @_[2];
my $range= @_[3];
my $title=$sysinf[$i][0];
my $name= $sysinf[$i][1];
printf(" <RRD::GRAPH $PAGENAME".$i.$range.".gif --start -$range $opts --title=\"$title\" $size \n");
my $start=2;
for (my $j=$start;$j<10+$start;$j++) {
my $val =$sysinf[$i][$j];
if ($val eq "") { next; }
printf(" DEF:val$j=$name:$val:AVERAGE\n");
}
for (my $j=$start;$j<10+$start;$j++) {
my $val =$sysinf[$i][$j];
if ($val eq "") { next; }
if ($j==$start) {
printf(" LINE2:val$j".$colors[$j-$start].":$val\n");
} else {
printf(" LINE2:val$j".$colors[$j-$start].":$val\n");
}
}
printf(" >\n\n");
} |
So we have the databases and the scripts. Now we need to
populate the database.
Verify the two scripts getmailtemp.pl and getstat.pl.
getmailtemp.pl is
| Code: | #!/usr/bin/perl
#
# bernds basic aproach to statistics gathering
# V0.1 GPL, no warranty, yadadada...
#
# this should run every 5 mins
# the line in crontab looks like this:
# */5 * * * * /home/httpd/htdocs/...../rrd/getstat.pl
use strict;
my $debug=0;
my $line,
my $t1=0;
my $t2=0;
my $dummy;
my $res1;
my $rrt="/usr/bin/rrdtool";
my $sns="/usr/bin/sensors";
my $mail="/var/log/mail/current";
if ($ARGV[0] ne "") {
printf("Debug mode, no data is written\n");
$debug=1;
}
# mail stats. requires postfix, metalog, spamassasin and amavis
my $msent= doexec("grep -c \"250 O\" $mail"); # sent mails, response 250
my $mrx= doexec("grep -c \"relay=local\" $mail"); # received mails
my $mspm= doexec("grep -c \"identified spam\" $mail"); # message from spamassasin
my $mvir= doexec("grep -c \"Virus found - quarantined\" $mail"); # message from amavis
if ($debug==0) {
$res1=system("$rrt update mail.rrd N:$mrx:$msent:$mspm:$mvir");
} else {
printf("Mail: Sent $mrx Rxed $msent Spam $mspm Virus $mvir\n");
}
# requires lm_sensors
#
# adapt to your needs: my board has two temp sensors
concat(FH,"$sns|");
while ($line=<FH>) {
# Apollo
#start of line followed by "temp1:"
if ($line =~ /^temp1:/) {
($dummy,$t1)=split(/ /,$line);
}
if ($line =~ /^temp2:/) {
($dummy,$t2)=split(/ /,$line);
}
# Via Eden
#start of line followed by "MB1: Temp"
if ($line =~ /^MB1 Temp:/) {
$t1=0; # no actual temp
}
if ($line =~ /^Proc Temp:/) {
$t2=substr $line,12,4;
}
}
if ($debug==0) {
$res1=system("$rrt update temp.rrd N:$t1:$t2");
} else {
print ("Temps $t1 $t2\n");
}
# execute external program and return retval as integer
sub doexec {
open(FH,"@_|") or die ("cant exec @_");
my $line =<FH>;
return $line+0;
}
|
getstat.pl is
| Code: |
#!/usr/bin/perl
#
# bernds basic aproach to data gathering
# V0.1 GPL, no warranty, yadadada...
#
# this should get called every 5 mins
# the line in crontab looks like this:
# */5 * * * * /home/httpd/htdocs/ ... /rrd/getstat.pl
use strict;
use Net::SNMP;
my $debug=0;
my $rrt="/usr/bin/rrdtool";
my $datapath="/usr/bin/rrdtool";
my $res1="";
my $res2="";
my $ld=0;
#
# Adjust to match your needs
#
my $s = Net::SNMP->session( -hostname => "127.0.0.1",
-community => "public" );
if ($ARGV[0] ne "") {
printf("Debug mode, no data is written\n");
$debug=1;
}
#
# Get OIDs by calling
#
#snmpwalk localhost public interfaces.ifTable.ifEntry.ifInOctets -On
#snmpwalk localhost public interfaces.ifTable.ifEntry.ifOutOctets -On
#
my $eth0in = '.1.3.6.1.2.1.2.2.1.10.2';
my $eth0out = '.1.3.6.1.2.1.2.2.1.16.2';
my $eth1in = '.1.3.6.1.2.1.2.2.1.10.3';
my $eth1out = '.1.3.6.1.2.1.2.2.1.16.3';
# fetch data from uptime
my $up=`/usr/bin/uptime` or die "Can't get uptime $!\n";
($res1,$res2) = split("average: ",$up);
($ld) = split(",",$res2);
if ($debug==1) {
printf("Load $ld ($res1) ($up)\n");
} else {
$res1=system("$rrt update load.rrd N:$ld");
}
# fetch data
my $res = $s->get_request(-varbindlist => [$eth0in,$eth0out,$eth1in,$eth1out]);
# throw data at rrd
if ($debug==1) {
printf("eth0 bytes in $res->{$eth0in} out $res->{$eth0out}\n");
printf("eth1 bytes in $res->{$eth1in} out $res->{$eth1out}\n");
} else {
$res1=system("$rrt update eth0.rrd N:$res->{$eth0in}:$res->{$eth0out}");
$res1=system("$rrt update eth1.rrd N:$res->{$eth1in}:$res->{$eth1out}");
} |
Test them by calling them with any parameter, thus enabling debug mode.
That is :
"./getmailtemp.pl -deb"
should return something like
| Code: | Debug mode, no data is written
Mail: Sent 6 Rxed 0 Spam 0 Virus 0
Temps 0 49.6 |
"./getstat.pl -deb"
should return something like
| Code: |
Debug mode, no data is written
Load 0.05 (0.05,)
eth0 bytes in 68454004 out 5938416
eth1 bytes in 39313841 out 126012954 |
Now add these scripts to cron by calling
"crontab -e" and pasting these two lines:
| Code: |
*/15 * * * * /home/httpd/htdocs/rrd/getmailtemp.pl > /dev/null 2>&1
*/5 * * * * /home/httpd/htdocs/rrd/getstat.pl > /dev/null 2>&1 |
Go have a party or something and after a while you should
see some lines when pointing mozilla to http://yourhost/rrd/index.cgi[url][/url]
_________________
sigs are boring |
|
| Back to top |
|
 |
togge
n00b
Joined: 17 Mar 2003
Posts: 14
|
| Posted: Thu Jun 19, 2003 9:21 pm Post subject: |
|
|
Nice post 
But could you tell us more about the snmp part ?
Maybe give an snmpd.conf example |
|
| Back to top |
|
|
smouge
n00b
Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands
|
| Posted: Thu Jun 26, 2003 8:03 pm Post subject: |
|
|
THe following link might be interesting for you as well:
http://hotsanic.sourceforge.net/
THis provides some easy ready made scripts to use. |
|
| Back to top |
|
|
RaymaN
Tux's lil' helper
Joined: 15 Aug 2002
Posts: 93
Location: Earth, Europe, Germany, Berlin
|
| Posted: Wed Jul 23, 2003 9:22 am Post subject: |
|
|
@ bernjuer:
Very nice  Works with the standard syslogd ??
ray |
|
| Back to top |
|
|
RaymaN
Tux's lil' helper
Joined: 15 Aug 2002
Posts: 93
Location: Earth, Europe, Germany, Berlin
|
| Posted: Wed Jul 23, 2003 9:22 am Post subject: |
|
|
@ bernjuer:
Very nice Works with the standard syslogd ??
ray |
|
| Back to top |
|
|
smouge
n00b
Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands
|
| Posted: Fri Jul 25, 2003 6:38 am Post subject: |
|
|
I'm not sure, but the ready made scripts/plugins are mostly in perl, so you can see what is happening.
A lot of info it gets from /proc , network traffic by snmp
I haven't been progrmming new plug-ins , but at leat this HotSaNIC will give you a nice frame to place more plugins, and you can use the included plugins like eth by snmp almost out from the box.
There is no ebuild, but you could install it from source |
|
| Back to top |
|
|
swimmer
Veteran
Joined: 15 Jul 2002
Posts: 1260
Location: Netherlands
|
| Posted: Fri Jul 25, 2003 9:10 am Post subject: |
|
|
Nice work Bernd ...
Greetz
Stefan |
|
| Back to top |
|
|
cmay4
Tux's lil' helper
Joined: 18 May 2002
Posts: 109
|
| Posted: Fri Jul 25, 2003 4:01 pm Post subject: Another consideration |
|
|
Take a look at Cacti:
Pretty full featured with an excellent interface for setting types of graphs and options. Unfortunately requires MySQL (no PostgreSQL support yet). |
|
| Back to top |
|
|
Jevin Sweval
n00b
Joined: 19 Jun 2003
Posts: 64
Location: Indiana
|
| Posted: Sat Jul 26, 2003 5:37 pm Post subject: Re: Another consideration |
|
|
| cmay4 wrote: | Take a look at Cacti:
Pretty full featured with an excellent interface for setting types of graphs and options. Unfortunately requires MySQL (no PostgreSQL support yet). |
exactly!
i even have it monitoring the number of users on a vbulletin forum i visit via a webscrapping perl script that uses WWW::Mechanize. i have it monitor the number of pages printed on the home office printer via SNMP! 
it is hosted on my Rackshack server. you can check it out at http://www.futagollc.com/cacti/static/
the interface to add new hosts, data sources, and graphs is very easy to use. I made a graph template so that I can monitor the number of users on ANY vbulletin forum with just a few clicks!
_________________
Help me!  |
|
| Back to top |
|
|
dylix
Apprentice
Joined: 20 Sep 2002
Posts: 261
Location: Atlanta
|
|
| Back to top |
|
|
smouge
n00b
Joined: 22 Jan 2003
Posts: 66
Location: Oosterhout, the Netherlands
|
|
| Back to top |
|
|
Guezz-Who
Tux's lil' helper
Joined: 15 Mar 2003
Posts: 90
Location: Norway::Bergen
|
| Posted: Sun Aug 03, 2003 7:14 pm Post subject: LRRD |
|
|
You might also want to take a look at lrrd, i run it here: http://lurium.guezz.net/lrrd/guezz.net/lurium.guezz.net.html
From lrrd's homepage ( http://lrrd.sf.net/ )
| Quote: | | Lrrd is a tool for graphing all sorts of information about one or more servers and displaying it in a web interface. It uses the execellent RRDTool (written by Tobi Oetiker) and is written in Perl. LRRD has a server/client architecture. The server connects to all the clients at regular intervals, and asks them for data. It then stores the data in RRD-files, and (if needed) updates the graphs. One of the main goals have been ease of creating own "plugins" (graphs). |
Too bad there are no ebuilds yet.. and some of the perl modules arn't even in portage (at least not when I installed it).. You can get them via CPAN though..
_________________
If you do something right once, someone will ask you to do it again. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
