| View previous topic :: View next topic |
| Author |
Message |
drwindows
n00b
Joined: 02 Apr 2004
Posts: 9
|
 Posted: Wed Oct 06, 2004 4:22 pm Post subject: Using SNMP+MRTG+RRDTool for system monitoring & stats |
 |
|
Using SNMP + MRTG + RRDTool to generate system performance statistics
What will we use
What did I want
Several months ago, I wondered how could I get monitorized my Gentoo system. I wanted to know my router's througput, my CPU usage, my memory usage, and some other useful values. I wanted historical statistics, too.
What did I search for
Retrieving system statistics involves, as I see, three subsystems:
- Publisher
- Subscriber
- Database
The publisher must generate the usage or performance data. The subscriber must collect this data and store it into the database.
There are so many ways to mount a system with this structure, but I wanted it to be:
- Light
- Scalable
- Simple
- Well documented
Light means "not heavy", in other words, I don't want to add extra load to my system. I know it will need some CPU, some memory and some disk space, but the less, the better.
Scalable means that while my system grows, I can add extra data collection in an easy way.
Simple means that installation must have few steps, with few processes and few files and databases, which make it easy to maintain, disable, and ever uninstall.
Well documented means that it means.
What did I found
Then I met
- net-snmp as publisher.
- MRTG as subscriber.
- RRDTool as database.
Really, MRTG can act itself as subscriber and database manager. But tool's author, Tobi Oetiker, decided to implement a replacement more efficient for MRTG database manager. And of course he did.
With such tools, we can setup a decent monitoring and statistics system with a limited and non growing diskpace. It means that, given the RRDTool nature, you can harvest as much data as you want without filling up diskspace. This magic is done thanks to Round Robin Database RRDTool provides.
At this point, I suggest you to take a walk on the tools' pages. It saves me time to explain what are they and it saves you time to ask for their details.
Now that you've visited that pages, let's continue with this doc.
You'll be noticed that there is not a full tutorial on how to integrate previous three tools in a light way. That's why I'm spending my time writting this. Unfortunately, there's not simple way to accomplish our objetives, so I'll try to keep it as simple as possible.
Now let's get down into the mine.
Installation
What and where
On many tutorials and how-to's you don't know what are you going to install and where are you going to copy and modify files. I'll try to show you where and what before you start. I think it will give you a safety sensation and it will act as some kind of documentation where you'll find fast the little setting you want to move.
Following these instructions you'll have files into the following places:
for net-snmp:
- /usr/lib/net-snmp/* #living dir
- /usr/bin/ #links to living dir
- /etc/snmp/snmpd.conf #config file
- /etc/init.d/snmpd #startup file
- /etc/conf.d/snmpd #config of startup file
for MRTG
- /usr/lib/mrtg/* #living dir
- /usr/bin/ #links to living dir
- /etc/mrtg/mrtg.cfg #config file
- /etc/mrtg/*.inc # includes to config file
- /etc/init.d/mrtg #startup file
- /etc/conf.d/mrtg #config of startup file
for RRDTool
Is really difficyult to know where RRDtool files live. Anyway, we are going to merge/unmerge with no config, so we'll have to live ignoring that.
for our data
- /var/lib/mrtg/*.rrd #databases
- /home/mrtg/*#scripts and documentation
- /var/www/htdocs/graphs/* / #our graphs and generator scripts
We will create also a helper user, mrtg.
As an option, we will deliver the graphs via PHP script, so you'll need to have Apache with PHP interpreter installed if you want this feature.
At this point, you will notice this is not simple at all. Now is the time for the cowards to retreat. My braves, go on.
net-snmp
Quite simple, just and you're done.
Now we have a server part and a client part, let's start with server part.
SNMP server provides diverse data about the system where it lives in a standard way. All we need is to have the snmpd daemon up and running. But first take a look to config file /etc/snmp/snmpd.conf:
| Code: | > cat /etc/snmp/snmpd.conf
###########################################################################
#
# snmpd.conf
#
# - created by the snmpconf configuration program
#
###########################################################################
# SECTION: Access Control Setup
#
# This section defines who is allowed to talk to your running
# snmp agent.
# rwuser: a SNMPv3 read-write user
# arguments: user [noauth|auth|priv] [restriction_oid]
rwuser snmpuserrw auth
# rouser: a SNMPv3 read-only user
# arguments: user [noauth|auth|priv] [restriction_oid]
rouser snmpuserro auth
# rocommunity: a SNMPv1/SNMPv2c read-only access community name
# arguments: community [default|hostname|network/bits] [oid]
rocommunity public foton
# rwcommunity: a SNMPv1/SNMPv2c read-write access community name
# arguments: community [default|hostname|network/bits] [oid]
rwcommunity publicrw foton
###########################################################################
# SECTION: Trap Destinations
#
# Here we define who the agent will send traps to.
# trapcommunity: Default trap sink community to use
# arguments: community-string
trapcommunity public
###########################################################################
# SECTION: Monitor Various Aspects of the Running Host
#
# The following check up on various aspects of a host.
# proc: Check for processes that should be running.
# proc NAME [MAX=0] [MIN=0]
#
# NAME: the name of the process to check for. It must match
# exactly (ie, http will not find httpd processes).
# MAX: the maximum number allowed to be running. Defaults to 0.
# MIN: the minimum number to be running. Defaults to 0.
#
# The results are reported in the prTable section of the UCD-SNMP-MIB tree
# Special Case: When the min and max numbers are both 0, it assumes
# you want a max of infinity and a min of 1.
proc X 1 0
proc httpd 0 0
# disk: Check for disk space usage of a partition.
# The agent can check the amount of available disk space, and make
# sure it is above a set limit.
#
# disk PATH [MIN=100000]
#
# PATH: mount path to the disk in question.
# MIN: Disks with space below this value will have the Mib's errorFlag set.
# Can be a raw byte value or a percentage followed by the %
# symbol. Default value = 100000.
#
# The results are reported in the dskTable section of the UCD-SNMP-MIB tree
disk /mnt/dos 5000000
disk / 100000000
# load: Check for unreasonable load average values.
# Watch the load average levels on the machine.
#
# load [1MAX=12.0] [5MAX=12.0] [15MAX=12.0]
#
# 1MAX: If the 1 minute load average is above this limit at query
# time, the errorFlag will be set.
# 5MAX: Similar, but for 5 min average.
# 15MAX: Similar, but for 15 min average.
#
# The results are reported in the laTable section of the UCD-SNMP-MIB tree
load 90 70 50
###########################################################################
# SECTION: System Information Setup
#
# This section defines some of the information reported in
# the "system" mib group in the mibII tree.
# syslocation: The [typically physical] location of the system.
# Note that setting this value here means that when trying to
# perform an snmp SET operation to the sysLocation.0 variable will make
# the agent return the "notWritable" error code. IE, including
# this token in the snmpd.conf file will disable write access to
# the variable.
# arguments: location_string
syslocation despacho
# syscontact: The contact information for the administrator
# Note that setting this value here means that when trying to
# perform an snmp SET operation to the sysContact.0 variable will make
# the agent return the "notWritable" error code. IE, including
# this token in the snmpd.conf file will disable write access to
# the variable.
# arguments: contact_string
syscontact donotspanme@hotmail.com |
So, you've just read that this file was created using snmpconf configuration program. It's so easy to use and it's well documented, try it yourself. The entries we are going to use mainly are rocommunity, disk and proc.
Once you're satisfied with your configuration, start server.
Just type | Code: | | /etc/init.d/snmpd start |
and it will be running (I hope). If you want to make this permanent, type | Code: | | rc-update add snmpd default |
This way service will automatically started at startup.
So, we have our publisher set up and rrunning. Now we will play a little with our new toy. With
| Code: | | snmpwalk -v 1 -c public foton |
we will see what kind of data we can retrieve from our SNMP server. This list is huge, but you will soon see what values are interesting for you and what are not.
If you have a router in your network, you can take a walk on it via
| Code: | | snmpwalk -v 1 -c public router |
where router is your router address or name. By the way, now is the moment of creating a new entry in your hosts file or your DNS server.
Let's see some pruned output from last snmpwalk:
| Code: | > snmpwalk -v 1 -c public router
...
IF-MIB::ifDescr.1 = STRING: enet0
IF-MIB::ifDescr.2 = STRING: pppoe
...
IF-MIB::ifInOctets.1 = Counter32: 284517416
IF-MIB::ifInOctets.2 = Counter32: 626621858
...
IF-MIB::ifOutOctets.1 = Counter32: 586469877
IF-MIB::ifOutOctets.2 = Counter32: 311859315
... |
We'll use this data later.
When you get tired of playing with snmpwalk, let's continue.
Important
SNMP provides a way to spying and remote controlling your computer. Read carefully the documentation ant take proper actions to prevent security hazards.
MRTG
Simple again, just But this time, afer emerging we will not have our service configured. That is because MRTG as ebuild version , is intended for use with cron scheduled execution. But MRTG has the daemon option, and I like it because config files only are parsed once and the proggie is loaded into memory once. I'm not lack of process, but I don't want to waste it for free. Furthermore, I don't want to implicate more subsystems than the minimum necesary. So the daemon mode idea is that I like.
So we have to write down our own init scripts.
/etc/init.d/mrtg
| Code: | #!/sbin/runscript
# MRTG rc management file
depend() {
need net
}
checkconfig() {
if [ ! -e ${CONFIGFILE} ] ; then
eerror "You need an ${CONFIGFILE} config file to run mrtg"
return 1
fi
}
start() {
#checkconfig || return 1
ebegin "Starting mrtg"
if [ ! -f ${LOCKFILE} ]
then
start-stop-daemon --start --exec /usr/bin/mrtg -- --lock-file ${LOCKFILE} --daemon --pid-file=${PIDFILE} --logging ${LOGFILE} ${CONFIGFILE} &>/dev/null
eend $? "mrtg did not start, error code $?"
else
eend 1 "${LOCKFILE} locks execution"
fi
}
stop() {
ebegin "Stopping mrtg"
start-stop-daemon --stop --pidfile ${PIDFILE}
mrtg_ecode=$?
eend $mrtg_ecode "Error stopping the mrtg daemon, error $mrtg_ecode"
ebegin "Removing lock file"
rm -f ${LOCKFILE}
eend $? "Failed to remove ${LOCKFILE} , error $?"
return $mrtg_ecode
}
restart() {
svc_stop
sleep 5
svc_start
} |
/etc/conf.d/mrtg
| Code: | # /etc/conf.d/mrtg
# Config file for mrtg control script
# Change the following vars only if you know
# what you're doing, there's no checking for
# invalid data yet!
# owner of mrtg process
USER="mrtg"
# configuration file
CONFIGFILE="/etc/mrtg/mrtg.cfg"
#Process id file
PIDFILE="/var/run/mrtg.pid"
# Lock file (don't change, must be existing)
LOCKFILE="/var/lock/subsys/mrtg"
# File where mrtg logs messages
LOGFILE="/var/log/mrtg" |
Watch out long lines.
Don't start the service by now. First we have to configure MRTG and create RRDTool databases, we later will see why if MRTG creates them atumatically.
First, we will edit /etc/mrtg/mrtg.cfg
I suggest you split config file into various incude files, one per device managed, only for clarity purposes.
/etc/mrtg/mrtg.cfg
| Code: | WorkDir: /var/lib/mrtg
Logdir: /var/log
LogFormat: rrdtool
#Search for perl libs
#LibAdd: /usr/local/rrdtool/lib/perl/
#Search for rrdtool path
#PathAdd: /usr/local/rrdtool/bin/
RunAsDaemon:Yes
Interval:5
#LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt,/usr/share/snmp/mibs/HOST-RESOURCES-MIB.txt
Include: router.inc
Include: foton.inc |
Watch out long lines again.
/etc/mrtg/foton.inc
| Code: | #/etc/mrtg/foton.inc
# Included configuration file from /etc/mrtg/mrtg.cfg
# for gentoo linux box
#User vs System CPU usage
LoadMIBs: /usr/share/snmp/mibs/UCD-SNMP-MIB.txt,/usr/share/snmp/mibs/TCP-MIB.txt,/usr/share/snmp/mibs/HOST-RESOURCES-MIB.txt
Target[foton.usrsys]: ssCpuRawUser.0&ssCpuRawSystem.0:public@foton
MaxBytes[foton.usrsys]: 100
Title[foton.usrsys]: CPU usr sys
Target[foton.idlenice]: ssCpuRawIdle.0&ssCpuRawNice.0:public@foton
MaxBytes[foton.idlenice]: 100
Title[foton.idlenice]: CPU idle nice
#Open TCP connections
#MIB: /usr/share/snmp/mibs/TCP-MIB.txt
# .1.3.6.1.2.1.6.9.0
Target[foton.tcpopen]: tcpCurrEstab.0&tcpCurrEstab.0:public@foton
MaxBytes[foton.tcpopen]: 1000000
Title[foton.tcpopen]: Open TCP connections
Options[foton.tcpopen]: gauge
#System processes
#MIB: /usr/share/snmp/mibs/HOST-RESOURCES-MIB.txt
# .1.3.6.1.2.1.25.1.6.0
Target[foton.proc]: hrSystemProcesses.0&hrSystemProcesses.0:public@foton
MaxBytes[foton.proc]: 1000
Title[foton.proc]: Number of running processes
Options[foton.proc]: gauge
# Free memory, RAM and RAM&Swap
#Already loaded in this file
#MIB: /usr/local/share/snmp/mibs/UCD-SNMP-MIB.txt
# .1.3.6.1.4.1.2021.4.11.0
#Target[foton.freemem]: memTotalFree.0&memTotalFree.0:public@foton
#MaxBytes[foton.freemem]: 1000000
#Title[foton.freemem]: Free Memory Total
#Options[foton.freemem]: gauge
# Free RAM VS Free Swap
# Reported by snmpd
#total swap: 265064
#total real: 191424
#Total : 456488
Target[foton.ram.swap]: memAvailReal.0&memAvailSwap.0:public@foton
MaxBytes[foton.ram.swap]: 456488
Title[foton.ram.swap]: RAM vs swap Free Memory
Options[foton.ram.swap]: gauge
# Disk percent usage (almost unuseful)
#Already loaded in this file
#MIB: /usr/local/share/snmp/mibs/UCD-SNMP-MIB.txt
#.1.3.6.1.4.1.2021.9.1.9 dskPercent
#Target[foton.diskspercent]: .1.3.6.1.4.1.2021.9.1.9.1&.1.3.6.1.4.1.2021.9.1.9.2:public@foton
#MaxBytes[foton.diskspercent]: 100
#Title[foton.diskspercent]: disk usage percent
#Options[foton.diskspercent]: gauge,nopercent
# Disk available
#Already loaded in this file
#MIB: /usr/local/share/snmp/mibs/UCD-SNMP-MIB.txt
#.1.3.6.1.4.1.2021.9.1.7 dskAvail
Target[foton.disks.usage]: .1.3.6.1.4.1.2021.9.1.7.1&.1.3.6.1.4.1.2021.9.1.7.2:public@foton
MaxBytes1[foton.disks.usage]: 24691824
MaxBytes2[foton.disks.usage]: 14048404
Title[foton.disks.usage]: disk available totals
Options[foton.disks.usage]: gauge,nopercent |
/etc/mrtg/router.inc
| Code: | #Router electron
Target[electron.traffic]: 2:public@router
#Interval[electron.traffic]: 5
#2Mb/s = 250000 B/s
MaxBytes1[electron.traffic]: 250000
#1Mb/s = 125000 B/s
MaxBytes2[electron.traffic]: 125000
Title[electron.traffic]: ADSL Traffic Analysis |
Notice most of targets are result of snmpwalk experimenting. Target names will convert into rrd databases in WorkDir referenced path. And we have selected options for systems de have not installed yet. So this time we will wait next installation before playing with this one.
RRDTool
Really this app is an upgrade of storaging subsystem of MRTG. It provides a way to save time-series data, and consolidation functions on the fly. As you have had a look to its website, I will not extend on its features. Again, emerging is quite simple:
it have not config options, so we can start playing with it now.
We don't need customization, but we like it, don't we? When MRTG tries to store data into a database, if such doesn't exists, it creates one. But it creates it "its own way". It creates two datasources, and three consolidation functions, AVERAGE, MAX and LAST. Sometimes we want other CF, like MIN. When? for example, measuring free memory. In these cases, it's better to create our own databases using rrdtool create. So we will create some databases:
| Code: | #!/bin/bash
#/home/mrtg/create.disk.available.database
#database creation
#Execute snmpwalk to collect information about your disks:
#
#snmpwalk -v 1 -c public foton .1.3.6.1.4.1.2021.9.1
#UCD-SNMP-MIB::dskPath.1 = STRING: /mnt/dos
#UCD-SNMP-MIB::dskPath.2 = STRING: /
#UCD-SNMP-MIB::dskDevice.1 = STRING: /dev/hda5
#UCD-SNMP-MIB::dskDevice.2 = STRING: /dev/hda4
#UCD-SNMP-MIB::dskTotal.1 = INTEGER: 24691824
#UCD-SNMP-MIB::dskTotal.2 = INTEGER: 14048404
#
#Start now (`date +"%s"`)
#Heartbeat: 600 (10 min)
#Values between 0 and partition size
#Update interval 300 (5 min)
#Gather average and minimum each 5min (1), 30 min (6)
# 2 hour (24) and 1 day (288)
if [ ! -e /var/lib/mrtg/foton.disks.usage.rrd ]
then
rrdtool create /var/lib/mrtg/foton.disks.usage.rrd --start `date +"%s"` \
DS:ds0:GAUGE:600:0:24691824 \
DS:ds1:GAUGE:600:0:14048404 \
--step 300 \
RRA:AVERAGE:0.5:1:800 \
RRA:AVERAGE:0.5:6:800 \
RRA:AVERAGE:0.5:24:800 \
RRA:AVERAGE:0.5:288:800 \
RRA:MIN:0.5:1:800 \
RRA:MIN:0.5:6:800 \
RRA:MIN:0.5:24:800 \
RRA:MIN:0.5:288:800 \
RRA:MAX:0.5:1:800 \
RRA:MAX:0.5:6:800 \
RRA:MAX:0.5:24:800 \
RRA:MAX:0.5:288:800
else echo database alerady exists!
fi
#EOF |
| Code: | #!/bin/bash
# /home/mrtg/create.electron.traffic.database
# database creation
# Execute snmpwalk to collect information about your disks:
#
# snmpwalk -v 1 -c public router
#
# Start now (`date +"%s"`)
# Heartbeat: 600 (10 min)
# Values between 0 and maximum throghput of interface
# We use 50K for income to leave space if one maximum exceeds
# physical limit of 32K due to a incorrect value obtained.
# Update interval 300 (5 min)
# Gather average and minimum each 5min (1), 30 min (6)
# 2 hour (24) and 1 day (288)
if [ ! -e /var/lib/mrtg/electron.traffic.rrd ]
then
rrdtool create /var/lib/mrtg/electron.traffic.rrd --start `date +"%s"` \
DS:ds0:COUNTER:600:0:50000 \
DS:ds1:COUNTER:600:0:25000 \
--step 300 \
RRA:AVERAGE:0.5:1:800 \
RRA:AVERAGE:0.5:6:800 \
RRA:AVERAGE:0.5:24:800 \
RRA:AVERAGE:0.5:288:800 \
RRA:MIN:0.5:1:800 \
RRA:MIN:0.5:6:800 \
RRA:MIN:0.5:24:800 \
RRA:MIN:0.5:288:800 \
RRA:MAX:0.5:1:800 \
RRA:MAX:0.5:6:800 \
RRA:MAX:0.5:24:800 \
RRA:MAX:0.5:288:800
else echo database alerady exists!
fi
#EOF |
I've created mrtg user. Acts as daemon owner, and holds at home motinoring configuration a scripts.
Hint: If someday your router speed grows or want to change some limits, don't get stuck as I did, use rrdtool tune. I've complicated my life until I found this simple solution. Don't forget to update your creation files, that way you can reinstall your monitoring system faster.
Making it work together
Now we can start MRTG on our known way, | Code: | | /etc/init.d/mrtg start |
You may find a lot of troubles, as I did. Be patient, you'll be rewarded with full-coloured statistics about your system that for sure will help you to take accurate decissions.
After a while of work, you'll begin to have some numbers. Exploit this time and read your installation documentation. For sure you will imagine new ideas for your system to optimize. After that, It is time to create the graphs. We'll do it using shell scripts, this way you can always execute processes form within your app.
Generating the graphs
| Code: | #~mrtg/traffic.adsl.router
# Generates interval graph for router traffic
#
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/electron.bandwith.$INTERVALSTR.png
echo Using $INTERVAL interval
rrdtool graph /var/www/localhost/htdocs/graphs/electron.bandwith.$INTERVALSTR.png \
-s -$INTERVAL \
-a PNG \
-z \
DEF:inspeed_bytes=/var/lib/mrtg/electron.traffic.rrd:ds0:AVERAGE \
DEF:outspeed_bytes=/var/lib/mrtg/electron.traffic.rrd:ds1:AVERAGE \
DEF:inspeed_bytes_max=/var/lib/mrtg/electron.traffic.rrd:ds0:MAX \
DEF:outspeed_bytes_max=/var/lib/mrtg/electron.traffic.rrd:ds1:MAX \
"CDEF:in_bytes=inspeed_bytes,0,256000,LIMIT,UN,0,inspeed_bytes,IF,$INTERVAL,*" \
"CDEF:out_bytes=outspeed_bytes,0,128000,LIMIT,UN,0,outspeed_bytes,IF,$INTERVAL,*" \
"CDEF:total_bytes=in_bytes,out_bytes,+" \
HRULE:32000#F0C0C0:"Top outcoming\n" \
COMMENT:" Max Avg Current Volume\n" \
AREA:inspeed_bytes#00FF00:"Incoming " \
GPRINT:inspeed_bytes:MAX:'%5.2lf %sb/s' \
GPRINT:inspeed_bytes:AVERAGE:"%5.2lf %Sb/s" \
GPRINT:inspeed_bytes:LAST:"%5.2lf %Sb/s" \
GPRINT:in_bytes:AVERAGE:"%8.2lf %sB \n" \
LINE1:inspeed_bytes#007F00 \
LINE1:outspeed_bytes#FF0000:"Outcoming" \
GPRINT:outspeed_bytes:MAX:'%5.2lf %sb/s' \
GPRINT:outspeed_bytes:AVERAGE:"%5.2lf %Sb/s" \
GPRINT:outspeed_bytes:LAST:"%5.2lf %Sb/s" \
GPRINT:out_bytes:AVERAGE:"%8.2lf %sB \n" \
GPRINT:total_bytes:AVERAGE:" total %8.2lf %sB\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
-v "Bytes/s" -t "Router traffic - last $INTERVALSTR" -l 0 |
| Code: |
###########################
#/home/mrtg/disks.available.foton
#!/bin/bash
#Generates a disk space availability graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/disks.available.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
rrdtool graph /var/www/localhost/htdocs/graphs/disks.available.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-b 1024 \
-a PNG \
-z \
DEF:dos_dsk_Kbytes=/var/lib/mrtg/foton.disks.usage.rrd:ds0:AVERAGE \
DEF:root_dsk_Kbytes=/var/lib/mrtg/foton.disks.usage.rrd:ds1:AVERAGE \
DEF:dos_dsk_Kbytes_min=/var/lib/mrtg/foton.disks.usage.rrd:ds0:MIN \
DEF:root_dsk_Kbytes_min=/var/lib/mrtg/foton.disks.usage.rrd:ds1:MIN \
DEF:dos_dsk_Kbytes_max=/var/lib/mrtg/foton.disks.usage.rrd:ds0:MAX \
DEF:root_dsk_Kbytes_max=/var/lib/mrtg/foton.disks.usage.rrd:ds1:MAX \
"CDEF:dos_dsk_bytes=dos_dsk_Kbytes,1024,*" \
"CDEF:root_dsk_bytes=root_dsk_Kbytes,1024,*" \
"CDEF:dos_dsk_bytes_min=dos_dsk_Kbytes_min,1024,*" \
"CDEF:root_dsk_bytes_min=root_dsk_Kbytes_min,1024,*" \
"CDEF:dos_dsk_bytes_max=dos_dsk_Kbytes_max,1024,*" \
"CDEF:root_dsk_bytes_max=root_dsk_Kbytes_max,1024,*" \
"CDEF:total_dsk_bytes=root_dsk_bytes,dos_dsk_bytes,+" \
COMMENT:" min Avg Max Current\n" \
LINE1:root_dsk_bytes#FF00FF:"/ " \
GPRINT:root_dsk_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:root_dsk_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:root_dsk_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:root_dsk_bytes:LAST:"%7.2lf %SBytes\n" \
LINE1:dos_dsk_bytes#0000FF:"/mnt/dos" \
GPRINT:dos_dsk_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:dos_dsk_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:dos_dsk_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:dos_dsk_bytes:LAST:"%7.2lf %SBytes\n" \
GPRINT:total_dsk_bytes:LAST:" Total %7.2lf %SBytes\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
COMMENT:"`uptime`\n" \
-v "Bytes" -t "Disk availability - last $INTERVALSTR" -l 0 |
| Code: | ###########################
#/home/mrtg/tcp.foton
#!/bin/bash
#Generates a tcp connections count graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/tcp.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
rrdtool graph /var/www/localhost/htdocs/graphs/tcp.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-a PNG \
-z \
DEF:num_processes=/var/lib/mrtg/foton.tcpopen.rrd:ds0:AVERAGE \
DEF:num_processes_max=/var/lib/mrtg/foton.tcpopen.rrd:ds0:MAX \
COMMENT:" Avg Max Current\n" \
LINE1:num_processes#FF0000:"TCP count " \
GPRINT:num_processes:AVERAGE:"%7.2lf %S" \
GPRINT:num_processes:MAX:'%7.2lf %S' \
GPRINT:num_processes:LAST:"%7.0lf %S\n" \
LINE1:num_processes_max#00FF00:"TCP count max " \
GPRINT:num_processes_max:AVERAGE:"%7.2lf %S" \
GPRINT:num_processes_max:MAX:'%7.2lf %S' \
GPRINT:num_processes_max:LAST:"%7.0lf %S\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
-v "TCP Connections" -t "TCP Connection count - last $INTERVALSTR" |
| Code: | ###########################
#/home/mrtg/processes.foton
#!/bin/bash
#Generates a process count graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/processes.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
rrdtool graph /var/www/localhost/htdocs/graphs/processes.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-a PNG \
-z \
DEF:num_processes=/var/lib/mrtg/foton.proc.rrd:ds0:AVERAGE \
DEF:num_processes_max=/var/lib/mrtg/foton.proc.rrd:ds0:MAX \
COMMENT:" Avg Max Current\n" \
LINE1:num_processes#FF0000:"Process count " \
GPRINT:num_processes:AVERAGE:"%7.2lf %S" \
GPRINT:num_processes:MAX:'%7.2lf %S' \
GPRINT:num_processes:LAST:"%7.0lf %S\n" \
LINE1:num_processes_max#00FF00:"Proc count max" \
GPRINT:num_processes_max:AVERAGE:"%7.2lf %S" \
GPRINT:num_processes_max:MAX:'%7.2lf %S' \
GPRINT:num_processes_max:LAST:"%7.0lf %S\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
-v "Processes" -t "Process count - last $INTERVALSTR" |
| Code: | ###########################
#/home/mrtg/memory.foton
#!/bin/bash
#Generates a memory availability / usage graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/memory.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
#total swap: 265064
TOTALSWAP=265064
rrdtool graph /var/www/localhost/htdocs/graphs/memory.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-b 1024 \
-a PNG \
-z \
DEF:ram_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds0:AVERAGE \
DEF:swap_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds1:AVERAGE \
DEF:ram_Kbytes_min=/var/lib/mrtg/foton.ram.swap.rrd:ds0:MIN \
DEF:swap_Kbytes_min=/var/lib/mrtg/foton.ram.swap.rrd:ds1:MIN \
DEF:ram_Kbytes_max=/var/lib/mrtg/foton.ram.swap.rrd:ds0:MAX \
DEF:swap_Kbytes_max=/var/lib/mrtg/foton.ram.swap.rrd:ds1:MAX \
"CDEF:ram_bytes=ram_Kbytes,1024,*" \
"CDEF:swap_bytes=$TOTALSWAP,swap_Kbytes,-,1024,*" \
"CDEF:ram_bytes_min=ram_Kbytes_min,1024,*" \
"CDEF:swap_bytes_min=$TOTALSWAP,swap_Kbytes_min,-,1024,*" \
"CDEF:ram_bytes_max=ram_Kbytes_max,1024,*" \
"CDEF:swap_bytes_max=$TOTALSWAP,swap_Kbytes_max,-,1024,*" \
COMMENT:" min Avg Max Current\n" \
LINE1:swap_bytes_max#FF8080 \
LINE1:swap_bytes_min#800000 \
LINE1:swap_bytes#FF0000:"swap usage " \
GPRINT:swap_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:swap_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:swap_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:swap_bytes:LAST:"%7.2lf %SBytes\n" \
LINE1:ram_bytes_max#80FF80 \
LINE1:ram_bytes_min#008000 \
LINE1:ram_bytes#00FF00:"real free " \
GPRINT:ram_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:ram_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:ram_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:ram_bytes:LAST:"%7.2lf %SBytes\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
COMMENT:"`uptime`\n" \
-v "Bytes" -t "Memory usage - last $INTERVALSTR" -l 0 |
| Code: | ###########################
#/home/mrtg/cpu.foton
#!/bin/bash
#Generates a CPU info graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/cpu.percent.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
rrdtool graph /var/www/localhost/htdocs/graphs/cpu.percent.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-a PNG \
-z \
DEF:user=/var/lib/mrtg/foton.usrsys.rrd:ds0:AVERAGE \
DEF:system=/var/lib/mrtg/foton.usrsys.rrd:ds1:AVERAGE \
DEF:idle=/var/lib/mrtg/foton.idlenice.rrd:ds0:AVERAGE \
DEF:nice=/var/lib/mrtg/foton.idlenice.rrd:ds1:AVERAGE \
"CDEF:total=100,idle,-" \
COMMENT:" Max Avg Current\n" \
AREA:system#FF4000:"System " \
GPRINT:system:MAX:'%7.2lf %%' \
GPRINT:system:AVERAGE:"%7.2lf %%" \
GPRINT:system:LAST:"%7.2lf %%\n" \
STACK:user#0080FF:"User " \
GPRINT:user:MAX:'%7.2lf %%' \
GPRINT:user:AVERAGE:"%7.2lf %%" \
GPRINT:user:LAST:"%7.2lf %%\n" \
STACK:nice#00FFFF:"Nice " \
GPRINT:nice:MAX:'%7.2lf %%' \
GPRINT:nice:AVERAGE:"%7.2lf %%" \
GPRINT:nice:LAST:"%7.2lf %%\n" \
LINE1:total#008080:"CPU " \
GPRINT:total:MAX:'%7.2lf %%' \
GPRINT:total:AVERAGE:"%7.2lf %%" \
GPRINT:total:LAST:"%7.2lf %%\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
COMMENT:"`uptime | sed 's/.*\(uptime[^,]*,[^,]*\).*/\1 hours/g'`\n" \
-v "%" -t "CPU usage - $INTERVALSTR" -l 0 |
| Code: | ###########################
#/home/mrtg/memory.stack.foton
#!/bin/bash
#Generates a memory stack graph
###########################
case $1 in
day)
INTERVAL=86400;;
week)
INTERVAL=604800;;
month)
INTERVAL=2678400;;
year)
INTERVAL=31622400;;
*)
INTERVAL=86400;;
esac
if [ $INTERVAL == 86400 ]; then
INTERVALSTR="day"
else
INTERVALSTR="$1"
fi
echo Generating graphs/memory.stack.foton.$INTERVALSTR.png
echo Using $INTERVAL interval
TOTALSWAP=265064
TOTALRAM=191424
rrdtool graph /var/www/localhost/htdocs/graphs/memory.stack.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-a PNG \
-z \
-b 1024 \
DEF:free_ram_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds0:AVERAGE \
DEF:free_swap_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds1:AVERAGE \
"CDEF:used_ram_bytes=$TOTALRAM,free_ram_Kbytes,-,1024,*" \
"CDEF:used_swap_bytes=$TOTALSWAP,free_swap_Kbytes,-,1024,*" \
"CDEF:used_mem_bytes=used_ram_bytes,used_swap_bytes,+" \
HRULE:$(($TOTALRAM * 1024))#FF8080:"Total RAM\n" \
COMMENT:" Max Avg Current\n" \
AREA:used_ram_bytes#40FFFF:"RAM " \
GPRINT:used_ram_bytes:MAX:'%7.2lf %SB' \
GPRINT:used_ram_bytes:AVERAGE:"%7.2lf %SB" \
GPRINT:used_ram_bytes:LAST:"%7.2lf %SB\n" \
STACK:used_swap_bytes#8080FF:"Swap " \
GPRINT:used_swap_bytes:MAX:'%7.2lf %SB' \
GPRINT:used_swap_bytes:AVERAGE:"%7.2lf %SB" \
GPRINT:used_swap_bytes:LAST:"%7.2lf %SB\n" \
LINE1:used_mem_bytes#0000FF:"Memory " \
GPRINT:used_mem_bytes:MAX:'%7.2lf %SB' \
GPRINT:used_mem_bytes:AVERAGE:"%7.2lf %SB" \
GPRINT:used_mem_bytes:LAST:"%7.2lf %SB\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
-v "Bytes" -t "Memory usage - $INTERVALSTR" |
This is "art-deco".
All scripts accepts one or none arguments.
| Code: | | traffic.adsl.router [day|week|month|year] |
generates last [period] graph. You can improve these adding arguments and refining intervals. It's your chance, but read the RRDtool graph manual first for best results.
Viewing the graphs
Now we'll wite a php script to see the results in a web page. Youl'll have to have installed php 3/4/5 and a web server if you want to make this script work.
Make links to previous files in /var/www/localhost/htdocs/graphs/gen/.
| Code: | > ls -la /var/www/localhost/htdocs/graphs/gen
total 6
drwxr-xr-x 2 root root 304 Oct 4 15:07 .
drwxr-xr-x 3 apache apache 1424 Oct 4 15:23 ..
lrwxrwxrwx 1 root root 30 May 17 15:16 1.traffic.adsl.router -> /home/mrtg/traffic.adsl.router
lrwxrwxrwx 1 root root 32 May 17 18:28 2.disks.available.foton -> /home/mrtg/disks.available.foton
lrwxrwxrwx 1 root root 20 May 17 18:29 3.tcp.foton -> /home/mrtg/tcp.foton
lrwxrwxrwx 1 root root 26 May 17 18:30 4.processes.foton -> /home/mrtg/processes.foton
lrwxrwxrwx 1 root root 23 May 17 18:31 5.memory.foton -> /home/mrtg/memory.foton
lrwxrwxrwx 1 root root 20 May 17 18:31 6.cpu.foton -> /home/mrtg/cpu.foton
lrwxrwxrwx 1 root root 29 May 17 18:32 7.memory.stack.foton -> /home/mrtg/memory.stack.foton |
Following script will execute the linked scripts and will show results in a HTML page.
| Code: | > cat /var/www/localhost/htdocs/graphs/index.php
<HTML>
<head>
<meta http-equiv="Pragma" content="no-cache">
<!-- Pragma content set to no-cache tells the browser not to cache the page
This may or may not work in IE -->
<meta http-equiv="expires" content="0">
<!-- Setting the page to expire at 0 means the page is immediately expired
Any vales less then one will set the page to expire some time in past and
not be cached. This may not work with Navigator -->
<title>System graphs</title>
</head>
<table width="100%">
<tr>
<td><a href="?interval=day">Day</a></td><td><a href="?interval=week">Week</a></td><td><a href="?interval=month">Month</a></td><td><a href="?interval=year">Year</a></td>
</tr>
</table>
<?php
include ("compat.php");
$gen = scandir("./gen");
sort($gen);
foreach($gen as $stat) {
if (substr($stat,0,1) != ".") {
$cmd = "./gen/" . $stat . " " . $_REQUEST["interval"];
echo ("<PRE>$cmd\n</PRE>");
$lines = array();
exec ("$cmd",$lines,$result);
//echo( passthru("/var/www/localhost/htdocs/graph/gen/$stat day",$result));
if ($result == 0) {
echo "<PRE>";
foreach($lines as $line) {
echo ("$line\n");
}
echo "</PRE>";
$line = explode("/",$lines[0]);
$pngfile = $line[1];
?>
<img src="<?=$pngfile ?>" alt="<?=$pngfile ?>" >
<h5> <?=$pngfile ?> </h5>
<?
} else {
?><h3> <?=$result?></h3>
<?
}
//flush();
}
}
?>
</HTML> |
php 4/5 compatibility script:
| Code: | > cat /var/www/localhost/htdocs/graphs/compat.php
<?php
if(!function_exists("scandir"))
{
function scandir($dirstr)
{
// php.net/scandir (PHP5)
$files = array();
$fh = opendir($dirstr);
while (false !== ($filename = readdir($fh)))
{
array_push($files, $filename);
}
closedir($fh);
return $files;
}
}
?> |
And that's all. Easy, huh?
Conclussion
Setting up this monitoring/statistics system is not a piece of cake. But if you do it, you'll be rewarded. Our objective is partially accomplished, because we have not achieved the simplicity. Anyway we have a good system and we can use it. You can optimize it for your system in many ways. You can edit generators and fit your time-period needs. You can spatiate samples acquisition to reduce system load and/or diskspace usage. You can improve graphs or create a CPU/motherboard temperature add-on. You are encouraged to improve this document with your ideas. One last advice: write down all steps you do, even the simplest. This is a complicated task and perhaps tomorow you can forget what you did to solve that tricky isuue you are experimenting.
Make me know if you find this doc useful.
Last edited by drwindows on Sat Nov 03, 2007 11:14 am; edited 2 times in total |
|
| Back to top |
|
 |
Lepaca Kliffoth
l33t
Joined: 28 Apr 2004
Posts: 737
Location: Florence, Italy
|
| Posted: Thu Oct 07, 2004 12:21 pm Post subject: |
|
|
Uhm snmpwalk says it gets no answer from host. Of course I put animebox instead of foton and I tried localhost as well but it still says no answer. Maybe I'll have to tinker with the configuration?
_________________
It isn't enough to win - everyone else must lose, and you also have to rub it in their face (maybe chop off an arm too for good measure).
Animebox! |
|
| Back to top |
|
|
Lepaca Kliffoth
l33t
Joined: 28 Apr 2004
Posts: 737
Location: Florence, Italy
|
| Posted: Thu Oct 07, 2004 12:28 pm Post subject: |
|
|
Sorry it was very easy. Stupid question. Forgive me.
_________________
It isn't enough to win - everyone else must lose, and you also have to rub it in their face (maybe chop off an arm too for good measure).
Animebox! |
|
| Back to top |
|
|
Lepaca Kliffoth
l33t
Joined: 28 Apr 2004
Posts: 737
Location: Florence, Italy
|
| Posted: Thu Oct 07, 2004 12:54 pm Post subject: |
|
|
You should mention that the /var/lib/mrtg directory must be created by hand.
_________________
It isn't enough to win - everyone else must lose, and you also have to rub it in their face (maybe chop off an arm too for good measure).
Animebox! |
|
| Back to top |
|
|
Caltex
n00b
Joined: 11 Oct 2003
Posts: 5
|
| Posted: Fri Oct 29, 2004 6:15 am Post subject: Can't start mrtg |
|
|
Hi
I got to the step of starting mrtg, but i get the error:
> /etc/init.d/mrtg start
* Starting mrtg...
* /var/lock/subsys/mrtg locks execution [ !! ]
why is that file locking execution?
Thanks for your help |
|
| Back to top |
|
|
drwindows
n00b
Joined: 02 Apr 2004
Posts: 9
|
| Posted: Fri Oct 29, 2004 2:07 pm Post subject: Re: Can't start mrtg |
|
|
MRTG daemon maintains a lock file (I guess it's in order to prevent executing the daemon twice or more) If daemon hangs, this file is left. I think it is safe to delete with | Code: | | rm -fv /var/lock/subsys/mrtg |
and start daemon. In fact, this is done by stop section of init script.
Make sure user has r/w permissions, too.
You can also refer to mrtg manual entry. |
|
| Back to top |
|
|
pjesi
n00b
Joined: 02 Apr 2004
Posts: 54
|
| Posted: Tue Nov 30, 2004 12:51 am Post subject: |
|
|
| Lepaca Kliffoth wrote: | | Uhm snmpwalk says it gets no answer from host. Of course I put animebox instead of foton and I tried localhost as well but it still says no answer. Maybe I'll have to tinker with the configuration? |
Care to tell what the stupid problem was?  |
|
| Back to top |
|
|
xergio
n00b
Joined: 22 Sep 2004
Posts: 23
|
| Posted: Wed Jan 26, 2005 3:02 pm Post subject: |
|
|
| pjesi wrote: | | Lepaca Kliffoth wrote: | | Uhm snmpwalk says it gets no answer from host. Of course I put animebox instead of foton and I tried localhost as well but it still says no answer. Maybe I'll have to tinker with the configuration? |
Care to tell what the stupid problem was? |
in the configuration file change "foton" to your hostname |
|
| Back to top |
|
|
sgtrock
Tux's lil' helper
Joined: 21 Feb 2003
Posts: 87
|
| Posted: Fri Jun 17, 2005 8:05 am Post subject: |
|
|
Did you check out cacti? It looks like a much more complete solution if you're interested in PHP based solutions. Personally, I'm looking for a Python based one to minimize security concerns. I'm not a good enough programmer to guarantee that I won't make mistakes with PHP.  |
|
| Back to top |
|
|
drwindows
n00b
Joined: 02 Apr 2004
Posts: 9
|
| Posted: Fri Jun 17, 2005 9:07 am Post subject: About cacti |
|
|
Yes, I've checked it; and it's a good solution if you don't want to complicate your life and don't care about inefficient usage of your resources.
I were worried about data harvesting more than data presentation, notice that while reviewing statistics is about 0.1% of the usage of system, data retrieval and storage is continuous. Cacti data retrieval solution were not as optimal as it could be.
Cacti approximation is via cron jobs. My approximation is via resident daemon. If somebody can attach cacti presentation to some kind of daemon-like backend, please post here. |
|
| Back to top |
|
|
bitwise
Tux's lil' helper
Joined: 20 Apr 2005
Posts: 104
Location: seattle
|
| Posted: Fri Jun 17, 2005 12:45 pm Post subject: |
|
|
| very nice, I'm going to be going through this later. I played with RRDTool a while ago on my webserver, but was having issues with the data being displayed properly. I suspect it was just my failure to properly adjust the times for the data retrieval/graphing. |
|
| Back to top |
|
|
sgtrock
Tux's lil' helper
Joined: 21 Feb 2003
Posts: 87
|
| Posted: Fri Jun 17, 2005 4:18 pm Post subject: |
|
|
Ah. I hadn't realized that they took that approach. You make a good point about the additional resources necessary to run that way.
Have you considered submitting your alternative solution to the cacti project? They've done a HUGE amount of work on the report generation section. It may be possible to adapt your daemon work to their back end, using some modified configuration files. If it is, it would marry what looks like a remarkably complete report generation front end with a lighter back end.
Thoughts? |
|
| Back to top |
|
|
ghettodev
n00b
Joined: 23 Jan 2005
Posts: 11
|
| Posted: Wed Jul 06, 2005 7:40 pm Post subject: |
|
|
| you should really add this to the gentoo wiki, good tutorial. |
|
| Back to top |
|
|
alinv
Guru
Joined: 19 Nov 2002
Posts: 395
Location: Bucharest
|
| Posted: Wed Jul 13, 2005 5:01 pm Post subject: |
|
|
First of all, great howto!
I have only one problem:
| Code: | # ./cpu.bursa
Generating graphs/cpu.percent.bursa.day.png
Using 86400 interval
ERROR: Garbage ':47:50 2005 \n' after command:
COMMENT:Generated Jul 13 19:47:50 2005 \n |
It seems that the colons have to be escaped in rrdtool COMMENTs
_________________
Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.
S.B. |
|
| Back to top |
|
|
alinv
Guru
Joined: 19 Nov 2002
Posts: 395
Location: Bucharest
|
| Posted: Wed Jul 13, 2005 7:39 pm Post subject: |
|
|
Correction, I have another problem
Building the graph for memory&swap:
| Code: | # ./memory.bursa
Generating graphs/memory.bursa.day.png
Using 86400 interval
ERROR: the RRD does not contain an RRA matching the chosen CF |
Any hints?
Thanks,
Alin
_________________
Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.
S.B. |
|
| Back to top |
|
|
ramgar
n00b
Joined: 10 Mar 2004
Posts: 6
|
| Posted: Fri Aug 12, 2005 7:16 am Post subject: |
|
|
| alinv wrote: | Correction, I have another problem
Building the graph for memory&swap:
| Code: | # ./memory.bursa
Generating graphs/memory.bursa.day.png
Using 86400 interval
ERROR: the RRD does not contain an RRA matching the chosen CF |
Any hints?
Thanks,
Alin |
Hi
the automatically created rrd-database for memory doesnt contains the MIN consolidation function (CF).
You can try something like this:
| Code: |
rrdtool create /path/to/rrd.rrd --start `date +"%s"` \
DS:ds0:GAUGE:600:0:[memory total as reported by snmpd] \
DS:ds1:GAUGE:600:0:[memory total as reported by snmpd]\
--step 300 \
RRA:AVERAGE:0.5:1:800 \
RRA:AVERAGE:0.5:6:800 \
RRA:AVERAGE:0.5:24:800 \
RRA:AVERAGE:0.5:288:800 \
RRA:MIN:0.5:1:800 \
RRA:MIN:0.5:6:800 \
RRA:MIN:0.5:24:800 \
RRA:MIN:0.5:288:800 \
RRA:MAX:0.5:1:800 \
RRA:MAX:0.5:6:800 \
RRA:MAX:0.5:24:800 \
RRA:MAX:0.5:288:800
|
hope that helps,
ramgar |
|
| Back to top |
|
|
TXTad
Tux's lil' helper
Joined: 15 Jan 2004
Posts: 108
Location: Texas
|
| Posted: Wed Aug 24, 2005 4:42 am Post subject: |
|
|
| Quote: | | So, you've just read that this file was created using snmpconf configuration program. It's so easy to use and it's well documented, try it yourself. |
Is a bit of a simplification. Can anyone walk through the steps (and terminology) of snmpconf? I tried snmpconfg -g basic_setup, and even that wasn't too clear.
Tad |
|
| Back to top |
|
|
alinv
Guru
Joined: 19 Nov 2002
Posts: 395
Location: Bucharest
|
| Posted: Fri Sep 02, 2005 9:14 pm Post subject: |
|
|
| ramgar wrote: | Hi
the automatically created rrd-database for memory doesnt contains the MIN consolidation function (CF).
You can try something like this:
| Code: |
rrdtool create /path/to/rrd.rrd --start `date +"%s"` \
DS:ds0:GAUGE:600:0:[memory total as reported by snmpd] \
DS:ds1:GAUGE:600:0:[memory total as reported by snmpd]\
--step 300 \
RRA:AVERAGE:0.5:1:800 \
RRA:AVERAGE:0.5:6:800 \
RRA:AVERAGE:0.5:24:800 \
RRA:AVERAGE:0.5:288:800 \
RRA:MIN:0.5:1:800 \
RRA:MIN:0.5:6:800 \
RRA:MIN:0.5:24:800 \
RRA:MIN:0.5:288:800 \
RRA:MAX:0.5:1:800 \
RRA:MAX:0.5:6:800 \
RRA:MAX:0.5:24:800 \
RRA:MAX:0.5:288:800
|
hope that helps,
ramgar |
It doesn't like it
| Code: | | ERROR: min must be less than max in DS definition |
Thanks,
Alin
_________________
Ever tried. Ever failed. No matter. Try Again. Fail again. Fail better.
S.B. |
|
| Back to top |
|
|
kyron
Apprentice
Joined: 26 Aug 2002
Posts: 198
Location: Montreal, Qc.
|
| Posted: Sun Sep 18, 2005 5:19 pm Post subject: Re: About cacti |
|
|
| drwindows wrote: | Yes, I've checked it; and it's a good solution if you don't want to complicate your life and don't care about inefficient usage of your resources.
I were worried about data harvesting more than data presentation, notice that while reviewing statistics is about 0.1% of the usage of system, data retrieval and storage is continuous. Cacti data retrieval solution were not as optimal as it could be.
Cacti approximation is via cron jobs. My approximation is via resident daemon. If somebody can attach cacti presentation to some kind of daemon-like backend, please post here. |
Have you checked out cactid (daemon)? From what I read, it's meant jsut for that purpose of lowering the overhead, running as a daemon and being written in native C + POSIX Threads...
_________________
M$ Windows: When in doubt, REBOOT
Linux: When in doubt, RTFM  |
|
| Back to top |
|
|
njcwotx
Guru
Joined: 25 Feb 2005
Posts: 386
Location: Texas
|
| Posted: Thu Oct 20, 2005 7:35 pm Post subject: |
|
|
I have been reading through the How to here and the docs with MRTG and RRDTOOL and I have some questions if someone can answer before I find the answers in the docs.
First, when creating a graph, with this command below as an example, there is a reference to "/var/lib/mrtg/foton.ram.swap.rrd" database.
| Code: |
rrdtool graph /var/www/localhost/htdocs/graphs/memory.foton.$INTERVALSTR.png \
-s -$INTERVAL \
-b 1024 \
-a PNG \
-z \
DEF:ram_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds0:AVERAGE \
DEF:swap_Kbytes=/var/lib/mrtg/foton.ram.swap.rrd:ds1:AVERAGE \
DEF:ram_Kbytes_min=/var/lib/mrtg/foton.ram.swap.rrd:ds0:MIN \
DEF:swap_Kbytes_min=/var/lib/mrtg/foton.ram.swap.rrd:ds1:MIN \
DEF:ram_Kbytes_max=/var/lib/mrtg/foton.ram.swap.rrd:ds0:MAX \
DEF:swap_Kbytes_max=/var/lib/mrtg/foton.ram.swap.rrd:ds1:MAX \
"CDEF:ram_bytes=ram_Kbytes,1024,*" \
"CDEF:swap_bytes=$TOTALSWAP,swap_Kbytes,-,1024,*" \
"CDEF:ram_bytes_min=ram_Kbytes_min,1024,*" \
"CDEF:swap_bytes_min=$TOTALSWAP,swap_Kbytes_min,-,1024,*" \
"CDEF:ram_bytes_max=ram_Kbytes_max,1024,*" \
"CDEF:swap_bytes_max=$TOTALSWAP,swap_Kbytes_max,-,1024,*" \
COMMENT:" min Avg Max Current\n" \
LINE1:swap_bytes_max#FF8080 \
LINE1:swap_bytes_min#800000 \
LINE1:swap_bytes#FF0000:"swap usage " \
GPRINT:swap_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:swap_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:swap_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:swap_bytes:LAST:"%7.2lf %SBytes\n" \
LINE1:ram_bytes_max#80FF80 \
LINE1:ram_bytes_min#008000 \
LINE1:ram_bytes#00FF00:"real free " \
GPRINT:ram_bytes_min:MIN:'%7.2lf %SBytes' \
GPRINT:ram_bytes:AVERAGE:"%7.2lf %SBytes" \
GPRINT:ram_bytes_max:MAX:'%7.2lf %SBytes' \
GPRINT:ram_bytes:LAST:"%7.2lf %SBytes\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
COMMENT:"`uptime`\n" \ -v "Bytes" -t "Memory usage - last $INTERVALSTR" -l 0 |
At what point does the database file get created and how does it get named? Does it get created with the follwing snippet of code in the mrtg config?
| Code: |
Target[foton.ram.swap]: memAvailReal.0&memAvailSwap.0:public@foton
MaxBytes[foton.ram.swap]: 456488
Title[foton.ram.swap]: RAM vs swap Free Memory
Options[foton.ram.swap]: gauge |
It appears that the "foton.ram.swap" may be my answer, but I just want to be clear that I see it the right way. In the same reguard, I will assume that any time I need a new 'default' database that this will be where I name the database.
Second, in the example above there are mrtg configs that refer to 2 other targets called, foton.freemem and foton.diskpercent, but have no example graph. I just assumed that the graph code was left out to shorten the how to, or is there some other reason these are there?
Thanks.
_________________
Drinking from the fountain of knowldege.
Sometimes sipping.
Sometimes gulping.
Always thirsting. |
|
| Back to top |
|
|
drwindows
n00b
Joined: 02 Apr 2004
Posts: 9
|
| Posted: Mon Oct 24, 2005 5:34 pm Post subject: |
|
|
| sgtrock wrote: | Have you considered submitting your alternative solution to the cacti project? They've done a HUGE amount of work on the report generation section. It may be possible to adapt your daemon work to their back end, using some modified configuration files. If it is, it would marry what looks like a remarkably complete report generation front end with a lighter back end.
|
| kyron wrote: | Have you checked out cactid (daemon)? From what I read, it's meant jsut for that purpose of lowering the overhead, running as a daemon and being written in native C + POSIX Threads...
|
Well, of course it would be a nice solution marrying this back-end with their front-end... if they were compatible. Our approach it is based on isolation and modularity. Cacti does not. Cacti is a monolithic app where you can plug some components(like cactid), but it has not any layer you can substitute using some interface.
Futhermore, cactid may be a C, POSIX, TURBO program, or whatever you want, but it generates graphs, and we return to the problem of making graphs nobody will never see. Bad resource administration again. Bottleneck is on channels, not in CPU rounds.
It will be harder to adapt cacti to work with this backend, but if someone wants, please do it. And be sure I will install such Cacti in my system. |
|
| Back to top |
|
|
drwindows
n00b
Joined: 02 Apr 2004
Posts: 9
|
| Posted: Mon Oct 24, 2005 5:49 pm Post subject: |
|
|
| alinv wrote: |
I have only one problem:
| Code: | # ./cpu.bursa
Generating graphs/cpu.percent.bursa.day.png
Using 86400 interval
ERROR: Garbage ':47:50 2005 \n' after command:
COMMENT:Generated Jul 13 19:47:50 2005 \n |
It seems that the colons have to be escaped in rrdtool COMMENTs |
Watch out quotes!
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
it was this?
Next, your
| alinv wrote: |
| Code: |
ERROR: min must be less than max in DS definition
|
|
It sounds that you have been copied ramgar's code directly. So max value would be zero. I'm not sure, but next time post exactly your files, that way we could help you better. |
|
| Back to top |
|
|
batjohan
n00b
Joined: 13 Oct 2004
Posts: 28
Location: Husum, Sweden
|
| Posted: Sat Nov 05, 2005 1:52 am Post subject: |
|
|
How do i fix the ERROR: Garbage thing?
| Code: |
Generating graphs/bell.bandwith.day.png
Using 86400 intervall
ERROR: Garbage ':11:38 2005 \n' after command:
COMMENT:Generated Nov 05 02:11:38 2005 \n
|
Last 3 lines in gen/1.traffic.adsl.bell
| Code: |
GPRINT:total_bytes:AVERAGE:" total %8.2lf %sB\n" \
COMMENT:"Generated `date +\"%b %d %H:%M:%S %Y\"` \n" \
-v "Bytes/s" -t "Router traffic - last $INTERVALSTR" -l 0
|
im running rrdtool 1.2.11-r2
EDIT:
When i have
COMMENT:"Generated `date +\"%b %d %H\:%M\:%S %Y\"` \n" \
it works |
|
| Back to top |
|
|
m27315
Apprentice
Joined: 10 Dec 2004
Posts: 226
Location: 2 workstations down
|
| Posted: Tue Feb 28, 2006 12:32 am Post subject: 2 questions: more garbage plus 127 error codes in PHP |
|
|
Thanks for the good guide. Unfortunately, I have run into two snags, which I do not think have been answered:- more garbage problems (btw, i also had to escape the colons in the comment lines):
| Code: | $ ./gen/memory.stack
Generating graphs/memory.stack.prime.day.png
Using 86400 interval
481x168
./gen/memory.stack: line 42: COMMENT: Max Avg Current\n: command not found |
The tail of the responsible code is:
| Code: | COMMENT:"Generated `date +\"%b %d %H\:%M\:%S %Y\"` \n" \
-v "Bytes" -t "Memory usage - $INTERVALSTR"
|
I have another Garbage error here:
| Code: | $ ./gen/memory
Generating graphs/memory.prime.day.png
Using 86400 interval
ERROR: Garbage ':28:07 up 1:00, 1 user, load average: 0.00, 0.08, 0.16\n' after command:
COMMENT: 18:28:07 up 1:00, 1 user, load average: 0.00, 0.08, 0.16\n |
which is produced by:
| Code: | COMMENT:"Generated `date +\"%b %d %H\:%M\:%S %Y\"` \n" \
COMMENT:"`uptime`\n" \
-v "Bytes" -t "Memory usage - last $INTERVALSTR" -l 0 |
Any hints? I believe my quotes are correct, because I copied and pasted from this web-page. What am I missing?
Whenever I access the web-page, I get '127' error codes. However, the scripts work fine if run from the command line. Is this a PHP safe-mode, or similar voodoo issue? Here's the resulting web-page (blank-lines deleted):
| Code: | Day Week Month Year
./gen/cpu day
127
./gen/disks.available day
127
./gen/memory day
127
./gen/memory.stack day
127
./gen/processes day
127
./gen/tcp day
127
./gen/traffic.adsl.router day
127 |
I really do not want to open up safe-mode. If that is the problem, can anyone suggest other alternatives?TIA
edit - 2006/2/28: Issue number #2 is safe-mode related. Disabling safe-mode fixes the 127 error code, displaying some graphs. However, all the scripts that produce the "garbage" error, produce an error-code of 1. The apache error_log also contains the full "garbage" error description. |
|
| Back to top |
|
|
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
Documentation, Tips & Tricks
