SNMP Monitoring and Alerting

You do not need to monitor devices such as switches and routers actively; you can configure them to send traps to the monitoring station whenever they encounter abnormal behavior such as a machine sending too many broadcast packets or too many packets being dropped at the router. The network-monitoring station can then generate alerts. 

Devices that support SNMP (Simple Network Management Protocol) can send SNMP traps to a monitoring station whenever errors or specific events as described above occur. The monitoring host can collect those traps and based on the type of trap generate alerts. We will discuss here how to use do this.

To configure devices to send SNMP traps, configure your device, which could be a router, printer, switch or a workstation supporting SNMP to send traps to the monitoring station. For this access your device's interface through telnet, Web browser or a device-management application and specify the IP addresses of the monitoring station where traps will be sent. For example, we configured our Intel switch to send traps to the IP 192.168.3.13, which is the address of our monitoring machine running Linux. Then configure the device for the types of events or errors for which traps will be sent. For our switch we set it such that when broadcast packets/sec from the port to which our file server was connected increases beyond a certain level, the switch will generate a trap. You can add more traps like if collisions on the switch increase beyond a limit it will generate a trap. Now you need to configure your monitoring host to take an action whenever it receives a trap.

To configure monitoring host to receive SNMP traps and generate alerts in Linux, you should have net-snmp and net-snmp-utils packages installed. The packages are available with most Linux distributions. Now start the snmptrapd daemon process by issuing the command.

#service snmptrapd start

To make the process start with system boot up issue the command.

#chkconfig snmptrapd on

Now that the trap collection program is running on the system, you have to specify what kinds of traps it will look for and what action it will take. This is specified in the snmptrapd.conf file. So create a file named snmptrapd.conf in the directory /etc/snmp, if it does not exist already. Open the file in any text editor and specify the actions to be taken. But before telling you the syntax for the actions that have to be specified in the file, first let's understand some SNMP basics. SNMP works with various variables with each variable having a unique identifier called Object Identifier (OID). The various variables and their structure along with their OIDs are defined in a Management Information Base (MIB), which is nothing but a collection of text files containing the SNMP variables' definitions. The default location for the MIBs is /usr/share/snmp/mibs. You can add additional MIBs specific to your device in this directory. Now back to the snmptrapd.conf file. Here you have to specify the action to be taken. So add a line in it with the following syntax.

traphandle the OID of the trap variable action to be taken

So, for our Intel switch which sends RMON traps to the monitoring machine we added a line like this.

traphandle RMON-MIB::risingAlarm /usr/bin/traptoemail -s 192.168.1.19 -f

sys@cmil.com admin@cmil.com 

The line says whenever the snmptrapd process receives a RMON rising alarm trap it will execute the /usr/bin/traptoemail program with some arguments. The RMON-MIB file is present in the /usr/share/snmp/mibs directory and contains the definitions and OIDs for various RMON variables including the risingAlarm variable. You can even replace the variable name RMON-MIB::risingAlarm by its OID, which is 1.3.6.1.2.1.16.0.1

Once you configure your hoist machine to receive alerts, the next step is to decide the means through which alerts will be sent.

These can be through e-mail, SMS, instant messenger, etc. We will talk about using e-mail as an alerting mean. 

The SNMP program comes with a script called traptoemail which can send email alerts describing the trap and the variables enclosed in it giving detailed information about the trap, like the IP address of the device generating the trap, the event which triggered it and the set threshold and actual values which raised the alarm. The arguments the script takes are the SMTP server address, the e-mail ID from which the e-mail will be sent and the e-mail ID to which the e-mail will be sent.

Restart the snmptrapd program.

#service snmptrapd restart

This is it, whenever there is any abnormal behavior in your network you will get a mail in your inbox so that you get to know of the problem first hand.

Sending SMS alerts

We have talked about sending e-mail alerts in case of problems in the network. But, you may not get any alerts if the network infrastructure is down or if the e-mail server is not working. So, you should also send alerts using an alternative way. And, the best way is to send alerts using

SMS. 

For that you'll need a cellphone, a cable to connect it to the monitoring host and a software that will let you send SMS through the system using the cellphone. We used a software called Gnokii (on this month's PCQXtreme CD) and integrated it with Nagios to send SMS alerts. Here we will show you how to do that. First, install the RPM by running the command.

#rpm -ivh gnokii-0.5.8-1.i386.rpm

This will install the required binaries and configuration files. Open the configuration file /etc/ gnokiirc in any text editor. You'll have to make two changes in this file. First is the port to which your cellphone cable is connected. If you are using a serial port cable then make an entry like this

port = /dev/ttys0

Or if you are using a USB cable make an entry like this.

port = /dev/usb/ttyUSB0

The next change that you have to make is for the model of your cellphone. Before doing that check if your cellphone model is listed in the supported models list on the gnokii website (www.gnokii.org). If it is supported then add a line in the file

model = 6110

This configures the software for a Nokia 6610 cellphone connected via a USB cable to our management host. If instead of the RPM you get a tarball of the Gnokii software, then compile the binaries from the source and place them in the location as specified in the readme file. 

Also there is a sample configuration file with it, which you can customize. Now that the gnokii program is configured, you will need to integrate it with Nagios. Open the file /usr/local/nagios/etc/misccommands.cfg and change the entries for the command notify-by-epager to look like:

define command{



command_name notify-by-epager


command_line /usr/bin/printf “%b” “Service: $SERVICEDESC$\nHost: $HOSTNAME$\nState: $SERVICESTATE$\nDate: $DATETIME$” | /usr/bin/gnokii -sendsms $CONTACTPAGER$


}

Also make the host-notify-by-epager command look like the above command.

After this open your /usr/local/nagios/etc/contacts.cfg file and enter the mobile numbers in the pager option for the contacts defined in the file. The contact definition should look something like this.

define contact{



contact_name nagiosadmin


..


..


..


service_notification_commands notify-by-email,notify-by-epager


host_notification_commands host-notify-by-email,host-notify-by-epager


email admin@cmil.com


pager 9811xxxxxx


}

Now force the Nagios process to relaod the changes made to the configuration files.

#service nagios reload

After this Nagios will alert you of network problems via both e-mail and SMS.

Not only Nagios, the Gnokii program can be integrated with other program as well. For example, in for sending SNMP traps, instead of using the traptoemail script to send e-mail alerts, you can specify in the snmptrapd.conf file that the Gnokii program be used to send SMS alerts.

SolarWinds: AN SNMP based nms 

SolarWinds is an SNMP-based complete network management and monitoring solution with well-organized and interlinked features. It is available in five editions: Standard, Professional, Professional Plus, Engineer's and Broadband Engineer's Edition. 

What we used is the Engineer's edition. It has nine sets of tools, viz Network performance-monitoring tools, Network discovery tools, Tools for Cisco routers, Fault-monitoring tools, MIB browser tools, Security and attack tools, Ping and diagnostic tools, IP address-management tools and Miscellaneous tools. The information is displayed in a user-friendly interface. It covers all the aspects of FCAPS (fault management, configuration, accounting, performance and security) and includes tools such as ping, subnet calculators and network management and address-management functions.

Under the Discovery set of tools, the IP Network Browser scans the subnet and searches for network devices in it. With Ping Sweep you can ping all the devices and view the result either collectively like machines that have responded or the ones that haven't, or separately. The software can discover MAC addresses of machines along with the details of their manufacturers. 

With the Network Sonar you get details of your network devices such as their configuration, statistics and charts showing the status of the nodes on the network. It can also continuously monitor the network and maintain a log. The MIB browser gives you access to specific details of the host with explanations. Bandwidth can be monitored using Bandwidth Gauge. It shows you how much traffic moving in/out at a particular port. To kill remote sessions on a switch/router/server you can use the remote TCP Session Reset tool.

In the Address Management part, you can resolve information about IP addresses and host names and find out if they are being used, when were they last used and response time to pings. With the Monitoring tool you can monitor and ping routers, servers and websites and get their response time and packet loss if any. From here, you can also monitor bandwidth usage by various devices. This is useful for identifying bottlenecks in your network. You can also check for DNS database errors by going for a reverse DNS lookup. 

Finally, the Network performance tool tracks the performance of the interface and sends alerts to the network monitoring console whenever there is a violation.

Vendor Solarwinds.net Inc  www.solarwinds.net