NETWORKING

  • 11/29/2017
    7:00 AM
  • Rating: 
    0 votes
    +
    Vote up!
    -
    Vote down!

Network Monitoring Using Python

Learn how to handle SNMP tasks with Python in this excerpt from "Mastering Python Networking."

Imagine you get a call at 2 a.m. in the morning. The person on the other end asks, "Hi, we are facing a difficult issue that is impacting production. We think it might be network-related. Can you check for us?" Where would you check first? Of course, you would look at your monitoring tool and confirm whether any of the metrics changed in the last few hours. Throughout the book, we have been discussing various ways to programmatically make predictable changes to our network, with the goal of keeping the network running as smoothly as possible. 

However, networks are not static. Far from it, they are probably one of the most fluent parts of the entire infrastructure. By definition, a network connects different parts together, constantly passing traffic back and forth. There are lots of moving parts that can cause your network to stop working as expected: hardware fails, software with bugs, human mistakes, despite their best intentions, and so on. We need ways to make sure our network works as expected and that we are hopefully notified when things are not. 

In the next two chapters, we will look at various ways to perform network monitoring tasks. Many of the tools we have looked at thus far can be tied together or directly managed by Python. Like everything we have looked at up to this point, network monitoring has to do with two parts. First, we need to know with what information the equipment is capable of transmitting. Second, we need to identify what useful information we can interpret from them.

We will look at a few tools that allow us to monitor the network effectively: 

  • Simple Network Management Protocol (SNMP)
  • Matplotlib and pygal visualization
  • MRTG and Cacti

This list is not exhaustive, and there is certainly no lack of commercial vendors in the space. The basics of network monitoring that we will look at, however, carry well for both open source and commercial tools. 

Lab Setup

The lab for this chapter is similar to the one in the last chapter but with this difference: both the network devices are IOSv devices. Here's an illustration of this:

 

The two Ubuntu hosts will be used to generate traffic across the network so we can look at some non-zero counters. 

SNMP

SNMP is a standardized protocol used to collect and manage devices. Although the standard allows you to use SNMP for device management, in my experience, most network administrators prefer to keep SNMP as an information collection mechanism only. Since SNMP operates on UDP that is connectionless and considering the relatively weak security mechanism in versions 1 and 2, making device changes via SNMP tend to make network operators a bit uneasy. SNMP Version 3 has added cryptographic security and new concepts and terminologies to the protocol, but the way it's adapted varies among network device vendors.

SNMP is widely used in network monitoring and has been around since 1988 and was part of RFC 1065. The operations are straightforward with the network manager sending GET and SET requests toward the device and the device with the SNMP agent responding with the information per request. The most widely adapted standard is SNMPv2c, which is defined in RFC 1901 - RFC 1908. It uses a simple community-based security scheme for security. It has also introduced new features, such as the ability to get bulk information.

The information residing in the device is structured in the Management Information Base (MIB). The MIB uses a hierarchical namespace containing an Object Identifier (OID), which represents the information that can be read and fed back to the requester. When we talk about using SNMP to query device information, we are really talking about using the management station to query the specific OID that represents the information we are after. You're required to put some efforts for consolidating basic common information into a common OID structure; however, the output of the effort varies in terms of how successful it is. At least in my experience, I typically need to consult with vendor documentation to find the OID that I need. 

Some of the main points to take away from the operation are:

  • The implementation heavily relies on the amount of information the device agent can provide. This, in turn, relies on how the vendor treats SNMP: as a core feature or an added feature.
  • SNMP agents generally require CPU cycles from the control plane to return a value. Not only is this inefficient for devices with, say, large BGP tables, it is also not feasible to use SNMP to constantly query the data. 
  • The user needs to know the OID in order to query the data. 

Since SNMP has been around for a while, my assumption is that you have some experience with it already. Let's jump directly into our first example. 

Setup

First, let's make sure the SNMP managing device and agent works in our setup. The SNMP bundle can be installed on either the hosts in our lab or the managing device on the management network. As long as the manager has IP reachability to the device and the managed device allows the connection, SNMP should work well. 

In my setup, I have installed SNMP on both the Ubuntu host on the management network as well as the client host in the lab to test security: 

$ sudo apt-get install snmp

There are many optional parameters you can configure on the network device, such as contact, location, chassis ID, and SNMP packet size. The options are device-specific and you should check the documentation on your device. For IOSv devices, we will configure an access list to limit only the desired host for querying the device as well as tying the access list with the SNMP community string. In our case, we will use the word secret as the community string and permit_snmp as the access list name:

    !

    ip access-list standard permit_snmp

     permit 172.16.1.173 log

     deny any log

    !

    !

    snmp-server community secret RO permit_snmp

    !

The SNMP community string is acting as a shared password between the manager and the agent; therefore, it needs to be included anytime you want to query the device.

We can use tools, such as the MIB locater , for finding specific OIDs to query. Alternatively, we can just start walking through the SNMP tree, starting from the top of Cisco's enterprise tree at .1.3.6.1.4.1.9: 

$ snmpwalk -v2c -c secret 172.16.1.189 .1.3.6.1.4.1.9

iso.3.6.1.4.1.9.2.1.1.0 = STRING: "

Bootstrap program is IOSv

"

iso.3.6.1.4.1.9.2.1.2.0 = STRING: "reload"

iso.3.6.1.4.1.9.2.1.3.0 = STRING: "iosv-1"

iso.3.6.1.4.1.9.2.1.4.0 = STRING: "virl.info"

...

We can be very specific about the OID we need to query as well: 

$ snmpwalk -v2c -c secret 172.16.1.189 .1.3.6.1.4.1.9.2.1.61.0

iso.3.6.1.4.1.9.2.1.61.0 = STRING: "cisco Systems, Inc.

170 West Tasman Dr.

San Jose, CA 95134-1706

U.S.A.

Ph +1-408-526-4000

Customer service 1-800-553-6387 or +1-408-526-7208

24HR Emergency 1-800-553-2447 or +1-408-526-7209

Email Address tac@cisco.com

World Wide Web http://www.cisco.com"

The last thing to check would be to make sure the access list would deny unwanted SNMP queries. Because we had the log keyword for both permit and deny entries, only 172.16.1.173 is permitted for querying the device: 

*Mar 3 20:30:32.179: %SEC-6-IPACCESSLOGNP: list permit_snmp permitted 0 172.16.1.173 -> 0.0.0.0, 1 packet

*Mar 3 20:30:33.991: %SEC-6-IPACCESSLOGNP: list permit_snmp denied 0 172.16.1.187 -> 0.0.0.0, 1 packet

As you can see, the biggest challenge in setting up SNMP is to find the right OID. Some of the OIDs are defined in standardized MIB-2; others are under the enterprise portion of the tree. Vendor documentation is the best bet, though. There are a number of tools that can help, such as the MIB Browser; you can add MIBs (again, provided by the vendors) to the browser and see the description of the enterprise-based OIDs. A tool such as Cisco's SNMP Object Navigator  proves to be very valuable when you need to find the correct OID of the object you are looking for. 

PySNMP 

PySNMP is a cross-platform Python SNMP engine implementation developed by Ilya Etingof. It abstracts a lot of SNMP details for you, as great libraries do, and supports both Python 2 and Python 3. 

PySNMP requires the PyASN1 package. According to Wikipedia: PyASN1 conveniently provides a Python wrapper around ASN.1. Let's install the package first: 

cd /tmp

git clone https://github.com/etingof/pyasn1.git

cd pyasn1/

sudo python3 setup.py install

Next, install the PySNMP package:

git clone https://github.com/etingof/pysnmp

cd pysnmp/

sudo python3 setup.py install

Note At the time of this writing, there are some bug fixes that have not been pushed to the PyPI repository yet. The packages can also be installed via pip; sudo pip3 install pysnmp will automatically install pyasn1. 

Let's look at how to use PySNMP to query the same Cisco contact information we used in the previous example, which is slightly modified from the PySNMP example. We will import the necessary module and create a CommandGenerator object first: 

>>> from pysnmp.entity.rfc3413.oneliner import cmdgen

>>> cmdGen = cmdgen.CommandGenerator()

>>> cisco_contact_info_oid = "1.3.6.1.4.1.9.2.1.61.0"

We can perform SNMP using the getCmd method. The result is unpacked into various variables; of these, we care most about varBinds that contain the query result: 

>>> errorIndication, errorStatus, errorIndex, varBinds = cmdGen.getCmd(

...     cmdgen.CommunityData('secret'),

...     cmdgen.UdpTransportTarget(('172.16.1.189', 161)),

...     cisco_contact_info_oid

... )

>>> for name, val in varBinds:

...     print('%s = %s' % (name.prettyPrint(), str(val)))

...

SNMPv2-SMI::enterprises.9.2.1.61.0 = cisco Systems, Inc.

170 West Tasman Dr.

San Jose, CA 95134-1706

U.S.A.

Ph +1-408-526-4000

Customer service 1-800-553-6387 or +1-408-526-7208

24HR Emergency 1-800-553-2447 or +1-408-526-7209

Email Address tac@cisco.com

World Wide Web http://www.cisco.com

>>> 

Note that the response values are PyASN1 objects. The prettyPrint() method will convert some of these values into a human-readable format, but since the result in our case was not converted, we will convert it into a string manually. 

We can put a script using the preceding interactive example into pysnmp_1.py with the referenced error checking if we run into problems. We can also include multiple OIDs in the getCmd() method: 

system_up_time_oid = "1.3.6.1.2.1.1.3.0"

cisco_contact_info_oid = "1.3.6.1.4.1.9.2.1.61.0"

 

errorIndication, errorStatus, errorIndex, varBinds = cmdGen.getCmd(

 cmdgen.CommunityData('secret'),

 cmdgen.UdpTransportTarget(('172.16.1.189', 161)),

 system_up_time_oid,

 cisco_contact_info_oid

)

The result will just be unpacked and listed out in our example: 

$ python3 pysnmp_1.py

SNMPv2-MIB::sysUpTime.0 = 16052379

SNMPv2-SMI::enterprises.9.2.1.61.0 = cisco Systems, Inc.

170 West Tasman Dr.

....

In the next example, we will persist the values we received from the queries so we can perform other functions, such as visualization, with the data.

We welcome your comments on this topic on our social media channels, or [contact us directly] with questions about the site.

Log in or Register to post comments