python & snmp

用python获取snmp信息有多个现成的库可以使用，其中比较常用的是netsnmp和pysnmp两个库。网上有较多的关于两个库的例子。

本文重点在于如何并发的获取snmp的数据，即同时获取多台机器的snmp信息。

netsnmp

先说netsnmp。python的netsnmp，其实是来自于net-snmp包。

python通过一个c文件调用net-snmp的接口获取数据。

因此，在并发获取多台机器的时候，不能够使用协程获取。因为使用协程，在get数据的时候，协程会一直等待net-snmp接口返回数据，而不会像socket使用时那样在等待数据时把CPU切换给其他协程使用。从这点上来说，使用协程和串行获取没有区别。

那么如何解决并发获取的问题呢？可以使用线程，多线程获取(当然也可以使用多进程)。多个线程同时调用net-snmp的接口获取数据，然后cpu在多个线程之间不停切换。当一个线程获取一个结果后，可以继续调用接口获取下一个snmp数据。

这里我写了一个样例程序。首先把所有的host和oid做成任务放到队列里，然后启动多个线程，去执行获取任务。程序样例如下：

import threadingimport timeimport netsnmpimport Queuestart_time = time.time()hosts = ["192.20.150.109", "192.20.150.110", "192.20.150.111", "192.20.150.112", "192.20.150.113", "192.20.150.114", "192.20.150.115", "192.20.150.116", "192.20.150.117", "192.20.150.118", "192.20.150.119", "192.20.150.120", "192.20.150.121", "192.20.80.148", "192.20.80.149", "192.20.96.59", "192.20.82.14", "192.20.82.15", "192.20.82.17", "192.20.82.19", "192.20.82.12", "192.20.80.139", "192.20.80.137", "192.20.80.136", "192.20.80.134", "192.20.80.133", "192.20.80.131", "192.20.80.130", "192.20.81.141", "192.20.81.140", "192.20.82.26", "192.20.82.28", "192.20.82.23", "192.20.82.21", "192.20.80.128", "192.20.80.127", "192.20.80.122", "192.20.81.159", "192.20.80.121", "192.20.80.124", "192.20.81.151", "192.20.80.118", "192.20.80.119", "192.20.80.113", "192.20.80.112", "192.20.80.116", "192.20.80.115", "192.20.78.62", "192.20.81.124", "192.20.81.125", "192.20.81.122", "192.20.81.121", "192.20.82.33", "192.20.82.31", "192.20.82.32", "192.20.82.30", "192.20.81.128", "192.20.82.39", "192.20.82.37", "192.20.82.35", "192.20.81.130", "192.20.80.200", "192.20.81.136", "192.20.81.137", "192.20.81.131", "192.20.81.133", "192.20.81.134", "192.20.82.43", "192.20.82.45", "192.20.82.41", "192.20.79.152", "192.20.79.155", "192.20.79.154", "192.25.76.235", "192.25.76.234", "192.25.76.233", "192.25.76.232", "192.25.76.231", "192.25.76.228", "192.25.20.96", "192.25.20.95", "192.25.20.94", "192.25.20.93", "192.24.163.14", "192.24.163.21", "192.24.163.29", "192.24.163.6", "192.18.136.22", "192.18.136.23", "192.24.193.2", "192.24.193.19", "192.24.193.18", "192.24.193.11", "192.20.157.132", "192.20.157.133", "192.24.212.232", "192.24.212.231", "192.24.212.230"]oids = [".1.3.6.1.4.1.2021.11.9.0",".1.3.6.1.4.1.2021.11.10.0",".1.3.6.1.4.1.2021.11.11.0",".1.3.6.1.4.1.2021.10.1.3.1", ".1.3.6.1.4.1.2021.10.1.3.2",".1.3.6.1.4.1.2021.10.1.3.3",".1.3.6.1.4.1.2021.4.6.0",".1.3.6.1.4.1.2021.4.14.0", ".1.3.6.1.4.1.2021.4.15.0"]myq = Queue.Queue()rq = Queue.Queue()#把host和oid组成任务for host in hosts: for oid in oids: myq.put((host,oid))def poll_one_host(): while True: try: #死循环从队列中获取任务，直到队列任务为空 host, oid = myq.get(block=False) session = netsnmp.Session(Version=2, DestHost=host, Community="cluster",Timeout=3000000,Retries=0) var_list = netsnmp.VarList() var_list.append(netsnmp.Varbind(oid)) ret = session.get(var_list) rq.put((host, oid, ret, (time.time() - start_time))) except Queue.Empty: breakthread_arr = []#开启多线程num_thread = 50for i in range(num_thread): t = threading.Thread(target=poll_one_host, kwargs={}) t.setDaemon(True) t.start() thread_arr.append(t)#等待任务执行完毕for i in range(num_thread): thread_arr[i].join()while True: try: info = rq.get(block=False) print info except Queue.Empty: print time.time() - start_time break

netsnmp除了支持get操作之外，还支持walk操作，即遍历某个oid。

但是walk使用的时候需要谨慎，以免导致高延时等问题，具体可以参见之前的一篇snmpwalk高延时问题分析的博客。

pysnmp

pysnmp是用python实现的一套snmp协议的库。其自身提供了对于异步的支持。

import timeimport Queuefrom pysnmp.hlapi.asyncore import *t = time.time()myq = Queue.Queue()#回调函数。在有数据返回时触发def cbFun(snmpEngine, sendRequestHandle, errorIndication, errorStatus, errorIndex, varBinds, cbCtx): myq.put((time.time()-t, varBinds))hosts = ["192.20.150.109", "192.20.150.110", "192.20.150.111", "192.20.150.112", "192.20.150.113", "192.20.150.114", "192.20.150.115", "192.20.150.116", "192.20.150.117", "192.20.150.118", "192.20.150.119", "192.20.150.120", "192.20.150.121", "192.20.80.148", "192.20.80.149", "192.20.96.59", "192.20.82.14", "192.20.82.15", "192.20.82.17", "192.20.82.19", "192.20.82.12", "192.20.80.139", "192.20.80.137", "192.20.80.136", "192.20.80.134", "192.20.80.133", "192.20.80.131", "192.20.80.130", "192.20.81.141", "192.20.81.140", "192.20.82.26", "192.20.82.28", "192.20.82.23", "192.20.82.21", "192.20.80.128", "192.20.80.127", "192.20.80.122", "192.20.81.159", "192.20.80.121", "192.20.80.124", "192.20.81.151", "192.20.80.118", "192.20.80.119", "192.20.80.113", "192.20.80.112", "192.20.80.116", "192.20.80.115", "192.20.78.62", "192.20.81.124", "192.20.81.125", "192.20.81.122", "192.20.81.121", "192.20.82.33", "192.20.82.31", "192.20.82.32", "192.20.82.30", "192.20.81.128", "192.20.82.39", "192.20.82.37", "192.20.82.35", "192.20.81.130", "192.20.80.200", "192.20.81.136", "192.20.81.137", "192.20.81.131", "192.20.81.133", "192.20.81.134", "192.20.82.43", "192.20.82.45", "192.20.82.41", "192.20.79.152", "192.20.79.155", "192.20.79.154", "192.25.76.235", "192.25.76.234", "192.25.76.233", "192.25.76.232", "192.25.76.231", "192.25.76.228", "192.25.20.96", "192.25.20.95", "192.25.20.94", "192.25.20.93", "192.24.163.14", "192.24.163.21", "192.24.163.29", "192.24.163.6", "192.18.136.22", "192.18.136.23", "192.24.193.2", "192.24.193.19", "192.24.193.18", "192.24.193.11", "192.20.157.132", "192.20.157.133", "192.24.212.232", "192.24.212.231", "192.24.212.230"]oids = [".1.3.6.1.4.1.2021.11.9.0",".1.3.6.1.4.1.2021.11.10.0",".1.3.6.1.4.1.2021.11.11.0",".1.3.6.1.4.1.2021.10.1.3.1", ".1.3.6.1.4.1.2021.10.1.3.2",".1.3.6.1.4.1.2021.10.1.3.3",".1.3.6.1.4.1.2021.4.6.0",".1.3.6.1.4.1.2021.4.14.0", ".1.3.6.1.4.1.2021.4.15.0"] snmpEngine = SnmpEngine()#添加任务for oid in oids: for h in hosts: getCmd(snmpEngine, CommunityData('cluster'), UdpTransportTarget((h, 161), timeout=3, retries=0,), ContextData(), ObjectType(ObjectIdentity(oid)), cbFun=cbFun)time1 = time.time() - t#执行异步获取snmpsnmpEngine.transportDispatcher.runDispatcher()#打印结果while True: try: info = myq.get(block=False) print info except Queue.Empty: print time1 print time.time() - t break

pysnmp本身只支持最基础的get和getnext命令，因此如果想使用walk，需要自己进行实现。

性能测试

在同一个环境下，对两者进行了性能测试。两者对198个host，10个oid进行采集。

测试组 耗时(sec) netsnmp(20线程) 6.252 netsnmp(50线程) 3.269 netsnmp(200线程) 3.265 pysnmp 4.812

可以看到netsnmp的采集速度跟线程数有关。当线程数增大到一定程度，采集时间不再缩短。因为开辟线程同样会消耗时间。而已有的线程已经足够处理。

pysnmp性能较之略差一下。详细分析pysnmp在添加任务(执行getCmd时)消耗了约1.2s，之后的采集约消耗3.3秒。

在增加了oid数，在进行实验。host仍然是198个，oid是42个。

测试组 耗时(sec) netsnmp(20线程) 30.935 netsnmp(50线程) 12.914 netsnmp(200线程) 4.044 pysnmp 11.043

可以看到差距被进一步拉大。在线程足够多的情况下，netsnmp的效率要明显强于pysnmp。

因为二者都支持可以并行采集多个host，从易用性来说，netsnmp更为简单一些，且netsnmp支持walk功能。本文更加推荐netsnmp。

安装netsnmp需要安装net-snmp。如果centos，则使用yum会较为方便。

以上就是本文的全部内容，希望对大家的学习有所帮助，也希望大家多多支持。