Details
-
Help
-
Resolution: Cannot Reproduce
-
Major
-
None
-
CE-21.10
-
Observium CE 21.10.11666 (13th October 2021)
OS Linux 3.10.0-1160.59.1.el7.x86_64 [amd64] (CentOS 7)
Apache 2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.6.40
PHP 5.6.40 (OPcache: ENABLED) (Memory: 1GB)
Python 2.7.5
MariaDB 5.5.68-MariaDB (extension: mysqli 5.5.68-MariaDB)
SNMP NET-SNMP 5.7.2
RRDtool 1.7.1
Fping 3.10 (IPv4 only)Observium CE 21.10.11666 (13th October 2021) OS Linux 3.10.0-1160.59.1.el7.x86_64 [amd64] (CentOS 7) Apache 2.4.6 (CentOS) OpenSSL/1.0.2k-fips PHP/5.6.40 PHP 5.6.40 (OPcache: ENABLED) (Memory: 1GB) Python 2.7.5 MariaDB 5.5.68-MariaDB (extension: mysqli 5.5.68-MariaDB) SNMP NET-SNMP 5.7.2 RRDtool 1.7.1 Fping 3.10 (IPv4 only)
Description
Hello!
I could really use some help. We noticed most of our graphs had disappeared. Unfortunately, exactly what went on to break this is a mystery for now and I'm trying to unravel the issue and fix it.
While manually running a discovery on all devices, I saw this error in the mibs module output for every single device:
Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS.
The system hosting Observium can snmpwalk any of our devices as normal, but the "mibs" module in Observium is unable to do the walks needed during a run of discovery.php. The result is all of our traffic graphs for all devices have basically disappeared and I just can't see why calls to the snmp_command() function is failing.
All the usual mibs are in the /opt/observium/mibs directory and permissions are all readable.
I've run out of ideas and need help on where to look next. Any clues would be super appreciated! If any other information is needed to figure this out, please let me know and I'll post.
Thanks!
Example of an snmpwalk working just fine outside Observium on one of our faithful old switches:
[root@nms observium]# snmpwalk -v1 -c redacted switch01 sysDescr
|
SNMPv2-MIB::sysDescr.0 = STRING: Juniper Networks, Inc. ex2200-48t-4g Ethernet Switch, kernel JUNOS 12.3R12.4, Build date: 2016-01-20 05:01:04 UTC Copyright (c) 1996-2016 Juniper Networks, Inc.
|
An example of one specific device running against only the mibs module, for output brevity's sake:
[root@nms observium]# /opt/observium/discovery.php -h 80 -m mibs
|
___ _ _
|
/ _ \ | |__ ___ ___ _ __ __ __(_) _ _ _ __ ___
|
| | | || '_ \ / __| / _ \| '__|\ \ / /| || | | || '_ ` _ \
|
| |_| || |_) |\__ \| __/| | \ V / | || |_| || | | | | |
|
\___/ |_.__/ |___/ \___||_| \_/ |_| \__,_||_| |_| |_|
|
Observium Community Edition 21.10.11666
|
https://www.observium.org
|
##### Starting discovery run at 2022-04-26 11:34:00 #####
|
##### switch01 [80] #####
|
+-------------+--+
|
| OID | |
|
+-------------+--+
|
| sysDescr | |
|
| sysObjectID | |
|
+-------------+--+
|
o OS Type generic
|
o OS Group unix
|
o SNMP Version v1
|
o Last discovery 2022-04-26 11:33:46
|
o Last duration 2.53 seconds
|
|
|
##### Module Start: mibs #####
|
Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS.
|
o MIBs discovered <empty>
|
o Duration 0.325s
|
|
|
##### switch01 [80] completed discovery modules at 2022-04-26 11:34:01 #####
|
o Discovery time 0.5958 seconds
|
|
|
##### Finished discovery run at 2022-04-26 11:34:01 #####
|
o Devices Discovered 1
|
o Discovery Time 0.627 secs
|
o Memory usage 32.5MB (peak: 32.5MB)
|
o MySQL Usage Cell[2/0s] Row[5/0.001s] Rows[6/0.002s] Column[1/0s] Update[1/0.002s] Insert[1/0.002s] Delete[2/0.002s]
|
o RRDTool Usage
|
Attachments
Activity
Attach debug discovery for mibs module or all:
./discovery.php -d -h 80
|
Hi Mike,
The device definitely responds to snmp v1, notice the response below on a simple "walk" for a single item "sysDescr":
| [root@nms observium]# snmpwalk -v1 -c redacted switch01 sysDescr |
| SNMPv2-MIB::sysDescr.0 = STRING: Juniper Networks, Inc. ex2200-48t-4g Ethernet Switch, kernel JUNOS 12.3R12.4, Build date: 2016-01-20 05:01:04 UTC Copyright (c) 1996-2016 Juniper Networks, Inc. |
In any case, here's the output you and the bot requested below. Debug output does nothing obviously useful.
[root@security observium]# ./discovery.php -d -h 80
DEBUG!
DEFINITIONS loaded by: 0.0872350 msDEFINITIONS in memory: 20.3MB
SQL[[root@security observium]# ./poller.php -d -h 80
DEBUG!
DEFINITIONS loaded by: 0.0661769 msDEFINITIONS in memory: 20.3MB{}
The myagent.snmpwalk file is attached here: [^myagent.snmpwalk]
Thanks!
General questions and device support can be discussed in our Discord channel, click here to join.
Please make and attach additional information about the device:
- full snmp dump from device:
snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalksnmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalkIf device not support SNMP version 2c, replace -v2c with -v1.
- If you have problems with discovery or poller processes, please do and attach these debugs:
./discovery.php -d -h <device>./poller.php -d -h <device>
- additionally attach device and/or vendor specific MIB files
This comment is added automatically.
This is the only output from the requested command:
[root@security ~]# cd /opt/observium
[root@security observium]# ./discovery.php -d -h 80
DEBUG!
DEFINITIONS loaded by: 0.0815599 ms
DEFINITIONS in memory: 20.3MB
SQL[[root@security observium]#