Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-4095

mibs module says during discovery.php run: "Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS."

Details

    • Help
    • Resolution: Cannot Reproduce
    • Major
    • None
    • CE-21.10
    • Discovery

    Description

      Hello!

      I could really use some help. We noticed most of our graphs had disappeared. Unfortunately, exactly what went on to break this is a mystery for now and I'm trying to unravel the issue and fix it.

      While manually running a discovery on all devices, I saw this error in the mibs module output for every single device:

       

      Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS.

       

      The system hosting Observium can snmpwalk any of our devices as normal, but the "mibs" module in Observium is unable to do the walks needed during a run of discovery.php. The result is all of our traffic graphs for all devices have basically disappeared and I just can't see why calls to the snmp_command() function is failing.

      All the usual mibs are in the /opt/observium/mibs directory and permissions are all readable.

      I've run out of ideas and need help on where to look next. Any clues would be super appreciated! If any other information is needed to figure this out, please let me know and I'll post.

      Thanks!

       

       

      Example of an snmpwalk working just fine outside Observium on one of our faithful old switches:

      [root@nms observium]# snmpwalk -v1 -c redacted switch01 sysDescr
      SNMPv2-MIB::sysDescr.0 = STRING: Juniper Networks, Inc. ex2200-48t-4g Ethernet Switch, kernel JUNOS 12.3R12.4, Build date: 2016-01-20 05:01:04 UTC Copyright (c) 1996-2016 Juniper Networks, Inc.
      

       

      An example of one specific device running against only the mibs module, for output brevity's sake:

      [root@nms observium]# /opt/observium/discovery.php -h 80 -m mibs
        ___   _                              _
       / _ \ | |__   ___   ___  _ __ __   __(_) _   _  _ __ ___
      | | | || '_ \ / __| / _ \| '__|\ \ / /| || | | || '_ ` _ \
      | |_| || |_) |\__ \|  __/| |    \ V / | || |_| || | | | | |
       \___/ |_.__/ |___/ \___||_|     \_/  |_| \__,_||_| |_| |_|
                          Observium Community Edition 21.10.11666
                                        https://www.observium.org
      #####  Starting discovery run at 2022-04-26 11:34:00  #####
      #####  switch01 [80]  #####
      +-------------+--+
      | OID         |  |
      +-------------+--+
      | sysDescr    |  |
      | sysObjectID |  |
      +-------------+--+
       o OS Type              generic
       o OS Group             unix
       o SNMP Version         v1
       o Last discovery       2022-04-26 11:33:46
       o Last duration        2.53 seconds
       
      #####  Module Start: mibs  #####
      Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS.
       o MIBs discovered      <empty>
       o Duration             0.325s
       
      #####  switch01 [80] completed discovery modules at 2022-04-26 11:34:01  #####
       o Discovery time       0.5958 seconds
       
      #####  Finished discovery run at 2022-04-26 11:34:01  #####
       o Devices Discovered   1
       o Discovery Time       0.627 secs
       o Memory usage         32.5MB (peak: 32.5MB)
       o MySQL Usage          Cell[2/0s] Row[5/0.001s] Rows[6/0.002s] Column[1/0s] Update[1/0.002s] Insert[1/0.002s] Delete[2/0.002s]
       o RRDTool Usage
      

      Attachments

        Activity

          [OBS-4095] mibs module says during discovery.php run: "Empty oids passed to snmp_command(). THIS SHOULD NOT HAPPEN. PLEASE REPORT TO DEVELOPERS."
          vic-sb Vic Breen added a comment - - edited

          This is the only output from the requested command:

           

           

           

          [root@security ~]# cd /opt/observium
          [root@security observium]# ./discovery.php -d -h 80
          DEBUG!
          DEFINITIONS loaded by: 0.0815599 ms

          DEFINITIONS in memory: 20.3MB

          SQL[[root@security observium]#

          vic-sb Vic Breen added a comment - - edited This is the only output from the requested command:       [root@security ~] # cd /opt/observium [root@security observium] # ./discovery.php -d -h 80 DEBUG! DEFINITIONS loaded by: 0.0815599 ms DEFINITIONS in memory: 20.3MB SQL[ [root@security observium] #

          Attach debug discovery for mibs module or all:

          ./discovery.php -d -h 80

          landy Mike Stupalov added a comment - Attach debug discovery for mibs module or all: ./discovery.php -d -h 80
          vic-sb Vic Breen added a comment -

          Hi Mike,

          The device definitely responds to snmp v1, notice the response below on a simple "walk" for a single item "sysDescr":

           

          [root@nms observium]# snmpwalk -v1 -c redacted switch01 sysDescr
          SNMPv2-MIB::sysDescr.0 = STRING: Juniper Networks, Inc. ex2200-48t-4g Ethernet Switch, kernel JUNOS 12.3R12.4, Build date: 2016-01-20 05:01:04 UTC Copyright (c) 1996-2016 Juniper Networks, Inc.

           

          In any case, here's the output you and the bot requested below. Debug output does nothing obviously useful.

           

          [root@security observium]# ./discovery.php -d -h 80
          DEBUG!
          DEFINITIONS loaded by: 0.0872350 msDEFINITIONS in memory: 20.3MB
          SQL[[root@security observium]# ./poller.php -d -h 80
          DEBUG!
          DEFINITIONS loaded by: 0.0661769 msDEFINITIONS in memory: 20.3MB{}

           

          The myagent.snmpwalk file is attached here: [^myagent.snmpwalk]

           

          Thanks!

          vic-sb Vic Breen added a comment - Hi Mike, The device definitely responds to snmp v1, notice the response below on a simple "walk" for a single item "sysDescr":   [root@nms observium] # snmpwalk -v1 -c redacted switch01 sysDescr SNMPv2-MIB::sysDescr.0 = STRING: Juniper Networks, Inc. ex2200-48t-4g Ethernet Switch, kernel JUNOS 12.3R12.4, Build date: 2016-01-20 05:01:04 UTC Copyright (c) 1996-2016 Juniper Networks, Inc.   In any case, here's the output you and the bot requested below. Debug output does nothing obviously useful.   [root@security observium] # ./discovery.php -d -h 80 DEBUG! DEFINITIONS loaded by: 0.0872350 ms DEFINITIONS in memory: 20.3MB SQL[ [root@security observium] # ./poller.php -d -h 80 DEBUG! DEFINITIONS loaded by: 0.0661769 ms DEFINITIONS in memory: 20.3MB { }   The myagent.snmpwalk file is attached here: [^myagent.snmpwalk]   Thanks!

          It seems as device not respond by snmp.

          landy Mike Stupalov added a comment - It seems as device not respond by snmp.

          Attach debug discovery for:

          ./discovery.php -d -m mibs -h 80
          

          landy Mike Stupalov added a comment - Attach debug discovery for: ./discovery.php -d -m mibs -h 80

          General questions and device support can be discussed in our Discord channel, click here to join.


          Please make and attach additional information about the device:

          • full snmp dump from device:

            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk
            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk

            If device not support SNMP version 2c, replace -v2c with -v1.

          • If you have problems with discovery or poller processes, please do and attach these debugs:

            ./discovery.php -d -h <device>
            ./poller.php -d -h <device>

          • additionally attach device and/or vendor specific MIB files

          This comment is added automatically.

          bot Observium Bot added a comment - General questions and device support can be discussed in our Discord channel, click here to join . Please make and attach additional information about the device: full snmp dump from device: snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk If device not support SNMP version 2c, replace -v2c with -v1. If you have problems with discovery or poller processes, please do and attach these debugs: ./discovery.php -d -h <device> ./poller.php -d -h <device> additionally attach device and/or vendor specific MIB files This comment is added automatically.

          People

            landy Mike Stupalov
            vic-sb Vic Breen
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: