Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-4107

Ubiquiti LTE device show corrupt serial numbers

Details

    • Vendor Bug
    • Resolution: Fixed
    • Major
    • None
    • None
    • Default

    Description

      Ubiquiti LTE device show corrupt serial numbers in the Event log and are been replaced perpetually in a stuck loop. Only happened after updating to 22.5 

      Attachments

        Activity

          [OBS-4107] Ubiquiti LTE device show corrupt serial numbers

          Thanks for looking at this and with the update installed, the repeated poling message have stopped and the record no longer shows a serial number.

          Thanks everyone

          HughesHall Paul Blaxill added a comment - Thanks for looking at this and with the update installed, the repeated poling message have stopped and the record no longer shows a serial number. Thanks everyone

          It seems that this device didn't respond to the serial (mac address) query this poll : 

          {{}}

          CMD[/usr/bin/snmpget -v2c -c *** -Pud -Ih -OQUvx --hexOutputLength=0 -m UBNT-AFLTU-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/ubiquiti 'udp':'hhhnroofap-a.hughes.ad.cam.ac.uk':'161' afLTUMac.0]

          CMD EXITCODE[0]
          CMD RUNTIME[0.0275s]

          STDOUT[{}}}{{{}No Such Object available on this agent at this OID{}}}{{{}]
          SNMP STATUS[FALSE]
          SNMP ERROR1000 - Failed response

           

          SQL[UPDATE `devices` set `uptime` ='21931026',`version` ='',`hardware` ='',`serial` ='',`last_polled` =NOW(),`last_polled_timetaken` ='1.5784' WHERE `device_id` = '38']SQL RUNTIME[0.00293589s] 
          o Updated Data         uptime, version, hardware, serial, last_polled, last_polled_timetaken

          Though it had a garbage serial in the database already from the previous poll:

            [serial]                    => string(8; UTF-8) "ô?¿i+"

          You should re-run the poll a couple of times to check for these parts and see what the device is returning for afLTUMac.0

          ./poller.php -h <device_id> -m os -d

          Or you can manually run the snmpget command a few times and see what it returns over time (might only change infrequently, or every poll).

          This definitely looks like a vendor bug that Ubnt need to fix.

          adam.

          adama Adam Armstrong added a comment - It seems that this device didn't respond to the serial (mac address) query this poll :  {{}} CMD [/usr/bin/snmpget -v2c -c *** -Pud -Ih -OQUvx --hexOutputLength=0 -m UBNT-AFLTU-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp:/opt/observium/mibs/ubiquiti 'udp':'hhhnroofap-a.hughes.ad.cam.ac.uk':'161' afLTUMac.0] CMD EXITCODE [0] CMD RUNTIME [0.0275s] STDOUT [{}}}{{{}No Such Object available on this agent at this OID{}}}{{{}] SNMP STATUS [FALSE] SNMP ERROR 1000 - Failed response   SQL [UPDATE `devices` set `uptime` ='21931026',`version` ='',`hardware` ='',`serial` ='',`last_polled` =NOW(),`last_polled_timetaken` ='1.5784' WHERE `device_id` = '38'] SQL RUNTIME [0.00293589s]   o Updated Data         uptime, version, hardware, serial, last_polled, last_polled_timetaken Though it had a garbage serial in the database already from the previous poll:   [serial]                     => string(8; UTF-8) "ô?¿i+" You should re-run the poll a couple of times to check for these parts and see what the device is returning for afLTUMac.0 ./poller.php -h <device_id> -m os -d Or you can manually run the snmpget command a few times and see what it returns over time (might only change infrequently, or every poll). This definitely looks like a vendor bug that Ubnt need to fix. adam.

          Your device seems have an memory leak and return random data on requests serial Oids.

          I added checks for valid serials for this os.
          Issue should be fixed in r12011.

          landy Mike Stupalov added a comment - Your device seems have an memory leak and return random data on requests serial Oids. I added checks for valid serials for this os. Issue should be fixed in r12011.

          New file uploaded

          HughesHall Paul Blaxill added a comment - New file uploaded

          This looks like the device has a memory leak or something and is returning random data for the serial number.

          As Mike said, the poller debug output starts half way through at the beginning of the mempools module. The serial is collected near the beginning in the system module. You can do a shorter debug with :

          ./poller.php -h <device_id> -m system -d

          adam.

          adama Adam Armstrong added a comment - This looks like the device has a memory leak or something and is returning random data for the serial number. As Mike said, the poller debug output starts half way through at the beginning of the mempools module. The serial is collected near the beginning in the system module. You can do a shorter debug with : ./poller.php -h <device_id> -m system -d adam.

          Please make snmpdump from an LTE device (as Bot requested).
          You attached not useful output (not full poller debug).

          landy Mike Stupalov added a comment - Please make snmpdump from an LTE device (as Bot requested). You attached not useful output (not full poller debug).

          General questions and device support can be discussed in our Discord channel, click here to join.


          Please make and attach additional information about the device:

          • full snmp dump from device:

            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk
            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk

            If device not support SNMP version 2c, replace -v2c with -v1.

          • If you have problems with discovery or poller processes, please do and attach these debugs:

            ./discovery.php -d -h <device>
            ./poller.php -d -h <device>

          • additionally attach device and/or vendor specific MIB files

          This comment is added automatically.

          bot Observium Bot added a comment - General questions and device support can be discussed in our Discord channel, click here to join . Please make and attach additional information about the device: full snmp dump from device: snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk If device not support SNMP version 2c, replace -v2c with -v1. If you have problems with discovery or poller processes, please do and attach these debugs: ./discovery.php -d -h <device> ./poller.php -d -h <device> additionally attach device and/or vendor specific MIB files This comment is added automatically.

          People

            landy Mike Stupalov
            HughesHall Paul Blaxill
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: