Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-1522

Rrd update crash - ifHCOutBroadcastPkts outsized

    XMLWordPrintable

Details

    • Bug
    • Resolution: Not A Bug
    • Blocker
    • None
    • Professional Edition
    • Poller
    • None
    • php 5.4

    Description

      When I poll some ports on my device, the value return is too big for the rrd to handle, and rrdcached crashes.
      I tried deleting the rrd & restarting rrdcached, but observium recreates it with the same max value, and the error comes back.

      When rrdtool crashes, all graphs are down, which is pretty annoying.

      Debug :

      ./poller.php -h 3 -m ports -d
      [...]
      ifXEntry DEBUG: SNMP Auth options = -v2c -c 'pouetpouet'

      CMD[/usr/bin/snmpbulkwalk -v2c -c 'pouetpouet' -Pu -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'mydevice.domain.tld':'161' ifXEntry]

      EXITCODE[0]
      CMD RUNTIME[0.0177s]
      STDOUT[
      ifName.1 = vmnic0
      ifName.2 = vmnic1
      [...]
      ifHCOutMulticastPkts.6 = 0
      ifHCOutMulticastPkts.7 = 0
      ifHCOutBroadcastPkts.1 = 101115
      ifHCOutBroadcastPkts.2 = 18446742978492891134
      ifHCOutBroadcastPkts.3 = 63455
      ifHCOutBroadcastPkts.4 = 538
      ifHCOutBroadcastPkts.5 = 0

      // The problematic value is ifHCOutBroadcastPkts.2

      [...]
      RRD CMD[update /opt/observium/rrd/mydevice.domain.tld/port-2.rrd N:2222411483:112999116:0:0:75870324490:73554100427:373958:1.8446742978497E+19:0:0:0:173621:18446742978492891134:200337:4540969 --daemon unix:/var/run/rrdcached.sock]
      RRD RUNTIME[0.0011s]
      RRD STDOUT[OK u:0.00 s:0.00 r:0.41]
      RRD_STATUS[TRUE]

      // This command generate this error in syslog :
      Oct 27 09:11:30 SUP11 rrdcached[12750]: queue_thread_main: rrd_update_r (/opt/observium/rrd/mydevice.domain.tld/port-2.rrd) failed with status -1. (/opt/observium/rrd/mydevice.domain.tld/port-2.rrd: not a simple signed integer: '1.8446742978497E+19')

      1. rrdtool info /opt/observium/rrd/mydevice.domain.tld/port-2.rrd|grep -i OutBroadcastPkts
        ds[OUTBROADCASTPKTS].index = 12
        ds[OUTBROADCASTPKTS].type = "DERIVE"
        ds[OUTBROADCASTPKTS].minimal_heartbeat = 600
        ds[OUTBROADCASTPKTS].min = 0.0000000000e+00
        ds[OUTBROADCASTPKTS].max = 1.2500000000e+10
        ds[OUTBROADCASTPKTS].last_ds = "18446742978492891134"
        ds[OUTBROADCASTPKTS].value = NaN
        ds[OUTBROADCASTPKTS].unknown_sec = 68

      Attachments

        Activity

          People

            landy Mike Stupalov
            fufroma Alan Garcia
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: