Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-5302

Collectd graphs broken with collectd 5.x: legacy DS names and redis "memory" meta graph

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Minor
    • None
    • Professional Edition
    • Graphs
    • Observium Professional 26.10.15293

    Description

      Summary

      collectd graphs: several definitions fail with rrdtool errors (legacy collectd 4.x DS names, redis "memory" type)

      Environment

      • Observium Professional 26.10.15293
      • collectd 5.x (redis, apache, ntpd, ... plugins)

      Bug 1 — redis memory graph: "ERROR: don't understand '_nnl'"

      ... '-b' '1024' 'COMMENT:Bytes Cur Min Avg Max\l' 'CDEF:_stk=_nnl' -s ... -e ...
      RRDTool Output: ERROR: don't understand '_nnl'
      

      Cause: the meta graph is selected by collectd type only ($MetaGraphDefs['memory'] = 'meta_graph_memory'). meta_graph_memory() only looks for the files written by the collectd "memory" plugin (memory-free.rrd, memory-used.rrd, ...). The redis plugin also uses the "memory" type but writes a single memory.rrd without type instance. No source is found, so collectd_draw_meta_stack() emits no DEF at all, and end($sources) on an empty array produces the orphan CDEF:_stk=_nnl.

      Fix (patch 01): when no type instance matches, fall back to the single <type>.rrd file:

          if (empty($sources)) {
              foreach ($config['datadirs'] as $datadir) {
                  if (is_file($datadir . '/' . $title . '.rrd')) {
                      $sources[] = ['name' => 'used', 'file' => $datadir . '/' . $title . '.rrd'];
                      break;
                  }
              }
          }
      

      Bug 2 — legacy collectd 4.x DS names: "ERROR: No DS called '...'"

      RRDTool Output: ERROR: No DS called 'percent' in '.../redis-localhost/percent-hitratio.rrd'
      

      Cause: collectd 5.0 renamed the data source of most single-value types to value. Several $GraphDefs still use the collectd 4.x names, so every graph of these types fails with current collectd. I checked all definitions against collectd 5 types.db:

      Type Old DS collectd 5 DS
      apache_bytes, apache_requests, apache_scoreboard (+ meta_graph_apache_scoreboard) count value
      percent percent value
      spam_check hits value
      entropy entropy value
      frequency frequency value
      time_offset, time_dispersion, delay (ntpd) seconds value
      frequency_offset (ntpd) ppm value
      ping ping value
      timeleft timeleft value
      users users value

      Fix (patch 01): use DS value for all of the above. load_percent is a different type and is left untouched. $GraphDefs['vs_threads'] and ['vs_memory'] also use old multi-DS layouts, but they are never reached because the meta graphs of the same name take precedence, so they are left as is.

      Testing: the redis memory/percent, apache, entropy, users fixes run in production here. All other renamed definitions were tested against r15092 with rrdtool 1.7.2 on synthetic RRDs created with the collectd 5 layout (DS "value"): each one fails before the patch and renders after it.

      Hardening (patch 02)

      The same class of bug as Bug 1 can hit any meta graph type (cpu, swap, ps_state, ...) when another plugin uses that type with different type instances. Patch 02 makes collectd_draw_meta_stack() / collectd_draw_meta_line() return FALSE when there is nothing to draw, and makes graphs/device/collectd.inc.php fall back to the generic single-file graph in that case. Returning FALSE alone is not enough, because the caller appends -s/-e to $rrd_cmd and rrdtool would fail with another error. Patch 02 alone (without 01) is also enough to render the redis memory graph, via $GraphDefs['memory'].

      Testing: patch 02 runs in production here, together with patch 01.

      Attachments

      • 01-collectd-memory-fallback-and-ds-names.patch — html/includes/collectd/definitions.php
      • 02-collectd-meta-graph-fallback.patch — html/includes/collectd/functions.php, html/includes/graphs/device/collectd.inc.php

      Attachments

        Activity

          People

            adama Adam Armstrong
            alexbat Alex B
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: