Details
-
Bug
-
Resolution: Not A Bug
-
Major
-
None
-
None
-
None
Description
Dear,
we have a subscription observium ver. 0.14.10.5902(stable) and more than 400 devices connected. We saw that for linux servers 'disk_free' parameter is not correctly shown. Example:
-on debian 7 server:
rootfs 67840600 63365072 1029412 99% /
Observium databases (storage-state table) sais:
storage_id storage_polled storage_size storage_units storage_used storage_free storage_perc :
343 1414504366 69468774400 4096 64885837824 4582936576 93
(storage table):
storage_id device_id storage_mib storage_index storage_type storage_descr storage_hc storage_deleted:
343 192 host-resources-mib 32 hrStorageFixedDisk / 0 0
Also on centos servers same issue.
In other monitoring tools I saw that ucd-snmp-mib MIB is used. Is it possible to change mib poller from host-resources-mib to ucd-snmp-mib?
So, the problem is that on the server is only 1006M (1 GB) free disk space, and on observium GUI is 4.27 GB free, and alert checks not working for us.
Thanks in advance!
Attachments
Activity
UCD values are definitely there.
First server above
UCD-SNMP-MIB::dskPath.1 = STRING: / |
UCD-SNMP-MIB::dskPath.2 = STRING: /dev |
UCD-SNMP-MIB::dskPath.3 = STRING: /usr/home/freenet_jail/usr/ports |
UCD-SNMP-MIB::dskPath.4 = STRING: /usr/home/tor_jail/dev |
UCD-SNMP-MIB::dskPath.5 = STRING: /usr/home/freenet_jail/dev |
UCD-SNMP-MIB::dskDevice.1 = STRING: /dev/mirror/root |
UCD-SNMP-MIB::dskDevice.2 = STRING: devfs |
UCD-SNMP-MIB::dskDevice.3 = STRING: /usr/ports |
UCD-SNMP-MIB::dskDevice.4 = STRING: devfs |
UCD-SNMP-MIB::dskDevice.5 = STRING: devfs |
...
|
Second server above
UCD-SNMP-MIB::dskPath.1 = STRING: / |
UCD-SNMP-MIB::dskPath.2 = STRING: /dev |
UCD-SNMP-MIB::dskPath.3 = STRING: /tmp |
<snip for space> |
UCD-SNMP-MIB::dskPath.11 = STRING: /var/tmp |
UCD-SNMP-MIB::dskPath.12 = STRING: /zroot |
UCD-SNMP-MIB::dskDevice.1 = STRING: zroot/ROOT/default |
UCD-SNMP-MIB::dskDevice.2 = STRING: devfs |
UCD-SNMP-MIB::dskDevice.3 = STRING: zroot/tmp |
<snip for space> |
UCD-SNMP-MIB::dskDevice.11 = STRING: zroot/var/tmp |
UCD-SNMP-MIB::dskDevice.12 = STRING: zroot |
...
|
Net-snmp has probably ceased to provide dskTable entries for these filesystems.
You can verify with snmpwalk.
All of my servers have both available, but observium is using HOST-RESOURCES for disk space calculations.
2017-01-26 04:55:22 /, type: ufs, dev: /dev/mirror/root Storage added: index 4, mib host-resources-mib, descr /, type: ufs, dev: /dev/mirror/root |
2017-01-26 04:55:17 marin Mempools: 4 added. |
2017-01-26 04:55:17 Swap:/dev/mirror/swap Memory pool added: mib host-resources-mib, index 3, descr Swap:/dev/mirror/swap |
2017-01-26 04:55:17 Shared Real Memory Metrics Memory pool added: mib host-resources-mib, index 2, descr Shared Real Memory Metrics |
2017-01-26 04:55:17 Real Memory Metrics Memory pool added: mib host-resources-mib, index 1, descr Real Memory Metrics |
2017-01-26 04:55:05 Physical memory Memory pool added: mib ucd-snmp-mib, index 0, descr Physical memory |
Interestingly, looking at a new server just added, it removes a UCD and replaces it with a HOST-RESOURCES a day after the server was added (no changes were made to the server)
2017-04-10 00:37:03 yukino Storage: 1 added, 1 deleted, 10 unchanged. |
2017-04-10 00:37:03 Storage removed: index 1, mib ucd-snmp-mib, descr / |
2017-04-10 00:37:03 /, type: zfs, dev: zroot/ROOT/default Storage added: index 7, mib host-resources-mib, descr /, type: zfs, dev: zroot/ROOT/default |
<Snip non-storage related stuff>
|
2017-04-09 22:50:54 yukino Storage: 11 added. |
2017-04-09 22:50:54 /zroot, type: zfs, dev: zroot Storage added: index 18, mib host-resources-mib, descr /zroot, type: zfs, dev: zroot |
2017-04-09 22:50:54 /var/tmp, type: zfs, dev: zroot/var/tmp Storage added: index 17, mib host-resources-mib, descr /var/tmp, type: zfs, dev: zroot/var/tmp |
<snip other partitions>
|
2017-04-09 22:50:54 /usr/home, type: zfs, dev: zroot/usr/home Storage added: index 10, mib host-resources-mib, descr /usr/home, type: zfs, dev: zroot/usr/home |
2017-04-09 22:50:54 /tmp, type: zfs, dev: zroot/tmp Storage added: index 9, mib host-resources-mib, descr /tmp, type: zfs, dev: zroot/tmp |
|
UCD-MIB is used in preference to HOST-RESOURCES-MIB where it is available and provides more accurate information.
This can be easily worked around by using ucd-snmp-mib instead of host-resources-mib.
Why is host-resources-mib used over ucd when it has known issues?
This is known "issue" for net-snmp on linux hosts.
But it can not be fixed in observium, because there is no universal mechanism for detect real fixed storages and virtual "memory" or so on storages.
I tried various ways, but without success.
This is a very bad solution. I'm also not sure if you're right, as we've never encountered this issue before.
Any changes will be overwritten on upgrade.
Thanks for your answer. Finally I am done with free storage checks and I can set alerts.
I found on some forum that linux uses offset for disk space, and found a nice formula: "free = (total*offset) - used" (offset is normarly 0.05 (or 5%) for linux filesystems, which means 5% of the filesystem is reserved).
So I have changed line in /opt/observium/includes/polling/storage/host-resources-mib.inc.php
from:
$storage['free'] = $storage['size'] - $storage['used'];
to:
$storage['free'] = $storage['size'] * 0.95 - $storage['used'];
I hope that on next stable observium svn update my modification will not be removed.
Thanks and best regards!
This makes no sense. HOST-RESOURCES-MIB storage values are for filesystems. Other partitions have no impact on other filesystems.
dskTable is only used when the data returned by it is different to what's in host-resources-mib.
This shouldn't change once things are discovered though.