Details
-
Bug
-
Resolution: Done
-
Major
-
None
-
None
-
None
-
/opt/observium$ ./discovery.php -VV
##### Observium #####
o Version 26.10.15295
o Train stable
o Released 7th October 2026
##### Software versions #####
o OS Linux 6.8.12-43-pve [amd64] (Ubuntu 22.04.5)
o VM LXC Container
o Apache 2.4.52
o PHP 8.1.2-1ubuntu2.26
o Python 3.10.12
o MySQL 8.0.46-0ubuntu0.22.04.4 (client: MySQLnd 8.1.2-1ubuntu2.26)
o SNMP NET-SNMP 5.9.1
o RRDtool 1.7.2
o Fping 5.1 (IPv4 and IPv6)
o Fetch cURL 7.81.0 (OpenSSL/3.0.2, LibZ 1.2.11, LibIDN 2.3.2)
##### PHP info #####
o OPcache DISABLED
o Memory Limit Unlimited
o Math BCMATH
##### DB info #####
o DB schema 551
o MySQL binlog OFF
o MySQL mode NO_ENGINE_SUBSTITUTION
##### Charset info #####
o PHP UTF-8
o MySQL utf8mb3
##### Timezones info #####
o Date Monday, 19-Oct-26 18:39:09 UTC
o PHP +00:00
o MySQL +00:00/opt/observium$ ./discovery.php -VV ##### Observium ##### o Version 26.10.15295 o Train stable o Released 7th October 2026 ##### Software versions ##### o OS Linux 6.8.12-43-pve [amd64] (Ubuntu 22.04.5) o VM LXC Container o Apache 2.4.52 o PHP 8.1.2-1ubuntu2.26 o Python 3.10.12 o MySQL 8.0.46-0ubuntu0.22.04.4 (client: MySQLnd 8.1.2-1ubuntu2.26) o SNMP NET-SNMP 5.9.1 o RRDtool 1.7.2 o Fping 5.1 (IPv4 and IPv6) o Fetch cURL 7.81.0 (OpenSSL/3.0.2, LibZ 1.2.11, LibIDN 2.3.2) ##### PHP info ##### o OPcache DISABLED o Memory Limit Unlimited o Math BCMATH ##### DB info ##### o DB schema 551 o MySQL binlog OFF o MySQL mode NO_ENGINE_SUBSTITUTION ##### Charset info ##### o PHP UTF-8 o MySQL utf8mb3 ##### Timezones info ##### o Date Monday, 19-Oct-26 18:39:09 UTC o PHP +00:00 o MySQL +00:00
Description
Hello,
We recently updated our Observium installation from 26.7.14966 to 26.10.15295 and noticed an impact to the polling times for our devices, both in our regular 5 minute polling cycles and in our quarter-daily discovery cycles.
Pre-upgrade our total polling times averaged about 180 seconds. After the upgrade our polling times are now averaging about 550 seconds. No devices have been added or removed between these two time periods. Some of the outliers for long poll times are listed below:
- mgmt00.auz1: ~150 seconds before upgrade to ~550 seconds after
- mgmt01.auz1: ~90 seconds before upgrade to ~190 seconds after
- mgmt02.auz1: ~80 seconds before upgrade to ~180 seconds after
- mgmt01.dca1: ~130 seconds before upgrade to ~190 seconds after
- mgmt03.dca1: ~70 seconds before upgrade to ~80 seconds after
I've run the debug poll command on two of these devices, mgmt00.auz1 & mgmt03.dca1 and for both devices the longest section was the port polling.
mgmt00.auz1:
CMD[/usr/bin/snmpbulkwalk -Cr'1' -t '4' -v3 -l 'authPriv' -n '' -x *** -X *** -a *** -A *** -u *** -Pud -Ir -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'mgmt00.auz1':'161' ifEntry] |
|
|
CMD EXITCODE[0] |
CMD RUNTIME[213.7908s] |
|
|
--- output cut ---
|
|
|
CMD[/usr/bin/snmpbulkwalk -Cr'1' -t '4' -v3 -l 'authPriv' -n '' -x *** -X *** -a *** -A *** -u *** -Pud -Ir -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'mgmt00.auz1.':'161' ifXEntry] |
|
|
CMD EXITCODE[0] |
CMD RUNTIME[193.5658s] |
mgmt03.dca1
CMD[/usr/bin/snmpbulkwalk -Cr'2' -v3 -l 'authPriv' -n '' -x *** -X *** -a *** -A *** -u *** -Pud -Ir -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'mgmt03.dca1':'161' ifEntry] |
|
|
CMD EXITCODE[0] |
CMD RUNTIME[41.2908s] |
|
|
--- output cut ---
|
|
|
CMD[/usr/bin/snmpbulkwalk -Cr'2' -v3 -l 'authPriv' -n '' -x *** -X *** -a *** -A *** -u *** -Pud -Ir -OQUs -m IF-MIB -M /opt/observium/mibs/rfc:/opt/observium/mibs/net-snmp 'udp':'mgmt03.dca1':'161' ifXEntry] |
|
|
CMD EXITCODE[0] |
CMD RUNTIME[38.7049s] |
Additionally we also started to receive the following message in the Event Log for Observium after the upgrade when we had not received it at all prior to the upgrade:
ERROR! snmpbulkwalk exit by timeout. If the device responds to other SNMP requests, try to decrease SNMP Max Repetitions on the device properties page or set to 0 to not use bulk snmp commands.
We attempted to reduce the maximum repetitions from the default (5) down to 2, and even 1, but that did not stop the error from occurring.
Can you please advise on what we can do to troubleshoot and resolve these significantly increased polling times? discovery.php -VV is attached in the Environment section of the form.