Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-4871

Problems polling OcNOS and Huawei devices

    XMLWordPrintable

Details

    • Bug
    • Resolution: Unresolved
    • Major
    • None
    • Professional Edition
    • Poller
    • None
    • Linux, nginx, php 7.4.33

    Description

      Hello all,

      Lately we are getting problems polling OcNOS and Huawei devices and it happens sporadically. We didn't changed anything on the device's side. The error from poller is below

       

        ___   _                              _
       / _ \ | |__   ___   ___  _ __ __   _() _   _  _ __ ___

            '_ \ / __ / _ | '__ \ \ / /         '_ ` _ \
        _     _) __ |  __/      \ V /     _            

       __/ |._/ |_/ _|||     _/  || _,||| || ||
                                Observium Professional 24.6.13582
                                        https://www.observium.org

              1.  Starting polling run at 2024-07-23 06:50:50  #####
              1.  caesium.del.eix.x3me.net [1484]  #####

       o OS                   ocnos
       o Last Polled
       o SNMP Version         v2c
       o Device status        Device is reachable by PING (0.30ms) and SNMP (10.28ms)
       o Modules Excluded     unix-agent, wmi, ipmi, junose-atm-vp, cisco-ipsec-flow-monitor, cisco-remote-access-monitor, cisco-cef,
                              arista-software-ip-forwarding,cipsec-tunnels, cisco-cbqos, cisco-eigrp, aruba-controller, cisco-vpdn,
                              netscaler-vsvr
       o Modules Disabled     applications, netstats, ucd-mib, ipSystemStats, printersupplies, ucd-diskio, wifi, p2p-radios, lsp, pseudowires,
                              mac-accounting,loadbalancer, entity-physical, fdb-table, oids, syslog, hr-mib
       o Modules Enabled      system, os, sensors, status, counter, processors, mempools, storage, ports, bgp-peers, ospf, sla, graphs, processes, probes

              1.  Module Start: system  #####

       o SNMPv2-MIB
       o Uptime               259 days, 19h 33m 6s
       o Last reboot          2023-11-06 11:17:44
       o Load average         0.5, 0.38, 0.42
       o sysObjectID          .1.3.6.1.4.1.36673.100.1.2.1.1.2
       o snmpEngineID         80001F880334EFB66A024C
       o sysDescr             Hardware Model:EC_AS7716-24SC, Software version: OcNOS,6.3.3.41
       o sysName              caesium.del

          ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣀⡀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⣠⡶⠋⠁⠙⣦⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⣠⡶⠛⠛⠲⣤⣀⠀⠀⠀⠀⡀⠀⠀⠀⠀⠀⠀⠀⣠⠄⣠⣾⣡⠀⠀⠀⠀⠸⡆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⢰⡏⠀⠀⠀⠀⠈⣿⣷⣤⡐⣄⣽⡾⠧⠤⠤⣤⣤⣾⣿⣾⣿⣯⣀⡀⠀⠀⠀⢠⡇⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⢷⡀⠀⠀⠀⠀⣿⣿⣿⣿⣿⣯⣤⣤⡀⣠⣈⣹⣽⣿⣿⣿⣿⣷⣦⣄⠀⢀⡾⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠈⠻⣆⠀⠀⠀⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡟⠀⣠⠞⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠈⢳⣆⠘⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⣾⣷⣶⣤⣀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⠀⣘⣿⣿⣿⣿⣿⣟⣿⣿⣿⣿⣿⣿⣿⣿⢡⡼⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⣾⣿⣿⣿⣿⣿⡏⣯⡇⣿⣿⣿⡏⣿⣿⣿⣜⣛⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡷⣄⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⣿⣿⣿⣿⣿⣿⣿⣭⣾⣿⣿⣿⣇⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡿⠃⠀⠘⢷⣄⠀⠀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⢠⣿⣿⡏⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⡏⢿⣿⣿⡿⠁⠀⠀⠀⠸⣿⣷⡀⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⢸⣿⣿⡇⠈⢻⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⠻⣿⣿⡟⢠⣿⣿⡟⠁⠀⠀⠀⠀⠙⣻⣿⣿⣆⠀⠀⠀⠀⠀
      ⠀⠀⠀⠈⣿⣿⣷⡈⠛⠿⣿⣿⡆⠻⢿⣿⣽⣿⣯⣿⠿⠃⠀⠻⠛⣰⣿⣿⠏⠀⠀⠀⠀⢀⣙⣿⣿⣿⣿⣿⣧⡀⠀⠀⠀
      ⠀⠀⠀⠀⢿⠈⠙⠻⠷⠤⠄⢉⡁⠀⠀⠙⢉⣵⣌⠀⠀⠀⣀⠄⠘⠛⠋⠁⢀⣀⣤⣤⣭⣭⣽⣿⣿⣿⣿⣿⣿⣷⡀⠀⠀
      ⠀⠀⠀⢀⣿⡄⠀⠀⠀⠀⠀⠀⠉⠓⢦⣄⡉⠉⢉⣩⠶⠋⠁⠀⠀⠀⣠⣾⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣷⡀⠀
      ⠀⠀⠀⣼⣿⣿⣷⣶⣶⣆⣀⡀⠀⠀⠀⠀⠉⠉⠉⠀⠀⠀⠀⠀⣠⣾⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣿⣧⠀
      ⠀⠀⠀⣿⣿⣿⣿⣿⣿⣿⣥⣄⡀⠀⠀⠀⠀⠀⠀⠀⠀⣸⣦⣾⣿⣿⣿⣿⣿⣿⡿⠿⠿⡟⠿⣿⣟⡿⢿⡛⠀⣘⣿⣿⡆
      ⠀⠀⠀⢿⣿⣿⠿⠿⠿⠿⢿⣿⣿⣷⣿⡆⠀⠀⣶⣶⣾⡿⠛⠛⠛⠛⠻⣿⣿⣿⡟⠉⠀⠀⠀⠈⠁⠁⢀⢀⠲⣬⣿⣿⡇
      ⠀⠀⠀⠘⡏⠳⣤⡄⠀⡄⠀⠀⠈⠙⢿⣧⠀⢀⣿⣿⠏⠀⠀⠀⠀⢀⣠⣿⣿⠋⠀⠀⠀⠀⠀⠀⠀⠀⠈⣯⣷⣿⣿⣿⡇
      ⠀⠀⠀⠀⢿⠀⠀⠉⠳⣇⠀⠀⠀⠀⠀⢻⠀⢸⣿⣿⣆⢠⠀⡆⣰⡿⠟⠉⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⡾⣿⣿⣿⣿⣿⡇
      ⠀⠀⠀⠀⠘⣧⠀⠀⠀⠹⣷⣤⣄⠀⡈⣾⡆⢸⡿⠋⠿⠾⠴⠷⠋⠀⠀⠀⠀⠀⠀⠀⠀⠀⢀⣠⠞⠉⣶⣾⣿⣿⣿⣿⠀
      ⠀⠀⠀⠀⠀⠈⠳⣄⠀⠀⠀⠉⠻⣿⣿⣿⣇⠸⡷⠀⠀⠀⢀⣠⢄⣠⣶⣤⣤⣤⡤⠤⠖⠛⠉⠀⠠⣤⣼⣿⣿⣿⣿⡏⠀
      ⠀⠀⠀⠀⠀⠀⠀⠉⢻⣷⠶⠒⠀⠀⠁⠈⢿⡄⣷⣆⣤⣶⡿⠛⠛⠛⠉⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠈⢿⣿⣿⣿⣿⠇⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠘⣏⠀⠀⠀⠀⠀⠀⠘⣧⣿⣿⡿⢋⡤⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠰⣾⣿⣿⣿⣿⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⢻⡄⠀⠀⠀⠀⠀⠀⢸⣿⣿⣶⣫⠆⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⡀⢦⣵⣿⣿⣿⣿⠇⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⢻⣆⠀⠀⠀⠀⠀⠀⣿⣿⣿⠇⠀⣀⠀⠀⠀⠀⠀⠀⠀⠀⠳⠦⣴⣿⣶⣿⣿⣿⣿⡿⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠛⢷⣄⠀⠀⠀⠀⢹⣿⣿⣾⣿⣋⣤⡄⠀⠀⠀⠀⠀⠀⠀⠠⣈⠻⣿⣿⣿⣿⣿⡇⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠈⠉⠛⣦⣤⣤⣾⣿⣿⣿⣿⣿⣯⡴⠂⣀⡀⠀⡀⡀⡀⢠⣬⣻⣿⣿⣿⣿⡿⠀⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⢀⣀⣀⣀⣀⣠⡴⠊⠉⠀⠈⢹⠿⣿⣿⣿⣿⣿⣷⣾⣿⠾⡿⡿⠿⣷⣦⣿⣿⣿⣿⣿⡟⠁⠀⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠛⠷⠦⠤⠤⠴⠶⠶⠶⠶⠚⠉⠀⠀⠈⣉⣩⠽⠟⠋⠁⠀⠀⣁⣠⠿⠛⠋⠉⠉⠉⠁⠀⠀⠀⠀⠀⠀
      ⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠉⠉⠉⠉⠉⠉⠁⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀
      Oops! Something went wrong! You may want to report this to the Observium developers.

      Call to undefined function mb_ord() in /opt/observium/includes/common.inc.php:4972

      Stack trace:

      #0: (unknown function) called at [/opt/observium/includes/common.inc.php:4972]
      4967: * "\u00cb\u0419\u0426\u0423\u041a\u0415\u041d\u0413\u0428\u0429\u0417\u0425\u042a\u0424\u042b\u0412\u0410\u041f\u0420\u041e\u041b\u0414\u0416\u042d\u042f\u0427\u0421\u041c\u0418\u0422\u042c\u0411\u042e"
      4968: */
      4969: function fix_json_unicode($string)
      4970:

      Unknown macro: { 4971}

      -\x{FFFF}])/u', function ($match)

      Unknown macro: { 4972}

      , $string);
      4974: }
      4975:
      4976: function safe_unserialize($str)
      4977:

      Unknown macro: { #1}

      called at [(unknown file):(unknown line)]

      #2: preg_replace_callback called at [/opt/observium/includes/common.inc.php:4973]
      4968: */
      4969: function fix_json_unicode($string)
      4970:

      Unknown macro: { 4971}

      -\x{FFFF}])/u', function ($match)

      Unknown macro: { 4972}

      , $string);
      4974: }
      4975:
      4976: function safe_unserialize($str)
      4977: {
      4978: if (is_array($str))

      Unknown macro: { #3}

      else

      Unknown macro: { 4945}

      4949: $json_fix = @json_decode($str_fix, TRUE, 512, $options);
      4950: if (json_last_error() === JSON_ERROR_NONE) {
      4951: //print_vars(smart_quotes(fix_json_unicode($str)));

      #4: safe_json_decode called at [/opt/observium/includes/http.inc.php:611]
      606:
      607: // If response return valid code and content, additional parse for specific defined tests
      608: if ($success) {
      609: // Decode if request OK
      610: if ($is_response_array)

      Unknown macro: { 611}

      613: // else additional formats?
      614:
      615: // Check if call succeeded
      616: if (isset($def['response_test'])) {

      #5: test_http_request called at [/opt/observium/includes/http.inc.php:712]
      707: $request_status = FALSE;
      708: // Send out API call and parse response
      709: for ($retry = 1; $retry <= (int)$request_retry; $retry++) {
      710: print_debug("Request [$url] #$retry:");
      711: $response = get_http_request($url, $options, $ratelimit);
      712: if ($request_status = test_http_request($def, $response))

      Unknown macro: { 713}

      716: // wait little time
      717: sleep($request_sleep);

      #6: process_http_request called at [/opt/observium/includes/geolocation.inc.php:339]
      334:
      335: // API URL to POST to
      336: $url = generate_http_url($geo_def, $tags);
      337:
      338: // First or second request
      339: if (process_http_request($geo_def, $url, $options, $mapresponse)) {
      340:
      341: switch ($geo_def['response_format']) {
      342: case 'xml':
      343: // Hrm, currently unused
      344: break;

      #7: get_geo_http_def called at [/opt/observium/includes/geolocation.inc.php:142]
      137: $address_second = $geo_type === 'forward' ? generate_location_alt($address) : NULL;
      138:
      139: // Generate geolocation tags, used for rewrites in definition
      140: $tags = generate_geolocation_tags($location['location_geoapi'], $location);
      141:
      142: $data = get_geo_http_def($geo_def, $geo_type, $tags, $debug_msg);
      143:
      144: if (!$data) {
      145: // Return an old array with new status (for later recheck)
      146: unset($geo_db['hostname'], $geo_db['location_updated']);
      147: $location['location_status']  = $debug_msg;

      #8: get_geolocation called at [/opt/observium/includes/polling/system.inc.php:311]
      306: print_debug_vars($geo_db);
      307:
      308: $geo_detect = geo_detect($device, $poll_device, $geo_db, $dns_only);
      309:
      310: if ($geo_detect || $dns_only) {
      311: $update_geo = get_geolocation($poll_device['sysLocation'], $geo_db, $dns_only);
      312: if ($update_geo) {
      313: print_debug_vars($update_geo, 1);
      314: if (is_numeric($update_geo['location_lat']) && is_numeric($update_geo['location_lon']) && $update_geo['location_country'] !== 'Unknown')

      Unknown macro: { 315}

      186:
      187: foreach (dbFetchColumn($query, $params) as $device_id)

      Unknown macro: { 188}

      192:
      193: $poller_time = elapsed_time($poller_start, 4);
      194:

      Attachments

        1. caesium.poller
          39 kB
        2. caesium.snmpwalk
          4.40 MB

        Activity

          People

            landy Mike Stupalov
            ymarinov Yavor Marinov
            Votes:
            1 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: