Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-1335

Storage snmp/wmi mislabeled volumes on Windows

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • None
    • Community Edition, Professional Edition
    • Discovery, Poller
    • Observium 6493 against 2012 R2 Datacenter with net-snmp 5.7.2 and wmic 4.0.0tp4-SVN-build-UNKNOWN

    Description

      Windows volumes have an extra backslash when polled via SNMP, and an incorrect rrd filename when polled via WMI. This causes issues with historic data when changing methods.

      Windows responds to host-resources-mib while polling via snmp, however, net-snmp output containing backslashes is escaped with an extra backslash. This patch removes the extra backslash from $descr so that paths are correct in the web interface, database, and rrd files. Existing Windows volumes should be removed or have both storage.storage_descr updated and rrd file renamed.

      It also uses the correct storage_mib as referenced in the database for rrd filenames. WMI output returns paths correctly, but rrd filenames contain hrstorage instead of host-resources-mib as stored in storage.storage_mib. This is noted in OBSERVIUM-938.

      Last, unknown volumes are skipped and device rediscovery is requested.

      Before:
      storage-host-resources-mib-C____Label___Serial_Number_9a842de8.rrd
      storage-hrstorage-C___Label___Serial_Number_9a842de8.rrd
      C:\\
       
      After:
      storage-host-resources-mib-C___Label___Serial_Number_9a842de8.rrd
      C:\
      

      Attachments

        Issue Links

          Activity

            [OBS-1335] Storage snmp/wmi mislabeled volumes on Windows

            Fixed in r6807.

            landy Mike Stupalov added a comment - Fixed in r6807.

            Alternatively, let the labels stay wrong and just add an extra backslash to WMI.

            jpmasseo Jonathan Masseo added a comment - Alternatively, let the labels stay wrong and just add an extra backslash to WMI.

            The problem: Windows storage volumes are mislabeled with an extra backslash, and no longer update after enabling WMI.


            • force rediscover - This check only occurs when parsing fixed local disks on Windows systems via WMI, and will always rediscover unless the mismatch between SNMP and WMI is corrected.
            • mib_name check - I was unaware if an earlier Microsoft or third party SNMP agent exposed ucd or another mib, and did that just in case.
            • net-snmp escapes output - All SNMP responses suffer an additional level of escaping, you can easily watch backslashes double in fields like sysContact.

            Full output from running discovery -d -m storage, poller -d -m storage, and poller -d -m wmi is attached, before and after changing stripslashes, along with output from the storage table and rrd filenames.


            In the case of hrStorageDescr, the escaping backslash from SNMP is escaped again along with the intended backslash, then passed into the database storage_descr field, so both the web interface incorrectly displays these volumes as

            C:\\, F:\\, and I:\\
            

            and the rrd filenames updated by the SNMP storage module contain an extra underscore to match. However, the first WMI storage pass disables the SNMP storage module and those files are no longer updated, as WMI updates the shorter rrd filenames as C:\, F:\, and I:\.

            Notable debug output from a storage discovery, storage poll, and wmi poll:

            HOST-RESOURCES-MIB 31 -> 1, hrStorageFixedDisk, HOST-RESOURCES-MIB, C:\\ Label:  Serial Number 46877b75, 4096, 498630389760, 276420460544, 0
            SQL[UPDATE `storage` set `storage_descr` ='C:\\\\ Label:  Serial Number 46877b75' WHERE `storage_id` = '1']
            RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C____Label___Serial_Number_46877b75.rrd N:276420460544:222209929216]
            RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276420460544:222209929216]
            

            Only the first rrd is referenced in the web interface, and only the second is updated on subsequent polls.

            Stripping the extra backslash from SNMP at the start corrects these problems, so the web interface shows Windows volumes without the extra backslash, as C:\, F:\, and I:\, and both the storage and wmi poller modules use identical rrd filenames, preventing a loss of data when changing the polling method.

            Debug output from the same commands after stripslashes():

            HOST-RESOURCES-MIB 31 -> 1, hrStorageFixedDisk, HOST-RESOURCES-MIB, C:\ Label:  Serial Number 46877b75, 4096, 498630389760, 276341080064, 0
            SQL[UPDATE `storage` set `storage_descr` ='C:\\ Label:  Serial Number 46877b75' WHERE `storage_id` = '1']
            RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276341080064:222289309696]
            RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276341080064:222289309696]
            

            jpmasseo Jonathan Masseo added a comment - The problem: Windows storage volumes are mislabeled with an extra backslash, and no longer update after enabling WMI. force rediscover - This check only occurs when parsing fixed local disks on Windows systems via WMI, and will always rediscover unless the mismatch between SNMP and WMI is corrected. mib_name check - I was unaware if an earlier Microsoft or third party SNMP agent exposed ucd or another mib, and did that just in case. net-snmp escapes output - All SNMP responses suffer an additional level of escaping, you can easily watch backslashes double in fields like sysContact. Full output from running discovery -d -m storage, poller -d -m storage, and poller -d -m wmi is attached, before and after changing stripslashes, along with output from the storage table and rrd filenames. In the case of hrStorageDescr, the escaping backslash from SNMP is escaped again along with the intended backslash, then passed into the database storage_descr field, so both the web interface incorrectly displays these volumes as C:\\, F:\\, and I:\\ and the rrd filenames updated by the SNMP storage module contain an extra underscore to match. However, the first WMI storage pass disables the SNMP storage module and those files are no longer updated, as WMI updates the shorter rrd filenames as C:\, F:\, and I:\. Notable debug output from a storage discovery, storage poll, and wmi poll: HOST-RESOURCES-MIB 31 -> 1, hrStorageFixedDisk, HOST-RESOURCES-MIB, C:\\ Label: Serial Number 46877b75, 4096, 498630389760, 276420460544, 0 SQL[UPDATE `storage` set `storage_descr` ='C:\\\\ Label: Serial Number 46877b75' WHERE `storage_id` = '1'] RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C____Label___Serial_Number_46877b75.rrd N:276420460544:222209929216] RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276420460544:222209929216] Only the first rrd is referenced in the web interface, and only the second is updated on subsequent polls. Stripping the extra backslash from SNMP at the start corrects these problems, so the web interface shows Windows volumes without the extra backslash, as C:\, F:\, and I:\, and both the storage and wmi poller modules use identical rrd filenames, preventing a loss of data when changing the polling method. Debug output from the same commands after stripslashes(): HOST-RESOURCES-MIB 31 -> 1, hrStorageFixedDisk, HOST-RESOURCES-MIB, C:\ Label: Serial Number 46877b75, 4096, 498630389760, 276341080064, 0 SQL[UPDATE `storage` set `storage_descr` ='C:\\ Label: Serial Number 46877b75' WHERE `storage_id` = '1'] RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276341080064:222289309696] RRD CMD[update /opt/observium/rrd/hyperv0/storage-host-resources-mib-C___Label___Serial_Number_46877b75.rrd N:276341080064:222289309696]

            rrd filename fixed in r6542.

            • force rediscover device from poller - will never be added, because can produce on some devices "rediscover every 5 min";
            • not understand why you need to get mib_name from db, windows always used 'host-resources-mib';
            • did not understand what the problem with slashes. Pls attach debug output for poller:

              ./poller.php -d -m storage -h <your_device>
              

            landy Mike Stupalov added a comment - rrd filename fixed in r6542. force rediscover device from poller - will never be added, because can produce on some devices "rediscover every 5 min"; not understand why you need to get mib_name from db, windows always used 'host-resources-mib'; did not understand what the problem with slashes. Pls attach debug output for poller: ./poller.php -d -m storage -h <your_device>

            No comments in the code.

            adama Adam Armstrong added a comment - No comments in the code.

            People

              landy Mike Stupalov
              jpmasseo Jonathan Masseo
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: