Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-2429

Allow poller-wrapper.py to exclude polling on device(s) based on CIDR parameters

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • None
    • Professional Edition
    • Poller

    Description

      Example: 

      poller-wrapper.py -exclude 10.1.0.0/24, 10.1.1.4/29

      This would make it possible for multiple pollers to be used in environments where each data center will have its own poller but share the same shared mysql server/rrdtool store.

      Attachments

        Activity

          [OBS-2429] Allow poller-wrapper.py to exclude polling on device(s) based on CIDR parameters

          Feature added in r9328.

          1. you can create device group with limit devices by IP/Network.
          2. than you can limit poller-wrapper by include/exclude specific group, ie:

            ./observium-wrapper -g 2 3 4 poller
            

            (multiple groups allowed)

          landy Mike Stupalov added a comment - Feature added in r9328. you can create device group with limit devices by IP/Network. than you can limit poller-wrapper by include/exclude specific group, ie: ./observium-wrapper -g 2 3 4 poller (multiple groups allowed)

          This will do based on include/exclude groups.

          landy Mike Stupalov added a comment - This will do based on include/exclude groups.

          Mike/Landy is currently working on implementing this featureset into the base poller wrapper. Be careful when updating, since this patch will likely break soon.

          adama Adam Armstrong added a comment - Mike/Landy is currently working on implementing this featureset into the base poller wrapper. Be careful when updating, since this patch will likely break soon.
          Tini Tini added a comment - - edited

          This feature was something we needed very, very, very, very much.

          We manages variety of servers and they are divided into completely different countries or regions.
          Therefore scale-up of the poller server was no longer able to handle issues such as server load and network delay, and it needed a way to separate poller.

          We could get a good idea about how to separate poller at the below URL.
          (http://blog.best-practice.se/2017/10/scaling-observium-horizontally.html)
          But what the URL introduced was how to divide the poller instance.
          We couldn't meet all of our requirements.
          Because we wanted to separate poller into different regions.

          Of course, it should be considered carefully to run a remote poller as Adam Armstrong suggests.
          it's because all the network environment isn't good.
          However, we think that we can prepare for loss of data by arrange the relay server in the middle.

          Anyway, we were able to solve the problem completely with this patch.
          We finally got the results that what we wanted.
          I'm very grateful to Fadly Tabrani for providing this patch.

          Tini Tini added a comment - - edited This feature was something we needed very, very, very, very much. We manages variety of servers and they are divided into completely different countries or regions. Therefore scale-up of the poller server was no longer able to handle issues such as server load and network delay, and it needed a way to separate poller. We could get a good idea about how to separate poller at the below URL. ( http://blog.best-practice.se/2017/10/scaling-observium-horizontally.html ) But what the URL introduced was how to divide the poller instance. We couldn't meet all of our requirements. Because we wanted to separate poller into different regions. Of course, it should be considered carefully to run a remote poller as Adam Armstrong suggests. it's because all the network environment isn't good. However, we think that we can prepare for loss of data by arrange the relay server in the middle. Anyway, we were able to solve the problem completely with this patch. We finally got the results that what we wanted. I'm very grateful to Fadly Tabrani for providing this patch.
          berserk2012 Taehee added a comment -

          This is amazing! I really needed this feature. Now I can poll by location, os, and even syscontact. Thanks a lot!

          berserk2012 Taehee added a comment - This is amazing! I really needed this feature. Now I can poll by location, os, and even syscontact. Thanks a lot!

          We would only accept such a feature if it was well thought out and useful to more than one person. It's structural suicide to accept every random idea.

          You don't seem to have added the ability to exclude a location, which would seem to be important.

          In any event. I think it's more useful to be able to do this sort of thing in the Web UI. We've previously discussed ways of doing this, but we've not really had any instances where it's been necessary.

          In general one should think very carefully about having remote pollers, since SNMP is designed to travel long distances, but MySQL and storage protocols are not.

          adama Adam Armstrong added a comment - We would only accept such a feature if it was well thought out and useful to more than one person. It's structural suicide to accept every random idea. You don't seem to have added the ability to exclude a location, which would seem to be important. In any event. I think it's more useful to be able to do this sort of thing in the Web UI. We've previously discussed ways of doing this, but we've not really had any instances where it's been necessary. In general one should think very carefully about having remote pollers, since SNMP is designed to travel long distances, but MySQL and storage protocols are not.

          You shouldn't be using IPs in Observium. Thus this whole concept makes little sense.

          adama Adam Armstrong added a comment - You shouldn't be using IPs in Observium. Thus this whole concept makes little sense.

          A simple patch to make the wrapper poll devices based on the locations,  already listed in the database.[^patch.diff]. Rusty coder alert, definitely needs review.

          fadly.tabrani@gmail.com Fadly Tabrani added a comment - A simple patch to make the wrapper poll devices based on the locations,  already listed in the database. [^patch.diff] . Rusty coder alert, definitely needs review.

          Just a thought, since location info is already stored in the device table, could it be used to limit polled devices instead since using CIDR as that would need additional IP resolution. 

          For example 

          poller-wrapper.py -locaton dc1 

          poller-wrapper.py -location dc3 dc4

          fadly.tabrani@gmail.com Fadly Tabrani added a comment - Just a thought, since location info is already stored in the device table, could it be used to limit polled devices instead since using CIDR as that would need additional IP resolution.  For example  poller-wrapper.py -locaton dc1  poller-wrapper.py -location dc3 dc4

          I believe the functionality to filter devices to be polled is already present based on host names and the reason in code explains it well. 

          poller-wrapper.py

          "Additionally, if a hostname wildcard is passed, add it to the where clause. This is
          important in cases where you have pollers distributed geographically and want to limit
          pollers to polling hosts matching their geographic naming scheme."
           
          It would be nice to extend this feature to based on CIDRs as this is normally how the network is segmented.
           
          For example, a poller servicing a location with the 10.1.0.0/16 subnet and another servicing the 10.2.0.0/16 subnet.
           

          fadly.tabrani@gmail.com Fadly Tabrani added a comment - I believe the functionality to filter devices to be polled is already present based on host names and the reason in code explains it well.  poller-wrapper.py "Additionally, if a hostname wildcard is passed, add it to the where clause. This is important in cases where you have pollers distributed geographically and want to limit pollers to polling hosts matching their geographic naming scheme."   It would be nice to extend this feature to based on CIDRs as this is normally how the network is segmented.   For example, a poller servicing a location with the 10.1.0.0/16 subnet and another servicing the 10.2.0.0/16 subnet.  

          People

            landy Mike Stupalov
            fadly.tabrani@gmail.com Fadly Tabrani
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: