Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-4057

95th percentile calculation on traffic accounting

Details

    • Help
    • Resolution: Unresolved
    • Major
    • None
    • Enterprise Edition
    • Web Interface
    • None
    • Latest Observium monitoring a Juniper SRX550

    Description

      I've attached a screenshot of the traffic accounting page for one of our customers.  It's calculated off the interface which connects that customer to the WAN/Internet.

      I just wanted to verify that these numbers look correct.  I don't see a way to export data from the accounting page.  But I can export data from the regular graph from the device page.

      I'm not quite sure how to interpret the data I'm getting.  I would like to sort this based on the Mbps at each data point collected (I believe polled every minute for accounting) and try to do the 95th percentile in Excel just to confirm.

      Any help is much appreciated.  I'm also attaching the CSV in case anyone can explain the headers in that file.

      Attachments

        Activity

          [OBS-4057] 95th percentile calculation on traffic accounting

          One last question just came up.  Would there be an easy way to have an alert in the billing section when the customer goes over X% of their commitment to send an alert? 

          jason@unitedlayer.com Jason Berenson added a comment - One last question just came up.  Would there be an easy way to have an alert in the billing section when the customer goes over X% of their commitment to send an alert? 

          Adam,

          The percentile graph looks awesome!  Thank you for that.  I also verified manually and everything looks accurate.  Much appreciated!

          Jason.

          jason@unitedlayer.com Jason Berenson added a comment - Adam, The percentile graph looks awesome!  Thank you for that.  I also verified manually and everything looks accurate.  Much appreciated! Jason.

          Hi Jason,

          This should now be visible in the UI. It doesn't have the cross lines (i'm not entirely sure how to do that with the graphing library used for that graph), but it's easy enough to zoom in to the period and find the closest datapoint to 95th.

          adam.

          adama Adam Armstrong added a comment - Hi Jason, This should now be visible in the UI. It doesn't have the cross lines (i'm not entirely sure how to do that with the graphing library used for that graph), but it's easy enough to zoom in to the period and find the closest datapoint to 95th. adam.

          I'll see if i can work out how to generate a graph like that. I think it should be simple.

          adama Adam Armstrong added a comment - I'll see if i can work out how to generate a graph like that. I think it should be simple.

          Thanks Adam.  I'll give this a try.  The customer is questioning the bandwidth usage which is why I would like to get the data points myself so I can draw the 95th in Excel and prove it out.

          A graph similar to the new screenshot I just attached showing the sorted samples and the 95th line to the right would be useful to prove it I think.

          jason@unitedlayer.com Jason Berenson added a comment - Thanks Adam.  I'll give this a try.  The customer is questioning the bandwidth usage which is why I would like to get the data points myself so I can draw the 95th in Excel and prove it out. A graph similar to the new screenshot I just attached showing the sorted samples and the 95th line to the right would be useful to prove it I think.

          Hi James,

          Don't use anything from any "rrd" based stuff for billing. It's averaged data and not reliable. The 5 minute values are stored in the mysql database.

          It looks like we might not currently have a clean way to export this data, though you can pull it from the same internal API that the javascript graphing code uses : 

          /billing-data.php?bill_id=27&bill_code=&from=1646092800&to=1648771199

          bill_id is obvious, from and to are unixtime start and end times.

          This file has the format of Date, Input, Output. In & out are bps. Date is in RFC format and should be readable by most languages.

          It looks like this customer has had just enough spikes (backups i guess) that have pushed their 95th way above their actual usage.

          I wonder if a graph that displays the measurements in size-order and then marks the 95th threshold would be useful.

          adam.

          adama Adam Armstrong added a comment - Hi James, Don't use anything from any "rrd" based stuff for billing. It's averaged data and not reliable. The 5 minute values are stored in the mysql database. It looks like we might not currently have a clean way to export this data, though you can pull it from the same internal API that the javascript graphing code uses :  /billing-data.php?bill_id=27&bill_code=&from=1646092800&to=1648771199 bill_id is obvious, from and to are unixtime start and end times. This file has the format of Date, Input, Output. In & out are bps. Date is in RFC format and should be readable by most languages. It looks like this customer has had just enough spikes (backups i guess) that have pushed their 95th way above their actual usage. I wonder if a graph that displays the measurements in size-order and then marks the 95th threshold would be useful. adam.

          Does anyone have any ideas on how to convert the CSV data to usable data?  I need to calculate the 95th percentile graph manually to make sure this is being done correctly.  But the data in the CSV is useless.

          jason@unitedlayer.com Jason Berenson added a comment - Does anyone have any ideas on how to convert the CSV data to usable data?  I need to calculate the 95th percentile graph manually to make sure this is being done correctly.  But the data in the CSV is useless.

          General questions and device support can be discussed in our Discord channel, click here to join.


          Please make and attach additional information about the device:

          • full snmp dump from device:

            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk
            snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk

            If device not support SNMP version 2c, replace -v2c with -v1.

          • If you have problems with discovery or poller processes, please do and attach these debugs:

            ./discovery.php -d -h <device>
            ./poller.php -d -h <device>

          • additionally attach device and/or vendor specific MIB files

          This comment is added automatically.

          bot Observium Bot added a comment - General questions and device support can be discussed in our Discord channel, click here to join . Please make and attach additional information about the device: full snmp dump from device: snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1 > myagent.snmpwalk snmpwalk -v2c -c <community> -t 3 -Cc --hexOutputLength=0 -ObentxU <hostname> .1.3.6.1.4.1 >> myagent.snmpwalk If device not support SNMP version 2c, replace -v2c with -v1. If you have problems with discovery or poller processes, please do and attach these debugs: ./discovery.php -d -h <device> ./poller.php -d -h <device> additionally attach device and/or vendor specific MIB files This comment is added automatically.

          People

            adama Adam Armstrong
            jason@unitedlayer.com Jason Berenson
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: