Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-3429

Make Ports API endpoint more memory efficient

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • Professional Edition
    • API
    • None

    Description

      We had an issue with always increasing memory demand of the ports endpoint on Observium, it was caused by loading the entire ports table into memory, modifying it in place, and then encoding the whole array as json, and only then finally sending it to the requester.

      Attached is code I made as a proof of concept. it is compatible with the current version with the only json difference being that the "count" key appears last, since json key order is not important that should be fine.

      The attached script selects the same ports, and then uses an iterator to fetch the result, encode and send it to the requester.

      I agree the attached script is a ... bit hacky, a proper script might include a streaming json encoder, but it's mostly a proof of concept, that works and is in production one instance.

      Attachments

        Activity

          People

            adama Adam Armstrong
            gunnarg Gunnar Guðvarðarson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated: