Details

    • Help
    • Resolution: Fixed
    • Critical
    • None
    • None
    • Default
    • None

    Description

      Keeos getting out of memory alert which is stopping the mariadb service 

      Attachments

        Activity

          [OBS-4828] Out of Memory

          Hi Adam, we've reduced it to 1M and truncated the logs. RAM usage has returned to normal. Thanks for all your help.

          alyssa.garcia Alyssa Mae Garcia added a comment - Hi Adam, we've reduced it to 1M and truncated the logs. RAM usage has returned to normal. Thanks for all your help.

          You should reduce the retention time for eventlog, it's by default 6 months, but honestly most of the data in there is not going to be useful to you at all, nevermind 6 months later. You can set it quite a bit smaller to reduce some query times.

          For vlans_fdb, you have a very very large number of entries. It's possible your particular network isn't suitable to have this module enabled (a lot of very large layer 2 devices with visibility of a very large number of stations on a large number of vlans, for example a university campus).

          In this instance you should disable the fdb module and truncate the table.

          For eventlog, you can truncate the table now too to reduce its size quickly.

          To recover the space used by mariadb (it won't free the space itself), you need to delete the tables and recreate them.

          adama Adam Armstrong added a comment - You should reduce the retention time for eventlog, it's by default 6 months, but honestly most of the data in there is not going to be useful to you at all, nevermind 6 months later. You can set it quite a bit smaller to reduce some query times. For vlans_fdb, you have a very very large number of entries. It's possible your particular network isn't suitable to have this module enabled (a lot of very large layer 2 devices with visibility of a very large number of stations on a large number of vlans, for example a university campus). In this instance you should disable the fdb module and truncate the table. For eventlog, you can truncate the table now too to reduce its size quickly. To recover the space used by mariadb (it won't free the space itself), you need to delete the tables and recreate them.

          How do we keep down the size of the vlans_fdb and eventlog tables? Currently, vlans_fdb has 40 million rows in it and 33 million rows in the eventlog table

          alyssa.garcia Alyssa Mae Garcia added a comment - How do we keep down the size of the vlans_fdb and eventlog tables? Currently, vlans_fdb has 40 million rows in it and 33 million rows in the eventlog table

          Hi Mike, 

          • Currently have 112 up, 77 down and 2 ignored devices total of 191
          • For syslog collection
          • Housekeeping (pls see attached screenshot)
          •  
          • Another thing
          alyssa.garcia Alyssa Mae Garcia added a comment - Hi Mike,  Currently have 112 up, 77 down and 2 ignored devices total of 191 For syslog collection Housekeeping (pls see attached screenshot)   Another thing

          14.5 Gb for mariadb process, so much.

          landy Mike Stupalov added a comment - 14.5 Gb for mariadb process, so much.

          Probably this is mariadb service trouble.

          • How many devices?
          • Do you use syslog collection?
          • Do you have configured housekeeping cron job?
          landy Mike Stupalov added a comment - Probably this is mariadb service trouble. How many devices? Do you use syslog collection? Do you have configured housekeeping cron job?

          We don't have recommendations, the amount of memory required is specific to your installation.

          You should check the size of the tables in your database, and if you're properly running housekeeping to make sure the database doesn't get too large.

          adama Adam Armstrong added a comment - We don't have recommendations, the amount of memory required is specific to your installation. You should check the size of the tables in your database, and if you're properly running housekeeping to make sure the database doesn't get too large.

          Hi Adam, it was 16GB before I opened a ticket and increased it to 24GB, but it still runs out of memory. How much memory do you recommend for the Observium server?

          alyssa.garcia Alyssa Mae Garcia added a comment - Hi Adam, it was 16GB before I opened a ticket and increased it to 24GB, but it still runs out of memory. How much memory do you recommend for the Observium server?

          Hi Adam, it was 16GB before I opened a ticket and increased it to 24GB, but it still runs out of memory. How much memory do you recommend for the Observium server?

          alyssa.garcia Alyssa Mae Garcia added a comment - Hi Adam, it was 16GB before I opened a ticket and increased it to 24GB, but it still runs out of memory. How much memory do you recommend for the Observium server?

          Your server/vm probably needs more memory.

          adama Adam Armstrong added a comment - Your server/vm probably needs more memory.

          People

            adama Adam Armstrong
            alyssa.garcia Alyssa Mae Garcia
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: