Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-3614

syslog messages appear truncated for Citrix ADC

Details

    • Bug
    • Resolution: Fixed
    • Minor
    • None
    • Professional Edition
    • Alerting
    • Centos 7

    Description

      We are using rsyslog to capture the logs in Observium.  The messages from our Citrix ADC virtual appliances seem to be truncated.

      Original message (tcpdump on port 514):

      10:49:36.461706 IP (tos 0x0, ttl 253, id 19565, offset 0, flags [none], proto UDP (17), length 228)
      10.250.128.20.32730 > 10.250.32.100.syslog: [udp sum ok] SYSLOG, length: 200
      Facility local0 (16), Severity notice (5)
      Msg: 01/19/2021:09:49:33 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEDOWN 35996673 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State DOWN\0x0a
      0x0000: 3c31 3333 3e20 3031 2f31 392f 3230 3231
      0x0010: 3a30 393a 3439 3a33 3320 474d 5420 4443
      0x0020: 5258 2d41 4e53 2d50 3030 3320 302d 5050
      0x0030: 452d 3020 3a20 6465 6661 756c 7420 4556
      0x0040: 454e 5420 4445 5649 4345 444f 574e 2033
      0x0050: 3539 3936 3637 3320 3020 3a20 2044 6576
      0x0060: 6963 6520 2273 6572 7665 725f 7365 7276
      0x0070: 6963 6547 726f 7570 5f4e 5353 5643 5f53
      0x0080: 534c 5f31 302e 3235 302e 3635 2e32 313a
      0x0090: 3836 3336 2853 5647 5f50 5244 5f48 5454
      0x00a0: 5053 5f44 533f 4443 5258 2d4c 444d 2d50
      0x00b0: 3030 323f 3836 3336 2922 202d 2053 7461
      0x00c0: 7465 2044 4f57 4e0a

      Syslog debug output (enabled #$config['syslog']['debug'] = TRUE:

      [2021/01/19 10:48:22 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:48:22|| 01/19/2021:09:48:19 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEDOWN 35993973 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State DOWN||
      [2021/01/19 10:48:59 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:48:59|| 01/19/2021:09:48:56 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEUP 35995288 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State UP||
      [2021/01/19 10:49:36 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:49:36|| 01/19/2021:09:49:33 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEDOWN 35996673 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State DOWN||
      [2021/01/19 10:50:08 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:50:08|| 01/19/2021:09:50:06 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEUP 35997958 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State UP||
      [2021/01/19 10:50:24 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:50:24|| 01/19/2021:09:50:22 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEDOWN 35998539 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State DOWN||
      [2021/01/19 10:50:57 +0100] syslog.php(84224): dcrx-ans-p003||16||5||5||||2021-01-19 10:50:57|| 01/19/2021:09:50:54 GMT DCRX-ANS-P003 0-PPE-0 : default EVENT DEVICEUP 35999829 0 : Device "server_serviceGroup_NSSVC_SSL_10.250.65.21:8636(SVG_PRD_HTTPS_DS?DCRX-LDM-P002?8636)" - State UP||

       

      The output in the Observium web interface:

       

      Attachments

        Issue Links

          Activity

            [OBS-3614] syslog messages appear truncated for Citrix ADC

            Fixed in r10995.

             

            For rsyslog need restart service after update.

            landy Mike Stupalov added a comment - Fixed in r10995.   For rsyslog need restart service after update.

            Hrm, but your messages stored differently

             

            landy Mike Stupalov added a comment - Hrm, but your messages stored differently  

            Just tested your debug syslog entries:

            I only can said - observium store full message which received from rsyslog :/

            landy Mike Stupalov added a comment - Just tested your debug syslog entries: I only can said - observium store full message which received from rsyslog :/
            mdwnn mdwnn added a comment -

            I added the debug output. I assume this is what you needed?

            mdwnn mdwnn added a comment - I added the debug output. I assume this is what you needed?
            mdwnn mdwnn added a comment -

            This is the debug output:

            [2021/01/26 13:47:08 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:07|| 01/26/2021:12:47:07 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431870 0 :  Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_TEST_LDAPS_ADMB?DC-BRU-150?636)" - State DOWN||
            [2021/01/26 13:47:08 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:08|| 01/26/2021:12:47:08 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431875 0 :  Device "server_serviceGroup_NSSVC_TCP_10.250.64.12:389(SVG_TEST_LDAP_LIANTIS?DCRX-WDC-P002?389)" - State DOWN||
            [2021/01/26 13:47:09 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:09|| 01/26/2021:12:47:09 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431923 0 :  Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_DEV_LDAPS_ADMB?DC-BRU-150?636)" - State DOWN||
            [2021/01/26 13:47:38 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:38|| 01/26/2021:12:47:38 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432797 0 :  Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_TEST_LDAPS_ADMB?DC-BRU-150?636)" - State UP||
            [2021/01/26 13:47:38 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:38|| 01/26/2021:12:47:38 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432807 0 :  Device "server_serviceGroup_NSSVC_TCP_10.250.64.12:389(SVG_TEST_LDAP_LIANTIS?DCRX-WDC-P002?389)" - State UP||
            [2021/01/26 13:47:40 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:40|| 01/26/2021:12:47:40 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432846 0 :  Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_DEV_LDAPS_ADMB?DC-BRU-150?636)" - State UP||
            

            mdwnn mdwnn added a comment - This is the debug output: [2021/01/26 13:47:08 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:07|| 01/26/2021:12:47:07 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431870 0 : Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_TEST_LDAPS_ADMB?DC-BRU-150?636)" - State DOWN|| [2021/01/26 13:47:08 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:08|| 01/26/2021:12:47:08 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431875 0 : Device "server_serviceGroup_NSSVC_TCP_10.250.64.12:389(SVG_TEST_LDAP_LIANTIS?DCRX-WDC-P002?389)" - State DOWN|| [2021/01/26 13:47:09 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:09|| 01/26/2021:12:47:09 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEDOWN 62431923 0 : Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_DEV_LDAPS_ADMB?DC-BRU-150?636)" - State DOWN|| [2021/01/26 13:47:38 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:38|| 01/26/2021:12:47:38 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432797 0 : Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_TEST_LDAPS_ADMB?DC-BRU-150?636)" - State UP|| [2021/01/26 13:47:38 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:38|| 01/26/2021:12:47:38 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432807 0 : Device "server_serviceGroup_NSSVC_TCP_10.250.64.12:389(SVG_TEST_LDAP_LIANTIS?DCRX-WDC-P002?389)" - State UP|| [2021/01/26 13:47:40 +0100] syslog.php(99197): dcrx-ans-n004||16||5||5||||2021-01-26 13:47:40|| 01/26/2021:12:47:40 GMT DCRX-ANS-N004 0-PPE-0 : default EVENT DEVICEUP 62432846 0 : Device "server_serviceGroup_NSSVC_TCP_172.16.200.150:636(SVG_DEV_LDAPS_ADMB?DC-BRU-150?636)" - State UP||
            mdwnn mdwnn added a comment -

            I restarted it (both restart and stop/start). I enabled debugging and I will update the case when I have some messages.

            mdwnn mdwnn added a comment - I restarted it (both restart and stop/start). I enabled debugging and I will update the case when I have some messages.

            did you restart rsyslog service?

            Show lines in Syslog debug output.

            landy Mike Stupalov added a comment - did you restart rsyslog service? Show lines in Syslog debug output.

            People

              landy Mike Stupalov
              mdwnn mdwnn
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: