Uploaded image for project: 'Observium'
  1. Observium
  2. OBS-2631

poller-wrapper.py fails to log when log file is a fifo pipe in python 3

Details

    • Improvement
    • Resolution: Fixed
    • Minor
    • None
    • Professional Edition
    • Poller
    • CentOS 7, Python 3.4.5, PHP 7.2 from remirepo

    Description

      When the log file is a named fifo pipe or stdout (for injecting logs directly into journald) and the poller wrapper is running on Python 3, it fails with an "OSError: [Errno 29] Illegal seek" on the `f = open(log_path,'a')` line.

      This is a known but unsolved python 3 bug as can be seen here https://bugs.python.org/issue27805 and https://bugs.python.org/issue20074.

      The solution we used for now is to overwrite the open function to the open mode if appropriate, we did not want to modify the poller-wrapper directly to avoid issues with upgrades.

      `python3 poller-wrapper-wrapper.py poller-wrapper.py -w 10`

       

      poller-wrapper-wrapper.py is attached.

      Attachments

        Activity

          [OBS-2631] poller-wrapper.py fails to log when log file is a fifo pipe in python 3
          landy Mike Stupalov added a comment - - edited

          we already use python3 for years, but never seen your case

          landy Mike Stupalov added a comment - - edited we already use python3 for years, but never seen your case
          gunnarg Gunnar Guðvarðarson added a comment -

          Hey, I attached a patch as discussed ... months ago on IRC. It basically fixes the issue our poller wrapper mitigated.

          p.s. maybe a good idea to start switching to python3?

          gunnarg Gunnar Guðvarðarson added a comment - Hey, I attached a patch as discussed ... months ago on IRC. It basically fixes the issue our poller wrapper mitigated. p.s. maybe a good idea to start switching to python3?
          gunnarg Gunnar Guðvarðarson added a comment -

          Hey Mike, did you see my response, ticket has been in Status:PENDING RESPONSE for a while now...

          gunnarg Gunnar Guðvarðarson added a comment - Hey Mike, did you see my response, ticket has been in  Status: PENDING RESPONSE  for a while now...
          gunnarg Gunnar Guðvarðarson added a comment -
          1. remove current poller log file
          2. run `mkfifo <log file path>`
          3. run `cat <log file path>`
          4. (in a different window) run poller-wrapper using python 2, observe that it works
          5. (in a different window) run poller-wrapper using python 3, observe that it crashes with an "Illegal seek"
          gunnarg Gunnar Guðvarðarson added a comment - remove current poller log file run `mkfifo <log file path>` run `cat <log file path>` (in a different window) run poller-wrapper using python 2, observe that it works (in a different window) run poller-wrapper using python 3, observe that it crashes with an "Illegal seek"
          landy Mike Stupalov added a comment -

          I asked, show me real case when required this wrapper.
          Ie, write steps to reproduce the error..

          I use python3 and not have such troubles when wrapper started from cron.

          landy Mike Stupalov added a comment - I asked, show me real case when required this wrapper. Ie, write steps to reproduce the error.. I use python3 and not have such troubles when wrapper started from cron.
          gunnarg Gunnar Guðvarðarson added a comment - - edited

          The only reason we use the wrapper is because your wrapper is expecting the 'append' mode to work for the configured log files, it does on files but when the logs are set to a fifo pipe or char device like /dev/stdout it results in an invalid seek because python 3 opens file in write mode and then seeks to the end which special files do not allow, python 2 does not do that, and opens special files correctly.

          Optimally python 3 would be fixed, but it hasn't, so my suggestion is something similar to our stat and conditional mode flag for the open call.

          gunnarg Gunnar Guðvarðarson added a comment - - edited The only reason we use the wrapper is because your wrapper is expecting the 'append' mode to work for the configured log files, it does on  files but when the logs are set to a fifo pipe or char device like /dev/stdout it results in an invalid seek because python 3 opens file in write mode and then seeks to the end which special files do not allow, python 2 does not do that, and opens special files correctly. Optimally python 3 would be fixed, but it hasn't, so my suggestion is something similar to our stat and conditional mode flag for the open call.
          landy Mike Stupalov added a comment -

          Which real case for use this additional wrapper?

          In our normal usage (from cron) this issue not happened..

          landy Mike Stupalov added a comment - Which real case for use this additional wrapper? In our normal usage (from cron) this issue not happened..

          People

            landy Mike Stupalov
            gunnarg Gunnar Guðvarðarson
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: