IPMI chassis en power supply support

Mike Stupalov added a comment - 2020/04/29 02:13 PM As I discovered in many instances, discrete sensors really more hard for detect correct status. This is bitwise status and not possible for correctly set as status by simple compare. Your patch will work for your device(s) only and not correct for any (many) others. I.e. for my supermicro platform this patch is incorrect!.. Now I complete not want improve this discrete statuses, until there is a unified method for various platforms.

Bernard van der Velden added a comment - 2020/04/29 01:26 PM - edited This change introduces code that compares the values $current and $state which are in practice the same values. This results in checking a discrete sensor will always report as healthy:   ipmitool output compared to Supermicro sensor readings from ipmi frontend:   Server 1 Chassis Intru | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> not healthy, not failing in Observium since r10395 while it should PS1 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy PS2 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy I did some testing on server 2:   Server 2 - normal VBAT | 0x4 | discrete | 0x04ff| na | na | na | na | na | na -> healthy Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na -> healthy PS1 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy PS2 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy, power supply is inserted and power cable is connected, reports as healthy in Observium   Server 2 - removed PS2 VBAT | 0x4 | discrete | 0x04ff| na | na | na | na | na | na -> healthy Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na -> healthy PS1 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy PS2 Status | 0x0 | discrete | 0x0000| na | na | na | na | na | na -> I would like to mark this as a failure but this depends on your own setup, reports as healthy in Observium   Server 2 - removed power cord PS2 VBAT | 0x4 | discrete | 0x04ff| na | na | na | na | na | na -> healthy Chassis Intru | 0x0 | discrete | 0x0000| na | na | na | na | na | na -> healthy PS1 Status | 0x1 | discrete | 0x0100| na | na | na | na | na | na -> healthy PS2 Status | 0xb | discrete | 0x0b00| na | na | na | na | na | na -> not healthy, a power supply should have power, reports as healthy in Observium since r10395  It seems to be required that the state is compared to the hard coded values related to the sensor names. Observium now reports my Chassis intrusion sensor as healthy while it is not, and probably will do the same in the case of a failing power supply. This is a big issue for us because we need to be able to check if a power supply is failing om our servers. From what I can find the following states or communicated by ipmitool for discrete sensors: Power supply: 0x0=power supply unit not present 0x1=status ok 0x3=power supply off or failed 0xB=Input out of range (ex. No AC input)   Chassis Intrusion: 0x0=No Intrusion 0x1=Intrusion   VBAT: 0x4=Healthy Can this issue be reopened? Edit: I made a patch which detects failing sensors correctly again in our setup: ipmi_patch_rework.diff

Andrew Bonney added a comment - 2020/04/27 01:17 PM Sure. I've attached the poller output, along with the output from calling 'ipmitool sensor' and 'ipmitool -v sdr' directly. The latter file shows that (for example) the sensor named 'PS1 PG Fail' is of the 'Voltage' type, not 'Power Supply', so its assertions don't match what the existing entPhysicalClass 'powersupply' matching does. ipmitool-sdr-r730.txt

Mike Stupalov added a comment - 2020/04/27 11:50 AM IPMI sensors polled without changes.

Andrew Bonney added a comment - 2020/04/27 08:03 AM Thanks Mike. Would the sensor output still be useful?

Mike Stupalov added a comment - 2020/04/25 01:12 PM Ok, now should be fixed in r10395.

Mike Stupalov added a comment - 2020/04/25 11:44 AM andrewbonney Please also attach debug for your devices (with fail description and dell hardware), I need to see more ipmi output examples: ./poller.php -d -m unix-agent,sensors,status -h <device>

Andrew Bonney added a comment - 2020/04/24 01:59 PM - edited The attached patch removes the state inversions from the previous version. Looking at the IPMI spec and the live sensor output I have access to I'm not sure it will be possible to improve the various discrete sensors without calling some additional IPMI commands. It's only when you know the sensor type behind each description that you can identify what their assertions really mean. These may be a good thing (power supply is present) or a bad thing (chassis intrusion), but can be represented by the same value. ipmi_sensors.diff

Andrew Bonney added a comment - 2020/04/24 09:55 AM - edited We're having the same issue, although I believe there's an additional bug which presents itself if you're using ipmitool remotely rather than in conjunction with the unix-agent. If this is the case then even with the suggested patch the values don't get processed due to a lookup for "$config ['*ipmi*'] ['states'] ['*unix-agent-state'*] " in includes/definitions/entities/status.inc.php. The attached patch resolves this for us, although note that this includes two additional changes to value handling. We have some sensors described as 'PS1 PG Fail' for example, which need the power supply status flag inverting. It also appears that the sensors classed as 'other' are inverted (noted on a Dell R730), but I don't have enough data to say if this will be consistent for every IPMI implementation. Edit: After a further look I'd certainly exclude the state inversions. I'm looking into whether this can be done any more consistently across implementations. ipmi_sensors.diff

Bernard van der Velden added a comment - 2020/01/06 01:55 PM I attached the logs