Overall CPU usage alerting - COMPLETE

Hello. Now LibreNMS can do alerting per-core CPU usage, but on highload servers, several CPU cores have a 99% usage, but overall CPU Usage is lower than 30%. I made a simple script for alerting based on overall CPU usage for Linux/Unix/Windows servers.

Take this script to cron on LibreNMS Server

#!/bin/bash

#MySQL Login/Pass/DB

USER=“root”
PASS=“123”
DBS=“librenms”

#Take a Device IDs

ID=`mysql -u $USER -p$PASS -D $DBS -e “SELECT device_id FROM processors WHERE processor_type=‘hr’” |sort -u |grep -v device_id`

echo $ID > /tmp/cpuid.txt

for X in `cat /tmp/cpuid.txt`
do

#Calculate AVG CPU usage per device

for Y in `mysql -u $USER -p$PASS -D $DBS -e “SELECT round(AVG(processor_usage)) FROM processors WHERE processor_type=‘hr’ AND device_id=’$X’” |grep -v processor_usage`
do

#Get a hrDeviceIndex and then check exists or not hrDeviceIndex 50000

DVI=`mysql -u $USER -p$PASS -D $DBS -e “SELECT hrDeviceIndex FROM hrDevice WHERE hrDeviceIndex=‘50000’ AND device_id=’$X’” |grep -v hrDeviceIndex`

#INSERT OR UPDATE Overall CPU usage per device in DB

if [ “$DVI” != “50000” ]; then

#Get a CPU Description from DB

HDSC=`mysql -u $USER -p$PASS -D $DBS -e “SELECT hrDeviceDescr FROM hrDevice WHERE hrDeviceType=‘hrDeviceProcessor’ AND device_id=’$X’” |sort -u |grep -v hrDeviceDescr`

mysql -u $USER -p$PASS -D $DBS -e “INSERT INTO hrDevice (device_id,hrDeviceIndex,hrDeviceDescr,hrDeviceType,hrDeviceErrors,hrDeviceStatus,hrProcessorLoad) VALUES(’$X’,‘50000’,’$HDSC’,‘hrDeviceProcessor’,‘0’,‘running’,’$Y’)”

elif [ “$DVI” = “50000” ]; then

mysql -u $USER -p$PASS -D $DBS -e “UPDATE hrDevice SET hrProcessorLoad=’$Y’ WHERE device_id=’$X’ AND hrDeviceIndex=‘50000’”

fi

done

done

#END

After that go to Rules page in NMS and make a simple alert Rule with conditions:

%hrDevice.hrProcessorLoad > “75” && %hrDevice.hrDeviceIndex = “50000”

P.S. Better if developers of LibreNMS can do it in php code, and not use my script.

3 Likes

Hi,

I’m having error just by manually running your command.

./overall_cpu.sh: line 35: syntax error near unexpected token (' ./overall_cpu.sh: line 35:mysql -u $USER -p$PASS -D $DBS -e “INSERT INTO hrDevice (device_id,hrDeviceIndex,hrDeviceDescr,hrDeviceType,hrDeviceErrors,hrDeviceStatus,hrProcessorLoad) VALUES(’$X’,‘50000’,’$HDSC’,‘hrDeviceProcessor’,‘0’,‘running’,’$Y’)”’

You can now put custom SQL queries into alert rules to achieve this. Please update and edit an existing rule -> Advanced. Docs have been updated as well. https://docs.librenms.org/#Alerting/Rules/#advanced