Slack Transport no longer working

Help
1

Our slack transport stopped providing any events sometime on August 4th, our custom alerts as well as default alerts aren’t populating into the slack hooks we have configured.

Also seeing sql errors in the librenms.log:

[2020-08-13 16:52:41] production.ERROR: SQLSTATE[42000]: Syntax error or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ‘~ “^cisco.*State$” && sensors.sensor_alert = “1”) = 1’ at line 1 (SQL: SELECT * FROM devices,sensors WHERE (devices.device_id = 93 AND devices.device_id = sensors.device_id) AND (sensors.sensor_current != “1” && sensors.sensor_current != “5” && sensors.sensor_type ~ “^cisco.*State$” && sensors.sensor_alert = “1”) = 1) (SQL: SELECT * FROM devices,sensors WHERE (devices.device_id = 93 AND devices.device_id = sensors.device_id) AND (sensors.sensor_current != “1” && sensors.sensor_current != “5” && sensors.sensor_type ~ “^cisco.*State$” && sensors.sensor_alert = “1”) = 1)#0 /opt/librenms/LibreNMS/Alert/AlertRules.php(75): dbFetchRows()

We pass a ./vaidate.php without any warnings.

Reviewing the logs, we’ve apparently been seeing these syntax errors as far back as at least 6-29.

Any recommendations on what to review next? Was there another alert syntax change in recent months?

One potential issue is we are still running librenms on debian 9, is that a known problem? We also have separated out the db to a dedicated server, which is running mariadb 10.3.24 on ubuntu 18.04.

1 Like
2

When I use the syntax being generated by the alert in mariadb I am getting back valid sql queries.

From the error, it lists the file:
/opt/librenms/LibreNMS/Alert/AlertRules.php

line 75, I also see on our installation that file was modified June 17th, unfortunately our logs don’t go back before this date to see if these syntax errors occurred prior to this modification.

3

I cleaned up some of logs and these sql errors seem unrelated.

I configured test rules that trigger and applied different transports (like email). Testing these transports work, applying these transports to simple rules that clearly trigger work, but whatever is responsible for firing these to the transports are not working, not with slack integration or email integration.

Is there any additional logging I can review, nothing relevant is appearing in the librenms logs directory.

any troubleshooting advice or links would be welcome.

4

update on our side, a reboot resolved the issue. Prior, we tried restarting services, but a full reboot was required.

Ideally some relevant error log would of populated, reboots fixing problems is better than not fixing the problem but is generally a last resort.