Currently all alert would be received in a single channel. Slack has (un)fortunately become the centralised place for all information. We currently got channels for notifications (low impact, looked after by junior staff) and alerts (high impact, managers like to keep an eye out in here).
Based on the channel, slack would alert + send appropriate push notifications to different people.
+1 for this. ie: Grafana
You can choose various notification “channels”, and for each you can select the transport, and various specifics.
If the transport.slack.php could also grab Alert Templates pre-assigned to each alert rule, this could possibly accomplish this? Then the template would just need to have “@mentions” and preferably the “channel” call?
+1 for this and more fine grained control over what slack channels are notified
Assuming you had that functionality and you set a channel for critical and a channel for warning messages, where would you want the ok status messages to appear when the service returns to normal operation?
Please note that the Transport will have no memory on where the previous messages were sent too.
I assume a recovery response for a critical issue would be sent to the same “critical transport”.
-Jeff
Unfortunately as I alluded in my previous post the Notification system has no memory of shorts that would allow it to post the recovery response to the appropriate channel.
If Alert rule X exists, and its severity is set to “warning” and I have a slack transport called “slack_warning” and the alert rule X transport is set to use “slack_warning” for transport, then all the warning alerts for alert rule X are sent to that slack channel, as well as the recovery alerts.
You could also create a transport group and send both the alert and recovery notifications to multiple slack channels at once.
I feel like this functionality is already present in the current release of LibreNMS.
Based on the original post at the top.
-J
This would require that the alert sub system stores in the database which alert was sent to which channel, also requires multiple lookups on the DB for each alert, this would not be an issue on a small installation but it can and will create a lot of load on installations with a few hundred (or more) alerts per hour.
If the alert logic understands that it only sends an “Ok” alert if the previous status was something other than “Ok”, the previous status must be tracked somewhere. Could it not select transport based on that?