Rules not triggering a transport alert

Tags: #<Tag:0x00007fdb873f1f78>

I am having issues where some of our Rules are being triggered, but not actually sending alerts to our transport (MSTeams)

We can manually run the alerts in CLI and it works. We get the alert on teams and everything. However if we let alerts generate naturally, it doesn’t ever send it to the transport. Only shows an active rule/alert on the dashboard.

That said our Devices up/down alerts correctly. The ones that seem to not are Sensor over or under limit, and any new rules we make.

Couple points to mention:

  • Version 1.50-41
  • PHP 7.2.17-1
  • Using MS Teams as the transport
  • All rules are set to use the transport
  • Testing the transport also works correctly.

Yeah i had this same issue. Change the Max from 1 to -1 and they should work fine. Or i think like 5 should be okay.

If you have a max set with ‘1’, then it will run once and only once. It will only run again after it has been cleared. So if you expect more than one notification for a single alert, then yes, you should change to a higher value, or to -1 for unlimited as @paintballer4lfe wrote .

Just a side note as to why i said set to -1 right out of the gate. I had this very same issue setting it to 1 as you did. Asked some of the guys in the Discord and the only working solution to send 1 alert, let alone many, was to set it to -1 or 5.

It appears the “run once” rule for my Telegram transport did not work unless it was set to multiple or unlimited.

We don’t even get it once is the issue. @PipoCanaja - Yes the alert shows in the dashboard but it never sends to transport.

Try test the rule with the debug flag on https://docs.librenms.org/Alerting/Testing/#transports

Did and it returns no errors. It sends to our teams channels when doing it manually via test-alert.php -r 1 -d -h 1 for example.

Just wont organically do it for some devices. We can’t figure out why.

Just for some more example of how it is properly triggering for some and not for others.

Here is a device that went down last night that worked.

Here is our test device that wouldn’t.

Is it just the test device that doesnt work? Does this work with Email transport?

can you create another alert rule:

with

Max: 1
Delay: 0
Interval: 300

assign to MSteams transport and test?

also can you check the UPS Bench device is not ignored or disabled in the device settings.
image

Also doing the same thing on other transports (Mail for example)



5 Min + has passed and no transports triggered.

To confirm,

your not getting alerts via email either?

the devices are not disabled or ignored?

“Device Up/Down” alert rule, is not mapped to specific devices or device groups ?

I find it strange that the UPS Bench was exactly 5 minutes from down/up looks like snmp down but for some reason is saying Down from icmp check.

your not getting alerts via email either?
Correct

the devices are not disabled or ignored?
Correct

“Device Up/Down” alert rule, is not mapped to specific devices or device groups ?
Correct

I can best describe it as if…everything is working except for the exact moment that a rule should be triggering sending to transport. Manually testing transport, alerts, rules, etc go through fine with no errors. But it appears when an rule is actually triggered it never even tries to send to transport, no matter our max/notification limits/settings.

2 Likes

Hi, facing the same issue. Did u manage to find the root cause?

Hello, I am facing exactly the same issue here. Tests are working fine, transport works when test button is pressed. but it doesn’t when triggered automatically. Were you able to solve this issue?

after checking another post, I have found the issue. When using MSTeams The transports templates must be changed to the MSteams template: https://docs.librenms.org/Alerting/Templates/

Problem solved!