Alerts for network latency?

Tags: #<Tag:0x00007f84b0661de0>

I am trying to set up alerts for network latency, ie. when a network device has an unusually large ping time. A google search brings up this:

which suggests using: %device_perf.avg Larger than or Equals ‘100’

So I’ve currently got a rule that reads

device_perf.avg >= 150 AND macros.device = 1

which, as I understand it, should only get tripped if the device is online, and if its recent ping exceeds 150ms.

Enabling that rule, and immediately several devices get flagged. But… when I go to look at those same devices and check their “Latency” tab, the graphs DO NOT show high latency. And worse, on the /alerts page none of the devices ever recover from the alerts, despite that when I check them all, their ping times are all quite normal.

It seems that the “device_perf.avg” is the wrong thing to check? But google isn’t giving me much advice on what I could be using instead.

Perhaps look at the built in alert rule
devices.last_ping_timetaken > 100