If you have syslog messages from the device coming to LibreNMS you can, there are existing rule examples you can modify to look at a time window for particular things and alert on them:
For example:
Begin oversharing:
I use the above to flag login failures to network devices etc. If you don’t then have a notification going somewhere for them though, blink and you’ll miss them on the dashboard as they will go away after the search window (5 minutes above).
I tend to use the LibreNMS dashboard and visible alerts for ‘status’ views of things and don’t send much to notifications to reduce the noise, but when I do need a notification somewhere that is really important to me, I either:
- Have a specific transport used for specific rules I really care about, for example I have a ‘security’ transport for specific alert rules to a slack channel to separate from the general noise.
- Integrate Graylog to handle events from syslog and from LibreNMS logs themselves and then customise notifications/escalations, or;
- Use the External Hook feature to process critical log entries as they come in and do something - such as send to a slack channel - I use this for firewall threat detection events:
See: Syslog - external-hooks
As I construct that house of cards, I then run scripts to fire test alerts once a week to test my notification chains so I can trust any silence … something as simple as this in cron that I trigger notifications from in my alert rules/handling:
58 8 * * * librenms echo XVERYTHING IS OK | tr X E | logger