Centralising Alerts from multiple sites

I currently have the opposite, but share a similar need. Some random thoughts/ideas …

I have a central server polling various sites remotely over lossy/latent links, works for the most part, but haven’t yet pushed to install local instances - when I do, I’m then the same as @Wayne_Kilner

For the alerting, after a lot of tuning and notification work we send critical things to Slack (teams just can’t keep up on mobile), but there is a lot of flapping noise and related non-notified alerts if you were to watch the UI all day.

I also have a disturbingly large number of weathermaps we leverage that I’d want to centralise if I pushed servers out to site - they are for a mix of audiences, local site people, and central operations. I could generate and ship them around via various methods and display centrally I suppose, but it would likely be done separate to any central LibreNMS instance dashboard/UI.

I’ve used Uptime Kuma at various sites to provide a local basic status dashboard with 10,30,60 etc. second ping or API polling update for site people to stare at - it can also notify in a large number of ways out of the box, relay to webhooks etc. worth a look.

In a former role with better connected sites, I’ve used a centralised graylog server to accept logs from devices, and the LibreNMS system logs themselves to parse out events/alerts and tune/escalate etc. The catch scaling this now is my log shipping from remote sites would be unreliable and drop events, so they would need to be handled locally if upstream site comms are down. Locally, any form of non-visual UI alert (email, chat etc.) would need upstream comms to work anyway - so the best we get is the site looks down from a central perspective, and site can keep rolling with the UI/dashboard etc.

Some of my alert musings from a while ago are here: BPDU Spanning Tree alerts - #2 by rhinoau