Good afternoon,
I have sucesfully set up LibreNMS for all my devices and I love it so far.
The one thing I cannot resolve several days and I dont know how to proceed further is email alerts. I have sucesfully set up transport rule, default contact, test mails are going through OK no problem.
But when I get an alert, no email is sent out. However when I acknowledge an alert, I get an ACK email about the alert. So by this I think that emailing is set up correctly, I can see that the transport rule is set on all alerts because it is the default one.
I have tried bilion of settings combinations such as use only default contact and several other combinations, but the mail never arrives.
Ive tried to run test-template.php, but I dont know the alerts.rule_id and seems Iam dumb enough to get past this and get it.
Ive run validate.php in GUI and everything is OK, no errors. I have read several forum topics here and seems like that everybody resolved their issue with emailing in their original way and nothing worked for me (e.g. correct timezone,…).
Could you please point me out what to look for next?
Running latest version of LibreNMS, master update channel, mailing is set up via SMTP to my Microsoft 365 Exchange Online via connector (and this is known good config working fine - test email going through OK).
Thanks a lot
Can you run /opt/librenms/alerts.php -d ?
Hi Paulie,
thanks for getting touch with me. Sure, I have run it. Heres the output.
root@SRV-LibreNMS:/opt/librenms# ./alerts.php -d
DEBUG!
SQL[insert into `cache_locks` (`key`, `owner`, `expiration`) values (?, ?, ?) ["laravel_cachealerts","LdI9jEqtSlk4nXy0",1613313147] 1.43ms]
Start: Sun, 14 Feb 2021 14:31:27 +0000
ClearStaleAlerts():
SQL[SELECT `alerts`.`id` AS `alert_id`, `devices`.`hostname` AS `hostname` FROM `alerts` LEFT JOIN `devices` ON `alerts`.`device_id`=`devices`.`device_id` RIGHT JOIN `alert_rules` ON `alerts`.`rule_id`=`alert_rules`.`id` WHERE `alerts`.`state`!=0 AND `devices`.`hostname` IS NULL [] 0.26ms]
RunFollowUp():
SQL[SELECT alerts.id, alerts.alerted, alerts.device_id, alerts.rule_id, alerts.state, alerts.note, alerts.info FROM alerts WHERE alerts.state > 0 && alerts.open = 0 [] 0.14ms]
RunAlerts():
SQL[SELECT alerts.id, alerts.alerted, alerts.device_id, alerts.rule_id, alerts.state, alerts.note, alerts.info FROM alerts WHERE alerts.state != 2 && alerts.open = 1 [] 0.15ms]
RunAcks():
SQL[SELECT alerts.id, alerts.alerted, alerts.device_id, alerts.rule_id, alerts.state, alerts.note, alerts.info FROM alerts WHERE alerts.state = 2 && alerts.open = 1 [] 0.14ms]
End : Sun, 14 Feb 2021 14:31:27 +0000
SQL[select * from `cache_locks` where `key` = ? limit 1 ["laravel_cachealerts"] 0.25ms]
SQL[delete from `cache_locks` where `key` = ? and `owner` = ? ["laravel_cachealerts","LdI9jEqtSlk4nXy0"] 4.51ms]
root@SRV-LibreNMS:/opt/librenms#
I think this is related to the alarm and how is configured.
Can you paste a screenshot of your alert configuration ?
Alert is matching OK, I have tried several combinations of Max alerts, Delay and Interval. On the screenshot its setup as I want it to be - notify once after 5 minutes.
Transports arent filled, because I have set up my email transport as the default one (can be seen on second image).
Also pasting settings of my email settings, but those are working fine.
Thanks!
Hi @tukanos
Have you purposefully removed the device/device groups as there is nothing to match on the screenshot?
The Transport/Group is also empty on the screenshot.
Kind regards
Hi @Nathanael_Pearson,
I assume that when you dont specify anything in the devices box, it automatically matches this rule on all devices. I can see in the alerts menu that it says “All devices”.
Thanks!
I have tried many nonsense moves like inverting alert triggering, manually typing in trnasport rule, but nothing have helped. Still I got email only when I acknowledge alert, not when it gets actually triggered. 
I have set up Telegram notifications to test, if notifications work at all. Seems like the Telegram notifications does not work too when alert gets triggered.
Also I have found some people who had the same issue and they reported that they have changed the timezone to their own in PHP, I have set mine to Europe/Prague. They said that the database had UTC set, so the time was off and no alerts were send. Is there any way to correct the DB timezone so it matches the PHP one? Maybe this should help.
Are you using dispatch service or cron?
I had the same issue few months ago. I ended up enable the cron just for alerts as a workaround.
* * * * * librenms /opt/librenms/alerts.php >> /dev/null 2>&1
Hi Paulie, my concerns about timezone was right. I tried to disable ping to one of my devices on firewall and then I let the alert triggered until Ive got an email! BUT - alert got triggered at 9:45 (few minutes after I disabled it and the poller run). At 11:51 Ive got an email (2 hours later) and in the alert the time of the issue is at 10:45.
So in the alert there is +1 hour and the email got send +2 hours after the issue. I have checked that in PHP there is Europe/Prague and in the system clock there is Europe/Prague (and I assume that the DB get time from system clock). But there must be still some settings that are wrong and needs to be adjusted to match all the clocks to alert me on time.
Unfortunately, I dont know where to search.
Thanks
Hi Paulie,
thanks for your guidance. Initially I run the ./validate.php under root user where everything was OK. I tried to su - librenms and then validate.php and I saw that my time was OFF. I have modified timezone only under php.ini in the FPM folder, not in the CLI folder. After making that change and restarting the nginx.service everything is now OK and the alerts are working as a charm! 
This should be added to the manual. Have had this problem some time ago and was just debugging another setup to figure out that it was the same problem.
All of it is in Ansible code now, but for sure it should at least be mentioned in the manual.
The dispatch service also got stuck on some long running tasks and the watchdogs didn’t seem to proberly restart the service, also causing no alerts to be send. So it’s not really production ready yet I guess.
What should be in the docs?
That the timezone also has to be configured in the CLI PHP.ini file and not just under FPM to make alerting work.
Additionally the WebUI validate check doesn’t mention wrong/missing timezone settings while the CLI PHP check does. This makes sense since different php settings are used when executing via CLI. But maybe there is a way the verify the CLI PHP config with the WebUI check to avoid such problems.
The docs do mention setting PHP timezone from both parts.
1 Like
Awesome. Just fixed the Ansible playbook to include the cli settings adjustment. Relying on other people to mantain their (Ansible) code seems to fail me yet again. If you want it done right do it yourself. 
