Alert emails aren’t being sent, but recovery emails are. Alerts were working two days ago. We’re on daily release.
- Steps to reproduce an issue.
- Stop the sshd daemon being monitored with the librenms (check_mk) agent on the remote device.
- Poll the device: ./poller.php -h myhost -r -d -m unix-agent
- Test the alert for the ssh process monitoring rule on that host (no email is sent): $ ./scripts/test-alert.php -r14 -h myhost -d
SQL[SELECTdevice_id
FROMdevices
WHEREhostname
= ‘spacewalk’]
SQL[SELECT alert_log.id,alert_log.rule_id,alert_log.device_id,alert_log.state,alert_log.details,alert_log.time_logged,alert_rules.rule,alert_rules.severity,alert_rules.extra,alert_rules.name FROM alert_log,alert_rules WHERE alert_log.rule_id = alert_rules.id && alert_log.device_id = ‘117’ && alert_log.rule_id = ‘14’ && alert_rules.disabled = 0 ORDER BY alert_log.id DESC LIMIT 1]
SQL[SELECT attrib_value FROM devices_attribs WHERE attrib_type = “disable_notify” && device_id = ‘117’]
SQL[]
SQL[SELECT hostname, sysName, sysDescr, hardware, version, location, purpose, notes, uptime FROM devices WHERE device_id = ‘117’]
SQL[SELECTtemplate
,title
,title_rec
FROMalert_templates
JOINalert_template_map
ONalert_template_map
.alert_templates_id
=alert_templates
.id
WHEREalert_template_map
.alert_rule_id
=‘14’]
SQL[SELECTtemplate
,title
,title_rec
FROMalert_templates
WHEREname
=‘Default Alert Template’]
Issuing Alert-UID #6131/1: ; ; mail => ERROR: You must provide at least one recipient email address.
SQL[SELECT * FROMdevices
WHEREdevice_id
= ‘117’]
SQL[SELECT * FROM devices_attribs WHEREdevice_id
= ‘117’]
SQL[SELECT * FROMvrf_lite_cisco
WHEREdevice_id
= ‘117’]
SQL[INSERT INTOeventlog
(host
,device_id
,reference
,type
,datetime
,severity
,message
,username
) VALUES (‘117’,‘117’,‘NULL’,‘error’,NOW(),‘5’,‘Could not issue critical alert for rule ‘ssh daemon is not running’ to transport ‘mail’ Error: You must provide at least one recipient email address.’,’’)]
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; - Poll the device again (recovery email is sent)
$ ./scripts/test-alert.php -r14 -h myhost -d
SQL[SELECTdevice_id
FROMdevices
WHEREhostname
= ‘spacewalk’]
SQL[SELECT alert_log.id,alert_log.rule_id,alert_log.device_id,alert_log.state,alert_log.details,alert_log.time_logged,alert_rules.rule,alert_rules.severity,alert_rules.extra,alert_rules.name FROM alert_log,alert_rules WHERE alert_log.rule_id = alert_rules.id && alert_log.device_id = ‘117’ && alert_log.rule_id = ‘14’ && alert_rules.disabled = 0 ORDER BY alert_log.id DESC LIMIT 1]
SQL[SELECT attrib_value FROM devices_attribs WHERE attrib_type = “disable_notify” && device_id = ‘117’]
SQL[]
SQL[SELECT hostname, sysName, sysDescr, hardware, version, location, purpose, notes, uptime FROM devices WHERE device_id = ‘117’]
SQL[SELECTtemplate
,title
,title_rec
FROMalert_templates
JOINalert_template_map
ONalert_template_map
.alert_templates_id
=alert_templates
.id
WHEREalert_template_map
.alert_rule_id
=‘14’]
SQL[SELECTtemplate
,title
,title_rec
FROMalert_templates
WHEREname
=‘Default Alert Template’]
SQL[SELECT alert_log.id,alert_log.time_logged,alert_log.details FROM alert_log WHERE alert_log.state != 2 && alert_log.state != 0 && alert_log.rule_id = ‘14’ && alert_log.device_id = ‘117’ && alert_log.id < ‘6139’ ORDER BY id DESC LIMIT 1]
Issuing Alert-UID #6139/0: ; ; mail => OKSQL[SELECT * FROMdevices
WHEREdevice_id
= ‘117’]
SQL[SELECT * FROM devices_attribs WHEREdevice_id
= ‘117’]
SQL[SELECT * FROMvrf_lite_cisco
WHEREdevice_id
= ‘117’]
SQL[INSERT INTOeventlog
(host
,device_id
,reference
,type
,datetime
,severity
,message
,username
) VALUES (‘117’,‘117’,‘NULL’,‘alert’,NOW(),‘1’,‘Issued recovery for rule ‘ssh daemon is not running’ to transport ‘mail’’,’’)]
; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ; ;
-
The output of
./validate.php
$ ./validate.php -sComponent Version LibreNMS 1.37-83-g607a7f8 DB Schema 246 PHP 7.0.28 MySQL 5.5.56-MariaDB RRDTool 1.4.8 SNMP NET-SNMP 5.7.2 ====================================
[OK] Composer Version: 1.6.3
[OK] Dependencies up-to-date.
Checking configuration: OK
Checking database: OK
[OK] Database connection successful
[OK] Database schema correct
Checking disk: OK
Checking php: OK
Checking poller: WARN
[WARN] Some devices have not been polled in the last 5 minutes. You may have performance issues.
[FIX] Check your poll log and see: http://docs.librenms.org/Support/Performance/
Devices:
// CUT THESE OUT - These devices are actually down
Checking programs: OK
Checking rrd: OK
Checking updates: OK
Checking user: OK
LibreNMS configuration settings in database:
MariaDB [librenms]> select config_id,config_name,config_value from config where config_group = ‘alerting’ and config_name like ‘%mail%’ or config_name like ‘%default%’;
±----------±---------------------------±----------------------------+
| config_id | config_name | config_value |
±----------±---------------------------±----------------------------+
| 452 | alert.default_only | false |
| 453 | alert.default_mail | foo@bar |
| 458 | email_backend | smtp |
| 462 | alert.transports.mail | true |
| 465 | email_from | foo@bar |
| 466 | email_user | LibreNMS |
| 467 | email_sendmail_path | /usr/sbin/sendmail |
| 468 | email_smtp_host | a.b.c.d |
| 469 | email_smtp_port | 25 |
| 470 | email_smtp_timeout | 10 |
| 471 | email_smtp_secure | |
| 472 | email_smtp_auth | false |
| 473 | email_smtp_username | NULL |
| 474 | email_smtp_password | NULL |
| 739 | oxidized.default_group | |
| 746 | email_html | true |
| 761 | webui.default_dashboard_id | 2 |
| 770 | email_auto_tls | true |
| 779 | alert.default_if_none | true |
| 802 | alert.default_copy | true |
±----------±---------------------------±----------------------------+
20 rows in set (0.00 sec)
If it’s an issue with the WebUI then please consider including the browser version you are using.
If you need to post any text longer than a few lines, please use a pastebin service such as https://p.libren.ms using non-expiring pastes.