Transport alerts are not emailing

Help
1

We have completely stopped receiving alerts via email.
I can run a manual alert-test using

./scripts/test-alert.php -r 2 -d -h 804

I was able to receive alert until I updated to master from release for the update_channel using the following I found on other post

./lnms config:set update_channel master
cd /opt/librenms && git fetch --tags && git checkout $(git describe --tags $(git rev-list --tags --max-count=1))
./daily.sh
./validate.php

I set the cron for alerts as follows
/opt/librenms/alerts.php -d>> /dev/null 2>&1 >>/opt/librenms/logs/alerts.log
I get a repeating error of the following. But do not know where else to look to see what is causing the issue

count(): Argument #1 ($value) must be of type Countable|array, null given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, null given at /opt/librenms/LibreNMS/Alert/RunAlerts.php:303)"}

In RunAlerts.php line 303:

  count(): Argument #1 ($value) must be of type Countable|array, null given

SQL[update `cache_locks` set `owner` = ?, `expiration` = ? where `key` = ? and (`owner` = ? or `expiration` <= ?) ["eer7Abr03Wq2f7UB",1688673181,"laravel_cache_alerts","eer7Abr03Wq2f7UB",1688673121] 0.58ms]

./validate.php

Component Version
LibreNMS 23.6.0-19-g9c6541f3a (2023-07-05T06:29:32-07:00)
DB Schema 2023_04_27_164904_update_slas_opstatus_tinyint (252)
PHP 8.2.7
Python 3.8.10
Database MariaDB 10.3.38-MariaDB-0ubuntu0.20.04.1
RRDTool 1.7.2
SNMP 5.8
===========================================

[OK] Composer Version: 2.5.8
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK] Database and column collations are correct
[OK] Database schema correct
[OK] MySQl and PHP time match
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[OK] Python poller wrapper is polling
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached

2

Check your alert rules, you might have a bad one.

3

How do you test them? Does enabling disabling them do anything to help test?
The only ones I have added lately are over 6 months old and alerts were working after that.

4

Give this a shot Testing - LibreNMS Docs

5

OK just went through all my active alerts had a couple that came up with Stale-Rule: #2/xxx (2 change based on rule)
So I went into the rules removed the transport added back in and ran test again stale part was gone.
Alerts still only send not when done manually.
I am seeing the following still in debug log

Reporting disabled by user setting
count(): Argument #1 ($value) must be of type Countable|array, null given {"exception":"[object] (TypeError(code: 0): count(): Argument #1 ($value) must be of type Countable|array, null given at /opt/librenms/LibreNMS/Alert/RunAlerts.php:303)"}

In RunAlerts.php line 303:

  count(): Argument #1 ($value) must be of type Countable|array, null given

I ran a loop for all my devices against alerts using the test-alert script it worked for sending alerts until device 533 then it stopped sending emails for some reason.

for j in {1..800}; do echo $j; for i in `cat /var/tmp/alert_list`;do ./scripts/test-alert.php -r $i -d -h $j;done; done
6

Any output if you run the test manually on that device from the cli?

7

No there was no output other than what I already posted. Sorry been away for a week and just now getting back into this.

8

Well I resolved it but not sure completely how.
I turned off all my alert rules. Waited for a couple cycles of the alerting cron to go through.
Then enabled one alert at a time waiting for it to go through cycle to make sure it was good before turning on next alert. Did that until all alerts were enabled it is functioning now with no issues so far. My only thought is something was stuck with a rule and having all the alerts off allowed it to clear out with the alert that was “hanging” on to it. @laf Thanks for the assistance in the troubleshooting.

9

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.