Some pollers not update statistic on pollers page

Help
1

Hi all.
I have a slightly strange problem.
I have about 70 pollers. these are virtual machines that are created from a template in which Librenms is regularly updated. Librenms on pollers is also updated daily.
Yesterday I noticed a strange thing on the / poller page (https://librenms.site/poller) - data on all pollers stopped updating, except for one. this is how it looks:

image
image1885×266 30.7 KB

But at the same time, all pollers work and monitored devices data are updated.

here is the output of the validate poller, which is green on the screen

-bash-4.2$ ./validate.php
===========================================
Component | Version
--------- | -------
LibreNMS  | 23.2.0-37-gcfb70f73d (2023-03-16T17:05:23+02:00)
DB Schema | 2023_03_14_130653_migrate_empty_user_funcs_to_null (252)
PHP       | 8.1.11
Python    | 3.6.8
Database  | MariaDB 10.5.18-MariaDB
RRDTool   | 1.7.1
SNMP      | 5.7.2
===========================================

[OK]    Composer Version: 2.5.4
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database Schema is current
[OK]    SQL Server meets minimum requirements
[OK]    lower_case_table_names is enabled
[OK]    MySQL engine is optimal
[OK]    Database and column collations are correct
[OK]    Database schema correct
[OK]    MySQl and PHP time match
[OK]    Distributed Polling setting is enabled globally
[OK]    Connected to rrdcached
[OK]    Active pollers found
[OK]    Dispatcher Service not detected
[OK]    Locks are functional
[FAIL]  Some poller nodes have not checked in recently
        Inactive Nodes:
         poller0_0
         poller27_0
         poller17_0
         poller22_0
         poller6_0
         poller20_0
         poller33_0
         poller12_0
         poller45_0
         poller51_1
         poller50_0
         poller13_1
         poller2_1
         poller21_1
         poller23_1
          and 53 more...
[OK]    Redis is unavailable
[OK]    rrdtool version ok
[OK]    Connected to rrdcached

and here is the output of the validate poller, which is red on the screen


-bash-4.2$ ./validate.php
===========================================
Component | Version
--------- | -------
LibreNMS  | 23.2.0-37-gcfb70f7 (2023-03-16T17:05:23+02:00)
DB Schema | 2023_03_14_130653_migrate_empty_user_funcs_to_null (252)
PHP       | 8.1.11
Python    | 3.6.8
Database  | MariaDB 10.5.18-MariaDB
RRDTool   | 1.7.1
SNMP      | 5.7.2
===========================================

[OK]    Composer Version: 2.5.4
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database Schema is current
[OK]    SQL Server meets minimum requirements
[OK]    lower_case_table_names is enabled
[OK]    MySQL engine is optimal
[OK]    Database and column collations are correct
[OK]    Database schema correct
[OK]    MySQl and PHP time match
[OK]    Distributed Polling setting is enabled globally
[OK]    Connected to rrdcached
[OK]    Active pollers found
[OK]    Dispatcher Service not detected
[OK]    Locks are functional
[FAIL]  Some poller nodes have not checked in recently
        Inactive Nodes:
         poller0_0
         poller27_0
         poller17_0
         poller22_0
         poller6_0
         poller20_0
         poller33_0
         poller12_0
         poller45_0
         poller51_1
         poller50_0
         poller13_1
         poller2_1
         poller21_1
         poller23_1
          and 53 more...
[OK]    Redis is unavailable
[OK]    rrdtool version ok
[OK]    Connected to rrdcached

tell me who knows where to dig to find the problem

2

problem found and fixed.
Thank you all for your passive participation :slight_smile:

3

Hi Roman, how did you fix it?

4

From his screenshot it looks like librenms service was not enabled. the validate shows that it currently isn’t running or enabled.

Dispatcher Service not detected

You could enable the librenms watchdog service instead of the librenms service or cron, as it should restart the service automatically if the service is running but the poller hasn’t logged anything.

It also looks like the redis master could not be reached so he got the redis failure.

Redis is unavailable

But he is running the cron based poller so there is something not working correctly there. you could run the poller line from the conjob via command line to see what is happening.

5

I’m not use Redis. I’m use memcached.
It was misleading that the poller sends collected data and data are recorded, and the poller does not send only statistics about its work.
in any case, the problem was solved from the side of the mysql server - increased

wait_timeout
net_read_timeout
net_write_timeout

and the problem is solved

1 Like
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.