Poller Not Running from Cron

Hi All,

We are having issues with one of LibreNMS servers. We 2 Ubuntu18.04 boxes running a distributed setup.

Server 02 - DB, Web, Poller
Server 03 - Poller

Server 03’s poller is running and updating the DB fine
Server 02’s isn’t running from the cron job, This stopped working on 16th of May.

This is the ./validate from Server 02

root@02:/opt/librenms# ./validate.php -distributed
Component | Version
--------- | -------
LibreNMS  | 1.63-152-ga18403584
DB Schema | 2020_04_06_001048_the_great_index_rename (166)
PHP       | 7.4.6
Python    | 3.6.9
MySQL     | 10.1.44-MariaDB-0ubuntu0.18.04.1
RRDTool   | 1.7.0
SNMP      | NET-SNMP 5.7.3

[OK]    Composer Version: 1.10.6
[OK]    Dependencies up-to-date.
Checking configuration: OK
Checking database: OK
[OK]    Database connection successful
[OK]    Database schema correct
Checking disk: OK
Checking php: OK
Checking programs: OK
Checking python: OK
Checking rrd: OK
Checking updates: OK
Checking user: OK

This is the cron job on Server 02, other jobs are running OK

root@02:/etc/cron.d# cat librenms
# Using this cron file requires an additional user on your system, please see install docs.

33 */6 * * * librenms /opt/librenms/discovery-wrapper.py 8 >> /dev/null 2>&1
#33  */6   * * *   librenms    /opt/librenms/discovery.php -h all >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/discovery.php -h new >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/cronic /opt/librenms/poller-wrapper.py 32
#*/5  *    * * *   librenms    /opt/librenms/cronic /opt/librenms/librenms-service.py 32
15   0    * * *   librenms    /opt/librenms/daily.sh >> /dev/null 2>&1
*    *    * * *   librenms    /opt/librenms/alerts.php >> /dev/null 2>&1
#*/5  *    * * *   librenms    /opt/librenms/poll-billing.php >> /dev/null 2>&1
#01   *    * * *   librenms    /opt/librenms/billing-calculate.php >> /dev/null 2>&1
#*/5  *    * * *   librenms    /opt/librenms/check-services.php >> /dev/null 2>&1
*/5 * * * * librenms /opt/librenms/html/plugins/Weathermap/map-poller.php >> /dev/null 2>&1

If I run the poller manually on Server 02 it runs fine;

./poller.php host07 2020-05-27 09:29:16 - 1 devices polled in 52.85 secs
SNMP [50/29.48s]: Get[18/0.85s] Getnext[4/0.17s] Walk[28/28.47s]
MySQL [285/2.51s]: Cell[17/0.01s] Row[-16/-0.01s] Rows[29/0.06s] Column[0/0.00s] 
Update[253/2.44s] Insert[2/0.01s] Delete[0/0.00s]
RRD [324/1.31s]: Other[162/1.28s] Update[162/0.03s]

If I try and run the poller wrapper as the librenms user I get the following error.

$ ./poller-wrapper.py
ERROR: missing memcache python module:
On deb systems: apt-get install python-memcache
On other systems: easy_install python-memcached
Disabling distributed poller.
ERROR: Could not connect to MySQL database!
ERROR: (1045, "Access denied for user 'librenms'@'' (using password: YES)")

The modules are installed

root@02:~# apt list php-memcached python-memcache
Listing... Done
php-memcached/bionic,now 3.1.4+2.2.0-1+ubuntu18.04.1+deb.sury.org+20191129 amd64 [installed]
python-memcache/bionic-updates,bionic-updates,now 1.57-2ubuntu0.18.04.1 all [installed]

I can also connect to the DB as the librenms user using the detail is config.php

root@02:/opt/librenms# su librenms
$ mysql -h localhost -u librenms -pstrongpassword librenms
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 8023
Server version: 10.1.44-MariaDB-0ubuntu0.18.04.1 Ubuntu 18.04

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [librenms]>

I don’t know whta else could be stopping the cron job running and polling devices.

quick fix (I think) in poller 2 set db_socket for both .env and config.php then try again

Hi TheGreatDoc,

Thanks for the response. I have looked at the docs for .env and it shows the following


Excuse my noob questions but what gets set after the equals?

If I do the same in config.php what would go in the quotes?

$config[‘db_socket’] = ’ ';

Had a search and found the following;

root@02:/opt/librenms#  mysqladmin -p -u root version
Enter password:
mysqladmin  Ver 9.1 Distrib 10.1.44-MariaDB, for debian-linux-gnu on x86_64
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Server version          10.1.44-MariaDB-0ubuntu0.18.04.1
Protocol version        10
Connection              Localhost via UNIX socket
UNIX socket             /var/run/mysqld/mysqld.sock
Uptime:                 2 hours 46 min 21 sec

Threads: 35  Questions: 4980260  Slow queries: 0  Opens: 432  Flush tables: 1  Open 
tables: 426  Queries per second avg: 498.974

So I put the following as the Unix socket variable?

$config['db_socket'] = '/var/run/mysqld/mysqld.sock';

Hi The Great Doc,

Thank you very much. The poller on Server 2 is running again after adding the variables.




$config['db_socket'] = '/var/run/mysqld/mysqld.sock';

1 Like