Poller stopping

Tags: #<Tag:0x00007f707dc48bd8>

We are starting to experience that our poller is stopping.

We are seeing the following traceback and after that a few polls already initiated will follow but no new jobs will be started.

After this error, the discovery and the billing continues to run.
To get the poller running again we have to restart the librenms process.

I dont see any pattern in timing or specific devices being polled before the error is thrown.

The frequency of the error:
Oct 01 10:05:01
Oct 02 07:12:19
Oct 02 08:17:30
Oct 02 10:52:34
Oct 03 04:04:31
Oct 05 07:55:38
Oct 05 17:38:14
Oct 05 20:37:06
Oct 06 06:57:14

Traceback (most recent call last):
File “/opt/librenms/librenms-service.py”, line 48, in
service.start()
File “/opt/librenms/LibreNMS/service.py”, line 404, in start
sleep(self.config.master_resolution)
File “/opt/librenms/LibreNMS/service.py”, line 550, in reap
if (signal(SIGCHLD, SIG_DFL) == SIG_DFL):
File “/usr/lib/python3.6/signal.py”, line 48, in signal
return _int_to_enum(handler, Handlers)
File “/usr/lib/python3.6/signal.py”, line 30, in _int_to_enum
return enum_klass(value)
File “/usr/lib/python3.6/enum.py”, line 293, in call
return cls.new(cls, value)
File “/usr/lib/python3.6/enum.py”, line 535, in new
return cls.missing(value)
File “/usr/lib/python3.6/enum.py”, line 548, in missing
raise ValueError("%r is not a valid %s" % (value, cls.name))
TypeError: ‘int’ object is not callable

./validate.php

Component Version
LibreNMS 1.68-26-g4068b04cc
DB Schema 2020_09_22_172321_add_alert_log_index (174)
PHP 7.2.31-1+ubuntu18.04.1+deb.sury.org+1
Python 3.6.9
MySQL 10.1.44-MariaDB-0ubuntu0.18.04.1
RRDTool 1.7.0
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.10.13
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct
[WARN] PHP version 7.3 is the minimum supported version as of November, 2020. We recommend you update PHP to a supported version (7.4 suggested) to continue to receive updates. If you do not update PHP, LibreNMS will continue to function but stop receiving bug fixes and updates.

The error doesn’t make much sense to me unless there is a bug in the Python version you are running. Can you try a new version of Python?

Updating Python to 3.8 fixed the problem.

@PatrickFrederiksen does the problem remain fixed for you?

I’m running the latest librenms release at 3.8 and we’re still seeing this issue.