Netonix Poll sensor suddenly started failing

After an update on Saturday, 16 of my Netonix switches’ sensors including voltage appear to no longer work. I have 97 of these switches in LibreNMS and I can’t see any commonality between the ones that no longer poll. Any assistance appreciated.

LIBRENMS.LOG

/opt/librenms/poller.php 148 2023-03-13 13:44:48 - 1 devices polled in 6.241 secs
[2023-03-13T13:44:49.156997+13:00] production.ERROR: Error polling sensors module for 172.16.163.15. LibreNMS\Exceptions\UserFunctionExistException: Invalid user function:  in /opt/librenms/LibreNMS/Util/UserFuncHelper.php:38
Stack trace:
#0 /opt/librenms/includes/polling/functions.inc.php(182): LibreNMS\Util\UserFuncHelper->__call()
#1 /opt/librenms/includes/polling/functions.inc.php(130): record_sensor_data()
#2 /opt/librenms/includes/polling/sensors.inc.php(28): poll_sensor()
#3 /opt/librenms/includes/polling/functions.inc.php(339): include('...')
#4 /opt/librenms/poller.php(126): poll_device()
#5 {main}
[2023-03-13T13:44:49.205560+13:00] production.ERROR: Invalid user function:  {"exception":"[object] (LibreNMS\\Exceptions\\UserFunctionExistException(code: 0): Invalid user function:  at /opt/librenms/LibreNMS/Util/UserFuncHelper.php:38)"}
/opt/librenms/poller.php 153 2023-03-13 13:44:50 - 1 devices polled in 7.762 secs

I don’t have any specific insights to help other than observing it is probably related to a fair amount of polling and function changes in this PR: APC: Add support for Battery Recommended Days Remaining by Jellyfrog · Pull Request #14653 · librenms/librenms · GitHub

This raised issue seem to confirm the same: ASR1001 environmental · Issue #14896 · librenms/librenms · GitHub

Please run this and check if the error disappears:

lnms tinker --execute="\App\Models\Sensor::where('user_func', '')->update(['user_func' => null])"
2 Likes

Worked like a charm. Thanks!

Sensor’s fixed. But server is now very sluggish.

Random PHP errors now showing too. Php is 8.1.17

I’ll migrate it to a new server. Centos 7.9 → Ubuntu LTS was on my todo list anyway.

Thanks for the assistance @Jellyfrog

So I’ve migrated to Ubuntu 22.04, setup a clean install then imported the DB. It’s fast but there’s still the same php errors.

Is there anything obvious I can do to clean this up or should I be looking at reverting from a database a couple of weeks ago?

“Exception: Undefined array key “device_id””
“/opt/librenms/includes/html/pages/ports/list.inc.php:19”
“Exception: Undefined array key “location””
“/opt/librenms/includes/html/pages/ports/list.inc.php:21”
“Exception: Undefined variable $int_link”
“/opt/librenms/includes/html/print-interface.inc.php:267”

librenms@librenms:~$ ./validate.php

Component Version
LibreNMS 23.2.0-41-g473a7d360 (2023-03-19T16:07:52+13:00)
DB Schema 2023_03_14_130653_migrate_empty_user_funcs_to_null (249)
PHP 8.1.2
Python 3.10.4
Database MariaDB 10.6.7-MariaDB-2ubuntu1
RRDTool 1.7.2
SNMP 5.9.1
===========================================

[OK] Composer Version: 2.5.4
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK] Database and column collations are correct
[OK] Database schema correct
[OK] MySQl and PHP time match
“Exception: stream_socket_client(): Unable to connect to tcp://127.0.0.1:6379 (Connection refused)”
“/opt/librenms/vendor/predis/predis/src/Connection/StreamConnection.php:127”
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[FAIL] Some poller nodes have not checked in recently
Inactive Nodes:
librenms.XXXXX.net.XX
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached
“Exception: rtrim(): Passing null to parameter #1 ($string) of type string is deprecated”
“/opt/librenms/LibreNMS/Validations/User.php:95”