Unable to update or validate

Help
1

Long time reader, first post here. I’ve been using LibreNMS for several years and it has worked great. I recently upgraded to Ubuntu 22.04.1 LTS and along came PHP8. The system itself runs fine, poller works and the graphs work. The problem is I can no longer update or validate. I could start fresh but I’d rather not setup everything again as well as lose all my data over the years.

Any thoughts or am I out of luck? Here are a few details about the system. Thanks!

OS: Ubuntu 22.04.1 LTS

Version 1.35-52-g798211401 - Mon Jan 29 2018 19:29:08 GMT-0600
Database Schema 2022_04_08_085504_isis_adjacencies_table_add_index (236)
Web Server Apache/2.4.52 (Ubuntu)
PHP 8.1.2
Python 3.10.4
MySQL 10.6.7-MariaDB-2ubuntu1.1
Laravel 8.80.0
RRDtool 1.7.2

./validate.php
PHP Fatal error: Uncaught Error: Call to undefined function starts_with() in /opt/librenms/validate.php:83
Stack trace:
#0 {main}
thrown in /opt/librenms/validate.php on line 83
PHP Fatal error: Uncaught Error: Class “LibreNMS\Util\Version” not found in /opt/librenms/includes/common.php:587
Stack trace:
#0 /opt/librenms/validate.php(64): version_info()
#1 [internal function]: {closure}()
#2 {main}
thrown in /opt/librenms/includes/common.php on line 587

./daily.sh
https://p.libren.ms/view/7c7fc668

2

According to the daily log you have modified files in librenms. If this is NOT the case and you want to restore it, Do something like

git fetch
git reset --hard origin/master
git checkout origin/master
./daily.sh
1 Like
3

Progress!

Validate works now.

====================================

Component Version
LibreNMS 22.8.0-25-g797d73ad1
DB Schema 2022_07_19_081224_plugins_unique_index (244)
PHP 8.1.2
Python 3.10.4
Database MariaDB 10.6.7-MariaDB-2ubuntu1.1
RRDTool 1.7.2
SNMP 5.9.1
====================================

[OK] Composer Version: 2.4.1
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK]
[OK] Database schema correct
[OK] MySQl and PHP time match
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[OK] Python poller wrapper is polling
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached
[WARN] Your local git branch is not master, this will prevent automatic updates.
[FIX]:
You can switch back to master with git checkout master

Still getting errors on ./daily.sh.

error: The following untracked working tree files would be overwritten by checkout:
LibreNMS/Snmptrap/Handlers/UpsTrapsOnBattery.php
app/Http/Controllers/Device/Tabs/MibController.php
app/Http/Controllers/Form/CopyDashboardController.php
app/Http/Controllers/Form/WidgetSettingsController.php
app/Models/Widget.php
database/seeders/DefaultWidgetSeeder.php
doc/js/docs-install-tab.js
html/images/logos/rockylinux.svg
html/images/os/rocky.png
includes/definitions/riello.yaml
includes/html/forms/add-dashboard.inc.php
includes/html/forms/delete-dashboard.inc.php
includes/html/forms/edit-dashboard.inc.php
includes/html/forms/update-dashboard-config.inc.php
includes/polling/sensors/state/fortigate.inc.php
mibs/comware/HH3C-SPLAT-DEVM-MIB
mibs/extreme/Brocade-TC
mibs/hp/HP-SN-MIBS
mibs/transition/TN-SYSUSER-MIB
mibs/transition/TN-SYSUSER-MIB2
tests/data/ray_ray.json
tests/data/riello.json
tests/data/riello_mct20.json
Please move or remove them before you switch branches.
Aborting
Updating to latest codebase FAIL
You are not currently on a branch.
Please specify which branch you want to merge with.
See git-pull(1) for details.

git pull <remote> <branch>

Updating Composer packages OK
Updating SQL-Schema OK
Updating submodules OK
Cleaning up DB OK
Fetching notifications OK
Caching PeeringDB data OK
Caching Mac OUI data OK

Here is the result of the git commands.

$ git fetch
$ git reset --hard origin/master
HEAD is now at 797d73ad1 Misc errors cleanup (#14257)
$ git checkout origin/master
HEAD is now at 797d73ad1 Misc errors cleanup (#14257)

Getting close now, just need to figure out the git issues. Thanks!

4

I ended up removing all the files it was complaining about and running ./daily.sh again. No errors this time! Thanks!

./validate.php

====================================

Component Version
LibreNMS 22.8.0-26-g6f5cf7727
DB Schema 2022_07_19_081224_plugins_unique_index (244)
PHP 8.1.2
Python 3.10.4
Database MariaDB 10.6.7-MariaDB-2ubuntu1.1
RRDTool 1.7.2
SNMP 5.9.1
====================================

[OK] Composer Version: 2.4.1
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK]
[OK] Database schema correct
[OK] MySQl and PHP time match
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[OK] Python poller wrapper is polling
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached

./daily.sh

Updating to latest codebase OK
Updating Composer packages OK
Updating SQL-Schema OK
Updating submodules OK
Cleaning up DB OK
Fetching notifications OK
Caching PeeringDB data OK
Caching Mac OUI data OK

5

Spoke too soon. Rebooted the server and now the poller isn’t working. Some sort of database error.

Cronic detected failure or error output for the command:
/opt/librenms/poller-wrapper.py 4

RESULT CODE: 1

ERROR OUTPUT:
Traceback (most recent call last):
File “/opt/librenms/poller-wrapper.py”, line 57, in
wrapper.wrapper(
File “/opt/librenms/LibreNMS/wrapper.py”, line 557, in wrapper
cursor = db_connection.query(query)
File “/opt/librenms/LibreNMS/init.py”, line 299, in query
cursor.execute(query, args)
File “/usr/lib/python3/dist-packages/pymysql/cursors.py”, line 148, in execute
result = self._query(query)
File “/usr/lib/python3/dist-packages/pymysql/cursors.py”, line 310, in _query
conn.query(q)
File “/usr/lib/python3/dist-packages/pymysql/connections.py”, line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File “/usr/lib/python3/dist-packages/pymysql/connections.py”, line 775, in _read_query_result
result.read()
File “/usr/lib/python3/dist-packages/pymysql/connections.py”, line 1156, in read
first_packet = self.connection._read_packet()
File “/usr/lib/python3/dist-packages/pymysql/connections.py”, line 692, in _read_packet
packet_header = self._read_bytes(4)
File “/usr/lib/python3/dist-packages/pymysql/connections.py”, line 748, in _read_bytes
raise err.OperationalError(
pymysql.err.OperationalError: (2013, ‘Lost connection to MySQL server during query’)

STANDARD OUTPUT:
2022-08-30 14:10:01,758 :: INFO :: starting the poller check at 2022-08-30 14:10:01 with 4 threads for 48 devices
2022-08-30 14:10:13,664 :: INFO :: worker Thread-8 (poll_worker) finished device 59 in 11 seconds
2022-08-30 14:10:14,066 :: INFO :: worker Thread-8 (poll_worker) finished device 72 in 0 seconds
2022-08-30 14:10:14,452 :: INFO :: worker Thread-8 (poll_worker) finished device 70 in 0 seconds
2022-08-30 14:10:14,874 :: INFO :: worker Thread-8 (poll_worker) finished device 99 in 0 seconds
2022-08-30 14:10:14,877 :: INFO :: worker Thread-3 (poll_worker) finished device 98 in 13 seconds
2022-08-30 14:10:15,284 :: INFO :: worker Thread-8 (poll_worker) finished device 100 in 0 seconds
2022-08-30 14:10:15,297 :: INFO :: worker Thread-3 (poll_worker) finished device 86 in 0 seconds
2022-08-30 14:10:15,302 :: INFO :: worker Thread-2 (poll_worker) finished device 107 in 13 seconds
2022-08-30 14:10:15,671 :: INFO :: worker Thread-8 (poll_worker) finished device 85 in 0 seconds
2022-08-30 14:10:15,713 :: INFO :: worker Thread-2 (poll_worker) finished device 89 in 0 seconds
2022-08-30 14:10:15,723 :: INFO :: worker Thread-3 (poll_worker) finished device 84 in 0 seconds
2022-08-30 14:10:16,054 :: INFO :: worker Thread-8 (poll_worker) finished device 91 in 0 seconds
2022-08-30 14:10:16,105 :: INFO :: worker Thread-2 (poll_worker) finished device 103 in 0 seconds
2022-08-30 14:10:16,111 :: INFO :: worker Thread-3 (poll_worker) finished device 109 in 0 seconds
2022-08-30 14:10:16,438 :: INFO :: worker Thread-8 (poll_worker) finished device 48 in 0 seconds
2022-08-30 14:10:16,499 :: INFO :: worker Thread-3 (poll_worker) finished device 63 in 0 seconds
2022-08-30 14:10:16,508 :: INFO :: worker Thread-2 (poll_worker) finished device 108 in 0 seconds
2022-08-30 14:10:16,823 :: INFO :: worker Thread-8 (poll_worker) finished device 32 in 0 seconds
2022-08-30 14:10:16,899 :: INFO :: worker Thread-3 (poll_worker) finished device 36 in 0 seconds
2022-08-30 14:10:16,906 :: INFO :: worker Thread-2 (poll_worker) finished device 6 in 0 seconds
2022-08-30 14:10:17,292 :: INFO :: worker Thread-8 (poll_worker) finished device 18 in 0 seconds
2022-08-30 14:10:17,332 :: INFO :: worker Thread-5 (poll_worker) finished device 56 in 15 seconds
2022-08-30 14:10:17,383 :: INFO :: worker Thread-3 (poll_worker) finished device 30 in 0 seconds
2022-08-30 14:10:17,390 :: INFO :: worker Thread-2 (poll_worker) finished device 16 in 0 seconds
2022-08-30 14:10:17,695 :: INFO :: worker Thread-8 (poll_worker) finished device 15 in 0 seconds
2022-08-30 14:10:17,770 :: INFO :: worker Thread-5 (poll_worker) finished device 13 in 0 seconds
2022-08-30 14:10:17,786 :: INFO :: worker Thread-2 (poll_worker) finished device 73 in 0 seconds
2022-08-30 14:10:17,786 :: INFO :: worker Thread-3 (poll_worker) finished device 3 in 0 seconds
2022-08-30 14:10:18,092 :: INFO :: worker Thread-8 (poll_worker) finished device 94 in 0 seconds
2022-08-30 14:10:18,182 :: INFO :: worker Thread-5 (poll_worker) finished device 40 in 0 seconds
2022-08-30 14:10:18,182 :: INFO :: worker Thread-3 (poll_worker) finished device 53 in 0 seconds
2022-08-30 14:10:18,188 :: INFO :: worker Thread-2 (poll_worker) finished device 95 in 0 seconds
2022-08-30 14:10:18,490 :: INFO :: worker Thread-8 (poll_worker) finished device 97 in 0 seconds
2022-08-30 14:10:18,584 :: INFO :: worker Thread-3 (poll_worker) finished device 92 in 0 seconds
2022-08-30 14:10:18,597 :: INFO :: worker Thread-2 (poll_worker) finished device 96 in 0 seconds
2022-08-30 14:10:18,602 :: INFO :: worker Thread-5 (poll_worker) finished device 42 in 0 seconds
2022-08-30 14:10:18,888 :: INFO :: worker Thread-8 (poll_worker) finished device 11 in 0 seconds
2022-08-30 14:10:18,994 :: INFO :: worker Thread-3 (poll_worker) finished device 88 in 0 seconds
2022-08-30 14:10:19,003 :: INFO :: worker Thread-2 (poll_worker) finished device 102 in 0 seconds
2022-08-30 14:10:19,012 :: INFO :: worker Thread-5 (poll_worker) finished device 21 in 0 seconds
2022-08-30 14:10:19,288 :: INFO :: worker Thread-8 (poll_worker) finished device 7 in 0 seconds
2022-08-30 14:10:19,398 :: INFO :: worker Thread-5 (poll_worker) finished device 45 in 0 seconds
2022-08-30 14:10:19,400 :: INFO :: worker Thread-3 (poll_worker) finished device 87 in 0 seconds
2022-08-30 14:10:19,402 :: INFO :: worker Thread-2 (poll_worker) finished device 90 in 0 seconds
2022-08-30 14:10:19,697 :: INFO :: worker Thread-8 (poll_worker) finished device 105 in 0 seconds
2022-08-30 14:10:19,792 :: INFO :: worker Thread-5 (poll_worker) finished device 106 in 0 seconds
2022-08-30 14:10:19,801 :: INFO :: worker Thread-3 (poll_worker) finished device 101 in 0 seconds
2022-08-30 14:10:19,802 :: INFO :: worker Thread-2 (poll_worker) finished device 93 in 0 seconds
2022-08-30 14:10:19,802 :: INFO :: poller-wrapper checked 48 devices in 18 seconds with 4 workers with 0 errors
2022-08-30 14:10:19,803 :: CRITICAL :: DB Connection exception (2013, ‘Lost connection to MySQL server during query’)

6

Seems fine now. I think it was just telling me I interrupted the database connection when I rebooted. It’s now green in the poller menu. Whew!

7

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.