Problem after moving libreNMS to another server

Hello,

We have move a few weeks a go LibreNMS to another server.
The Old one was a CENTOS 7 system and we go to a Redhat 8.9 system.
Both systems had the same hardware. (HyperV-systems)
We have used the information for moving LibreNMS from this link:
FAQ - LibreNMS Docs

After we done this we had some problems with the new server. No graph was updated only the old rrd-data (copy from de old server) was showing.
Somtimes we have a bad connection with the backend.

The New server is busy CPU 100%
The Old server has a poller cluster health.
The New server had Standard Pollers.
Both pollers are working local on the server.
We don’t use distributed pollers.

Both systems are running parallel for now. So we don’t mis anything.
I will give you a vailidate.php of both systems.

Old-server:
./validate.php

Component Version
LibreNMS 22.9.0-3-g06c361c2b
DB Schema 2022_09_03_091314_update_ports_adsl_table_with_defaults (246)
PHP 7.4.33
Python 3.6.8
Database MariaDB 10.3.28-MariaDB
RRDTool 1.7.0
SNMP 5.8
====================================

[OK] Composer Version: 2.6.6
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK]
[OK] Database schema correct
[OK] MySQl and PHP time match
[WARN] PHP version 8.1 is the minimum supported version as of September, 2022. We recommend you update PHP to a supported version (8.1 suggested) to continue to receive updates. If you do not update PHP, LibreNMS will continue to function but stop receiving bug fixes and updates.
[OK] Active pollers found
[OK] Dispatcher Service is enabled
[OK] Locks are functional
[OK] Python wrapper cron entry is not present
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached

New-server:
./validate.php

Component Version
LibreNMS 24.1.0-67-gd6ce29c05 (2024-02-06T16:42:36+01:00)
DB Schema 2024_01_09_223927_bill_data_updated_indexes (288)
PHP 8.3.2
Python 3.6.8
Database MariaDB 10.3.39-MariaDB
RRDTool 1.7.0
SNMP 5.8
===========================================

[OK] Composer Version: 2.6.6
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK] Database and column collations are correct
[OK] Database schema correct
[OK] MySQL and PHP time match
[OK] Distributed Polling setting is enabled globally
[OK] Connected to rrdcached
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[OK] Python poller wrapper is polling
[OK] Redis is functional
[OK] rrdtool version ok
[OK] Connected to rrdcached

Sometimes the New-server show something els:

./validate.php

Component Version
LibreNMS 24.1.0-67-gd6ce29c05 (2024-02-06T16:42:36+01:00)
DB Schema Not Connected (0)
PHP 8.3.2
Python 3.6.8
Database Not Connected
RRDTool 1.7.0
SNMP 5.8
===========================================

[OK] Composer Version: 2.6.6
[OK] Dependencies up-to-date.
Could not connect to database, check logs/librenms.log.

Or

./validate.php

Component Version
LibreNMS 24.1.0-67-gd6ce29c05 (2024-02-06T16:42:36+01:00)
DB Schema 2024_01_09_223927_bill_data_updated_indexes (288)
PHP 8.3.2
Python 3.6.8
Database MariaDB 10.3.39-MariaDB
RRDTool 1.7.0
SNMP 5.8
===========================================

[OK] Composer Version: 2.6.6
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK] Database and column collations are correct
[OK] Database schema correct
[OK] MySQL and PHP time match
[OK] Distributed Polling setting is enabled globally
[OK] Connected to rrdcached
[FAIL] No active polling method detected
[OK] Dispatcher Service not detected
[OK] Locks are functional
[FAIL] No active python wrapper pollers found
[OK] Redis is functional
[OK] rrdtool version ok
[OK] Connected to rrdcached

LibreNMS.log say the following:

PDOException: SQLSTATE[HY000] [1040] Too many connections in /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Connectors/Connector.php

SQLSTATE[22003]: Numeric value out of range: 1264 Out of range value for column ‘ifOutBroadcastPkts_rate’ at row 1 (Connection: mysql, SQL: UPDATE ports_statistics set ifInNUcastPkts=0,ifInNUcastPkts_prev=0,ifInNUcastPkts_rate=0,ifInNUcastPkts_delta=0,ifOutNUcastPkts=0,ifOutNUcastPkts_prev=0,ifOutNUcastPkts_rate=0,ifOutNUcastPkts_delta=0,ifInDiscards=0,ifInDiscards_prev=0,ifInDiscards_rate=0,ifInDiscards_delta=0,ifOutDiscards=0,ifOutDiscards_prev=0,ifOutDiscards_rate=0,ifOutDiscards_delta=0,ifInUnknownProtos=0,ifInUnknownProtos_prev=0,ifInUnknownProtos_rate=0,ifInUnknownProtos_delta=0,ifInBroadcastPkts=345888628,ifInBroadcastPkts_prev=0,ifInBroadcastPkts_rate=1213644.3087719,ifInBroadcastPkts_delta=345888628,ifOutBroadcastPkts=18446744073440041575,ifOutBroadcastPkts_prev=0,ifOutBroadcastPkts_rate=3.2362708901245E+16,ifOutBroadcastPkts_delta=9223372036854775807,ifInMulticastPkts=859949919,ifInMulticastPkts_prev=0,ifInMulticastPkts_rate=3017368.1368421,ifInMulticastPkts_delta=859949919,ifOutMulticastPkts=273527650,ifOutMulticastPkts_prev=0,ifOutMulticastPkts_rate=959746.14035088,ifOutMulticastPkts_delta=273527650 WHERE port_id = 64209) (Connection: dbFacile, SQL: UPDATE ports_statistics set ifInNUcastPkts=0,ifInNUcastPkts_prev=0,ifInNUcastPkts_rate=0,ifInNUcastPkts_delta=0,ifOutNUcastPkts=0,ifOutNUcastPkts_prev=0,ifOutNUcastPkts_rate=0,ifOutNUcastPkts_delta=0,ifInDiscards=0,ifInDiscards_prev=0,ifInDiscards_rate=0,ifInDiscards_delta=0,ifOutDiscards=0,ifOutDiscards_prev=0,ifOutDiscards_rate=0,ifOutDiscards_delta=0,ifInUnknownProtos=0,ifInUnknownProtos_prev=0,ifInUnknownProtos_rate=0,ifInUnknownProtos_delta=0,ifInBroadcastPkts=345888628,ifInBroadcastPkts_prev=0,ifInBroadcastPkts_rate=1213644.3087719,ifInBroadcastPkts_delta=345888628,ifOutBroadcastPkts=18446744073440041575,ifOutBroadcastPkts_prev=0,ifOutBroadcastPkts_rate=3.2362708901245E+16,ifOutBroadcastPkts_delta=9223372036854775807,ifInMulticastPkts=859949919,ifInMulticastPkts_prev=0,ifInMulticastPkts_rate=3017368.1368421,ifInMulticastPkts_delta=859949919,ifOutMulticastPkts=273527650,ifOutMulticastPkts_prev=0,ifOutMulticastPkts_rate=959746.14035088,ifOutMulticastPkts_delta=273527650 WHERE port_id = 64209)#0 /opt/librenms/includes/polling/ports.inc.php(936): dbUpdate()

I hope that someone kan help us.

regards,

Jorg

For your PDO Exception: Increase open-files-limit in MariaDB on CentOS 7 with systemd

Thank you for your replay but it doesn’t seems to beworking.

I stil have the same issues.
No graphs
No stabiel connection with my Backend (MariaDB)
About 400 unpolled devices

Are there other options?

Please check here: https://example.com/poller
What is your Total poll time? It has to be <300s. If it isn’t, try to increase the thread number in /etc/cron.d/librenms, the line starting with */5 * * * * librenms /opt/librenms/cronic /opt/librenms/poller-wrapper.py.

Also did you apply optimizations?
Mainly mysql one?

Also you can run mysql-tuner and try to tune a little your mysql config.

1 Like