Librenms Bad performance (timeouts)

Hi,

I have had serious performance problems and my service is constantly down with errors on the webserver (nginx timeout) and sometimes in the database. I have already changed the PHP child processes, etc., and nothing has worked!
When it is available, clicking on any element takes a long time and often returns in timeout.

Does anyone have an optimized performance configuration they can share?
Thanks

Best Regards

Ubuntu 24.04.2 LTS

librenms@librenms:~$ ./validate.php
===========================================
Component | Version
--------- | -------
LibreNMS  | 25.3.0-73-g0b2ce6332 (2025-03-31T08:45:35+01:00)
DB Schema | 2025_03_19_205700_fix_ospfv3_ports_table (331)
PHP       | 8.3.6
Python    | 3.12.3
Database  | MariaDB 10.11.8-MariaDB-0ubuntu0.24.04.1
RRDTool   | 1.7.2
SNMP      | 5.9.4.pre2
===========================================

[OK]    Composer Version: 2.8.6
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database connection successful
[OK]    Database Schema is current
[OK]    SQL Server meets minimum requirements
[OK]    lower_case_table_names is enabled
[OK]    MySQL engine is optimal
[OK]    Database and column collations are correct
[OK]    Database schema correct
[OK]    MySQL and PHP time match
[OK]    Active pollers found
[OK]    Dispatcher Service not detected
[OK]    Locks are functional
[OK]    Python poller wrapper is polling
[OK]    Redis is unavailable
[OK]    rrdtool version ok
[OK]    Connected to rrdcached


System information as of Mon Mar 31 10:48:44 AM WAT 2025

  System load:  1.37              Temperature:           67.0 C
  Usage of /:   10.3% of 1.61TB   Processes:             599
  Memory usage: 2%                Users logged in:       0
  Swap usage:   0%                IPv4 address for eno1: 10.x.x.x


Have you gone through the documentation’s recommendations?

Hi @murrant ,

Thanks for your quickly response!

I had already done this optimization after installation, I did the review and everything is fine!
I changed the PHP child and enabled tuning in the MySQL settings, the system started working again and responding better, but it started to show errors in the icon and deformation of the pop-up graph and the global settings page is blank, I saw the logs and I have the errors below;

root@librenms:~# tail -f /var/log/php8.3-fpm.log
[01-Apr-2025 10:47:06] NOTICE: Terminating ...
[01-Apr-2025 10:47:06] NOTICE: exiting, bye-bye!
[01-Apr-2025 10:47:06] NOTICE: fpm is running, pid 2181613
[01-Apr-2025 10:47:06] NOTICE: ready to handle connections
[01-Apr-2025 10:47:06] NOTICE: systemd monitor interval set to 10000ms
[01-Apr-2025 10:47:35] NOTICE: Terminating ...
[01-Apr-2025 10:47:35] NOTICE: exiting, bye-bye!
[01-Apr-2025 10:47:35] NOTICE: fpm is running, pid 2198156
[01-Apr-2025 10:47:35] NOTICE: ready to handle connections
[01-Apr-2025 10:47:35] NOTICE: systemd monitor interval set to 10000ms
^C

root@librenms:~# tail -f /var/log/mysql/error.log
2025-03-31 18:37:26 0 [Note] Plugin 'FEEDBACK' is disabled.
2025-03-31 18:37:26 0 [Note] Server socket created on IP: '127.0.0.1'.
2025-03-31 18:37:26 0 [Note] /usr/sbin/mariadbd: ready for connections.
Version: '10.11.11-MariaDB-0ubuntu0.24.04.2-log'  socket: '/run/mysqld/mysqld.sock'  port: 3306  Ubuntu 24.04
2025-03-31 18:37:41 0 [Note] InnoDB: Buffer pool(s) load completed at 250331 18:37:41
2025-04-01  0:48:02 69844 [Warning] Aborted connection 69844 to db: 'librenms' user: 'librenms' host: 'localhost' (Got an error reading communication packets)
2025-04-01  6:48:02 139067 [Warning] Aborted connection 139067 to db: 'librenms' user: 'librenms' host: 'localhost' (Got an error reading communication packets)
2025-04-01 10:47:06 189280 [Warning] Aborted connection 189280 to db: 'librenms' user: 'librenms' host: 'localhost' (Got an error reading communication packets)
2025-04-01 10:47:35 189476 [Warning] Aborted connection 189476 to db: 'librenms' user: 'librenms' host: 'localhost' (Got an error reading communication packets)
2025-04-01 10:47:39 189472 [Warning] Aborted connection 189472 to db: 'librenms' user: 'librenms' host: 'localhost' (Got an error writing communication packets)
^C

root@librenms:~# tail -f /var/log/php8.3-fpm.log
[01-Apr-2025 10:47:06] NOTICE: Terminating ...
[01-Apr-2025 10:47:06] NOTICE: exiting, bye-bye!
[01-Apr-2025 10:47:06] NOTICE: fpm is running, pid 2181613
[01-Apr-2025 10:47:06] NOTICE: ready to handle connections
[01-Apr-2025 10:47:06] NOTICE: systemd monitor interval set to 10000ms
[01-Apr-2025 10:47:35] NOTICE: Terminating ...
[01-Apr-2025 10:47:35] NOTICE: exiting, bye-bye!
[01-Apr-2025 10:47:35] NOTICE: fpm is running, pid 2198156
[01-Apr-2025 10:47:35] NOTICE: ready to handle connections
[01-Apr-2025 10:47:35] NOTICE: systemd monitor interval set to 10000ms
^C

root@librenms:~# tail -f /var/log/nginx/error.log
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?id=75895&type=port_errors&legend=no&absolute_size=0&width=150&height=21&from=-1d HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?device=105&from=1743414455&to=1743500855&width=150&height=21&type=device_bits&legend=no&absolute=0 HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?device=106&from=1743414455&to=1743500855&width=150&height=21&type=device_bits&legend=no&absolute=0 HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?device=265&from=1743414455&to=1743500855&width=150&height=21&type=device_bits&legend=no&absolute=0 HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?device=225&from=1743414455&to=1743500855&width=150&height=21&type=device_bits&legend=no&absolute=0 HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:47:35 [crit] 1486#1486: *3928 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.6.6.22, server: librenms.domain.com, request: "GET /graph.php?device=190&from=1743414455&to=1743500855&width=150&height=21&type=device_bits&legend=no&absolute=0 HTTP/2.0", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.domain.com", referrer: "https://librenms.domain.com/"
2025/04/01 10:50:28 [alert] 1486#1486: *4282 pread() read only 5602 of 5716 from "/opt/librenms/html/plugins/Weathermap/output/edge1.png" while sending response to client, client: 10.6.6.41, server: librenms.domain.com, request: "GET /plugins/Weathermap/output/edge1.png?1405796415 HTTP/2.0", host: "librenms.domain.com", referrer: "https://librenms.domain.com/overview"
![Untitled|690x459](upload://iiEolhelxXRvx2E4UEWlmeycb4l.png)


server {
 listen      80;
 listen 443 ssl http2;
 server_name librenms.domain.com;
 root        /opt/librenms/html;
 index       index.php;

 ssl_certificate /home/administrator/certificate/wildcard.pem;
 ssl_certificate_key /home/administrator/certificate/wildcard_key.key;

 charset utf-8;
 gzip on;
 gzip_types text/css application/javascript text/javascript application/x-javascript image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;
 location / {
  try_files $uri $uri/ /index.php?$query_string;
 }
 location ~ [^/]\.php(/|$) {
  fastcgi_pass unix:/run/php-fpm-librenms.sock;
  fastcgi_split_path_info ^(.+\.php)(/.+)$;
  include fastcgi.conf;
 }
 location ~ /\.(?!well-known).* {
  deny all;
 }
}


librenms@librenms:~$ ./validate.php
===========================================
Component | Version
--------- | -------
LibreNMS  | 25.3.0-78-g2edcaaed9 (2025-04-01T07:11:59+01:00)
DB Schema | 2025_03_19_205700_fix_ospfv3_ports_table (331)
PHP       | 8.3.6
Python    | 3.12.3
Database  | MariaDB 10.11.11-MariaDB-0ubuntu0.24.04.2-log
RRDTool   | 1.7.2
SNMP      | 5.9.4.pre2
===========================================

[OK]    Composer Version: 2.8.6
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database connection successful
[OK]    Database Schema is current
[OK]    SQL Server meets minimum requirements
[OK]    lower_case_table_names is enabled
[OK]    MySQL engine is optimal
[OK]    Database and column collations are correct
[OK]    Database schema correct
[OK]    MySQL and PHP time match
[OK]    Active pollers found
[OK]    Dispatcher Service not detected
[OK]    Locks are functional
[OK]    Python poller wrapper is polling
[OK]    Redis is unavailable
[OK]    rrdtool version ok
[OK]    Connected to rrdcached

The errors you are showing are not normal and are really outside the scope of LibreNMS. (in other words these are generic system problems)

It looks like many of your processes including mariadb and php-fpm are crashing or being killed.

Check /var/log/syslog, you might find it’s OOM killer

Hi @laf

Thanks for reply!

2025-04-02T10:59:00.629331+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T10:59:01.282126+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T10:59:01.282372+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T10:59:01.765928+01:00 librenms CRON[3759687]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T10:59:43.537805+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:00:00.166132+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:00:00.167947+01:00 librenms systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
2025-04-02T11:00:00.210475+01:00 librenms systemd[1]: sysstat-collect.service: Deactivated successfully.
2025-04-02T11:00:00.210747+01:00 librenms systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
2025-04-02T11:00:00.751960+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:00:00.752566+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:00:01.820853+01:00 librenms CRON[3776979]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:00:01.822320+01:00 librenms CRON[3776980]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:00:01.822793+01:00 librenms CRON[3776982]: (librenms) CMD (/opt/librenms/html/plugins/Weathermap/map-poller.php >> /dev/null 2>&1)
2025-04-02T11:00:01.823244+01:00 librenms CRON[3776981]: (librenms) CMD (   /opt/librenms/check-services.php >> /dev/null 2>&1)
2025-04-02T11:00:01.824606+01:00 librenms CRON[3776983]: (librenms) CMD (   /opt/librenms/poll-billing.php >> /dev/null 2>&1)
2025-04-02T11:00:01.825392+01:00 librenms CRON[3776987]: (librenms) CMD (   /opt/librenms/discovery.php -h new >> /dev/null 2>&1)
2025-04-02T11:00:01.825902+01:00 librenms CRON[3776988]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:00:43.537937+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:00:56.258029+01:00 librenms CRON[3626535]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:00:57.625161+01:00 librenms CRON[3626540]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:01:00.251364+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:01:01.050767+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:01:01.051055+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:01:01.635843+01:00 librenms CRON[3799675]: (librenms) CMD (   /opt/librenms/billing-calculate.php >> /dev/null 2>&1)
2025-04-02T11:01:01.638610+01:00 librenms CRON[3799677]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:01:43.539460+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:02:00.621255+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:02:01.232420+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:02:01.232664+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:02:01.300533+01:00 librenms CRON[3820414]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:02:43.538967+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:02:57.758819+01:00 librenms CRON[3697190]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:02:58.296438+01:00 librenms CRON[3697194]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:03:00.622251+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:03:01.242904+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:03:01.243103+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:03:01.302215+01:00 librenms CRON[3830869]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:03:43.539694+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:04:00.302198+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:04:01.001007+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:04:01.001348+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:04:01.544101+01:00 librenms CRON[3844896]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:04:43.540166+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:05:00.147379+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:05:00.878419+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:05:00.879240+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:05:01.664546+01:00 librenms CRON[3864058]: (librenms) CMD (/opt/librenms/html/plugins/Weathermap/map-poller.php >> /dev/null 2>&1)
2025-04-02T11:05:01.665035+01:00 librenms CRON[3864059]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:05:01.667486+01:00 librenms CRON[3864060]: (librenms) CMD (   /opt/librenms/check-services.php >> /dev/null 2>&1)
2025-04-02T11:05:01.668435+01:00 librenms CRON[3864063]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2025-04-02T11:05:01.669618+01:00 librenms CRON[3864064]: (librenms) CMD (   /opt/librenms/poll-billing.php >> /dev/null 2>&1)
2025-04-02T11:05:01.670583+01:00 librenms CRON[3864065]: (librenms) CMD (   /opt/librenms/discovery.php -h new >> /dev/null 2>&1)
2025-04-02T11:05:01.671073+01:00 librenms CRON[3864067]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:05:01.671519+01:00 librenms CRON[3864066]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:05:43.541008+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:06:00.395418+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:06:01.383101+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:06:01.383505+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:06:01.488799+01:00 librenms CRON[3888494]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:06:43.541447+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:07:00.459305+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:07:01.018249+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:07:01.018456+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:07:01.171890+01:00 librenms CRON[3900694]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:07:43.541733+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:07:58.121127+01:00 librenms CRON[3776972]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:07:58.377204+01:00 librenms CRON[3776976]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:08:00.099132+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:08:00.715579+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:08:00.716027+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:08:01.385784+01:00 librenms CRON[3910522]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:08:43.541953+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:09:00.632176+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:09:00.633704+01:00 librenms systemd[1]: Starting phpsessionclean.service - Clean php session files...
2025-04-02T11:09:01.003144+01:00 librenms systemd[1]: phpsessionclean.service: Deactivated successfully.
2025-04-02T11:09:01.003437+01:00 librenms systemd[1]: Finished phpsessionclean.service - Clean php session files.
2025-04-02T11:09:01.242818+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:09:01.243032+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:09:01.449586+01:00 librenms CRON[3923250]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
2025-04-02T11:09:01.449938+01:00 librenms CRON[3923251]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:09:43.542756+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:10:00.040191+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:10:00.042260+01:00 librenms systemd[1]: Starting sysstat-collect.service - system activity accounting tool...
2025-04-02T11:10:00.083179+01:00 librenms systemd[1]: sysstat-collect.service: Deactivated successfully.
2025-04-02T11:10:00.083620+01:00 librenms systemd[1]: Finished sysstat-collect.service - system activity accounting tool.
2025-04-02T11:10:00.760817+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:10:00.761020+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:10:01.391392+01:00 librenms CRON[3940535]: (librenms) CMD (/opt/librenms/html/plugins/Weathermap/map-poller.php >> /dev/null 2>&1)
2025-04-02T11:10:01.392061+01:00 librenms CRON[3940536]: (librenms) CMD (   /opt/librenms/check-services.php >> /dev/null 2>&1)
2025-04-02T11:10:01.392386+01:00 librenms CRON[3940537]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:10:01.393278+01:00 librenms CRON[3940538]: (librenms) CMD (   /opt/librenms/poll-billing.php >> /dev/null 2>&1)
2025-04-02T11:10:01.393750+01:00 librenms CRON[3940539]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:10:01.394868+01:00 librenms CRON[3940540]: (librenms) CMD (   /opt/librenms/discovery.php -h new >> /dev/null 2>&1)
2025-04-02T11:10:01.395324+01:00 librenms CRON[3940541]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:10:43.543889+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:11:00.196932+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:11:01.056744+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:11:01.057602+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:11:01.734049+01:00 librenms CRON[3964787]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:11:43.543788+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:12:00.090584+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:12:00.818008+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:12:00.818240+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:12:01.376495+01:00 librenms CRON[3979520]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:12:43.544022+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:12:59.432688+01:00 librenms CRON[3864054]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:12:59.631505+01:00 librenms CRON[3864050]: (CRON) info (No MTA installed, discarding output)
2025-04-02T11:13:00.098109+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:13:00.726297+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:13:00.726537+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:13:01.637973+01:00 librenms CRON[3988530]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:13:43.545169+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:13:56.464819+01:00 librenms systemd[1]: Created slice user-1000.slice - User Slice of UID 1000.
2025-04-02T11:13:56.467657+01:00 librenms systemd[1]: Starting [email protected] - User Runtime Directory /run/user/1000...
2025-04-02T11:13:56.518677+01:00 librenms systemd[1]: Finished [email protected] - User Runtime Directory /run/user/1000.
2025-04-02T11:13:56.541083+01:00 librenms systemd[1]: Starting [email protected] - User Manager for UID 1000...
2025-04-02T11:13:58.287745+01:00 librenms systemd[4002210]: Queued start job for default target default.target.
2025-04-02T11:13:58.314435+01:00 librenms systemd[4002210]: Created slice app.slice - User Application Slice.
2025-04-02T11:13:58.314603+01:00 librenms systemd[4002210]: Started launchpadlib-cache-clean.timer - Clean up old files in the Launchpadlib cache.
2025-04-02T11:13:58.314685+01:00 librenms systemd[4002210]: Reached target paths.target - Paths.
2025-04-02T11:13:58.314761+01:00 librenms systemd[4002210]: Reached target timers.target - Timers.
2025-04-02T11:13:58.315738+01:00 librenms systemd[4002210]: Starting dbus.socket - D-Bus User Message Bus Socket...
2025-04-02T11:13:58.315995+01:00 librenms systemd[4002210]: Listening on dirmngr.socket - GnuPG network certificate management daemon.
2025-04-02T11:13:58.316208+01:00 librenms systemd[4002210]: Listening on gpg-agent-browser.socket - GnuPG cryptographic agent and passphrase cache (access for web browsers).
2025-04-02T11:13:58.316489+01:00 librenms systemd[4002210]: Listening on gpg-agent-extra.socket - GnuPG cryptographic agent and passphrase cache (restricted).
2025-04-02T11:13:58.317538+01:00 librenms systemd[4002210]: Starting gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation)...
2025-04-02T11:13:58.317731+01:00 librenms systemd[4002210]: Listening on gpg-agent.socket - GnuPG cryptographic agent and passphrase cache.
2025-04-02T11:13:58.317962+01:00 librenms systemd[4002210]: Listening on keyboxd.socket - GnuPG public key management service.
2025-04-02T11:13:58.318174+01:00 librenms systemd[4002210]: Listening on pk-debconf-helper.socket - debconf communication socket.
2025-04-02T11:13:58.318418+01:00 librenms systemd[4002210]: Listening on snapd.session-agent.socket - REST API socket for snapd user session agent.
2025-04-02T11:13:58.332817+01:00 librenms systemd[4002210]: Listening on dbus.socket - D-Bus User Message Bus Socket.
2025-04-02T11:13:58.348468+01:00 librenms systemd[4002210]: Listening on gpg-agent-ssh.socket - GnuPG cryptographic agent (ssh-agent emulation).
2025-04-02T11:13:58.348652+01:00 librenms systemd[4002210]: Reached target sockets.target - Sockets.
2025-04-02T11:13:58.348776+01:00 librenms systemd[4002210]: Reached target basic.target - Basic System.
2025-04-02T11:13:58.348889+01:00 librenms systemd[4002210]: Reached target default.target - Main User Target.
2025-04-02T11:13:58.349005+01:00 librenms systemd[4002210]: Startup finished in 1.766s.
2025-04-02T11:13:58.349452+01:00 librenms systemd[1]: Started [email protected] - User Manager for UID 1000.
2025-04-02T11:13:58.351973+01:00 librenms systemd[1]: Started session-3195.scope - Session 3195 of User administrator.
2025-04-02T11:14:00.142490+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:14:00.779627+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:14:00.779964+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:14:01.676694+01:00 librenms CRON[4003903]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:14:43.544884+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:15:00.041336+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:15:00.794765+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:15:00.794957+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:15:01.820748+01:00 librenms CRON[4022530]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:15:01.821217+01:00 librenms CRON[4022529]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:15:01.822541+01:00 librenms CRON[4022531]: (librenms) CMD (/opt/librenms/html/plugins/Weathermap/map-poller.php >> /dev/null 2>&1)
2025-04-02T11:15:01.823919+01:00 librenms CRON[4022532]: (librenms) CMD (   /opt/librenms/check-services.php >> /dev/null 2>&1)
2025-04-02T11:15:01.825237+01:00 librenms CRON[4022533]: (librenms) CMD (   /opt/librenms/poll-billing.php >> /dev/null 2>&1)
2025-04-02T11:15:01.825495+01:00 librenms CRON[4022534]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
2025-04-02T11:15:01.825935+01:00 librenms CRON[4022536]: (librenms) CMD (   /opt/librenms/discovery.php -h new >> /dev/null 2>&1)
2025-04-02T11:15:01.826448+01:00 librenms CRON[4022535]: (librenms) CMD (   /opt/librenms/cronic /opt/librenms/poller-wrapper.py 16)
2025-04-02T11:15:43.545282+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:15:45.850676+01:00 librenms systemd[1]: Stopping nginx.service - A high performance web server and a reverse proxy server...
2025-04-02T11:15:47.276904+01:00 librenms systemd[1]: nginx.service: Deactivated successfully.
2025-04-02T11:15:47.277223+01:00 librenms systemd[1]: Stopped nginx.service - A high performance web server and a reverse proxy server.
2025-04-02T11:15:47.277427+01:00 librenms systemd[1]: nginx.service: Consumed 13.816s CPU time, 110.9M memory peak, 0B memory swap peak.
2025-04-02T11:15:47.292840+01:00 librenms systemd[1]: Starting nginx.service - A high performance web server and a reverse proxy server...
2025-04-02T11:15:47.362464+01:00 librenms systemd[1]: Started nginx.service - A high performance web server and a reverse proxy server.
2025-04-02T11:16:00.206388+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:16:01.081786+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:16:01.082127+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:16:02.002717+01:00 librenms CRON[4046008]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:16:43.545519+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:17:00.622463+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:17:01.159848+01:00 librenms CRON[4059282]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)
2025-04-02T11:17:01.160293+01:00 librenms CRON[4059283]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:17:01.230375+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:17:01.230583+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.
2025-04-02T11:17:43.545832+01:00 librenms snmpd[1335]: systemstats_linux: unexpected header length in /proc/net/snmp. 237 != 224
2025-04-02T11:18:00.630351+01:00 librenms systemd[1]: Starting librenms-scheduler.service - LibreNMS Scheduler...
2025-04-02T11:18:01.219472+01:00 librenms CRON[4069638]: (librenms) CMD (   /opt/librenms/alerts.php >> /dev/null 2>&1)
2025-04-02T11:18:01.301042+01:00 librenms systemd[1]: librenms-scheduler.service: Deactivated successfully.
2025-04-02T11:18:01.301359+01:00 librenms systemd[1]: Finished librenms-scheduler.service - LibreNMS Scheduler.

Hi @murrant ,

Would it be a good idea to format the server and do a new implementation from scratch?
Thanks

Really depends where the issue is.

Is this a VM and have you allocated enough resources to it if so?

I have a physical server dedicated exclusively to librenms, it is a Dell server with the Ubuntu 24.04.2 LTS operating system;
Mem: 62 Gi
HDD: 1.3T
CPU: Intel(R) Xeon(R) CPU E5-2430 0 @ 2.20GHz

Decent enough spec without knowing how many devices, ports and sensors you are monitoring.

Like Murrant said though, this is more a system issue so a bit out of scope for us to help


I currently have the system this way!
Devices: 468
Ports: 28368
Services: 0
Health: 18230
Yes, I have enough capacity to make the system work normally.

Your screen shot shows an issue with downloading assets from the web server (such as css). Sometimes this is as simple as bad APP_URL or base_url.

I have the settings as defined, but it doesn’t work!

/opt/librenms/.env

APP_KEY=base64:xxxxxxxxxxxxxxxxxXXXXXXXXXXxxxxx=

DB_HOST=localhost
DB_DATABASE=librenms
DB_USERNAME=librenms
DB_PASSWORD=librenms

APP_URL=https://librenms.domain.com
NODE_ID=66d892abce3b0
VAPID_PUBLIC_KEY=XXXXXXXXXXXXXXXXXXXX
VAPID_PRIVATE_KEY=XXXXXXXXXXXXXXXXXXX

#New
SESSION_SECURE_COOKIE=true

What about lnms config:get base_url? If you need to change that you can do lnms config:set base_url https://xxxxxx

root@librenms:~# sudo su - librenms
librenms@librenms:~$ lnms config:set base_url https://librenms.domain.com

It didn’t work, everything remains the same, after the change I restarted the webserver and it remains the same

Run php artisan optimize:clear

librenms@librenms:~$ php artisan optimize:clear

   INFO  Clearing cached bootstrap files.

  events ................................................................................................................................. 22ms DONE
  views .................................................................................................................................. 70ms DONE
  cache ............................................................................................................................... 2,426ms DONE
  route .................................................................................................................................. 11ms DONE
  config .................................................................................................................................. 2ms DONE
  compiled ................................................................................................................................ 2ms DONE

librenms@librenms:~$

Thank you very much for your help!
It doesn’t work either, I’ll continue with formatting the server and installing it from scratch!
This is the second time I have had to go this route. I believe that after a certain amount of growth the system starts to show poor performance and errors, but it is difficult to get to the root of the problem and resolve it to return to normal.

That might fix the issue right now but that’s not normal behaviour unless you’re hitting resource limits.

I have a dedicated server and can add all the resources needed for the service. This shouldn’t be a problem!

Hello @murrant and @Iaf

Thank you very much for your support and patience.
I reinstalled the system and implemented Libremms on the server again. Everything went back to normal and worked perfectly.
However, after the number of devices increased, I started to experience performance errors, such as timeouts on the webserver and long waiting times for pages to respond.
This means that the librenms setup needs to be improved in terms of performance to support a greater number of devices.