502 Bad Gateway connect() to unix:/run/php-fpm-librenms.sock failed

Woke up this morning to a 502 Bad Gateway for the LibreNMS web ui. Restarted the server, nothing. Absolutely nothing was changed on the server before this started happening. I restored back to a snapshot from a few days ago when the web UI was working - same issue. I’m stumped.

/var/log/nginx/error.log

2025/05/29 17:18:48 [crit] 1351#1351: *3 connect() to unix:/run/php-fpm-librenms.sock failed (2: No such file or directory) while connecting to upstream, client: 10.x.x.x, server: librenms.x.x, request: "GET /alert-schedule HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock:", host: "librenms.x.x", referrer: "https://librenms.x.x/overview?dashboard=16"

systemctl status php8.3-fpm (same error after bouncing the service)

ERROR: unable to bind listening socket for address '/run/php-fpm-librenms.sock': Permission denied (13)
ERROR: FPM initialization failed
php8.3-fpm.service: Main process exited, code=exited, status=78/CONFIG
php8.3-fpm.service: Failed with result 'exit-code'.
Failed to start php8.3-fpm.service - The PHP 8.3 FastCGI Process Manager.

./validate.php

===========================================
Component | Version
--------- | -------
LibreNMS  | 25.5.0-60-g105923b65 (2025-05-29T10:33:49-04:00)
DB Schema | 2025_05_25_183627_drop_view_port_mac_link (341)
PHP       | 8.3.6
Python    | 3.12.3
Database  | MariaDB 10.11.11-MariaDB-0ubuntu0.24.04.2
RRDTool   | 1.7.2
SNMP      | 5.9.4.pre2
===========================================

[OK]    Composer Version: 2.8.9
[OK]    Dependencies up-to-date.
[OK]    Database Connected
[OK]    Database Schema is current
[OK]    SQL Server meets minimum requirements
[OK]    lower_case_table_names is enabled
[OK]    MySQL engine is optimal
[OK]    Database and column collations are correct
[OK]    Database schema correct
[OK]    MySQL and PHP time match
[OK]    Active pollers found
[OK]    Dispatcher Service not detected
[OK]    Locks are functional
[OK]    Python poller wrapper is polling
[OK]    Redis is unavailable
[OK]    rrdtool version ok
[OK]    Connected to rrdcached

Something has changed to cause that and it’s not from a LibreNMS update :slight_smile:

Check your php-fpm pools, ideally you just have the librenms.conf one if that’s all you run on the server. Check the config against the install docs.

Yup, I only have a single librenms.conf file in my pool. Here’s my config file (just the pieces from the install docs). I haven’t touched this file since the initial install.

/etc/php/8.3/fpm/pool.d/librenms.conf

user = librenms
group = librenms


listen = /run/php-fpm-librenms.sock

listen.owner = www-data
listen.group = www-data
;listen.mode = 0660

Check the sock file:

ERROR: unable to bind listening socket for address '/run/php-fpm-librenms.sock': Permission denied (13)

It shouldn’t exist if php-fpm isn’t running.

Correct, there is no sock file. I tried to create my own using touch, and chown librenms:librenms and chmod 777, same issue. Also tried pointing config files to the /run/php/ subfolder, then chown librenms:librenms chmod that folder instead. No dice.

try and run it manually as root:

/usr/sbin/php-fpm8.3 --nodaemonize --fpm-config /etc/php/8.3/fpm/php-fpm.conf

sudo /usr/sbin/php-fpm8.3 --nodaemonize --fpm-config /etc/php/8.3/fpm/pool.d/librenms.conf

MIB search path: /root/.snmp/mibs:/usr/share/snmp/mibs:/usr/share/snmp/mibs/iana:/usr/share/snmp/mibs/ietf
Cannot find module (SNMPv2-MIB): At line 0 in (none)
Cannot find module (IF-MIB): At line 0 in (none)
Cannot find module (IP-MIB): At line 0 in (none)
Cannot find module (TCP-MIB): At line 0 in (none)
Cannot find module (UDP-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-MIB): At line 0 in (none)
Cannot find module (NOTIFICATION-LOG-MIB): At line 0 in (none)
Cannot find module (DISMAN-EVENT-MIB): At line 0 in (none)
Cannot find module (DISMAN-SCHEDULE-MIB): At line 0 in (none)
Cannot find module (HOST-RESOURCES-TYPES): At line 0 in (none)
Cannot find module (MTA-MIB): At line 0 in (none)
Cannot find module (NETWORK-SERVICES-MIB): At line 0 in (none)
Cannot find module (UCD-DISKIO-MIB): At line 0 in (none)
Cannot find module (UCD-DLMOD-MIB): At line 0 in (none)
Cannot find module (LM-SENSORS-MIB): At line 0 in (none)
Cannot find module (UCD-SNMP-MIB): At line 0 in (none)
Cannot find module (UCD-DEMO-MIB): At line 0 in (none)
Cannot find module (SNMP-TARGET-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-AGENT-MIB): At line 0 in (none)
Cannot find module (SNMP-FRAMEWORK-MIB): At line 0 in (none)
Cannot find module (SNMP-MPD-MIB): At line 0 in (none)
Cannot find module (SNMP-USER-BASED-SM-MIB): At line 0 in (none)
Cannot find module (SNMP-VIEW-BASED-ACM-MIB): At line 0 in (none)
Cannot find module (SNMP-COMMUNITY-MIB): At line 0 in (none)
Cannot find module (IPV6-ICMP-MIB): At line 0 in (none)
Cannot find module (IPV6-MIB): At line 0 in (none)
Cannot find module (IPV6-TCP-MIB): At line 0 in (none)
Cannot find module (IPV6-UDP-MIB): At line 0 in (none)
Cannot find module (IP-FORWARD-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-PASS-MIB): At line 0 in (none)
Cannot find module (NET-SNMP-EXTEND-MIB): At line 0 in (none)
Cannot find module (SNMP-NOTIFICATION-MIB): At line 0 in (none)
Cannot find module (SNMPv2-TM): At line 0 in (none)
Cannot find module (NET-SNMP-VACM-MIB): At line 0 in (none)
[30-May-2025 12:21:23] ERROR: unable to bind listening socket for address '/run/php-fpm-librenms.sock': Permission denied (13)
[30-May-2025 12:21:23] ERROR: FPM initialization failed

AppArmour / SELinux in the way?

You also look to have something set in /etc/snmp/snmp.conf, that should be empty really or just contain mibs:

1 Like

Ugh, it was indeed AppArmor. I put PHP in complain mode, restarted the service, and the web ui came back instantly.

sudo aa-complain /usr/sbin/php-fpm8.3

I’m running Ubuntu 24.04.2 LTS, perhaps there was a critical update pushed and applied overnight. If anyone else comes across this issue, try the above command and restart php.

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.