Random 504 Gateway Time-out nginx/1.18.0 (Ubuntu)

Help
1

Hi
I am randomly getting this error.

504 Gateway Time-out
nginx/1.18.0 (Ubuntu)

the server is adequatlely sized with 12cores and 12gbram.
both of these parameters stay in 50-60% of usage when the problem occurs.

though i have set 1 minute polling on the devices. these are hardly 20 devices.

2

You need to check the error log for nginx, typically in /var/log/nginx/

3

/var/log$ sudo tail php8.3-fpm.log

[01-Dec-2024 21:01:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:27:03] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:27:47] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:37:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:40:01] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:45:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:47:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[01-Dec-2024 21:50:01] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[02-Dec-2024 04:02:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
[02-Dec-2024 04:04:46] WARNING: [pool librenms] server reached pm.max_children setting (5), consider raising it
4

tail /var/log/nginx/error.log


2024/12/02 15:17:44 [error] 2438990#2438990: *238 upstream timed out (110: Unknown error) while reading response header from upstream, client: 10.20.5.77, server: 10.20.7.140, request: "GET /graph.php?from=-30d&legend=yes&text=&id=122&type=port_bits&absolute_size=0&width=300&height=150 HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm-librenms.sock", host: "10.20.7.140", referrer: "http://10.20.7.140/dashboard/1"
5

You need to fix that.