Adding/Editing Devices Error 502

Help
1

I upgraded to PHP v7.3.2, Since this update when I try to add a new device through web interface or update the snmp settings on an existing device I get a TimeOut 502 error page that comes up. The device is never added or updated.

Could this be because Im using php 7.3 instead of the recommended 7.2?

image

2

Unlikely related to version. Look for a related error in librenms.log and post it here.

3

In lebrenms.log Im seeing the below over and over

> [2019-02-21 13:08:33] production.ERROR: SQLSTATE[42000]: Syntax erro or access violation: 1064 You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'L:\\ Label:Logs Serial Number 906bf7dc"" AND storage.storage_descr != ""/vmfs/vo' at line 1 (SQL: SELECT * FROM devices,storage WHERE (devices.device_id = 70 AND devices.device_id = storage.device_id) AND devices.type = "Server" AND storage.storage_descr NOT LIKE '%memory%' AND storage.storage_perc >= 90 AND storage.storage_descr != ""L:\\ Label:Logs Serial Number 906bf7dc"" AND storage.storage_descr != ""/vmfs/volumes/55916302-0f9fa955-4b78-8cdcd4ae1b40"") (SQL: SELECT * FROM devices,storage WHERE (devices.device_id = 70 AND devices.device_id = storage.device_id) AND devices.type = "Server" AND storage.storage_descr NOT LIKE '%memory%' AND storage.storage_perc >= 90 AND storage.storage_descr != ""L:\\ Label:Logs Serial Number 906bf7dc"" AND storage.storage_descr != ""/vmfs/volumes/55916302-0f9fa955-4b78-8cdcd4ae1b40"")
>     /opt/librenms/includes/alerts.inc.php:186
>     /opt/librenms/poller.php:162
>     /opt/librenms/poller.php 70 2019-02-21 13:08:33 - 1 devices polled in 9.854 secs

And in the error_log Im seeing this when the 504 error page comes up

>  2019/02/21 12:51:26 [crit] 1297#1297: *95838 open() "/var/lib/nginx/fastcgi/0/01/0000000010" failed (13: Permission denied) while reading upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /ajax/table/eventlog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "SERVERNAME", referrer: "http://SERVERNAME/eventlog/"
>         2019/02/21 12:51:33 [crit] 1297#1297: *95830 open() "/var/lib/nginx/fastcgi/1/01/0000000011" failed (13: Permission denied) while reading upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /ajax/table/eventlog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "SERVERNAME", referrer: "http://SERVERNAME/eventlog/"
>     2019/02/21 12:51:44 [error] 1297#1297: *95840 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /opt/librenms/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php on line 158PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 69632 bytes) in /opt/librenms/vendor/composer/ClassLoader.php on line 444" while reading response header from upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /ajax/table/eventlog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "SERVERNAME", referrer: "http://SERVERNAME/eventlog/"
>     2019/02/21 12:51:50 [crit] 1297#1297: *95840 open() "/var/lib/nginx/fastcgi/2/01/0000000012" failed (13: Permission denied) while reading upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /ajax/table/eventlog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "SERVERNAME", referrer: "http://SERVERNAME/eventlog/"
>     2019/02/21 12:57:02 [crit] 1297#1297: *96304 open() "/var/lib/nginx/fastcgi/3/01/0000000013" failed (13: Permission denied) while reading upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /ajax/table/eventlog HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock:", host: "SERVERNAME", referrer: "http://SERVERNAME/eventlog/"
>     2019/02/21 13:07:44 [error] 1294#1294: *96332 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 10.6.5.153, server: SERVERNAME, request: "POST /addhost/ HTTP/1.1", upstream: "fastcgi://unix:/var/run/php/php7.3-fpm.sock", host: "SERVERNAME", referrer: "http://SERVERNAME/addhost/"

The web interface eventlog doesnt seem to show anything excpet events on the monitored devices.

This second line is 186 from alerts.inc.php

$sql = $rule['query'];
$qry = dbFetchRows($sql, array($device_id));
$cnt = count($qry);
for ($i = 0; $i < $cnt; $i++) {
    if (isset($qry[$i]['ip'])) {
        $qry[$i]['ip'] = inet6_ntop($qry[$i]['ip']);
    }
}
4

#1 is an invalid alert rule
#2 says you are out of php memory

5

Below is GUI for the alert rule, what would make it invalid. Because there are 2 storage.storage_descr does it require OR instead on AND?

image
image800×801 27.9 KB

As for the PHP memory do I need to add RAM or is there a limit I need to set somewhere?

6

The quotes around the descr values…

It is a php setting in php.ini

7

That resolved those errors, and I can update devices now. Now I’m seeing this on the error log
Looks like my ipaddress is getting permission denied for somthing. I am the Admin though lvl 10.

2019/02/21 14:38:34 [error] 1291#1291: *96665 upstream timed out (110: Connection timed out) while reading response header from upstream, client: MYIPADDRESS, server: SERVERNAME, request: “POST /addhost/ HTTP/1.1”, upstream: “fastcgi://unix:/var/run/php/php7.3-fpm.sock”, host: “SERVERNAME”, referrer: “http://SERVERNAME/addhost/”

8

The connection timeout is nginx trying to connect to php-fpm I believe.

Check browser developer console for errors and librenms.log again.

9

That was it thanks for your help.