Unable to delete devices. Nginx Timeout or PHP memory limit error

Hello,

I am trying to delete some devices but I get a 504 gateway timeout nginx error when I try by webui

and If i try by delhost.php I get this error:

PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 8388616 bytes) in /opt/librenms/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Concerns/HasRelationships.php on line 632

In HasRelationships.php line 632:

Allowed memory size of 536870912 bytes exhausted (tried to allocate 8388616 bytes)

Php memory limit was sert to 128MB by default, I never changed this value but I am still getting this error changing this value to 512MB.

I have updated to the latest version and still getting this error.

–Validate Output:

./validate.php

Component Version
LibreNMS 22.7.0-68-ga40e1d5
DB Schema 2022_07_19_081224_plugins_unique_index (244)
PHP 7.3.33
Python 3.6.8
Database MariaDB 10.5.13-MariaDB
RRDTool 1.4.8
SNMP 5.7.2
====================================

[OK] Composer Version: 2.3.7
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database Schema is current
[OK] SQL Server meets minimum requirements
[OK] lower_case_table_names is enabled
[OK] MySQL engine is optimal
[OK]
[OK] Database schema correct
[OK] MySQl and PHP time match
[OK] Active pollers found
[OK] Dispatcher Service not detected
[OK] Locks are functional
[OK] Python poller wrapper is polling
[OK] Redis is unavailable
[OK] rrdtool version ok
[OK] Connected to rrdcached

Any clue? Could be related to some data missing in the database about this devices?
Does anyone knows how to remove this devices manually from the database?

Thanks in advance.

Something with that particular device. I would temporarily bump the memory up to 2G or something and see if it works then.

it looks that it gets stucked for hours without any output

/opt/librenms/delhost.php ssw27240006.domain.es

top
top - 22:31:08 up 1 day, 3:00, 1 user, load average: 7.71, 7.88, 7.97
Tasks: 129 total, 6 running, 123 sleeping, 0 stopped, 0 zombie
%Cpu(s): 50.9 us, 24.1 sy, 0.0 ni, 23.2 id, 0.9 wa, 0.0 hi, 0.9 si, 0.0 st
KiB Mem : 12137132 total, 269944 free, 5854252 used, 6012936 buff/cache
KiB Swap: 1048572 total, 1046524 free, 2048 used. 5919572 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
16052 librenms 20 0 2126368 1.6g 11632 R 96.0 14.2 239:12.98 php
27354 librenms 20 0 880804 424412 12300 S 10.6 3.5 9:32.33 php
10973 librenms 20 0 428952 41556 11960 R 9.6 0.3 0:00.29 php
11137 librenms 20 0 428696 40612 11776 S 9.3 0.3 0:00.28 php
11084 librenms 20 0 428696 40604 11776 S 8.6 0.3 0:00.26 php


I have deleted other devices without any problem but there are about 7 devices that i can not remove.

Is there any option to delete them directly on database?

I’m guessing there is a circular dependency issue.
Something points to each other and it tries to resolve it forever :slight_smile:

1 Like

Could be it, remove all device dependencies first.

I had similar issue in my environment with 2000+ devices
i used a dirty fix by adding below time out setting in nginx’s librenms.conf

fastcgi_read_timeout 240;

There are no dependencies for this devices:

I will try but it also fails running from command line and with php interpreter. I think this way does not use nginx, isn´t it?

Configured like these:

server {
listen 80;
server_name librenms.domain.es;
root /opt/librenms/html;
index index.php;
access_log /opt/librenms/logs/access_log;
error_log /opt/librenms/logs/error_log;
location / {
try_files $uri $uri/ @librenms;
}
location ~ .php {
fastcgi_param PATH_INFO $fastcgi_path_info;
include fastcgi.conf;
fastcgi_split_path_info ^(.+.php)(/.+)$;
fastcgi_pass unix:/var/run/php-fpm/php7.2-fpm.sock;
fastcgi_read_timeout 240;
}
location ~ /.ht {
deny all;
}
location @librenms {
rewrite api/v0(.*)$ /api_v0.php/$1 last;
rewrite ^(.+)$ /index.php/$1 last;
}
}

And same output:

yes, if running from cli, does not use nginx

Try commenting this line and try again;

it worked!!!

[[email protected] Observers]$ /opt/librenms/delhost.php ssw27240001.domain.es
Removed device ssw27240001.domain.es

Do i have to run anythinng else? or just uncoment once all the problematic devices have been removed…

Thanks for all

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.