Sluggish web-ui and polling slowed down

[adammmmm]

librenms@x:~$ ./validate.php

Component	Version
LibreNMS	1.43-61-gc222d0e
DB Schema	267
PHP	7.0.30-0ubuntu0.16.04.1
MySQL	10.0.36-MariaDB-0ubuntu0.16.04.1
RRDTool	1.5.5
SNMP	NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Since a day back the web-gui is extremely sluggish, every time I click myself to a new page the librenms.log is filled with thousands (14000 in one second) of logs like:

[2018-09-12 14:45:33] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-12 14:45:33] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given
[2018-09-12 14:45:33] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-12 14:45:33] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given
[2018-09-12 14:45:33] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-13 09:54:30] production.ERROR: 8 /opt/librenms/html/pages/front/tiles.php:35 Undefined offset: 1
[2018-09-13 09:54:30] production.ERROR: 8 /opt/librenms/html/pages/front/tiles.php:60 Undefined index: dashboard

Poller cpu increased about 10% since this, and it’s having trouble finishing its device polls.

[Jarvis_Meier]

Same problem here. Same errors in the log and I’m now getting un-polled device messages on the webui. The polling is finishing it’s just taking a lot linger to complete.

[dbray925]

You are not alone. Seeing same thing in our /var/log/messages. Thousands of:

httpd-scl-wrapper: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
httpd-scl-wrapper: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given

[bwilli]

Seeing similar issues here. Tons of these messages in the librenms.log file:

[2018-09-13 16:07:54] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-13 16:07:54] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given

Have tried solutions offered in the following but no luck yet:

• Backend broken, update->"whoops,looks like sth went wrong.Chek...."
• Poller process hangs (albeit slightly different issue from mine)

Far as I can tell, the problem began with the update which switched to laravel. Here’s my ./validate.php output:

====================================
Component | Version
--------- | -------
LibreNMS  | 1.43-61-gc222d0e
DB Schema | 267
PHP       | 7.0.31
MySQL     | 5.5.60-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.7.2
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct

[titanismo]

Same situation at my installation. After upgrade UI is not working at all. Show “Loading messages” at almost all pages and full of following log messages:

[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: from
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/pages/devices.inc.php:76 Undefined index: to

[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:163 ksort() expects parameter 1 to be array, null given
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 Illegal offset type
[2018-09-14 09:00:30] production.ERROR: 2 /opt/librenms/LibreNMS/Component.php:162 ksort() expects parameter 1 to be array, null given

[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/legacy_index.php:113 Undefined index: custom_css
[2018-09-14 09:00:30] production.ERROR: 8 /opt/librenms/html/legacy_index.php:271 Undefined variable: no_refresh

root@librenms:/opt/librenms# ./validate.php

Component	Version
LibreNMS	1.43-70-g9aa429c8d
DB Schema	267
PHP	7.2.7-0ubuntu0.18.04.2
MySQL	10.1.34-MariaDB-0ubuntu0.18.04.1
RRDTool	1.7.0
SNMP	NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

[David_Waldrop]

I am having similar issues. Keep getting the popup that says poller hasn’t completed in 5 minutes, tons of errors in the librenms.log especially when moving around the console. Started Tuesday. I built out a brand new install on Ubuntu 18.04 and it is doing the exact same thing. Could it be the change that was set to happen on the 11th?

[laf]

You should all update and try again

[titanismo]

Just updated, but situations stay the same:

root@librenms:/opt/librenms# ./validate.php

Component	Version
LibreNMS	1.43-74-g4bd1f6711
DB Schema	267
PHP	7.2.7-0ubuntu0.18.04.2
MySQL	10.1.34-MariaDB-0ubuntu0.18.04.1
RRDTool	1.7.0
SNMP	NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

[David_Waldrop]

I have updated several times manually with no resolution to the problem.

[awaum]

I had exactly the same issue.

At least I can confirm that these huge bloating of librenms.log is no longer occuring on my system.

====================================

Component	Version
LibreNMS	1.43-82-gcf31776
DB Schema	267
PHP	7.0.31
MySQL	5.5.60-MariaDB
RRDTool	1.4.8
SNMP	NET-SNMP 5.7.2
====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Thank you!
-awaum

[titanismo]

What did you do to solve your problem ?

[adammmmm]

The latest update, probably the “Component Errors” one, seemed to have done the trick.

[awaum]

Nothing by myself… ./daily.sh fixed it throughout the weekend…

[titanismo]

I can confirm that errors in logs are missing with latest update, but UI still show me “Problem with backend”.
I can not find errors in any log, but this messages is still there. Also tried to clear the case and different browser but the same.

Could someone give me a hint how to solve the problem?

root@librenms:/opt/librenms# ./validate.php

Component	Version
LibreNMS	1.43-82-gcf31776a6
DB Schema	267
PHP	7.2.7-0ubuntu0.18.04.2
MySQL	10.1.34-MariaDB-0ubuntu0.18.04.1
RRDTool	1.7.0
SNMP	NET-SNMP 5.7.3
====================================

[OK] Composer Version: 1.7.2
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

[titanismo]

I found the problem in my installation. Now everything is working as expected.
Thanks to all!

[bwilli]

I can confirm that the Component.php ksort() errors are gone. Thanks!

====================================
Component | Version
--------- | -------
LibreNMS  | 1.43-82-gcf31776
DB Schema | 267
PHP       | 7.0.31
MySQL     | 5.5.60-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.7.2
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct

[mattie47]

What was your issue you fixed?

[titanismo]

Initially when I tried to troubleshoot the problem I was added “#$config[‘secure_cookies’] = “true”;”
After removed it everything goes fine.

[laf]

Well that’s a commented out piece of code so with or without it, nothing would have changed.

[titanismo]

Actually was copy at the moment when was already commented. Before that it wasn’t commented.
Sorry for misunderstanding.