Intermittently slow, gaps in graphs, any more optimizations?

Help
1

Hello

I’m now monitoring 185 devices, 50,000 ports, I’m using the default 16 threads for polling., and my LibreNMS has started struggling, Gaps on graphs and intermittent slow behaviour.

image

I’ve followed the performance guide here Home - LibreNMS Docs , but unfortunately I haven’t seen any improvements.

Does anyone have some more suggestions? Should i offload mysql to another server?

Here are some htop’s :

note Poller CPUs

image
image1597×308 29.9 KB

note mysql CPUs

image
image1587×642 69.9 KB

Thanks in advance

2

where able to get RRD caching working?

3

also you could play with this a little see what your hardware can handle.

4

Yeah should tell us what you have tried in the performance doc.

5

Thanks guys

Yes I’m using RRDCached, It had already come with this docker container https://hub.docker.com/r/jarischaefer/docker-librenms/ which I’m using.

I can increase the thread count from 16, but all my 8 cores were maxing out already intermittently, i think this is what caused the gaps. I could decrease, do you think this could cause problems?

For now I’ve turned off Sensor polling with $config[‘poller_modules’][‘sensors’] = 0; , and it’s made a world of difference, no gaps anymore, and much faster. Not sure if theres a way i can keep temperature for the core chasis, instead of every port. Also monitoring light loss only for certain ports, not sure if that’s possible?

6

For now i’ve tried

RRDCached
innodb_flush_log_at_trx_commit = 0
$config[‘poller_modules’][‘ospf’] = 0;

Then i tried $config[‘poller_modules’][‘sensors’] = 0; and that fixed the problem for now.

I’m not sure what SNMP Max Repeaters does, If I have a switch with lots of ports, how should i start to work with this?

SNMP Max OIDs - Do i still need to look at this? here are my polling performance stats:

image
image1511×449 15.9 KB

Thanks

7

what are the hardware specs you are running librenms on?

1 Like
8

I would look at max oids as it can help speed things up. If you’ve got a lot of ports then enable per port polling.

As for specific sensors, it’s not possible I’m afraid. However now you’ve disabled the sensors module you should enable it on certain devices if that’s what you need.

That screenshot is odd, I’m assuming you only have one install and that the poller name changes when you re-launch the docker image. If so, delete all the pollers from the pollers table. See what that says after a full poller run, i.e how long it took.

1 Like