Nice , data.
I will also test and share my results.
Thanks.
I had a similar issue,
In my case I had too many devices for the host to handle during polling and it appeared as though it was start polling again before finishing the current polling.
To alleviate this, I set up distributed polling onto a bunch of Raspberry Pis (I had a surplus) and then assigned devices (by site) to these, this can also be achieved with Docker containers
Hi Leonardo,
First you might want to check those suspected devices rather than LNMS for load and other performance issues that might cause them to respond to the poller slower than before.
I would also go with Jellyfrog’s suggestion above and adjust the poller threads. You might wanna do that few times until you reach a good balance.
I would also go with Adam’s suggestion on creating distributed pollers closer to the problematic machines (if they are in a different location that requires longer path/latency). Then assign your problematic machines to be polled by those distributed pollers.
As you can see above, our master poller is polling 240+ devices in our facility in under 100 sec. While our distributed pollers, physically located in a remote office, are polling 40+ devices in that remote office in under 200 sec. So the 5 minutes poll cycle is met in both cases.
Before setting up the distributed pollers, I had really issues polling the remote devices and caused a lot of gaps as the poller cycles were not able to finish, and might overlap, causing other load and memory issues in LNMS.
Best Regards
Hi @kalamchi75 , glad you write to me.
I toke all your suggestions and improved my graphs a lot.
I have never paid attention to poller section / performance before , now I understand it impotance.
About grouping problematic machines … this is my next goal.
In fact I posted a forum questiong regarding poller groups.
I just need to create 2 o 3 pollers on same server and asign devices according to specific criteria.
Then I would like to set poller time interval on those pollers.
Can you provide some example / configs for that ?
Hi Leandro,
I am not sure how you would create pollers on the same server itself.
In my case, I have my remotes pollers physically located at the remote office. They poll the remote devices there and send them to the master.
Below is the configuration of the distributed on the master server:
# Enable Distributed Pollers
$config['distributed_poller'] = true;
$config['rrdcached'] = "localhost:42217";
$config['distributed_poller_memcached_host'] = 'localhost';
$config['distributed_poller_memcached_port'] = '11211';
And below are the bits of configuration required on the remote poller:
1- You need to point the database connection to your master poller DB. Make sure you allow the privileges for the DB user if the connection is coming from a different IP address (assuming the remote poller is on a different server with a different IP address in my case):
$config['db_host'] = 'librenms-master.xxx.xxx';
$config['db_port'] = '3306';
$config['db_user'] = 'librenms';
$config['db_pass'] = 'xxxxxxxxxx';
$config['db_name'] = 'librenms';
Now, the distributed poller config lines:
$config['distributed_poller_name'] = 'librenms-poller01';
$config['distributed_poller_group'] = '2';
$config['distributed_poller_memcached_host'] = "librenms-master.xxx.xxx";
$config['distributed_poller_memcached_port'] = 11211;
$config['distributed_poller'] = true;
$config['rrdcached'] = "librenms-master.xxx.xxx:42217";
$config['update'] = 0;
You might also want to install rrdcached and memcached on the master poller as it is required if you want to use distributed pollers. As you can see above, you will need to point your distributed poller to the memcached and rrdcached host/ports
Once this is done, the distributed poller should pop automatically in the GUI.
Now, notice the line:
$config['distributed_poller_group'] = '2';
This is the group number (ID) that your distributed poller(s) are assigned to. It can be any number. You will use this number to create your poller groups in the GUI. See below, ID 2
So if you have more that one distributed poller, you can group them by configuring the same number. or you can split them into multiple groups by assigning different numbers.
The last thing is that you specify which Poller group would poll a device in the device’s config in the GUI.
I hope this was helpful 
Best Regards
@Leandro_Roggerone
Polling interval cannot be changed within a LibreNMS deployment, even on multiple pollers. This is not possible without adding/changing a lot of code.
Of course, pathes are accepted 
Bye
thank you @kalamchi75 .
I apreciate your detailed answer.
However this is not what I want to accomplish now.
For the moment I have a single device architecture.
Hope it helps to someone trying to create a distributed scheme.
Leandro.
@PipoCanaja :
Understood about having different polling intervals, it is not possible currently.
So, is it good idea to isolate heavy loaded devices into separeted pollers ?
Im talkin about creating multiple pollers on same machine.
At least if this poller fails or , device within it takes too long to respond , It would not break / create gaps in other device´s graphs.
Does it make sense ?
Regards.
Leandro.
The polling time depends on the quantity and nature of the device you are polling.
In my case I have 2 very problematic devices.
Huawei main core, with a lot of interfaces , routes and bgp peers.
And.
Huawei OLT , with a lot of interface (it creates one per gpon interface), processors , health , and so …
I would like to handle them in a different way from the rest of the devices, thats why I wan to create poller groups, but allwais on same machine.
Leandro.
You are most welcome, Leandro.
Unfortunately I haven’t setup multiple pollers on the same server, not sure how is this achieved.
Are using a VM or a bare-metal server for your LNMS ?
If you are using a VM, then I would recommend using a second VM as a distributed poller. It might make your deployment easier since you don’t have to fiddle with multi-poller single-sever situation.
Just a thought !
Best Regards
@kalamchi75 , my lnms is running on VM (under proxmox).
So yes , I can try creating a second poller on a new vm , that is a great idea.
I will try a little bit more creating one or two more pollers on same machine.
If can not get it working I will follow your advice.
Leandro.
Good luck.
If you manage to make two separate pollers work on the same server, kindly share the procedure.
Best regards
of course , I will:
There is an open post :
here
I will also try to make it work on a testing enviroment.
Regards.
Hi @Leandro_Roggerone
Yes, having multiple pollers will ensure that if one poller is failing, it won’t impact devices that are handled by the other.
For that purpose, the pollers must run in different VMs. Cause if they are on same, and if one of the pollers is taking more than $polling-interval to do its job, it will start killing the machine sooner or later, which will impact the other poller running on the same VM.
So I would not even spend time on trying to run the pollers on the same VM.
@Leandro_Roggerone I just noticed that i’ve been calling you Leonardo instead of Leandro the whole time.
Sorry man. 
I should learn to use the @ more often hhh.
All the best
jaja … happens all the time.
@PipoCanaja : thanks for your feedback.
It is very clear about isolaing devices on different pollers can prevent to crash all graphs.
but … I dont totally agree about not running multiple pollers on same machine.
Because:
Distrubuting pollers on diferent machines is not allways possible and it difficults the maintenace.
In my opinion a middle solution should be possible … meaning that a multi-thread scheme should handle multiple pollers.
But.
Thats my opinion.
Last:
Lets supose I deside to grow my platform.
I will begin adding devices.
What polling time / when do you think , I should consider creating an extra pooller ?
Thanks!!!
Leandro
Having both poller instances on the same machine does not bring a single advantage over one poller instance, because the poller instance is anyway a wrapper that calls the real actions in a multithreaded way, already.
So “one poller instance” is in fact “one wrapper process calling multiple PHP pollers in // to do the job”. Just run “top” to get convinced and have a look at /etc/cron.d/librenms to see how this wrapper is configured (and you can increase/decrease the number of threads).
So the only usefull case is to run 2 pollers on different hardware, and adapt the number of // threads they generate according to the number of devices they have to poll, the polling interval, the latency between poller and devices, the number of ports etc etc…
There is no answer to your “growing” question. It just depends on so many parameters that we cannot tell in advance. What you can do is monitor the polling time of each of your poller instances and compare it to your polling interval. As long as it remains below a reasonnable amount (let say 70% of your polling interval) and remains stable (so does not grow more than 80% peak ever), you should be safe without a new poller instance. To achieve this, before creating a new poller instance, you can add VCPUs (if Virtual machine), increase RAM, tweak your DB and RRDCached, run on SSD drives instead of HDD, etc etc etc.
1 Like
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.