Distributed polling

Trying one last time here, even though I don’t know how active this forum is. My previous two questions did not get any replies at all so maybe I’m at the wrong place, asking for help/support?

Anyway, I have questions regarding distributed polling. At the moment, I have approximately 2,500 devices and two pollers (same poller group). They tell me that during the polling interval, they service around 1,600 - 1,700 devices (each) and I wonder how that could be, considering I only have 2,500 devices.

Each poller is set to 64 workers for polling and consumed/maximum is at around 10,000/19,200 for both (4 vCPU, 4 GB RAM). During most intervals, no devices can be seen in the pending state but every now and then I have 15-50 devices there, even though none of the poller actually uses all of its resources. I have set max connections on the MariaDB (separate server with 8 vCPU and 8 GB RAM) to 256 instead of the default 151.

You are maybe double polling some. What is your group setup? none?
Also, is memcache/redis working? (depending on which dist polling you are using)

I have two pollers and currently two groups but both pollers poll from both groups. I just haven’t yet merged all devices from before I started with distributed polling (still learning the product). Redis is working as far as I know, would polling even work if Redis didn’t? Memcached should also work as it should, being configured on both pollers and I have not seen any errors anywhere. How can I see that it works as expected?

An interesting part is that the numbers are not consistent. Today I’ve seen numbers between 2706 to 2807 devices actioned (I have currently 2551 devices) quite evenly distributed between the two pollers.

Forgot to mention that I have removed all but one IP to poll where switches have multiple interfaces, did that from the beginning.

You don’t use both redis and memcached only one or the other… also you didn’t say what type of distributed poller you are using, poller-wrapper.py or librenms-service.py…

I see. Well, I used poller-wrapper.py before so that’s why memcached is still there. I’m using the service.