Slow polling

Good day.
Sorry for my English.
Is it possible to do polling as in Observium.
Those with the help of snmpbulkwalk, we solve only ifname, ifalias, iftype, ifoperstatus, ifdescr. Then we filter the bad interfaces (~10k PPPoE interfaces with ifType=iso88022llc on my Juniper BRAS), and then with the help of snmpget we collect the counters of each good interface.
Librenms collects data module ports more than 5 minutes.
It takes about 40 seconds for the observium.

./validate.php

====================================

Component Version
LibreNMS 1.48.1-63-gae7e262
DB Schema 2019_01_16_195644_add_vrf_id_and_bgpLocalAs (131)
PHP 7.2.14-1+ubuntu16.04.1+deb.sury.org+1
MySQL 10.0.36-MariaDB-0ubuntu0.16.04.1
RRDTool 1.5.5
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.8.4
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct
[FAIL] Some devices have not completed their polling run in 5 minutes, this will create gaps in data.
[FIX]:
Check your poll log and see: http://docs.librenms.org/Support/Performance/
Devices:
192.168.249.103
[WARN] Your local git contains modified files, this could prevent automatic updates.
[FIX]:
You can fix this with ./scripts/github-remove
Modified Files:
includes/definitions/junos.yaml

cat includes/definitions/junos.yaml

os: junos
text: ‘Juniper JunOS’
type: network
over:
- { graph: device_bits, text: ‘Device Traffic’ }
- { graph: device_processor, text: ‘CPU Usage’ }
- { graph: device_mempool, text: ‘Memory Usage’ }
discovery_modules:
vrf: true
discovery:
- sysObjectID: .1.3.6.1.4.1.2636
- sysDescr:
- ‘kernel JUNOS’
- ‘kernel Junos’
bad_iftype:
- iso88022llc
bad_ifname_regexp:
- /^pp0.*/

Hi,

Im not sure where your problem is.

Also, to keep getting updates I suggest you to configure bad_ifnames/iftypes in config.php as per docs https://docs.librenms.org/Support/Configuration/#interfaces-to-be-ignored

Hi
You may also want to check selective port polling as described here (search ‘selected port polling’):

Hi
I think the problem is that librenms first caches all data, including all interface counters, and only then selects “good” or “bad”. In my case, there are> 10k interfaces on the router and this process takes a very long time. At the same time, some of the interfaces marked as “bad” are still marked as good.librenms3

Sorry, I can not attach more than one screenshot in the post

At the next survey, they are marked as ports for deletion, but other ports “break through” and further around. As a result, about 20k ports for deletion accumulated over the weekend, the disk space was over (because rrd files were created for these ports) and the system hung. For 4 hours, such interfaces have accumulated 3,5k

Per port polling in my case is not the way out, because configuration changes are constantly taking place on the router — interfaces are added or removed and a bad idea manually tracking them.
The obserium has a more thoughtful polling mechanism. First, we cache the data by which we will conclude that the interface is “bad” or “good”, and only then we query only the “good” interfaces.

But the observium is not an open source …