Discovery arp-table module in large production environment

Help
1

Hello,

I am running LibreNMS in a large production environment in a distributed poller setup.

We have several routers on campus which on which arp-tables are correctly collected.

One router however, specifically a Cisco Cat6500 running in VSS performing the vast majority of routing for our user community and as such, contains a very large ARP table, I cannot get the discovery arp-table module to collect this data

Running the arp-table discovery module in debug mode has identified that the following error is being generated

SQL Error! SQLSTATE[HY000]: General error: 1390 Prepared statement contains too many placeholders (SQL: insert into ipv4_mac ( context_name , device_id , ipv4_address , mac_address , port_id ) values

A bit of digging online suggests that there is a limitation in MariaDB of 65,535 placeholders

The output of validate.php is as follows

====================================

Component Version
LibreNMS 1.47
DB Schema 275
PHP 7.2.10-0ubuntu0.18.04.1
MySQL 10.1.34-MariaDB-0ubuntu0.18.04.1
RRDTool 1.7.0
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.8.0
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Has anyone else hit this issue and if so - does anyone have a possible workaround?

Any help would be much appreciated!

Thanks
Adam

2

Need to add array_chunk at line 104 of /opt/librenms/includes/discovery/arp-table.inc.php:

// add new entries
if (!empty($insert_data)) {
    $insert_data_chunks = array_chunk($insert_data, 1000, true);
    foreach ($insert_data_chunks as $insert_data_chunk) {
        dbBulkInsert($insert_data_chunk, 'ipv4_mac');
    }
}

// remove stale entries

BTW, on my Cat6500 polling ARP over SNMP = Cisco CPU to 99% :roll_eyes:

3

Nice work, If you send a pull request we’ll merge it upstream.

Not much we can do about the Cisco CPU :slight_smile: Luckily, that usually doesn’t impact traffic.