Route db file is 130G big with cisco devices

Hi !

We monitor aur Cisco Backone devices in our librenms !

The route db file ist extremly big 130GB

We have seen that there are millions of ospf lines in this database avery 5 min new lines !

We think that this is a bug ?!

Which further information we need to solve this problem ?

====================================

Component Version
LibreNMS 1.57-39-gb4fc84888
DB Schema 2019_10_03_211702_serialize_config (146)
PHP 7.2.24
MySQL 5.7.27
RRDTool 1.7.2
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.9.1
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Thanks

kind regards

kristoferus75

OSPF routes are fetched during the polling process so if you have very big ospf clouds this can happen. Other routing info is done during the discovery instead of polling.

Not sure if you can set the data to be purged better or adjust the ospf polling separately.

Hi Elias !

This is a big problem and this could not be normal behavour !

is not only ospf also bgp

    592 |              | 0.0.0.0      | 24             | 12            | 195.202.133.57  | indirect    | ospf         |   1571159380 |             |

| 592 | | 0.0.0.0 | 24 | 12 | 195.202.133.57 | indirect | ospf | 1571180990 | |
| 592 | | 0.0.0.0 | 24 | 12 | 195.202.133.57 | indirect | ospf | 1571202570 | |

| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572286208 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572307804 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572329408 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572350993 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572372643 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572394236 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572415844 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572437442 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572459051 | |
| 592 | | 1.0.136.0 | 0 | 20 | 195.202.156.3 | indirect | bgp | 1572480638 | |

BGP peers are within the poller reach but BGP routes should only be pulled during discovery, are you running the discovery process with the default 6h interval? If you look at your global config in the gui, what value do you have for route_purge?

I do remember someone else having the same issue with the table growing like crazy on cisco devices, I have only non-cisco devices and my routes table is 1.5MB so I don’t see this issue. OSPF and BGP are both enabled.

Hi Elias !

There is no route_purge entry in my global in the gui ?!

Sould a add it in the config.php ?

Thanks

kristoferus75

I can see only this purge entries:

https://docs.librenms.org/Support/Cleanup-options/

$config[‘syslog_purge’] = 30;
$config[‘eventlog_purge’] = 30;
$config[‘authlog_purge’] = 30;
$config[‘perf_times_purge’] = 30;
$config[‘device_perf_purge’] = 7;
$config[‘rrd_purge’] = 90;// Not set by default
$config[‘ports_purge’] = true;// Set to false by default

You can indeed try to add it manually:
$config[‘route_purge’] = 10;

Hi ! I already tested this but no effect after i start dail.sh script ! Could it be possible that this feature is very new ? because we doesnt update our librenms daily ?! Thanks kristoferus75

Hmmm that could also be, I’m on the daily release tree so not sure when it could’ve been introduced.

The route table discovery feature is less than one month old. And the route table discovery is limited to 1000 routes with most of the devices, unless you change manually this value. So if you have a different behaviour, please let us know more:

  • number of routes per device
  • frequency of discovery
  • stability of your routes
    and any other thing that might help.

we will update the librenms version next week -> hope that this solve the issue !

the problem ist that millions of lines with the same route are installed ->always a new line but same route ( i dont know after a polling or after a discovery freqency )

bgp and also ospf !

our devices have from full bgp table to only 100 Routes and that very stable

@kristoferus75
Yes, you are still running the old code. The new code will wipe the existing “route” table, so the 130G of data will be deleted.
Then, during each discovery, the routes will be polled and should not get duplicated anymore (this was tested with Cisco devices successfully).
Bye

@PipoCanaja

Thanks -> we will test it next week on monday !!!