Troubleshoot impossible traffic spikes

Help
1

We are using LibreNMS to monitor our Calix E7-2 GPON headend.

We are noticing occasional bandwidth spikes being recorded and shown on the graphs in LibreNMS. These spikes are anomalies because the speeds shown are simply not possible on the hardware that we have.

Our links are all 1Gbit, but as seen in this screenshot the graph is showing 9Gbit transfer: https://i.imgur.com/fug1FSI.png

These situations are infrequent and happen approximately once or twice a week.
Calix E7-2 headend, firmware version 3.1.40.1
LibreNMS details:

$ ./validate.php

Component Version
LibreNMS 1.38-80-g492a973
DB Schema 247
PHP 7.0.28-0ubuntu0.16.04.1
MySQL 5.7.22-0ubuntu0.16.04.1
RRDTool 1.5.5
SNMP NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.6.4
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Where do you suggest I begin when troubleshooting this issue?

2

Try run rrdtune script on it

https://docs.librenms.org/#Extensions/RRDTune/

and turn on RRDTune on the device, Edit Device -> Misc

3

Thank so much for the fast response, I have enabled this and will see how it goes.

Kind regard
Rob

4

https://docs.librenms.org/Support/FAQ/#why-do-i-see-traffic-spikes-in-my-graphs

5

Hello again, so with RRDTune enabled, I have seen one spike (100Gbps on a 1Gbps link) since. This is a lot less than previously but I’l look into it a bit more today and see if I can find the cause.

6

you have to run the scripts also.

image
image1467×162 8.99 KB

7

Thanks Kevin sorry I missed that step, assumed that enabling them would automatically run them.

Have done now and will see how it goes.

Appreciate the help :slight_smile:

8

Hello again, so I enabled the setting globally, as well as on the port affected (only one port, our main uplink) and have run the scripts but I am still seeing the same. Which logs would be helpful to see what is causing it?

9

You can try running the poller with debug and see if it is failing to return data sometimes or what. ./poller.php -h HOSTNAME -r -f -d -m ports