Weird spikes in network traffic on cisco 3650 and 3850 switches

Enabling RRDTune should help.

https://docs.librenms.org/#Extensions/RRDTune/

1 Like

check here -> https://docs.librenms.org/#Support/FAQ/#faq15

Then follow this doc.
https://docs.librenms.org/#Extensions/RRDTune/
make sure to run the scripts too.

Thank You, Will give it a try

I enabled RRDTune in the global settings, it seems like it fixed most ports but there are still a few that are showing as 10g ports.

I went to the devices settings and enabled the RRDTuning manually and then ran the script on the specific devices. Still seems to show 10G.

I think you are off by a zero. I see 1g and 100m ports on that screenshot. Those values are pulled directly from SNMP from the device.

1 Like

OK thank you, must have did the math wrong.

Hi @Kevin_Krumm I enabled the RDD Tuning and still see some traffic spikes on the Cisco 3850 and 3650s graphs. For some reason when they are in trunk mode to another cisco switch (ex.3560) Librenms shows spikes but not on the trunked connected switch. I see the same spikes to all the trunk switches but only on the 3850 side.
3850 Trunk Port

3560 Over all usage

image

Did run the RRD tune scripts also?

https://docs.librenms.org/#Support/FAQ/#faq15

You could try running the remove spikes script.

@Kevin_Krumm I did run both.
How do I use the removespikes.php, im not familiar with what values are needed.
I was just trying to figure out if it was actually a broadcast storm, but doesn’t seem so if its not on the reflecting switch.

How do I use the removespike.php I am trying to input ./removespike.php -R “ip”??

I have no idea, nerver used the script.

Good Morning,

Any chance you’ve had some luck with this? I’m seeing the exact same issues even with RRDTune enabled globally and am also struggling with the removespike.php script. It seems to be coming from a wide range of versions of the 3850’s for me.

Thank you

1 Like

You should make sure your polls are finishing as this was our issue. I can’t remember all the tuning I tried, but here’s a 60 second bash script I used to use to fix devices:

$ cat fix_em.sh 

!/bin/sh
 
for i in /opt/librenms/rrd/my-brokendevice/*;
do
         ./removespikes.php -M=variance A=nan- R=$i;
done;

Feel free to put your device in it and mess with the options as necessary. (I was too lazy to use argv)

1 Like

No I wasn’t able to fix them. Its weird that it’s only happens the newer switches. Cisco 3650 and 3850.

1 Like

Ok, thank you. Ya, I’m seeing the same thing (only 3850’s and 3650’s) and haven’t been able to get the removespikes script to work either. I’ll post back if I find the magic.

2 Likes

Do you ever notice gaps between graphs? Seems like SNMP on the switches are dying or locking up for a few seconds here and there…

I thought I was seeing gaps, but if I zoom in (shorten the time interval) it appears to be ok on mine. I think the spike is so much that the graph ends up with low spots that look like gaps.

Im actually seeing gaps

Gaps like that are usually performance issue with the device you are monitoring (SNMP timing out) or your networking monitoring server is struggling to keep up. You will need to troubleshoot further check the performce docs and run a debug on that device and see if you have any time outs.