NXOS Model/OS Version Detection broken

twelch · 22 September 2020 12:55

NXOS version/model detection appears to be broken. I suspect this is due to the UCS changes a while back, but haven’t been able to nail it down. We have multiple nexus 3064, 9660, and 93240 switches exhibiting this same behavior. From the event log on one of the devices:

2020-09-21 14:48:57 system redacted OS Version: 7.0(3)I7(6) System
2020-09-21 14:48:57 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-21 14:48:57 system redacted Serial: redacted System
2020-09-21 14:46:00 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-21 14:46:00 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-21 14:46:00 system redacted Serial: redacted -> System
2020-09-21 13:19:05 system redacted OS Version: 7.0(3)I7(6) System
2020-09-21 13:19:05 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-21 13:19:05 system redacted Serial: redacted System
2020-09-21 13:15:12 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-21 13:15:12 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-21 13:15:12 system redacted Serial: redacted -> System
2020-09-21 07:19:44 system redacted OS Version: 7.0(3)I7(6) System
2020-09-21 07:19:44 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-21 07:19:44 system redacted Serial: redacted System
2020-09-21 07:15:17 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-21 07:15:17 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-21 07:15:17 system redacted Serial: redacted -> System
2020-09-21 01:15:25 system redacted OS Version: 7.0(3)I7(6) System
2020-09-21 01:15:25 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-21 01:15:25 system redacted Serial: redacted System
2020-09-21 01:14:59 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-21 01:14:59 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-21 01:14:59 system redacted Serial: redacted -> System
2020-09-20 19:15:51 system redacted OS Version: 7.0(3)I7(6) System
2020-09-20 19:15:51 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-20 19:15:51 system redacted Serial: redacted System
2020-09-20 19:15:49 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-20 19:15:49 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-20 19:15:49 system redacted Serial: redacted -> System
2020-09-20 13:16:39 system redacted OS Version: 7.0(3)I7(6) System
2020-09-20 13:16:39 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-20 13:16:39 system redacted Serial: redacted System
2020-09-20 13:15:17 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-20 13:15:17 system redacted Serial: redacted -> System
2020-09-20 13:15:16 system redacted OS Version: 7.0(3)I7(6) -> System
2020-09-20 07:17:14 system redacted OS Version: 7.0(3)I7(6) System
2020-09-20 07:17:14 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX
Any suggestions as to what may have caused this?

Thanks,
Tim

bwilli37 · 22 September 2020 14:39

Seeing the same. It began on 9/19. Seeing the problem on all NX-OS devices.

2020-09-22 08:32:33 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-22 08:32:33 system redacted Serial: redacted System
2020-09-22 08:28:42 system redacted OS Version: 7.0(3)I4(8b) -> System
2020-09-22 08:28:42 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-22 08:28:42 system redacted Serial: redacted -> System
2020-09-22 02:27:27 system redacted OS Version: 7.0(3)I4(8b) System
2020-09-22 02:27:27 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-22 02:27:27 system redacted Serial: redacted System
2020-09-22 02:26:20 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-22 02:26:20 system redacted Serial: redacted -> System
2020-09-22 02:26:19 system redacted OS Version: 7.0(3)I4(8b) -> System
2020-09-21 20:32:37 system redacted OS Version: 7.0(3)I4(8b) System
2020-09-21 20:32:37 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX System
2020-09-21 20:32:37 system redacted Serial: redacted System
2020-09-21 20:28:05 system redacted OS Version: 7.0(3)I4(8b) -> System
2020-09-21 20:28:05 system redacted Hardware: N3K-C3064PQ-10GX -> ciscoModules.3.1.3.1163 System
2020-09-21 20:28:05 system redacted Serial: redacted -> System
2020-09-21 14:27:35 system redacted OS Version: 7.0(3)I4(8b) System
2020-09-21 14:27:35 system redacted Hardware: ciscoModules.3.1.3.1163 -> N3K-C3064PQ-10GX

twelch · 22 September 2020 15:23

github.com/librenms/librenms

Added support for OS detection from NXOS and UCS

librenms:master ← magnuslarsen:NXOS_UCS_fix

opened 07:48AM - 28 May 20 UTC

magnuslarsen

+26 -3

This PR adds support for LibreNMS to differentiate between Cisco UCS and Cisco N…XOS type devices. Currently both UCS server hardware and NXOS network devices gets detected as `NXOS` - which is wrong :-) If you use Oxidized, this leads to the wrong model configuration being used; and Oxidized fails to load the config. If someone could help me figure this one out, please help me. When changing to a the proper UCS os (`ucos`) - the hardware field changes from (the correct) `UCS 6248 Series Fabric Interconnect` to ` ciscoModules.3.1.3.1062` - how is this decided, and where can I change it? FROM: ![image](https://user-images.githubusercontent.com/7879747/83113572-5ee8dd00-a0b7-11ea-8326-f5fedfe4ebeb.png) TO: ![image](https://user-images.githubusercontent.com/7879747/83113584-63ad9100-a0b7-11ea-9b00-cbcbe9e00e8f.png) #### Please note > Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting. - [x] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/) - [x] If my Pull Request does some changes/fixes/enhancements in the WebUI, I have inserted a screenshot of it. #### Testers If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 5926` After you are done testing, you can remove the changes with `./scripts/github-remove`. If there are schema changes, you can ask on discord how to revert.

Likely this guy. It was committed on the 18th. Not sure if there’s any automated way to roll back that far.

Hans_Erasmus · 31 March 2021 16:43

Not sure if this LibreNMS related though. I found issues with pure SNMP walks.

Example:
(Working)
.1.3.6.1.2.1.1.1.0 = Cisco NX-OS(tm) nxos.7.0.3.I6.2.bin, Software (nxos), Version 7.0(3)I6(2), RELEASE SOFTWARE Copyright (c) 2002-2016 by Cisco Systems, Inc. Compiled 10/17/2017 18:00:00

(Broken) - Software 7.0(3)I7(6)
.1.3.6.1.2.1.1.1.0 = Usage:

This was what I got from SNMPBULKWALKS. So I reckon the issue lies with CISCO?

twelch · 31 March 2021 16:47

Well, we see the same issue on all versions we have running - over 80 nexus 3064, 3048, 93140, 93240, 9336, etc, running 9.3(4), 7.0(3)I7(6), 9.3(5), 9.3(6) and 7.0(3)I6. I think the UCS change caused it, but haven’t had time to dig into it yet.

snmpwalk -v2c -c .1.3.6.1.2.1.1.1.0
iso.3.6.1.2.1.1.1.0 = STRING: “Cisco NX-OS™ nxos.9.3.4.bin, Software (nxos), Version 9.3(4), RELEASE SOFTWARE Copyright © 2002-2020 by Cisco Systems, Inc. Compiled 4/28/2020 20:00:00”
snmpwalk -v2c -c .1.3.6.1.2.1.1.1.0
iso.3.6.1.2.1.1.1.0 = STRING: “Cisco NX-OS™ nxos.9.3.5.bin, Software (nxos), Version 9.3(5), RELEASE SOFTWARE Copyright © 2002-2020 by Cisco Systems, Inc. Compiled 7/20/2020 20:00:00”
snmpwalk -v2c -c .1.3.6.1.2.1.1.1.0
iso.3.6.1.2.1.1.1.0 = STRING: “Cisco NX-OS™ nxos.7.0.3.I7.6.bin, Software (nxos), Version 7.0(3)I7(6), RELEASE SOFTWARE Copyright © 2002-2019 by Cisco Systems, Inc. Compiled 3/5/2019 12:00:00”

system · 31 March 2023 16:47

This topic was automatically closed 730 days after the last reply. New replies are no longer allowed.