Huawei NE40E-MK2b - Peer bgp disappeared after system update

mrmarcio31rs · 17 January 2020 18:42

Guys, after updating my router bgp sessions are no longer identified by Librenms.
Is there any tips to solve this issue?

PipoCanaja · 17 January 2020 20:00

They changed the MIBs and/or the SNMP replies. And BGP support right now is not optimal. I am working on it right now (on CloudEngine and Campus family). I’ll contact you to test it on your NE.

mrmarcio31rs · 18 January 2020 14:26

OK, thanks for the help. Much success for this extraordinary tool!

mrmarcio31rs · 3 February 2020 12:49

Hello guys, anything new in this issue?

PipoCanaja · 3 February 2020 13:33

Hi @mrmarcio31rs
The code is already posted here.

github.com/librenms/librenms

Improve Huawei VRP bgp discovery

librenms:master ← PipoCanaja:vrp-bgp

opened 09:27PM - 20 Jan 20 UTC

PipoCanaja

+2718 -2082

This PR is improving multi-VRF BGP discovery on Huawei CloudEngine and AR device…s. Polling is not touched for the moment (so BGP data is still missing during polling) because a more important change (DB) is probably needed here (that will necessarily impact other OSes). DO NOT DELETE THIS TEXT #### Please note > Please read this information carefully. You can run `./scripts/pre-commit.php` to check your code before submitting. - [X] Have you followed our [code guidelines?](http://docs.librenms.org/Developing/Code-Guidelines/) #### Testers If you would like to test this pull request then please run: `./scripts/github-apply <pr_id>`, i.e `./scripts/github-apply 11054` After you are done testing, you can remove the changes with `./scripts/github-remove`. If there are schema changes, you can ask on discord how to revert.

You can test it with this command :
./scripts/github-apply 11054

Let me know the results.
Thanx

mrmarcio31rs · 4 February 2020 13:08

./scripts/github-apply 11054
:2264: trailing whitespace.

:2265: trailing whitespace.
– Description: HUAWEI-BGP-VPN-MIB , this Mib module for management
:2267: trailing whitespace.
– Reference: This MIB was extracted from RFC 4271
:2275: trailing whitespace.

:2280: trailing whitespace.
FROM HUAWEI-MIB
Checking patch includes/discovery/bgp-peers.inc.php…
error: while searching for:
if (!empty($af_data)) {
$af_list = build_cbgp_peers($device, $peer, $af_data, $peer2);
}
if ($device[‘os’] == ‘vrp’) {
d_echo(“VRP:”);
$af_data = snmpwalk_cache_oid($device, ‘hwBgpPeers’, $af_data, ‘HUAWEI-BGP-VPN-MIB’);
$af_list = build_cbgp_peers($device, $peer, $af_data, $peer2);
}

            if ($device['os_group'] === 'brocade') {
                // Brocade doesn't return MP-BGP SAFIs in the BGP4V2-MIB data, so it only displays the unicast SAFI

error: patch failed: includes/discovery/bgp-peers.inc.php:144
error: includes/discovery/bgp-peers.inc.php: patch does not apply
Checking patch includes/discovery/functions.inc.php…
Hunk #1 succeeded at 1213 (offset -18 lines).
Checking patch includes/polling/bgp-peers.inc.php…
Hunk #1 succeeded at 395 (offset -21 lines).
Checking patch mibs/huawei/HUAWEI-BGP-VPN-MIB…

PipoCanaja · 4 February 2020 23:38

Hi @mrmarcio31rs
Was your LibreNMS instance up to date ?

Please run :

./validate.php
./daily.php
./validate.php

mrmarcio31rs · 5 February 2020 11:49

Hi @PipoCanaja

root@cqnet-librenms:/home/camaquanet# su - librenms
$
$ ./validate.php

Component	Version
LibreNMS	1.60
DB Schema	2019_12_28_180000_add_overwrite_ip_to_devices (156)
PHP	7.2.24-0ubuntu0.18.04.2
MySQL	10.1.43-MariaDB-0ubuntu0.18.04.1
RRDTool	1.7.0
SNMP	NET-SNMP 5.7.3

====================================

[OK] Composer Version: 1.9.3
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct
$
$
$ ./daily.php
-su: 5: ./daily.php: Permission denied
$
$

mrmarcio31rs · 5 February 2020 11:56

$ ./scripts/github-apply 11054
:2263: trailing whitespace.

:2264: trailing whitespace.
– Description: HUAWEI-BGP-VPN-MIB , this Mib module for management
:2266: trailing whitespace.
– Reference: This MIB was extracted from RFC 4271
:2274: trailing whitespace.

:2279: trailing whitespace.
FROM HUAWEI-MIB
Checking patch includes/discovery/bgp-peers.inc.php…
Checking patch includes/discovery/functions.inc.php…
Checking patch includes/polling/bgp-peers.inc.php…
Checking patch mibs/huawei/HUAWEI-BGP-VPN-MIB…
Applied patch includes/discovery/bgp-peers.inc.php cleanly.
Applied patch includes/discovery/functions.inc.php cleanly.
Applied patch includes/polling/bgp-peers.inc.php cleanly.
Applied patch mibs/huawei/HUAWEI-BGP-VPN-MIB cleanly.
warning: squelched 344 whitespace errors
warning: 349 lines add whitespace errors.

PipoCanaja · 5 February 2020 12:34

OK. It looks better now. The patch is applied. Let me know if you have a better display of the BGP peers.

Concerning this :

$ ./daily.php
-su: 5: ./daily.php: Permission denied
$

This is not expected. You seems to have a right issue here. Please check the permissions and owner here.

mrmarcio31rs · 5 February 2020 13:49

Hello @PipoCanaja
success! everything ok with the bgp pairs now. Thanks for everything!

PipoCanaja · 5 February 2020 15:21

All right. The PR will be merged in a few days, weeks, as soon as I can collect the test data (for regression tests). When it is merged, you’ll have an error in your daily.sh update (it will complain the files are modified). You’ll have to revert the files to their original state and then daily.sh will run smoothly.

jozefrebjak · 22 April 2020 08:55

Hello @PipoCanaja thank you for your work. We own HUAWEI40E-MK2B too and we can see peers and status of them, but what we missing is number of prefixes. Updates are working, but number of prefixes not. Question is why, because we alredy have that informations in discovery run in SNMP['/usr/bin/snmpbulkwalk' '-v2c' '-c' 'COMMUNITY' '-OQUs' '-m' 'HUAWEI-BGP-VPN-MIB' '-M' '/opt/librenms/mibs:/opt/librenms/mibs/huawei' '-t' '5' 'udp:HOSTNAME:161' 'hwBgpPeers'] like this

hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 806549
hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 386
hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 3
hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 73662
hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 73662
hwBgpPeerPrefixRcvCounter.0.ipv4.unicast.ipv4."*" = 2
hwBgpPeerPrefixRcvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:11" = 33004
hwBgpPeerPrefixRcvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:12" = 33003
hwBgpPeerPrefixRcvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:02:38" = 2
hwBgpPeerPrefixRcvCounter.0.ipv6.unicast.ipv6."2a:01:03:90:06:66:04:25:00:00:00:00:00:00:00:01" = 85695
hwBgpPeerPrefixRcvCounter.0.ipv6.unicast.ipv6."2a:03:27:60:00:01:00:01:00:00:00:00:00:00:00:02" = 3
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 733001
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 385
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 2
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 73548
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 0
hwBgpPeerPrefixActiveCounter.0.ipv4.unicast.ipv4."*" = 0
hwBgpPeerPrefixActiveCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:11" = 32986
hwBgpPeerPrefixActiveCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:12" = 0
hwBgpPeerPrefixActiveCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:02:38" = 0
hwBgpPeerPrefixActiveCounter.0.ipv6.unicast.ipv6."2a:01:03:90:06:66:04:25:00:00:00:00:00:00:00:01" = 52392
hwBgpPeerPrefixActiveCounter.0.ipv6.unicast.ipv6."2a:03:27:60:00:01:00:01:00:00:00:00:00:00:00:02" = 2
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 8
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 2
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 1
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 8
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 8
hwBgpPeerPrefixAdvCounter.0.ipv4.unicast.ipv4."*" = 8
hwBgpPeerPrefixAdvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:11" = 1
hwBgpPeerPrefixAdvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:12" = 1
hwBgpPeerPrefixAdvCounter.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:02:38" = 1
hwBgpPeerPrefixAdvCounter.0.ipv6.unicast.ipv6."2a:01:03:90:06:66:04:25:00:00:00:00:00:00:00:01" = 1
hwBgpPeerPrefixAdvCounter.0.ipv6.unicast.ipv6."2a:03:27:60:00:01:00:01:00:00:00:00:00:00:00:02" = 2

And in SQL I can see only 0 values in Prefixes.

SQL[INSERT IGNORE INTO `bgpPeers_cbgp` (`device_id`,`bgpPeerIdentifier`,`afi`,`safi`,`context_name`,`AcceptedPrefixes`,`DeniedPrefixes`,`PrefixAdminLimit`,`PrefixThreshold`,`PrefixClearThreshold`,`AdvertisedPrefixes`,`SuppressedPrefixes`,`WithdrawnPrefixes`,`AcceptedPrefixes_delta`,`AcceptedPrefixes_prev`,`DeniedPrefixes_delta`,`DeniedPrefixes_prev`,`AdvertisedPrefixes_delta`,`AdvertisedPrefixes_prev`,`SuppressedPrefixes_delta`,`SuppressedPrefixes_prev`,`WithdrawnPrefixes_delta`,`WithdrawnPrefixes_prev`)  VALUES (:device_id,:bgpPeerIdentifier,:afi,:safi,:context_name,:AcceptedPrefixes,:DeniedPrefixes,:PrefixAdminLimit,:PrefixThreshold,:PrefixClearThreshold,:AdvertisedPrefixes,:SuppressedPrefixes,:WithdrawnPrefixes,:AcceptedPrefixes_delta,:AcceptedPrefixes_prev,:DeniedPrefixes_delta,:DeniedPrefixes_prev,:AdvertisedPrefixes_delta,:AdvertisedPrefixes_prev,:SuppressedPrefixes_delta,:SuppressedPrefixes_prev,:WithdrawnPrefixes_delta,:WithdrawnPrefixes_prev)

{"device_id":470,"bgpPeerIdentifier":"86.110.242.90","afi":"ipv4","safi":"unicast","context_name":"","AcceptedPrefixes":0,"DeniedPrefixes":0,"PrefixAdminLimit":0,"PrefixThreshold":0,"PrefixClearThreshold":0,"AdvertisedPrefixes":0,"SuppressedPrefixes":0,"WithdrawnPrefixes":0,"AcceptedPrefixes_delta":0,"AcceptedPrefixes_prev":0,"DeniedPrefixes_delta":0,"DeniedPrefixes_prev":0,"AdvertisedPrefixes_delta":0,"AdvertisedPrefixes_prev":0,"SuppressedPrefixes_delta":0,"SuppressedPrefixes_prev":0,"WithdrawnPrefixes_delta":0,"WithdrawnPrefixes_prev":0}

there is also missing info about Family but afi ans safi are added in sql so I don’t understand why it’s not possible to view it in UI. Also I don’t know if poller is working fine because I can see un UI State of some peers loke stop or nothing but in snmpwalk are all in start admin status

hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv4.unicast.ipv4."*" = start
hwBgpPeerAdminStatus.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:11" = start
hwBgpPeerAdminStatus.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:00:12" = start
hwBgpPeerAdminStatus.0.ipv6.unicast.ipv6."20:01:07:f8:00:91:00:00:00:00:00:00:00:00:02:38" = start
hwBgpPeerAdminStatus.0.ipv6.unicast.ipv6."2a:01:03:90:06:66:04:25:00:00:00:00:00:00:00:01" = start
hwBgpPeerAdminStatus.0.ipv6.unicast.ipv6."2a:03:27:60:00:01:00:01:00:00:00:00:00:00:00:02" = start

My install is always on newest master version of librenms.

jozefrebjak · 22 April 2020 09:26

@PipoCanaja So i looked into sql database and there I can see that I don’t have a data in bgpPeers_cbgp table at all for my Huawei NE router with device_id 470

Here is an output of

SELECT * FROM bgpPeers WHERE device_id = 470;

+------------+-----------+--------+----------------------------------+---------------------+-----------------+--------------+--------------------+----------------+---------------------+--------------+------------------+-------------------+------------------------+-------------------------+---------------------------+----------------------------+--------------+
| bgpPeer_id | device_id | vrf_id | astext                           | bgpPeerIdentifier   | bgpPeerRemoteAs | bgpPeerState | bgpPeerAdminStatus | bgpLocalAddr   | bgpPeerRemoteAddr   | bgpPeerDescr | bgpPeerInUpdates | bgpPeerOutUpdates | bgpPeerInTotalMessages | bgpPeerOutTotalMessages | bgpPeerFsmEstablishedTime | bgpPeerInUpdateElapsedTime | context_name |
+------------+-----------+--------+----------------------------------+---------------------+-----------------+--------------+--------------------+----------------+---------------------+--------------+------------------+-------------------+------------------------+-------------------------+---------------------------+----------------------------+--------------+
|        457 |       470 |   NULL | VNET-AS, SK                      | 86.110.242.90       |           29405 | established  | start              | 86.110.242.91  | 86.110.242.90       |              |          3622579 |              2866 |                3684211 |                   76538 |                   1882921 |                         17 | NULL         |
|        458 |       470 |   NULL | EMAX_AS1 http://www.e-max.sk, SK | 188.120.0.0         |           49115 | established  | start              | 188.120.0.1    | 188.120.0.0         |              |              763 |                13 |                  38112 |                   43171 |                   2241175 |                        278 | NULL         |
|        461 |       470 |   NULL | NIXSK-RS NIX.CZ z.s.p.o., SK     | 194.30.187.1        |           47627 | established  | start              | 194.30.187.167 | 194.30.187.1        |              |          7473581 |              4147 |                7511065 |                   41984 |                   1962280 |                          7 | NULL         |
|        462 |       470 |   NULL | NIXSK-RS NIX.CZ z.s.p.o., SK     | 194.30.187.2        |           47627 | established  | start              | 194.30.187.167 | 194.30.187.2        |              |          1447867 |               186 |                1457482 |                    9900 |                    503443 |                          7 | NULL         |
|        463 |       470 |   NULL | ROOTSERV, US                     | 194.30.187.238      |             112 | established  | start              | 194.30.187.167 | 194.30.187.238      |              |                3 |              1642 |                  37404 |                   39471 |                   1962280 |                    1211116 | NULL         |
|        524 |       470 |   NULL | NIXSK-RS NIX.CZ z.s.p.o., SK     | 2001:7f8:91::11     |           47627 | established  |                    |                | 2001:7F8:91::11     |              |          1891251 |               192 |                1928689 |                   38028 |                   1962281 |                          0 | NULL         |
|        525 |       470 |   NULL | NIXSK-RS NIX.CZ z.s.p.o., SK     | 2001:7f8:91::12     |           47627 | established  |                    |                | 2001:7F8:91::12     |              |           454149 |                38 |                 463750 |                    9754 |                    503466 |                          0 | NULL         |
|        526 |       470 |   NULL | ROOTSERV, US                     | 2001:7f8:91::238    |             112 | established  |                    |                | 2001:7F8:91::238    |              |                3 |                68 |                  37383 |                   37895 |                   1962274 |                          0 | NULL         |
|        567 |       470 |   NULL | EMAX_AS1 http://www.e-max.sk, SK | 188.120.0.145       |           49115 | established  | start              | 188.120.0.144  | 188.120.0.145       |              |                6 |                 2 |                  29994 |                   34708 |                   1799459 |                     645603 | NULL         |
|        603 |       470 |   NULL | EMAX_AS1 http://www.e-max.sk, SK | 2a03:2760:1:1::2    |           49115 | established  |                    |                | 2A03:2760:1:1::2    |              |                3 |                71 |                  27360 |                   31736 |                   1641593 |                          0 | NULL         |
|        825 |       470 |   NULL | VNET-AS, SK                      | 2a01:390:666:425::1 |           29405 | established  | stop               | 0.0.0.0        | 2A01:390:666:425::1 |              |           181895 |                 2 |                 184498 |                    3496 |                     89087 |                          0 | NULL         |
+------------+-----------+--------+----------------------------------+---------------------+-----------------+--------------+--------------------+----------------+---------------------+--------------+------------------+-------------------+------------------------+-------------------------+---------------------------+----------------------------+--------------+
11 rows in set (0.00 sec)

where you can see that some data are missing for ipv6 peers.

and when I run

SELECT * FROM bgpPeers_cbgp WHERE device_id = 470;

I got

Empty set (0.00 sec)

So that is a reason why I can’t see Family and also prefixes with value 0 for clear graphs.

PipoCanaja · 22 April 2020 09:37

Provide me snmpwalk so I can simulate your device, and I’ll fix it. I don’t have any device providing prefixes numbers here so I have to work on simulated devices only.

jozefrebjak · 22 April 2020 10:07

I can see that problem also on Huawei S5720I-12X-SI-ACthere is also no data in bgpPeers_cbgp table. But i think that in past I can see prefixes numbers. snmpwalk fo ne40 is really so big and it taking so long because we have 2x Full BGP and it will consume more cpu to take it.

PipoCanaja · 30 April 2020 22:07

@jozefrebjak
Again, without the device, I cannot do much . I need the snmpwalk to replicate it. Do you have some spare/test device on which you could produce the behaviour, collect the snmp data and send it ?

jozefrebjak · 1 May 2020 08:37

@PipoCanaja Do you have some S Series switches with BGP ? Because that problem is there for all VRP devices not only NE40xx devices. There is no data added to bgpPeers_cbgp table. So that is a reason why we cant’s see prefixes etc. for VRP devices.

Here is the snmpwalk of Huawei NE40E-M2K-B which is using HUAWEI-BGP-VPN-MIB

huawei_ne40e-m2k-b_snmpwalk

But there is also problem with S-Series switches which are using generic BGP MIB

PipoCanaja · 1 May 2020 09:01

OK. I’ll see if I can find out on an S5700 switch. Will update you. Which firmware version are you running ?

jozefrebjak · 5 May 2020 19:19

@PipoCanaja Here is the list of my Huawei swittches

Huawei NE40E-MK2b - Peer bgp disappeared after system update

./scripts/github-apply 11054 :2264: trailing whitespace.

root@cqnet-librenms:/home/camaquanet# su - librenms $ $ ./validate.php

$ ./scripts/github-apply 11054 :2263: trailing whitespace.

./scripts/github-apply 11054
:2264: trailing whitespace.

root@cqnet-librenms:/home/camaquanet# su - librenms
$
$ ./validate.php

$ ./scripts/github-apply 11054
:2263: trailing whitespace.