Network maps and neighbors are not working

After the recent discovery refactor, maps and neighbors are not showing. They should come from LLDP.

I checked the code and after the refactor it still expects the results from the snmp query in the form of [key][subkey][subkey] while the new snmp query outputs [key.subkey.subkey].

The end result of this is that the links are not discovered and hence nothing is shown.

If I use the old query, it works again ($lldp_array = snmpwalk_cache_threepart_oid($device, 'lldpRemoteSystemsData', array(), 'LLDP-MIB');)

Component Version
LibreNMS 1.32-46-g9774acd
DB Schema 211
PHP 5.4.45-1~dotdeb+7.1
MySQL 5.1.73-1+deb6u1
RRDTool 1.4.3
SNMP NET-SNMP 5.4.3

@murrant can be related to the precedent fix for this issue?

@tijmenNL Can you post the discovery debug output?

./discovery.php -h HOSTNAME -d -m discovery-protocols

It is quite big, can I post it here or create a paste somewhere?

Anyway here it is, I’ve replaced the sys descriptions by ****

LibreNMS Discovery
===================================
Version info:
Commit SHA: 9774acd0551050d19490bbdfb44046b664c856a7
Commit Date: 1508129391
DB Schema: 211
PHP: 5.4.45-1~dotdeb+7.1
MySQL: 5.1.73-1+deb6u1
RRDTool: 1.4.3
SNMP: NET-SNMP 5.4.3
==================================DEBUG!
SQL[SELECT * FROM `devices` WHERE status = 1 AND disabled = 0 AND `hostname` LIKE 'node05.dns-hosting.info' ORDER BY device_id DESC]
SQL[SELECT * FROM devices_attribs WHERE `device_id` = '19']
node05.dns-hosting.info 19 linux SQL[INSERT INTO `device_perf` (`xmt`,`rcv`,`loss`,`min`,`max`,`avg`,`exitcode`,`device_id`,`timestamp`)  VALUES ('3','3','0','1.36','1.38','1.36','0','19',NOW())]
SNMP Check response code: 0
SQL[SELECT attrib_value FROM devices_attribs WHERE `device_id` = '19' AND `attrib_type` = 'poll_mib' ]

Modules status: Global+ OS+ Device
#### Load disco module discovery-protocols ####
 CISCO-CDP-MIB: SNMP[/usr/bin/snmpbulkwalk -v2c -c COMMUNITY -OQUsetX -m CISCO-CDP-MIB -M /opt/librenms/mibs:/opt/librenms/mibs/supermicro:/opt/librenms/mibs/dell udp:HOSTNAME:161 cdpCache]
cdpCache = No Such Object available on this agent at this OID


 LLDP-MIB: SNMP[/usr/bin/snmpbulkwalk -v2c -c COMMUNITY -OQUsetX -m LLDP-MIB -M /opt/librenms/mibs:/opt/librenms/mibs/supermicro:/opt/librenms/mibs/dell udp:HOSTNAME:161 lldpRemTable]
lldpRemChassisIdSubtype[.0.2.5 = 4
lldpRemChassisIdSubtype[.0.2.6 = 4
lldpRemChassisIdSubtype[.0.2.8 = 4
lldpRemChassisIdSubtype[.0.3.7 = 7
lldpRemChassisIdSubtype[.0.3.9 = 7
lldpRemChassisIdSubtype[.52307900.20.17 = 4
lldpRemChassisIdSubtype[.292392600.2.20 = 4
lldpRemChassisId[.0.2.5 = "00 1C 23 E1 82 11 "
lldpRemChassisId[.0.2.6 = "00 1C 23 E1 86 76 "
lldpRemChassisId[.0.2.8 = "00 13 72 F9 A4 76 "
lldpRemChassisId[.0.3.7 = "node08.dns-hosting.info"
lldpRemChassisId[.0.3.9 = "ce24.ams-01.nl.leaseweb.net"
lldpRemChassisId[.52307900.20.17 = "00 16 3E 33 8F 48 "
lldpRemChassisId[.292392600.2.20 = "10 98 36 AA E9 95 "
lldpRemPortIdSubtype[.0.2.5 = 3
lldpRemPortIdSubtype[.0.2.6 = 3
lldpRemPortIdSubtype[.0.2.8 = 3
lldpRemPortIdSubtype[.0.3.7 = 5
lldpRemPortIdSubtype[.0.3.9 = 5
lldpRemPortIdSubtype[.52307900.20.17 = 3
lldpRemPortIdSubtype[.292392600.2.20 = 3
lldpRemPortId[.0.2.5 = "00 1C 23 E1 82 11 "
lldpRemPortId[.0.2.6 = "00 1C 23 E1 86 76 "
lldpRemPortId[.0.2.8 = "00 13 72 F9 A4 76 "
lldpRemPortId[.0.3.7 = "eth1"
lldpRemPortId[.0.3.9 = "FastEthernet3/45"
lldpRemPortId[.52307900.20.17 = "00 16 3E 33 8F 48 "
lldpRemPortId[.292392600.2.20 = "10 98 36 AA E9 95 "
lldpRemPortDesc[.0.2.5 = eth0
lldpRemPortDesc[.0.2.6 = eth0
lldpRemPortDesc[.0.2.8 = eth1
lldpRemPortDesc[.0.3.7 = eth1
lldpRemPortDesc[.0.3.9 = FastEthernet3/45
lldpRemPortDesc[.52307900.20.17 = eth0
lldpRemPortDesc[.292392600.2.20 = eth0
lldpRemSysName[.0.2.5 = node09.dns-hosting.info
lldpRemSysName[.0.2.6 = node08.dns-hosting.info
lldpRemSysName[.0.2.8 = node07.dns-hosting.info
lldpRemSysName[.0.3.7 = node08.dns-hosting.info
lldpRemSysName[.0.3.9 = ce24.ams-01.nl.leaseweb.net
lldpRemSysName[.52307900.20.17 = sylk01.dns-hosting.info
lldpRemSysName[.292392600.2.20 = node06.dns-hosting.info
lldpRemSysDesc[.0.2.5 = ****
lldpRemSysDesc[.0.2.6 = ****
lldpRemSysDesc[.0.2.8 = ****
lldpRemSysDesc[.0.3.7 = ****
lldpRemSysDesc[.0.3.9 = ****
lldpRemSysDesc[.52307900.20.17 = ****
lldpRemSysDesc[.292392600.2.20 = ****
lldpRemSysCapSupported[.0.2.5 = "38 "
lldpRemSysCapSupported[.0.2.6 = "38 "
lldpRemSysCapSupported[.0.2.8 = "38 "
lldpRemSysCapSupported[.0.3.7 = "01 "
lldpRemSysCapSupported[.0.3.9 = "28 "
lldpRemSysCapSupported[.52307900.20.17 = "39 "
lldpRemSysCapSupported[.292392600.2.20 = "39 "
lldpRemSysCapEnabled[.0.2.5 = "08 "
lldpRemSysCapEnabled[.0.2.6 = "00 "
lldpRemSysCapEnabled[.0.2.8 = "08 "
lldpRemSysCapEnabled[.0.3.7 = "01 "
lldpRemSysCapEnabled[.0.3.9 = "28 "
lldpRemSysCapEnabled[.52307900.20.17 = "01 "
lldpRemSysCapEnabled[.292392600.2.20 = "28 "

SNMP[/usr/bin/snmpbulkwalk -v2c -c COMMUNITY -OQUsetX -m BRIDGE-MIB -M /opt/librenms/mibs:/opt/librenms/mibs/supermicro:/opt/librenms/mibs/dell udp:HOSTNAME:161 dot1dBasePortIfIndex]
dot1dBasePortIfIndex = No Such Object available on this agent at this OID

SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
4SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
00 1C 23 E1 82 11SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
3SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
00 1C 23 E1 82 11SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth0SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
node09.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
38SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
08SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
4SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
00 1C 23 E1 86 76SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
3SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
00 1C 23 E1 86 76SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth0SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
node08.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
38SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
00SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
4SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
00 13 72 F9 A4 76SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
3SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
00 13 72 F9 A4 76SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth1SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
node07.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
38SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
08SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
7SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
node08.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
5SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
eth1SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth1SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
node08.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
01SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
01SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
7SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
ce24.ams-01.nl.leaseweb.netSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
5SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
FastEthernet3/45SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
FastEthernet3/45SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
ce24.ams-01.nl.leaseweb.netSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
28SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
28SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
4SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
00 16 3E 33 8F 48SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
3SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
00 16 3E 33 8F 48SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth0SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
sylk01.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
39SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
01SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisIdSubtype']
4SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemChassisId']
10 98 36 AA E9 95SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortIdSubtype']
3SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortId']
10 98 36 AA E9 95SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemPortDesc']
eth0SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysName']
node06.dns-hosting.infoSQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysDesc']
**** SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapSupported']
39SQL[SELECT * FROM `ports` WHERE `device_id` = '19' AND `ifIndex` = 'lldpRemSysCapEnabled']
28
 OSPF Discovery: SQL[SELECT DISTINCT(`ospfNbrIpAddr`),`device_id` FROM `ospf_nbrs` WHERE `device_id`='19']

SQL[SELECT * FROM `links` AS L, `ports` AS I WHERE L.local_port_id = I.port_id AND I.device_id = '19']
SQL[DELETE `l` FROM `links` `l` LEFT JOIN `devices` `d` ON `d`.`device_id` = `l`.`local_device_id` WHERE `d`.`device_id` IS NULL]
  orphaned links deleted

>> Runtime for discovery module 'discovery-protocols': 0.1010 seconds with 5304 bytes
#### Unload disco module discovery-protocols ####

SQL[SELECT attrib_value FROM devices_attribs WHERE `device_id` = '19' AND `attrib_type` = 'poll_mib' ]
SQL[UPDATE `devices` set `last_discovered` =NOW(),`last_discovered_timetaken` ='0.523' WHERE `device_id` = '19']
Discovered in 0.523 seconds

SQL[INSERT INTO `perf_times` (`type`,`doing`,`start`,`duration`,`devices`,`poller`)  VALUES ('discover','node05.dns-hosting.info','1508161654.8772','0.668','1','node16\n')]
./discovery.php node05.dns-hosting.info 2017-10-16 15:47:35 - 1 devices discovered in 0.668 secs
SNMP: Get[1/0.01s] Walk [0/0.00s]
MySQL: Cell[2/0.00s] Row[65/0.01s] Rows[6/0.00s] Column[0/0.00s] Update[1/0.12s] Insert[2/0.02s] Delete[0/0.00s]

Can you upgrade your net-snmp package and see if that helps?

Upgrading net-snmp is a bit of a problem on that machine, so I deployed a fresh Debian stretch and copied librenms to that.

With the newer net-snmp it seems resolved. I now get this:
lldpRemChassisIdSubtype[0][2][5] = 4
instead of this:
lldpRemChassisIdSubtype[.0.2.5 = 4

And therefore it can be parsed correctly. Does this mean NET-SNMP 5.4.3 is not supported anymore?

1 Like

It might. I’ll look into it and see if I can find anything. Thanks for reporting.

Hmm, bug with net-snmp -X option in versions at least <= 5.5, which is used by snmpwalk_group()

CentOS 6, 5, Ubuntu 12.04 and Debian 7 (Wheezy) all have bad versions

Centos 7 - 5.7.2
CentOS 6 - 5.5
CentOS 5 - 5.3.2

Debian 9 - 5.7.3
Debian 8 - 5.7.2
Debian 7 - 5.4.3
​​
Ubuntu 17.04 - 5.7.3
Ubuntu 16.10 - 5.7.3
Ubuntu 16.04 - 5.7.3
Ubuntu 14.04 - 5.7.2
Ubuntu 12.04 - 5.4.3

1 Like

Right so the bug is net-snmp. Can this be fixed, or should I move to Debian 9?

Probably, but it’s out of the scope of LibreNMS :confused:

Thanks, I will move to Debian 9 anyway, but what I actually meant was: will this be fixed, or do we have to accept that it’s a bug in net-snmpd and that it will not work with versions <=5.5.

As example, CentOS 6 (Red hat 6) is always maintained and supported. So it’s an important bug and it will be probably fixed especially for some distributions.

We will be evaluating what to do. If you can, I suggest you move to a newer OS.

Thanks for the info, I will.