LibreNMS broken graphs on devices with mixed 32-bit and 64-bit interface counters

Interface (“Port”) graphs in LibreNMS for some devices on Cisco routers are showing all values as NaN. This appears to be the result of:

Correspondingly, Cisco IOS® Software does not support 64-bit counters for interface speeds of less than 20 Mbps. This means that 64-bit counters are not supported on 10 Mb Ethernet ports, only 100 Mb Fast-Ethernet and other high speed ports support 64-bit counters.

So on devices with a mix of interfaces both faster and slower than 20 Mbps, the graphs are broken for the interfaces slower than 20 Mbps

Output of poller.php -h HOSTNAME -r -f -d is at Untitled - LibreNMS (abbreviated and anonymized)

Taking a look at the pastebin output, we’ll compare interface indexes 1 and 20:

Lines 56-76

ifName.1 = Gi0/0/0
ifName.2 = Gi0/0/1
ifName.3 = Gi0/1/0
ifName.4 = Gi0/1/1
ifName.5 = Gi0/1/2
ifName.6 = Gi0/1/3
ifName.7 = Ce0/2/0
ifName.8 = Ce0/2/1
ifName.9 = Nu0
ifName.10 = Vl1
ifName.11 = Vl10
ifName.12 = Tu10265025
ifName.13 = Vi1
ifName.14 = Vl20
ifName.15 = Vl30
ifName.16 = Vl40
ifName.17 = Vl50
ifName.18 = Vl60
ifName.19 = Vl70
ifName.20 = Di1
ifName.21 = Vi2

Lines 161-180

ifHCInOctets.1 = 257266749330
ifHCInOctets.2 = 0
ifHCInOctets.3 = 97063394276
ifHCInOctets.4 = 0
ifHCInOctets.5 = 0
ifHCInOctets.6 = 0
ifHCInOctets.7 = 7873353472
ifHCInOctets.8 = 0
ifHCInOctets.9 = 0
ifHCInOctets.10 = 44985051421
ifHCInOctets.11 = 9910106267
ifHCInOctets.12 = 6775825316
ifHCInOctets.13 = 0
ifHCInOctets.14 = 32565756742
ifHCInOctets.15 = 2656603946
ifHCInOctets.16 = 796302
ifHCInOctets.17 = 3701794728
ifHCInOctets.18 = 464811
ifHCInOctets.19 = 16936460
ifHCInOctets.21 = 248153523016

You can see there’s no OID for ifHCInOctets.20. LibreNMS doesn’t appear to be polling for plain ifInOctets.20, and so it never gets inserted in the DB for the graphs:

The relevant interface arrays create for index 1 and index 20:

Lines 920-961

  1 =>
  array (
    'ifName' => 'Gi0/0/0',
    'ifInMulticastPkts' => '310565',
    'ifInBroadcastPkts' => '5',
    'ifOutMulticastPkts' => '450741',
    'ifOutBroadcastPkts' => '11164',
    'ifHCInOctets' => '257266749330',
    'ifHCInUcastPkts' => '452317874',
    'ifHCInMulticastPkts' => '310565',
    'ifHCInBroadcastPkts' => '5',
    'ifHCOutOctets' => '94316401791',
    'ifHCOutUcastPkts' => '402286902',
    'ifHCOutMulticastPkts' => '450741',
    'ifHCOutBroadcastPkts' => '11164',
    'ifLinkUpDownTrapEnable' => 'enabled',
    'ifHighSpeed' => '1000',
    'ifPromiscuousMode' => 'false',
    'ifConnectorPresent' => 'true',
    'ifAlias' => '',
    'ifCounterDiscontinuityTime' => '0:0:00:26.87',
    'ifDescr' => 'GigabitEthernet0/0/0',
    'ifAdminStatus' => 'up',
    'ifOperStatus' => 'up',
    'ifLastChange' => '26602752',
    'ifType' => 'ethernetCsmacd',
    'ifPhysAddress' => 'f8:7a:41:ea:6a:0',
    'ifMtu' => '1500',
    'ifInErrors' => '0',
    'ifOutErrors' => '0',
    'ifInDiscards' => '0',
    'ifOutDiscards' => '1594336',
    'dot3StatsDuplexStatus' => 'fullDuplex',
    'cieIfInRuntsErrs' => '0',
    'cieIfInGiantsErrs' => '0',
    'cieIfInFramingErrs' => '0',
    'cieIfInOverrunErrs' => '0',
    'cieIfInIgnored' => '0',
    'cieIfInAbortErrs' => '0',
    'cieIfInputQueueDrops' => '0',
    'cieIfOutputQueueDrops' => '1594336',
  ),

Lines 1713-1745

  20 =>
  array (
    'ifName' => 'Di1',
    'ifInMulticastPkts' => '0',
    'ifInBroadcastPkts' => '0',
    'ifOutMulticastPkts' => '0',
    'ifOutBroadcastPkts' => '0',
    'ifLinkUpDownTrapEnable' => 'enabled',
    'ifHighSpeed' => '1',
    'ifPromiscuousMode' => 'false',
    'ifConnectorPresent' => 'false',
    'ifAlias' => 'PROVIDER PPPoE []',
    'ifCounterDiscontinuityTime' => '0:0:00:21.06',
    'ifDescr' => 'Dialer1',
    'ifAdminStatus' => 'up',
    'ifOperStatus' => 'up',
    'ifLastChange' => '448422807',
    'ifType' => 'ppp',
    'ifPhysAddress' => '',
    'ifMtu' => '1492',
    'ifInErrors' => '0',
    'ifOutErrors' => '0',
    'ifInDiscards' => '0',
    'ifOutDiscards' => '17217',
    'cieIfInRuntsErrs' => '0',
    'cieIfInGiantsErrs' => '0',
    'cieIfInFramingErrs' => '0',
    'cieIfInOverrunErrs' => '0',
    'cieIfInIgnored' => '0',
    'cieIfInAbortErrs' => '0',
    'cieIfInputQueueDrops' => '0',
    'cieIfOutputQueueDrops' => '17217',
  ),

You’ll note the lack of ifHC counters for index 20.

And the SQL inserts for indexes 1 and 20:

Lines 1812-1856

Port Gi0/0/0: GigabitEthernet0/0/0 (1 / #515) ifHCInOctets
ifHCOutOctets
ifHCInUcastPkts
ifHCOutUcastPkts
ifHCInBroadcastPkts
ifHCOutBroadcastPkts
ifHCInMulticastPkts
ifHCOutMulticastPkts
ifHighSpeed (1000)
dot3Duplex VLAN =   Using ifDescr as ifAlias
GigabitEthernet0/0/0
 ifInOctets (18749792 B) 81520.834782609 Bps 230 secs
 
 ifOutOctets (6669649 B) 28998.473913043 Bps 230 secs
 
 ifInErrors (0 B) 0 Bps 230 secs
 
 ifOutErrors (0 B) 0 Bps 230 secs
 
 ifInUcastPkts (26822 B) 116.61739130435 Bps 230 secs
 
 ifOutUcastPkts (22318 B) 97.034782608696 Bps 230 secs
 
 ifInNUcastPkts (0 B) 0 Bps 230 secs
 
 ifOutNUcastPkts (0 B) 0 Bps 230 secs
 
 ifInDiscards (0 B) 0 Bps 230 secs
 
 ifOutDiscards (134 B) 0.58260869565217 Bps 230 secs
 
 ifInUnknownProtos (0 B) 0 Bps 230 secs
 
 ifInBroadcastPkts (0 B) 0 Bps 230 secs
 
 ifOutBroadcastPkts (0 B) 0 Bps 230 secs
 
 ifInMulticastPkts (8 B) 0.034782608695652 Bps 230 secs
 
 ifOutMulticastPkts (13 B) 0.056521739130435 Bps 230 secs
bps(652.17 Kbps/231.99 Kbps)bytes(17.88 MiB/6.36 MiB)pkts(116.62 pps/97.03 pps)SQL[UPDATE `ports` set `ifInOctets`=?,`ifInOctets_prev`=?,`ifInOctets_rate`=?,`ifInOctets_delta`=?,`ifOutOctets`=?,`ifOutOctets_prev`=?,`ifOutOctets_rate`=?,`ifOutOctets_delta`=?,`ifInErrors`=?,`ifInErrors_prev`=?,`ifInErrors_rate`=?,`ifInErrors_delta`=?,`ifOutErrors`=?,`ifOutErrors_prev`=?,`ifOutErrors_rate`=?,`ifOutErrors_delta`=?,`ifInUcastPkts`=?,`ifInUcastPkts_prev`=?,`ifInUcastPkts_rate`=?,`ifInUcastPkts_delta`=?,`ifOutUcastPkts`=?,`ifOutUcastPkts_prev`=?,`ifOutUcastPkts_rate`=?,`ifOutUcastPkts_delta`=?,`poll_time`=?,`poll_prev`=?,`poll_period`=? WHERE `port_id` = ? ["257266749330",257247999538,81520.8347826087,18749792,"94316401791",94309732142,28998.47391304348,6669649,"0",0,0,0,"0",0,0,0,"452317874",452291052,116.61739130434782,26822,"402286902",402264584,97.03478260869565,22318,1647887356,1647887126,230,515] 0.72ms]
 
SQL[UPDATE `ports_statistics` set `ifInNUcastPkts`=?,`ifInNUcastPkts_prev`=?,`ifInNUcastPkts_rate`=?,`ifInNUcastPkts_delta`=?,`ifOutNUcastPkts`=?,`ifOutNUcastPkts_prev`=?,`ifOutNUcastPkts_rate`=?,`ifOutNUcastPkts_delta`=?,`ifInDiscards`=?,`ifInDiscards_prev`=?,`ifInDiscards_rate`=?,`ifInDiscards_delta`=?,`ifOutDiscards`=?,`ifOutDiscards_prev`=?,`ifOutDiscards_rate`=?,`ifOutDiscards_delta`=?,`ifInUnknownProtos`=?,`ifInUnknownProtos_prev`=?,`ifInUnknownProtos_rate`=?,`ifInUnknownProtos_delta`=?,`ifInBroadcastPkts`=?,`ifInBroadcastPkts_prev`=?,`ifInBroadcastPkts_rate`=?,`ifInBroadcastPkts_delta`=?,`ifOutBroadcastPkts`=?,`ifOutBroadcastPkts_prev`=?,`ifOutBroadcastPkts_rate`=?,`ifOutBroadcastPkts_delta`=?,`ifInMulticastPkts`=?,`ifInMulticastPkts_prev`=?,`ifInMulticastPkts_rate`=?,`ifInMulticastPkts_delta`=?,`ifOutMulticastPkts`=?,`ifOutMulticastPkts_prev`=?,`ifOutMulticastPkts_rate`=?,`ifOutMulticastPkts_delta`=? WHERE `port_id` = ? [0,0,0,0,0,0,0,0,"0",0,0,0,"1594336",1594202,0.5826086956521739,134,0,0,0,0,"5",5,0,0,"11164",11164,0,0,"310565",310557,0.034782608695652174,8,"450741",450728,0.05652173913043478,13,515] 0.6ms]
 
2 updated

Lines 2535-2578

Port Di1: Dialer1 (20 / #302437) ifInOctets
ifOutOctets
ifInUcastPkts
ifOutUcastPkts
ifInBroadcastPkts
ifOutBroadcastPkts
ifInMulticastPkts
ifOutMulticastPkts
ifHighSpeed (1)
VLAN =  PROVIDER PPPoE []
 ifInOctets (0 B) 0 Bps 230 secs
 
 ifOutOctets (0 B) 0 Bps 230 secs
 
 ifInErrors (0 B) 0 Bps 230 secs
 
 ifOutErrors (0 B) 0 Bps 230 secs
 
 ifInUcastPkts (0 B) 0 Bps 230 secs
 
 ifOutUcastPkts (0 B) 0 Bps 230 secs
 
 ifInNUcastPkts (0 B) 0 Bps 230 secs
 
 ifOutNUcastPkts (0 B) 0 Bps 230 secs
 
 ifInDiscards (0 B) 0 Bps 230 secs
 
 ifOutDiscards (0 B) 0 Bps 230 secs
 
 ifInUnknownProtos (0 B) 0 Bps 230 secs
 
 ifInBroadcastPkts (0 B) 0 Bps 230 secs
 
 ifOutBroadcastPkts (0 B) 0 Bps 230 secs
 
 ifInMulticastPkts (0 B) 0 Bps 230 secs  
 
 ifOutMulticastPkts (0 B) 0 Bps 230 secs
bps(0 bps/0 bps)bytes(0 B/0 B)pkts(0 pps/0 pps)SQL[UPDATE `ports` set `ifInOctets`=?,`ifInOctets_prev`=?,`ifInOctets_rate`=?,`ifInOctets_delta`=?,`ifOutOctets`=?,`ifOutOctets_prev`=?,`ifOutOctets_rate`=?,`ifOutOctets_delta`=?,`ifInErrors`=?,`ifInErrors_prev`=?,`ifInErrors_rate`=?,`ifInErrors_delta`=?,`ifOutErrors`=?,`ifOutErrors_prev`=?,`ifOutErrors_rate`=?,`ifOutErrors_delta`=?,`ifInUcastPkts`=?,`ifInUcastPkts_prev`=?,`ifInUcastPkts_rate`=?,`ifInUcastPkts_delta`=?,`ifOutUcastPkts`=?,`ifOutUcastPkts_prev`=?,`ifOutUcastPkts_rate`=?,`ifOutUcastPkts_delta`=?,`poll_time`=?,`poll_prev`=?,`poll_period`=? WHERE `port_id` = ? [0,0,0,0,0,0,0,0,"0",0,0,0,"0",0,0,0,0,0,0,0,0,0,0,0,1647887356,1647887126,230,302437] 0.46ms]
 
SQL[UPDATE `ports_statistics` set `ifInNUcastPkts`=?,`ifInNUcastPkts_prev`=?,`ifInNUcastPkts_rate`=?,`ifInNUcastPkts_delta`=?,`ifOutNUcastPkts`=?,`ifOutNUcastPkts_prev`=?,`ifOutNUcastPkts_rate`=?,`ifOutNUcastPkts_delta`=?,`ifInDiscards`=?,`ifInDiscards_prev`=?,`ifInDiscards_rate`=?,`ifInDiscards_delta`=?,`ifOutDiscards`=?,`ifOutDiscards_prev`=?,`ifOutDiscards_rate`=?,`ifOutDiscards_delta`=?,`ifInUnknownProtos`=?,`ifInUnknownProtos_prev`=?,`ifInUnknownProtos_rate`=?,`ifInUnknownProtos_delta`=?,`ifInBroadcastPkts`=?,`ifInBroadcastPkts_prev`=?,`ifInBroadcastPkts_rate`=?,`ifInBroadcastPkts_delta`=?,`ifOutBroadcastPkts`=?,`ifOutBroadcastPkts_prev`=?,`ifOutBroadcastPkts_rate`=?,`ifOutBroadcastPkts_delta`=?,`ifInMulticastPkts`=?,`ifInMulticastPkts_prev`=?,`ifInMulticastPkts_rate`=?,`ifInMulticastPkts_delta`=?,`ifOutMulticastPkts`=?,`ifOutMulticastPkts_prev`=?,`ifOutMulticastPkts_rate`=?,`ifOutMulticastPkts_delta`=? WHERE `port_id` = ? [0,0,0,0,0,0,0,0,"0",0,0,0,"17217",17217,0,0,0,0,0,0,"0",0,0,0,"0",0,0,0,"0",0,0,0,"0",0,0,0,302437] 0.39ms]
 
1 updated

Which leads to the question, is there any way to have LibreNMS fall back to 32-bit counters for interfaces where 64-bit counters are unavailable?

$ ./validate.php 
====================================
Component | Version
--------- | -------
LibreNMS  | 21.12.0
DB Schema | 2021_11_29_165436_improve_ports_search_index (233)
PHP       | 7.4.27
Python    | 3.6.8
MySQL     | 10.5.13-MariaDB
RRDTool   | 1.7.1
SNMP      | 5.7.2
====================================

[OK]    Installed from package; no Composer required
[OK]    Database connection successful
[OK]    Database schema correct
[INFO]  Detected Dispatcher Service
[WARN]  Non-git install, updates are manual or from package

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.