Poller not writing MySQL or RRD

Help
1

Hi,

got a problem with my LibreNMS.
Looks like the poller is running regularaly, but not writing any kind of data to MySQL of RRD.
The WebUI also says that polling of devices hasn’t completed the last 15 minutes.

Output of validate:

bash-4.2$ ./validate.php
====================================
Component | Version
--------- | -------
LibreNMS  | 1.63-139-g73e2649
DB Schema | 2020_04_06_001048_the_great_index_rename (165)
PHP       | 7.2.14
Python    | 3.6.8
MySQL     | 5.5.65-MariaDB
RRDTool   | 1.4.8
SNMP      | NET-SNMP 5.7.2
====================================

[OK]    Composer Version: 1.10.6
[OK]    Dependencies up-to-date.
[OK]    Database connection successful
[OK]    Database schema correct
bash-4.2$

When running
./poller.php -h 4050-testfloor-5142-1

it results in following:

bash-4.2$ ./poller.php -h 4050-testfloor-5142-1
LibreNMS Poller
Starting polling run:

Hostname:    4050-testfloor-5142-1
Device ID:   80
OS:          ciena-sds
Resolved IP: 172.25.0.4



#### Load poller module core ####
Uptime:  180 days 17 hours 2 minutes 54 seconds

>> Runtime for poller module 'core': 0.0615 seconds with 46800 bytes
>> SNMP: [2/0.06s] MySQL: [0/0.00s] RRD: [1/0.00s]
#### Unload poller module core ####


#### Load poller module unix-agent ####

>> Runtime for poller module 'unix-agent': 0.0005 seconds with 8248 bytes
>> SNMP: [0/0.00s] MySQL: [0/0.00s] RRD: [1/0.00s]
#### Unload poller module unix-agent ####


#### Load poller module os ####
Generic :(
Location: TESTFLOOR [48.995968, 8.463728]
Hardware:
OS Version:
OS Features:
Serial:

>> Runtime for poller module 'os': 0.0355 seconds with 255936 bytes
>> SNMP: [1/0.03s] MySQL: [0/0.00s] RRD: [1/0.00s]
#### Unload poller module os ####


#### Load poller module ipmi ####

>> Runtime for poller module 'ipmi': 0.0010 seconds with 35416 bytes
>> SNMP: [0/0.00s] MySQL: [1/0.00s] RRD: [1/0.00s]
#### Unload poller module ipmi ####


#### Load poller module sensors ####

>> Runtime for poller module 'sensors': 0.0006 seconds with 1464 bytes
>> SNMP: [0/0.00s] MySQL: [1/0.00s] RRD: [1/0.00s]
#### Unload poller module sensors ####


#### Load poller module processors ####

>> Runtime for poller module 'processors': 0.0017 seconds with 58192 bytes
>> SNMP: [0/0.00s] MySQL: [1/0.00s] RRD: [1/0.00s]
#### Unload poller module processors ####


#### Load poller module mempools ####

>> Runtime for poller module 'mempools': 0.0006 seconds with 1792 bytes
>> SNMP: [0/0.00s] MySQL: [1/0.00s] RRD: [1/0.00s]
#### Unload poller module mempools ####


#### Load poller module storage ####

>> Runtime for poller module 'storage': 0.0006 seconds with 1792 bytes
>> SNMP: [0/0.00s] MySQL: [1/0.00s] RRD: [1/0.00s]
#### Unload poller module storage ####


#### Load poller module netstats ####
 ICMP IP IP-FORWARD SNMP TCP TCPHC UDP

>> Runtime for poller module 'netstats': 0.2841 seconds with 4120 bytes
>> SNMP: [7/0.28s] MySQL: [0/0.00s] RRD: [6/0.00s]
#### Unload poller module netstats ####


#### Load poller module hr-mib ####


>> Runtime for poller module 'hr-mib': 0.0331 seconds with 1552 bytes
>> SNMP: [1/0.03s] MySQL: [0/0.00s] RRD: [1/0.00s]
#### Unload poller module hr-mib ####


#### Load poller module ucd-mib ####

>> Runtime for poller module 'ucd-mib': 0.0971 seconds with 2864 bytes
>> SNMP: [3/0.10s] MySQL: [0/0.00s] RRD: [1/0.00s]
#### Unload poller module ucd-mib ####


#### Load poller module ipSystemStats ####
ipv4 ipv6

>> Runtime for poller module 'ipSystemStats': 0.0641 seconds with 17360 bytes
>> SNMP: [1/0.06s] MySQL: [0/0.00s] RRD: [3/0.00s]
#### Unload poller module ipSystemStats ####

SNMP [35/2.36s]: Get[11/0.38s] Getnext[4/0.13s] Walk[20/1.86s]
MySQL [159/0.08s]: Cell[49/0.02s] Row[-48/-0.02s] Rows[33/0.03s] Column[0/0.00s] Update[111/0.05s] Insert[14/0.01s] Delete[0/0.00s]
RRD [70/0.01s]: Update[70/0.01s]
bash-4.2$

I stripped some lines in the middle to shorten it a liitle bit.

Anyone an idea what goes wrong?

2

Run ./poller.php -d -h 4050-testfloor-5142-1

-d argument does debug the output, so you will see why is not writting anything.

3

Hi,

that’s the whole output:

https://pastebin.com/P0q6tQvU

I am not that deep into it to understand the whole output.

But are the “?” in the UPDATE state correct?

4

That output looks goods to me… go to the graph and click in show rrd command, check the output

5

Output of the RRD:

RRDTool Command

rrdtool graph /tmp/qM7kN8XnoFL1X9kh --alt-autoscale-max --rigid -E --start 1590306000 --end 1590392400 --width 1728 --height 312 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c SHADEB#EEEEEE00 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c FRAME#5e5e5e -c ARROW#5e5e5e -R normal -c FONT#000000 --font LEGEND:8:DejaVuSansMono --font AXIS:7:DejaVuSansMono --font-render-mode normal DEF:outoctets=4050-testfloor-5142-1/port-id2066.rrd:OUTOCTETS:AVERAGE DEF:inoctets=4050-testfloor-5142-1/port-id2066.rrd:INOCTETS:AVERAGE DEF:outoctets_max=4050-testfloor-5142-1/port-id2066.rrd:OUTOCTETS:MAX DEF:inoctets_max=4050-testfloor-5142-1/port-id2066.rrd:INOCTETS:MAX CDEF:octets=inoctets,outoctets,+ CDEF:doutoctets=outoctets,1,* CDEF:outbits=outoctets,8,* CDEF:outbits_max=outoctets_max,8,* CDEF:doutoctets_max=outoctets_max,1,* CDEF:doutbits=doutoctets,8,* CDEF:doutbits_max=doutoctets_max,8,* CDEF:inbits=inoctets,8,* CDEF:inbits_max=inoctets_max,8,* VDEF:totin=inoctets,TOTAL VDEF:totout=outoctets,TOTAL VDEF:tot=octets,TOTAL CDEF:dpercentile_outn=doutbits,1,* VDEF:dpercentile_outnp=dpercentile_outn,95,PERCENT CDEF:dpercentile_outnpn=doutbits,doutbits,-,dpercentile_outnp,1,*,+ VDEF:dpercentile_out=dpercentile_outnpn,FIRST VDEF:percentile_in=inbits,95,PERCENT VDEF:percentile_out=outbits,95,PERCENT COMMENT:‘bps Now Ave Max 95th %\n’ AREA:inbits_max#D7FFC788: AREA:inbits#90B04088: LINE:inbits#608720:‘In ’ GPRINT:inbits:LAST:%6.2lf%s GPRINT:inbits:AVERAGE:%6.2lf%s GPRINT:inbits_max:MAX:%6.2lf%s GPRINT:percentile_in:%6.2lf%s\n AREA:doutbits_max#E0E0FF88: AREA:doutbits#8080C088: LINE:doutbits#606090:‘Out’ GPRINT:outbits:LAST:%6.2lf%s GPRINT:outbits:AVERAGE:%6.2lf%s GPRINT:outbits_max:MAX:%6.2lf%s GPRINT:percentile_out:%6.2lf%s\n GPRINT:tot:‘Total %6.2lf%sB’ GPRINT:totin:’(In %6.2lf%sB’ GPRINT:totout:‘Out %6.2lf%sB)\l’ LINE1:percentile_in#aa0000 LINE1:dpercentile_out#aa0000 --imgformat=SVG --daemon unix:/run/rrdcached.sock
RRDTool Output

1809x409
OK u:0.04 s:0.01 r:0.05

But in the screenshot you can see that the values ane NaN

image
image1881×873 216 KB

6

Does that happens in all interfaces?

7

This happens on all graphs except the performance graph, as this seem to be no RRD.

I have seen this in rrdcached status:

Mai 25 11:35:25 ciena-librenms rrdcached[2013]: queue_thread_main: rrd_update_r (/rrd-data/ciena-librenms/services-117.rrd) failed with status -1. (/rrd-data/ciena-librenms/services-117.rrd: found extra data on update argument: 3:3:0:0)

8

Ok. I think it could be related to this https://github.com/librenms/librenms/pull/11636

9

Hm,

Issues also appear on normal Linux servers and juniper switches.

So I don’t think it’s related to the Ciena PR.

10