We had some maintenance last night and part of that was to reboot some servers. I noticed when I came in this morning that there were some issues that I had to clean up and get things working again. It seemed like libre was working properly but then I noticed that the processor and total traffic graphs were in error on all the devices.
Since this new environment was still being tested, I went back to our original setup and noticed the graphs all seemed okay. I showed the rrd command on the bad server and it had this:
RRDTool Command
rrdtool graph /tmp/uocW6xLz9uTz0oLs -g --alt-autoscale-max --rigid -E --start 1500405600 --end 1500492000 --width 1728 --height 320 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c SHADEB#EEEEEE00 -c FONT#000000 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c FRAME#5e5e5e -c ARROW#5e5e5e -R normal --font LEGEND:8:DejaVuSansMono --font AXIS:7:DejaVuSansMono --font-render-mode normal COMMENT:’ Now Avg Max’ COMMENT:‘\n’ CDEF:inB= CDEF:outB= CDEF:octets=inB,outB,+ CDEF:doutB=outB,-1,* CDEF:inbits=inB,8,* CDEF:outbits=outB,8,* CDEF:bits=inbits,outbits,+ CDEF:doutbits=doutB,8,* VDEF:95thin=inbits,95,PERCENT VDEF:95thout=outbits,95,PERCENT CDEF:d95thoutn=doutbits,-1,* VDEF:d95thoutn95=d95thoutn,95,PERCENT CDEF:d95thoutn95n=doutbits,doutbits,-,d95thoutn95,-1,*,+ VDEF:d95thout=d95thoutn95n,FIRST VDEF:totin=inB,TOTAL VDEF:avein=inbits,AVERAGE VDEF:totout=outB,TOTAL VDEF:aveout=outbits,AVERAGE VDEF:tot=octets,TOTAL COMMENT:’ \n’ HRULE:999999999999999#FFFFFF:‘Total In ’ GPRINT:inbits:LAST:%6.2lf%s GPRINT:inbits:AVERAGE:%6.2lf%s GPRINT:inbits:MAX:%6.2lf%s GPRINT:totin:%6.2lf%sB COMMENT:’\n’ HRULE:999999999999990#FFFFFF:’ Out’ GPRINT:outbits:LAST:%6.2lf%s GPRINT:outbits:AVERAGE:%6.2lf%s GPRINT:outbits:MAX:%6.2lf%s GPRINT:totout:%6.2lf%sB COMMENT:‘\n’ HRULE:999999999999990#FFFFFF:’ Agg’ GPRINT:bits:LAST:%6.2lf%s GPRINT:bits:AVERAGE:%6.2lf%s GPRINT:bits:MAX:%6.2lf%s GPRINT:tot:%6.2lf%sB COMMENT:‘\n’ HRULE:0#999999 --imgformat=SVG
RRDTool Output
ERROR: can not parse an empty rpn expression
I noticed it wasn’t actually pointing to anything so I double checked our old environment and sure enough the new environment wasn’t actually querying and rrd file unlike the new one. I could run the good rrdtool command from the old environment on the new environment and it showed no errors. And as far as I can tell its only affecting those two graphs the Processor and the Total Traffic graphs.
Example of the good command from the old environment:
RRDTool Command
rrdtool graph /tmp/UavLUNr2u1HVtdYu -g --alt-autoscale-max --rigid -E --start 1500405600 --end 1500492000 --width 1728 --height 320 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c SHADEB#EEEEEE00 -c FONT#000000 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c FRAME#5e5e5e -c ARROW#5e5e5e -R normal --font LEGEND:8:DejaVuSansMono --font AXIS:7:DejaVuSansMono --font-render-mode normal COMMENT:’ Now Avg Max’ COMMENT:‘\n’ DEF:inB0=/opt/librenms/rrd/ipam-dev.domain.tld/port-id1368720.rrd:INOCTETS:AVERAGE DEF:outB0=/opt/librenms/rrd/ipam-dev.domain.tld/port-id1368720.rrd:OUTOCTETS:AVERAGE CDEF:octets0=inB0,outB0,+ CDEF:inbits0=inB0,8,* CDEF:outbits0=outB0,8,* CDEF:outbits0_neg=outbits0,-1,* CDEF:bits0=inbits0,outbits0,+ VDEF:totinB0=inB0,TOTAL VDEF:totoutB0=outB0,TOTAL VDEF:tot0=octets0,TOTAL AREA:inbits0#B6D14B:‘eno167800 In ’ GPRINT:inbits0:LAST:%6.2lf%s GPRINT:inbits0:AVERAGE:%6.2lf%s GPRINT:inbits0:MAX:%6.2lf%s COMMENT:’\n’ HRULE:999999999999999#A0A0E5:’ Out’ GPRINT:outbits0:LAST:%6.2lf%s GPRINT:outbits0:AVERAGE:%6.2lf%s GPRINT:outbits0:MAX:%6.2lf%s COMMENT:‘\n’ CDEF:inB=inB0,UN,0,inB0,IF CDEF:outB=outB0,UN,0,outB0,IF CDEF:octets=inB,outB,+ CDEF:doutB=outB,-1,* CDEF:inbits=inB,8,* CDEF:outbits=outB,8,* CDEF:bits=inbits,outbits,+ CDEF:doutbits=doutB,8,* VDEF:95thin=inbits,95,PERCENT VDEF:95thout=outbits,95,PERCENT CDEF:d95thoutn=doutbits,-1,* VDEF:d95thoutn95=d95thoutn,95,PERCENT CDEF:d95thoutn95n=doutbits,doutbits,-,d95thoutn95,-1,*,+ VDEF:d95thout=d95thoutn95n,FIRST VDEF:totin=inB,TOTAL VDEF:avein=inbits,AVERAGE VDEF:totout=outB,TOTAL VDEF:aveout=outbits,AVERAGE VDEF:tot=octets,TOTAL COMMENT:’ \n’ HRULE:999999999999999#FFFFFF:‘Total In ’ GPRINT:inbits:LAST:%6.2lf%s GPRINT:inbits:AVERAGE:%6.2lf%s GPRINT:inbits:MAX:%6.2lf%s GPRINT:totin:%6.2lf%sB COMMENT:’\n’ HRULE:999999999999990#FFFFFF:’ Out’ GPRINT:outbits:LAST:%6.2lf%s GPRINT:outbits:AVERAGE:%6.2lf%s GPRINT:outbits:MAX:%6.2lf%s GPRINT:totout:%6.2lf%sB COMMENT:‘\n’ HRULE:999999999999990#FFFFFF:’ Agg’ GPRINT:bits:LAST:%6.2lf%s GPRINT:bits:AVERAGE:%6.2lf%s GPRINT:bits:MAX:%6.2lf%s GPRINT:tot:%6.2lf%sB COMMENT:‘\n’ AREA:outbits0_neg#A0A0E5: HRULE:0#999999 --imgformat=SVG
RRDTool Output
1809x359
OK u:0.03 s:0.02 r:0.04
-bash-4.2$ php validate.php
====================================
Component | Version
--------- | -------
LibreNMS | 1.29-103-gc592f90
DB Schema | 199
PHP | 7.0.18
MySQL | 5.5.47-MariaDB
RRDTool | 1.6.0
SNMP | NET-SNMP 5.7.2
====================================
[OK] Database connection successful
[OK] Database schema correct
[WARN] Your local git contains modified files, this could prevent automatic updates.
Modified files:
config.php.default
Any ideas would be appreciated!
Thanks!