Issue with LibreNMS Extension - Apache SNMP Extend

prad · 21 September 2016 13:42

Hi,

I installed LibreNMS in my network and it’s working great. Then I installed Apache snmp extend as out line in here. But it doesn’t work as expected. This is what I’ve done so far,

Installed the extension on one of my (LAMP) node.
Enabled Apache “status” module.
included /etc/snmp/apache-stats.py script to /etc/snmp/snmpd.conf, so it looks like this now - extend apache /etc/snmp/apache-stats.py
/etc/snmp/apache-stats.py script can be manually run and doesn’t give any error.
Added my linux node to LibreNMS and enabled “Apache” under “Applications” tab.

However, my graphs under “Apps” are not populated due to some reason. I just ran following snmpwalk command from my linux node to check whether my snmp extend work,

snmpwalk -v 1 -c TEST localhost NET-SNMP-EXTEND-MIB::nsExtendObjects

And output looks okay,

NET-SNMP-EXTEND-MIB::nsExtendNumEntries.0 = INTEGER: 3 NET-SNMP-EXTEND-MIB::nsExtendCommand."apache" = STRING: /etc/snmp/apache-stats.py NET-SNMP-EXTEND-MIB::nsExtendCommand."qSize_1" = STRING: /bin/bash NET-SNMP-EXTEND-MIB::nsExtendCommand."puppetRunTime.sh" = STRING: /bin/bash NET-SNMP-EXTEND-MIB::nsExtendArgs."apache" = STRING: NET-SNMP-EXTEND-MIB::nsExtendArgs."qSize_1" = STRING: -c \"/opt/scripts/qSize.sh\" NET-SNMP-EXTEND-MIB::nsExtendArgs."puppetRunTime.sh" = STRING: -c \"sudo /opt/scripts/puppetRunTime.sh\" NET-SNMP-EXTEND-MIB::nsExtendInput."apache" = STRING: NET-SNMP-EXTEND-MIB::nsExtendInput."qSize_1" = STRING: NET-SNMP-EXTEND-MIB::nsExtendInput."puppetRunTime.sh" = STRING: NET-SNMP-EXTEND-MIB::nsExtendCacheTime."apache" = INTEGER: 5 NET-SNMP-EXTEND-MIB::nsExtendCacheTime."qSize_1" = INTEGER: 5 NET-SNMP-EXTEND-MIB::nsExtendCacheTime."puppetRunTime.sh" = INTEGER: 5 NET-SNMP-EXTEND-MIB::nsExtendExecType."apache" = INTEGER: exec(1) NET-SNMP-EXTEND-MIB::nsExtendExecType."qSize_1" = INTEGER: exec(1) NET-SNMP-EXTEND-MIB::nsExtendExecType."puppetRunTime.sh" = INTEGER: exec(1) NET-SNMP-EXTEND-MIB::nsExtendRunType."apache" = INTEGER: run-on-read(1) NET-SNMP-EXTEND-MIB::nsExtendRunType."qSize_1" = INTEGER: run-on-read(1) NET-SNMP-EXTEND-MIB::nsExtendRunType."puppetRunTime.sh" = INTEGER: run-on-read(1) NET-SNMP-EXTEND-MIB::nsExtendStorage."apache" = INTEGER: permanent(4) NET-SNMP-EXTEND-MIB::nsExtendStorage."qSize_1" = INTEGER: permanent(4) NET-SNMP-EXTEND-MIB::nsExtendStorage."puppetRunTime.sh" = INTEGER: permanent(4) NET-SNMP-EXTEND-MIB::nsExtendStatus."apache" = INTEGER: active(1) NET-SNMP-EXTEND-MIB::nsExtendStatus."qSize_1" = INTEGER: active(1) NET-SNMP-EXTEND-MIB::nsExtendStatus."puppetRunTime.sh" = INTEGER: active(1) NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."apache" = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."qSize_1" = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutput1Line."puppetRunTime.sh" = STRING: 12.149 NET-SNMP-EXTEND-MIB::nsExtendOutputFull."apache" = STRING: U 1926144 U U U U U U U 7 0 0 1 0 0 0 0 0 0 248 NET-SNMP-EXTEND-MIB::nsExtendOutputFull."qSize_1" = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutputFull."puppetRunTime.sh" = STRING: 12.149 NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."apache" = INTEGER: 20 NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."qSize_1" = INTEGER: 1 NET-SNMP-EXTEND-MIB::nsExtendOutNumLines."puppetRunTime.sh" = INTEGER: 1 NET-SNMP-EXTEND-MIB::nsExtendResult."apache" = INTEGER: 0 NET-SNMP-EXTEND-MIB::nsExtendResult."qSize_1" = INTEGER: 1 NET-SNMP-EXTEND-MIB::nsExtendResult."puppetRunTime.sh" = INTEGER: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".1 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".2 = STRING: 1926144 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".3 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".4 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".5 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".6 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".7 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".8 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".9 = STRING: U NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".10 = STRING: 7 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".11 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".12 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".13 = STRING: 1 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".14 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".15 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".16 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".17 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".18 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".19 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."apache".20 = STRING: 248 NET-SNMP-EXTEND-MIB::nsExtendOutLine."qSize_1".1 = STRING: 0 NET-SNMP-EXTEND-MIB::nsExtendOutLine."puppetRunTime.sh".1 = STRING: 12.149

Other than these graphs under “Apps”, rest of the graphs and features are working nicely. Appreciate if someone could give some direction on this.

Thanks,

Prad

laf · 21 September 2016 13:43

Run ./validate.php and pastebin the output

prad · 21 September 2016 13:50

Hi Laf,

Thanks for the quick reply. Ran the validate.php and output is,

====================================
Version info:
[WARN] Your install is out of date: d1ae83a378a6016a6619f6f13d49209334baf2c0 (Sun, 11 Sep 2016 15:23:16 +0000)
[WARN] Your local git contains modified files, this could prevent automatic updates.
Modified files:
…

list large number of files…

…

DB Schema: 118
PHP: 7.0.8-0ubuntu0.16.04.2
MySQL: 5.7.13-0ubuntu0.16.04.2
RRDTool: 1.5.5
SNMP: NET-SNMP version: 5.7.3

====================================
[OK] Database connection successful

I commented out $config[‘update’] = 0; from my config.php to avoid daily updates.

Regards,

Prad

prad · 21 September 2016 14:14

Hi Laf,

Complete validate.php out put is here - http://pastebin.com/ATHKL2x0

Thanks,

Prad

laf · 21 September 2016 17:58

You really should fix the permissions and see what happens.

prad · 21 September 2016 22:49

Hi Laf,

Fixed the permission issue. Issue is still there
Outcome of the validate.php is here.

Thanks,

Prad

laf · 22 September 2016 21:17

That output still shows a load of file permissions

prad · 22 September 2016 22:18

Hi Laf,

I’m sorry, other than git warning ("[WARN] Your local git contains modified files, this could prevent automatic updates.") I’m not seen any wrong in the out put

Just wondering whether you’re looked at my previous out put by any chance. The out put after fixing permission is here - http://pastebin.com/px9TKfRu

Thanks,

Prad

prad · 22 September 2016 22:19

In fact, I changed the permission to 777 but that didn’t solve the problem either. Could it be something else?

Regards,

laf · 23 September 2016 19:15

Sorry it’s modified files, you entire git looks broken.

Try git stash save and re-run validate.php

prad · 25 September 2016 21:36

Hi Laf,

Thanks for that. It fixed my git issue. So validate.php gives me some positive out come,

root@librenms:/opt/librenms# ./validate.php 
====================================
Version info:
Commit SHA: 8b68429a0619c627256c31299765ffddb2bdd9dd
DB Schema: 136
PHP: 7.0.8-0ubuntu0.16.04.2
MySQL: 5.7.13-0ubuntu0.16.04.2
RRDTool: 1.5.5
SNMP: NET-SNMP version: 5.7.3

====================================
[OK]      Database connection successful
root@librenms:/opt/librenms#

However, original problem is still there - blank graphs and it says “Error Drawing Graph”. I just click on one of these graphs and click “show rrd command”. Command looks okay to me,

RRDTool Command
rrdtool graph /tmp/fkvLLacDuuQPzGiP.png -l 0 -E --start 1474752300 --end 1474838700 --width 1680.3 --height 495 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c SHADEB#EEEEEE00 -c FONT#000000 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c FRAME#5e5e5e -c ARROW#5e5e5e -R normal --font LEGEND:8:DejaVuSansMono --font AXIS:7:DejaVuSansMono --font-render-mode normal -l 0 -E --start 1474752300 --end 1474838700 --width 1680.3 --height 495 -c BACK#EEEEEE00 -c SHADEA#EEEEEE00 -c SHADEB#EEEEEE00 -c FONT#000000 -c CANVAS#FFFFFF00 -c GRID#a5a5a5 -c MGRID#FF9999 -c FRAME#5e5e5e -c ARROW#5e5e5e -R normal --font LEGEND:8:DejaVuSansMono --font AXIS:7:DejaVuSansMono --font-render-mode normal DEF:kbyte_o=/opt/librenms/rrd/lamptest.taitradio.net/app-apache-46.rrd:kbyte:AVERAGE CDEF:kbyte=kbyte_o,8,* AREA:kbyte#CDEB8B: COMMENT:'Kbps Now Ave Max\n' LINE1.25:kbyte#006600:' ' GPRINT:kbyte:LAST:%6.2lf%s GPRINT:kbyte:AVERAGE:%6.2lf%s GPRINT:kbyte:MAX:%6.2lf%s\n COMMENT:\n

RRDTool Output
1761x564 OK u:0.17 s:0.03 r:0.20

Regards,

Prad

prad · 25 September 2016 21:40

BTW, I just installed MySQL script as doc suggested here.
Again, graphs are blank with “Error Drawing Graph” message. However, I can manually run mysql script,

root@lamptest:/opt/librenms-agent# /usr/lib/check_mk_agent/local/mysql
<<<mysql>>>
a0:54
a1:5
a2:0
a3:0
a4:-1
a5:-1
a6:-1
a7:-1
a8:-1
a9:-1
aa:-1
ab:-1
ac:-1
ad:-1
ae:-1
af:-1
ag:-1
ah:-1
ai:-1
aj:-1
ak:-1
al:-1
am:-1
an:-1
ao:-1
ap:-1
aq:-1
ar:-1
as:-1
at:-1
au:-1
av:-1
aw:-1
ax:-1
ay:-1
az:-1
b0:-1
b1:-1
b2:-1
b3:-1
b4:0
b5:2939
b6:0
b7:36
b8:176
b9:5740
ba:431
bb:1024
bc:431
bd:0
be:2
bf:2
bg:0
bh:1
bi:1
bj:2
bk:1
bl:151
bm:8
bn:876
bo:-1
bp:-1
bq:-1
br:-1
bs:0
bt:1
bu:16760152
bv:0
bw:0
bx:0
by:23
bz:0
c0:1
c1:16777216
c2:14973
c3:0
c4:0
c5:1439
c6:0
c7:0
c8:0
c9:0
ca:0
cb:0
cc:0
cd:27
ce:0
cf:0
cg:0
ch:6705
ci:0
cj:0
ck:0
cl:18
cm:6207
cn:2283
co:6
cp:153774354
cq:735094
cr:16777216
cs:-1
ct:-1
cu:-1
cv:-1
cw:32768
cx:0
cy:0
cz:-1
d0:-1
d1:-1
d2:-1
d3:-1
d4:-1
d5:-1
d6:-1
d7:-1
d8:-1
d9:-1
da:-1
db:-1
dc:-1
dd:-1
de:-1
df:-1
dg:-1
dh:1
di:395
dj:0
dk:0
dl:0
dm:2555
dn:2562
do:7768
dp:0
dq:0
dr:3348087
ds:0
dt:0
du:0
dv:0
dw:741134
dx:-1
dy:-1
dz:-1
e0:-1
e1:-1
e2:-1
e3:-1
e4:-1
e5:-1
e6:-1
e7:-1
e8:-1
e9:-1
ea:-1
eb:-1
ec:-1
ed:-1
ee:-1
ef:-1
eg:0
eh:3061760
ei:16777216
ej:0
ek:0
root@lamptest:/opt/librenms-agent#

What could be the issue?

Regards,

Prad

laf · 29 September 2016 11:43

Check you have the latest agent files and that your install is fully up to date.