Current setup:
Polling communicates as expected with NMS-DB, RRDCached, Memcached.
I’ve noticed however that the pollers (NMS-Poll-A,NMS-Poll-B) saved RRD files locally under /opt/librenms/rrd … even though its configured to use NMS-DB … is this correct functioning?
Hi @jongalli
I am not at all familiar with Distributed Poller, but can you confirm RRDCached is operationnal on all servers and communicating well ?
If it is, then shared storage for RRD should not be necessary.
Hi PipoCanaja,
In short: Yes can confirm it is operational on all servers as can graphs are being updated… see config below - can provide pastebin urls for debugging if required.
is there anything below i may be missing something??
** NMS-DB **
/etc/systemd/system/rrdcached.service
ExecStart=/usr/sbin/rrdcached -p /u01/rrdcached/run/rrdcached.pid -w 1800 -z 900 -f 3600 -s librenms -U librenms -G librenms -B -R -j /u01/rrdcached/journaling -l 0:42217 -t 8 -F -b /u01/rrdcached/librenms/
[@nms-db ~]$ sudo netstat -an | grep 42217
tcp 0 0 0.0.0.0:42217 0.0.0.0:* LISTEN
/etc/sysconfig/memcached
PORT=“11211”
USER=“memcached”
MAXCONN=“1024”
CACHESIZE=“64”
OPTIONS=""
[@nms-db ~]$ sudo netstat -an | grep 11211
tcp 0 0 0.0.0.0:11211 0.0.0.0:* LISTEN
NMS-Poll A Configuration
/opt/librenms/config.php
$config[‘rrdtool_version’]=‘1.6.0’;
$config[‘distributed_poller_name’] = ‘nms-dchc.dc.local’;
$config[‘distributed_poller_group’] = ‘0,2,3’;
$config[‘distributed_poller_memcached_host’] = ‘nms-memcached.dc.local’;
$config[‘distributed_poller_memcached_port’] = 11211;
$config[‘distributed_poller’] = true;
$config[‘rrdcached’]=‘nms-rrdcached.dc.local:42217’;
NMS-Poll B Configuration
/opt/librenms/config.php
$config[‘rrdtool_version’]=‘1.6.0’;
$config[‘distributed_poller_name’] = ‘nms-itld.dc.local’;
$config[‘distributed_poller_group’] = ‘1’;
$config[‘distributed_poller_memcached_host’] = ‘nms-memcached.dc.local’;
$config[‘distributed_poller_memcached_port’] = 11211;
$config[‘distributed_poller’] = true;
$config[‘rrdcached’]=‘nms-rrdcached.dc.local:42217’;
rrdcached looks off in the poller devices. They should be running and communicating with the one on DB server in order to have a working setup, I would say.
Is there any supporting documentation to suggest what rrdcached service script, in particular ‘rrdcached -l 0:42217’ should be in place on the pollers?
Reason I ask is that the rrdcached listening address (on each poller) cannot be bound to another server IP (such as NMS-RRDCACHED.
This above my experience of rrdcached 
NMS-Poll A Configuration
/etc/systemd/system/rrdcached.service
ExecStart=/usr/sbin/rrdcached -w 1800 -z 900 -f 3600 -s librenms -U librenms -G librenms -B -R -j /var/tmp -l 0:42217 -t 8 -F -b /opt/librenms/rrd/
NMS-Poll B Configuration
/etc/systemd/system/rrdcached.service
ExecStart=/usr/sbin/rrdcached -w 1800 -z 900 -f 3600 -s librenms -U librenms -G librenms -B -R -j /var/tmp -l 0:42217 -t 8 -F -b /opt/librenms/rrd/
The query however remains regarding whether pollers should be saving RRD files locally under /opt/librenms/rrd … as this is the current case. Removing the folder /opt/librenms/rrd displays an error in when running the rrdtool cmd (generated by librenms GUI) on the poller(s).
Any suggestions on what the expected RRDservice configuration on Poller servers should be set as when working with a distributed poller setup?
@jongalli
did you manage to run this?
Reading the docs I thought that we need one, central, rrdcached server listening on a tcp port and all pollers just connecting to this rrdcached server. The pollers shouldn’t need rrdcached service running.
So, did you manage to run this setup?
all poller servers (as it stands) seem to create rrd folders locally with the rrd files saved on the rrdcached server.
am unsure as to what function within librenms is actually calling the rrdservice on the poller servers.
I found that if you set in /opt/librenms/config.php
$config[‘rrdtool_version’] = ‘1.5.5’;
Then the rrdtool is using only the remote rrdcached service and not touching the local storage.
Of course, I also have this line:
$config[‘rrdcached’] = “MY_RRDCACHED_SERVER_HOSTNAME:42217”;
Hi Danis
For completeness, I have similar:
$config[‘rrdtool_version’]=‘1.7.1’;
$config[‘distributed_poller_name’] = ‘nms-p1.server’;
$config[‘distributed_poller_memcached_host’] = “nms-memcached.server”;
$config[‘distributed_poller_memcached_port’] = 11211;
$config[‘distributed_poller’] = true;
$config[‘rrdcached’]=‘nms-rrdcached.server:42217’;
I realize this is sort of an old thread, but I did not see a solution posted and I have struggled with this issue, so…
I believe, the RRD files are written locally and then transmitted to rrdcached. The issue I had was I though all the data storage would take place on the primary server and the RRD files would be transmitted directly to the primary server, but it would seem this is not the case. My remote poller had a 20 GB HDD and kept filling up fast and I didn’t get graphs for the devices polled by that poller. In the end, I put a second HDD of 100GB mounted at /rrd and created a symlink from /opt/librenms/rrd to /rrd and changed ownership on both locations to librenms:librenms. Now everything is all rainbows and butterflies; I have graphs for all my devices and everything is getting polled.
I hope that helps someone having the same issues save a little time and headache.