Centos 7 - RRDCached

kidn3ys · 30 August 2019 16:54

Hello,

I can’t seem to get rrdcached running on my Centos7 box. I currently have mariadb running on a separate box and am simply trying to get rrdcached running on the web front end after updating to 1.7.0. I followed the instructions that user Rick posted here: Distributed Poller not working - #13 by Ayeisha_Bean. More Specifically I did this:

As root:

rpm -e rrdtool rrdtool-devel rrdtool-perl
cd /opt ; wget RRDtool - Sorry this page does not exist
tar xvfz rrdtool-1.7.0.tar.gz
cd rrdtool-1.7.0
./configure
make ; make install
ln -s /opt/rrdtool-1.7.0/bin/rrdcached /usr/bin/rrdcached
ln -s /opt/rrdtool-1.7.0/bin/rrdtool rrdtool
ln -s /opt/rrdtool-1.7.0/bin/rrdcreate rrdcreate
ln -s /opt/rrdtool-1.7.0/bin/rrdupdate rrdupdate
ln -s /opt/rrdtool-1.7.0/bin/rrdcgi rrdcgi

cd /usr/bin

ln -s /opt/rrdtool-1.7.0/bin/rrdcached rrdcached
ln -s /opt/rrdtool-1.7.0/bin/rrdtool rrdtool
ln -s /opt/rrdtool-1.7.0/bin/rrdcreate rrdcreate
ln -s /opt/rrdtool-1.7.0/bin/rrdupdate rrdupdate
ln -s /opt/rrdtool-1.7.0/bin/rrdcgi rrdcgi

I created the /etc/systemd/system/rrdcached.service file per the instructions here: RRDCached - LibreNMS Docs

I then ran validate.php:

[root@lab-nms-web0 etc]# sudo su librenms
bash-4.2$ cd /opt/librenms
bash-4.2$ ./validate.php

Component Version

LibreNMS 1.54-64-ga0ba412

DB Schema 2019_07_09_150217_update_users_widgets_settings (140)

PHP 7.2.21

MySQL 5.5.60-MariaDB

RRDTool 1.7.0

SNMP NET-SNMP 5.7.2

====================================

[OK] Composer Version: 1.9.0
[OK] Dependencies up-to-date.
[OK] Database connection successful
[OK] Database schema correct

Before uncommenting “$config[‘rrdcached’] = “unix:/var/run/rrdcached.sock”;” in config.php I verified that graphs were still being drawn. Based on what I’ve read I need to make sure that the rrdcached service will start and then I can point my distributed pollers at the address/port running the service. However, I can’t seem to get it to start properly.

Attempting to start it results in the following output from “systemctl status rrdcached”

[root@lab-nms-web0 etc]# systemctl status rrdcached
● rrdcached.service - Data caching daemon for rrdtool
Loaded: loaded (/etc/systemd/system/rrdcached.service; enabled; vendor preset: disabled)
Active: failed (Result: timeout) since Fri 2019-08-30 09:21:32 PDT; 23min ago
Process: 17760 ExecStart=/bin/rrdcached -w 1800 -z 1800 -f 3600 -s librenms -U librenms -G librenms -B -R -j /var/tmp -l unix:/run/rrdcached.sock -t 4 -F -b /opt/librenms/rrd/ (code=exited, status=0/SUCCESS)
Main PID: 1986 (code=exited, status=0/SUCCESS)

Aug 30 09:20:01 lab-nms-web0.lab systemd[1]: Starting Data caching daemon for rrdtool…
Aug 30 09:20:01 lab-nms-web0.lab systemd[1]: PID file /run/rrdcached.pid not readable (yet?) after start.
Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: rrdcached.service start operation timed out. Terminating.
Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: Failed to start Data caching daemon for rrdtool.
Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: Unit rrdcached.service entered failed state.
Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: rrdcached.service failed.

And this output from journalctl -xe:

Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: rrdcached.service start operation timed out. Terminating.
Aug 30 09:21:32 lab-nms-web0.lab systemd[1]: Failed to start Data caching daemon for rrdtool.
– Subject: Unit rrdcached.service has failed
– Defined-By: systemd
– Support: systemd-devel Info Page

– Unit rrdcached.service has failed.

– The result is failed.
Aug 30 09:21:32 lab-nms-web0.lab.l systemd[1]: Unit rrdcached.service entered failed state.
Aug 30 09:21:32 lab-nms-web0.lab. systemd[1]: rrdcached.service failed.
Aug 30 09:21:32 lab-nms-web0.lab. polkitd[546]: Unregistered Authentication Agent for unix-process:17750:59003280 (system bus name :1.36739, object path /org/freedesktop/PolicyKit1/Authent
Aug 30 09:22:01 lab-nms-web0.lab. systemd[1]: Created slice User Slice of librenms.

After doing some digging I came across this post: RRDCached CENTOS 7 which determined that rrdcached wasn’t located in /usr/bin/ as the service definition file suggested.

In my case the output of ‘which rrdcached’ returned ‘/bin/rrdcached’ but updating the service definition file with this path didn’t resolve the issue.

I’ve also tried temporarily disabling SELinux but don’t see any change in the log output.

Does anyone have any other suggestions?
Thanks in advance.

Chris_Smith · 6 November 2019 13:58

Trying to set up a Centos 7 Instance here and hitting exactly the same thing - did anyone manage to resolve this issue?

alanbboyd · 6 November 2019 14:43

I missed this the first time - these are my notes for our Centos 7 install which is working fine. HTH.

RRDCached installation CentOS 7

This example is based on a fresh LibreNMS install, on a minimal CentOS 7.x installation.

Install the EPEL and update the repos and RRDtool.

sudo yum install epel-release
sudo yum update
sudo yum update rrdtool

Create the needed directories, set ownership and permissions.

sudo mkdir /var/run/rrdcached
sudo chown librenms:librenms /var/run/rrdcached
sudo chmod 755 /var/run/rrdcached

Create an rrdcached service for easy daemon management.

sudo touch /etc/systemd/system/rrdcached.service

Edit rrdcached.service and paste the example config:

[Unit]
Description=Data caching daemon for rrdtool
After=network.service

[Service]
Type=forking
PIDFile=/run/rrdcached.pid
ExecStart=/usr/bin/rrdcached -w 1800 -z 1800 -f 3600 -s librenms -U librenms -G librenms -B -R -j /var/tmp -l unix:/var/run/rrdcached/rrdcached.sock -t 4 -F -b /data/librenms/rrd/

[Install]
WantedBy=default.target

Reload the systemd unit files from disk, so it can recognize the newly created rrdcached.service. Enable the rrdcached.service on boot and start the service.

sudo systemctl daemon-reload
sudo systemctl enable --now rrdcached.service
sudo systemctl restart rrdcached
sudo systemctl status rrdcached.service

Edit /opt/librenms/config.php to include:

$config['rrdcached'] = "unix:/var/run/rrdcached/rrdcached.sock";

Check to see if the graphs are being drawn in LibreNMS. This might take a few minutes. After at least one poll cycle (5 mins), check the LibreNMS disk I/O performance delta. Disk I/O can be found under the menu Devices>All Devices>[localhost hostname]>Health>Disk I/O.

Depending on many factors, you should see the Ops/sec drop by ~30-40%.

To check if rrdcached is working:

sudo yum install socat

then:

while true; do clear ; echo STATS | sudo socat - /var/run/rrdcached/rrdcached.sock; sleep 1; done

Chris_Smith · 6 November 2019 15:04

Thanks for that -

In the interim I actually found a ‘fix’ - although i’m keen to hear if anything adverse will come of it

I simply commented out PIDFile=/run/rrdcached.pid in the service file and it ran fine after that, which is strange, I assume it must of defaulted back to something else

edit 1 # This is now working and populating graphing btw

alanbboyd · 6 November 2019 15:30

The rrdcached docs (RRDtool - rrdcached) state that if not specified, the default, $localstatedir/run/rrdcached.pid will be used.

cheers

Chris_Smith · 7 November 2019 08:48

Just incase anyone wonders whether it is worth implementing rrdcached…you can see the difference it made to my IO here

bluesky · 20 April 2021 10:35

Hi

Failing at the first step
yum install epel-release All good
yum update All good
yum update rrdtool No packages marked for update

I believe RRDCached needs 1.5 min

Sorry for to add CentOS 7.9 LibreNMS 21.4.0

alanbboyd · 20 April 2021 11:19

RRDtool 1.5 and above gives more features (see RRDCached - LibreNMS Docs) but it runs fine on 1.4.8.

I’m a little behind on LibreNMS on my production box (1.70.1-1-ga3635d0 - Thu Dec 03 2020) but it is also running on CentOS 7.9 (7.9.2009) with RRDtool 1.4.8 without any problems.

Please be aware my notes are for a system I installed a number of years ago.

HTH

bluesky · 20 April 2021 14:40

Hi

Think something is wrong with the setup.
RRDCache seems to run OK, but I have no graphs,
have “Error drawing graph” on all of them?

alanbboyd · 20 April 2021 14:51

Ok, things to check:

Are you seeing .rrd files created in /opt/librenms/rrd/<device_name>/, if so, do the permissions look ok, do you have librenms:librenms for owner:group?
Have a look at https://github.com/librenms/librenms/issues/3259, specifically the debug option on the image URL

bluesky · 20 April 2021 15:03

Thanks, sorry have to go view a apartment with my wife, will come back to this later.

bluesky · 21 April 2021 08:47

Hi

Got it working!!

Used the /etc/systemd/system/rrdcached.service from here
https://docs.librenms.org/Extensions/RRDCached/#rrdcached-installation-centos-7-or-8
and the config line
$config[‘rrdcached’] = “unix:/var/run/rrdcached.sock”;

system · 28 April 2021 08:47

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.

Component	Version
LibreNMS	1.54-64-ga0ba412
DB Schema	2019_07_09_150217_update_users_widgets_settings (140)
PHP	7.2.21
MySQL	5.5.60-MariaDB
RRDTool	1.7.0
SNMP	NET-SNMP 5.7.2
====================================

Centos 7 - RRDCached

[root@lab-nms-web0 etc]# sudo su librenms bash-4.2$ cd /opt/librenms bash-4.2$ ./validate.php

– Unit rrdcached.service has failed.

[root@lab-nms-web0 etc]# sudo su librenms
bash-4.2$ cd /opt/librenms
bash-4.2$ ./validate.php