Validate Errors, when New RRD Created

Hi,

I have seen this several times, finally getting a chance to report it - as it seems to happen quite reliably. I find that when LibreNMS creates a new RRD file (in this case for SMART RRD), I get permission errors for the new file(s), shown using validate.

[FAIL]  Some folders have incorrect file permissions, this may cause issues.
	[FIX]: 
	sudo chown -R librenms:librenms /opt/librenms
	sudo setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
	sudo chmod -R ug=rwX /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
	Files:
	 /opt/librenms/rrd/linuxServer/app-smart-59-.rrd

Thoughts?

Thanks!

1 Like

OK, pulling my hair out on this … :-(. Got the errors below (again, this is happening all the time) => checked the noted file before and after. Any idea why it’s being created with the wrong mode? I know I’m not creating the file … LOL!

[FAIL] Some folders have incorrect file permissions, this may cause issues.
[FIX]:
sudo chown -R librenms:librenms /opt/librenms
sudo setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
sudo chmod -R ug=rwX /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
Files:
/opt/librenms/rrd/wap-study/poller-perf-bgp-peers.rrd

And,

Before …
-rw-r–r-- 1 librenms librenms 125000 Jul 7 17:40 /opt/librenms/rrd/wap-study/poller-perf-bgp-peers.rrd

After …
-rw-rw-r-- 1 librenms librenms 125000 Jul 7 17:40 /opt/librenms/rrd/wap-study/poller-perf-bgp-peers.rrd

Thoughts?

Thanks!

OK, still having this issue :frowning_face:. I did some more searching, came across the link below, but the solution isn’t working for me (and I did confirm, ACL’s are enabled, running Ubuntu 20.04),

Any time a new rrd file gets created (like adding a new device, sensor, etc), the permissions are not set correctly, and I get these errors (so running these 3 commands quite often). An example of the permissions are below (“old” and “new” files … old have -rwxrwxr-- permissions, new are -rw-r–r–). Thoughts?

-rwxrwxr--   1 librenms librenms  125000 Jun  6 10:25 ucd_ssRawContexts.rrd*
-rwxrwxr--   1 librenms librenms  125000 Jun  6 10:25 ucd_ssRawInterrupts.rrd*
-rw-rw-r--   1 librenms librenms  125000 Jun  6 10:25 ucd_ssRawSwapIn.rrd
-rw-rw-r--   1 librenms librenms  125000 Jun  6 10:25 ucd_ssRawSwapOut.rrd
-rwxrwxr--   1 librenms librenms  125000 Jun  6 10:25 uptime.rrd*
-rw-r--r--   1 librenms librenms  125000 Jun  6 10:25 wireless-sensor-clients-openwrt-1.rrd
-rw-r--r--   1 librenms librenms  125000 Jun  6 10:25 wireless-sensor-clients-openwrt-2.rrd
-rw-r--r--   1 librenms librenms  125000 Jun  6 10:25 wireless-sensor-clients-openwrt-3.rrd

Thanks!

Having this problem too; i don’t think its every time, but pretty consistently when you add a new device and RRD files are created, the files are not getting owned by librenms. I believe they are getting created under www-data. This causes errors in Validate.php, as well as prevents data from being recorded in those graphs.

I think it comes down to doing a manual discovery run after adding the device. Perhaps a check should be done to see if a discovery is running as the user specified in config.php? There isn’t really a way to square that circle, short of waiting for Cron to run the discovery as the librenms user, or running libre under the www-data user (assuming Ubuntu as the OS). Open to other ideas if the Devs have any @murrant or @laf?

Often this is from people going to the debug capture and running things from there. The purpose is not to be a way to manually run discover/poll. There is a rediscover button on the device settings, click that :wink:

Also, on newly added devices they are automatically discovered. Just give them 5 minutes. If you don’t like waiting that long, run the dispatcher service.

If I could come up with a way for the debug to not actually make changes to the device, that would be ideal.

Well, i’m thinking a simple existence check on the manual discovery script; if you’re going to create a file that doesn’t exist, and you’re not running as $config[user], bail and toss a message saying what you just said: wait for the cron’d discovery process, or go kick it manually from the CLI.

Save people from themselves; even old hands who should know better but have a hard time breaking habits.

In my case I’ve added the permissions fix to a script in /usr/local/bin/ to run every 5 minutes via cron. Pretty blunt and/or brutish, but I’m getting too old to be subtle and my “empathy for the machine” dwindles each day.

Also, you can run the web server php as the librenms user :smiley:

So the expectation is that this will “clean up” the device, no longer the file permission errors (for that device)? If that’s the case, I’ll give it a go, let you know if I see further issues.

Thanks!

No, the expectation is that using device->capture causes the issue.

Right, or manual (CLI) discover.php, right?

I thought you were saying that Rediscover undoes the problem, no? Trying to figure out how to “repair” it.

Thanks!

Run the chown commands listed by validate.php

I have! Probably 20 times now - that’s the part I’m trying to figure out. Wondering now if discovery ran in between, messed it up. Let me keep an eye out for that.

Thanks!

Don’t use the capture and it won’t break it again :smiley:

I won’t! And I don’t actually… :laughing:. But does discovery.php (CLI) also cause this? That would be my issue.

Thanks!

no, unless you run it as root or any user other than librenms

1 Like

Awesome - thanks!

FYI, this just happened again … and I haven’t run discovery, polling, etc. manually :laughing:. Something seems to be “broken”, I think when a new file is created? Thoughts?

Here is the output from validate,

[FAIL]  Some folders have incorrect file permissions, this may cause issues.
        [FIX]:
        sudo chown -R librenms:librenms /opt/librenms
        sudo setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
        sudo chmod -R ug=rwX /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/
        Files:
         /opt/librenms/rrd/linuxServer/ucd_diskio-loop27.rrd

And if it helps, I did check … yep, this file is new (created just recently),

ls -alF /opt/librenms/rrd/linuxServer/ucd_diskio-loop27.rrd
-rw-r--r-- 1 librenms librenms 494576 Jun 25 06:30 /opt/librenms/rrd/linuxServer/ucd_diskio-loop27.rrd

Thanks!

FYI, another warning today - and I ran those noted commands (yesterday), definitely no manual discovery or poller in between!

Thanks.

Are you using rrdcached? Settings wrong there?