Pruning syslog items from LibreNMS DBS

Help
1

Hi Team,

Last week I added a Sonicwall to LibreNMS monitoring and allowed for syslogs from the firewall to go to LibreNMS.

It’s been running for a few days now and the disk of the OS already needs to be expanded because of how many syslog entries have been ingested.

When I got to the device, then to logs, then to syslog I now get a “504 Gateway Time-out” for nginx.

So my questions are:

  1. How can I stop the 504 Gateway time-out?

  2. How can I prune the syslog entries in the DB for this device?

  3. How can I get LibreNMS to auto-prune the syslogs from the DB, say for every 2 days?

Thanks,

Michael.

2

For that part use this https://docs.librenms.org/#Support/Configuration/#cleanup-options

3

For this part, you need to configure what is being sent by your Sonicwall you could try setup a priority on the sys logging on the SonicWall.

Also, check this https://docs.librenms.org/#Support/FAQ/#faq6
your php could need more memory.

4

Hi Kevin. Yes you were spot on with the PHP memory limit issue. It was set to 128M (default - I use the CentOS VM download for this server), increased to 256M and all OK.

Thank you.

1 Like
5

Hi Kevin,

With the cleanup options, I added this:

$config['syslog_purge'] = 1;

to:

/opt/librenms/config.php

then waited a day for the daily.sh run to go through, but didn’t seem to go through as the daily.log shows:

Cleaning up DB
Refreshing alert rules queries
Refreshing OS cache
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Syslog cleared for entries over 30 days 1000 limit
Eventlog cleared for entries over 30 days
Authlog cleared for entries over 30 days
Performance poller times cleared for entries over 30 days
Device performance times cleared for entries over 7 days
Returned: 0
Fetching notifications
[ Mon, 30 Oct 2017 00:15:16 +0000 ] http://www.librenms.org/notifications.rss (21)
[ Mon, 30 Oct 2017 00:15:16 +0000 ] misc/notifications.rss (28)
[ Mon, 30 Oct 2017 00:15:16 +0000 ] Updating DB Done
Returned: 0
Caching PeeringDB data

Returned: 0

So for some reason, the “1” day setting in config.php isn’t seen? did I do something wrong here?

Thanks.

Michael.

6

are trying to just clear the all of the syslog?

7

Hi Kevin. No, am happy to keep a number of days actually, as I’m interested now in learning how to add rules for syslog entries we’re interested in.

The initial problem was that the firewall logs being ingested were filling the disk too quickly. Have expanded the disk so this isn’t a problem now, and the PHP memory limit increase also fixed the Web UI timeout.

Now the only problem seems to be the “syslog_purge” setting not being read, which is why I asked if there’s anything I’ve done wrong.

Thanks.

Michael.

8

try running ./daily.sh manullly

9

Hi Kevin. Yes running manually I saw it with:

Checking PHP version

Returned: 0
Updating to latest release
HEAD is now at e55c68d... docs: Added changelog for 1.33 (#7580)
Returned: 0
Updating SQL-Schema

Returned: 0
Updating submodules

Returned: 0
Cleaning up DB
Refreshing alert rules queries
Refreshing OS cache
Syslog cleared for entries over 1 days 1000 limit
Syslog cleared for entries over 1 days 1000 limit
Syslog cleared for entries over 1 days 1000 limit
Syslog cleared for entries over 1 days 1000 limit
Syslog cleared for entries over 1 days 1000 limit

So it’s worked manually.

Does this mean I should keep an eye out tomorrow on the daily.log to see if it runs, and if so, whether it shows “Syslog cleared for entries over 1 days 1000 limit” ?

I can see this CentOS VM should correctly have this running each day at 12:15am:

# cat /etc/cron.d/librenms
# Using this cron file requires an additional user on your system, please see install docs.

33  */6   * * *   librenms    /opt/librenms/discovery.php -h all >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/discovery.php -h new >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/cronic /opt/librenms/poller-wrapper.py 4
15   0    * * *   librenms    /opt/librenms/daily.sh >> /dev/null 2>&1
*    *    * * *   librenms    /opt/librenms/alerts.php >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/poll-billing.php >> /dev/null 2>&1
01   *    * * *   librenms    /opt/librenms/billing-calculate.php >> /dev/null 2>&1
*/5  *    * * *   librenms    /opt/librenms/check-services.php >> /dev/null 2>&1

and it runs with the librenms user, so I’d expect it to work as it should.

I’ll keep an eye on this tomorrow.

Michael.

10

Hi. Just to update this issue, all is working fine and the pruning is properly occurring each day.

Thank you.

Michael.

1 Like
11

Hi Guys,

I have configured sonic firewall log via syslog-ng. But still i didnt get any logs in librenms syslog. Kindly help me anyone know this.

Thanks,
Gopal.

12

Please don’t hyjack threads. Please start your own thread.

13

ok sure. i will create own thread