Syslog not consumed by LibreNMS

I’ve recently deployed a new VM (Ubuntu) for LibreNMS to replace an old, not-really-working deployment (CentOS) and am having an issue after configuring syslog.

I’ve done a very similar build in a home lab and it works flawlessly, though the devices sending syslog are different (Ubiquiti at home, Peplink in the office).

I’ve used my home lab as a rough template as far as syslog is concerned and they match. The issue is I see syslog being received by the server from a tcpdump, but it’s not being pulled/processed/parsed by LibreNMS and I’m not quite sure what the outlier is here.

I’ve read that so long as the device is within LibreNMS and the hostname/IP match, it should be a no brainer, and in this particular instance, that is definitely the case, but beyond that, I’m at a loss.

I don’t know what anyone would want as far as output, so I’ll preempt by popping some data here.

The message received by the server…(some data redacted)

09:09:08.853125 IP (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto UDP (17), length 190)
xxx.xxx.48.68.46508 > xxx.xxx.50.173.514: SYSLOG, length: 162
Facility local0 (16), Severity notice (5)
Msg: Dec 21 09:09:08 gb-xxx-xxxx-1350-05 SpeedFusion: GB-XXXXX-CNNNGTWN-CPE-01 (GB-XXXXXXXX-Canning-Town, sn:XXXX-XXXX-XXXX) connected to GB-XXXXX-CNNNGTWN-CPE-01

In the case of the IP matching the host, the LibreNMS server does not point to internal DNS, so we’ve created host entries for the devices we’re managing (something I’m working to resolve). And the device most certainly is managed properly.

The config for /etc/syslog-ng/syslog-ng.conf is verbatim to the documentation. The LibreNMS config is as it should be to enable syslog.

I don’t quite know what else to look at so I appreciate any assistance you all can provide.

Things to check the syslog config, make sure names line up in librenms. Make sure syslog-ng is actually running.
Also post your syslog -ng config

Yep…I’ve done just that…as mentioned everything is verbatim to the documentation in both instances. Syslog is most certainly running.

As far as the syslog config, looking at a few aspects of that, in the case of the host, it’s certainly configured. It had previously been sending to Elasticsearch without issue, but as the output in the OP shows, it’s certainly being sent to the LibreNMS server now. There’s not much else to configure elsewhere really.

As far as the config goes…

@version: 3.5
@include “scl.conf”
@includescl-root/system/tty10.conf”

First, set some global options.

options {
chain_hostnames(off);
flush_lines(0);
use_dns(no);
use_fqdn(no);
owner(“root”);
group(“adm”);
perm(0640);
stats_freq(0);
bad_hostname(“^gconfd$”);
};

########################

Sources

########################
source s_sys {
system();
internal();
};

source s_net {
tcp(port(514) flags(syslog-protocol));
udp(port(514) flags(syslog-protocol));
};

########################

Destinations

########################
destination d_librenms {
program(“/opt/librenms/syslog.php” template (“$HOST||$FACILITY||$PRIORITY||$LEVEL||$TAG||$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC||$MSG||$PROGRAM\n”) template-escape(yes));
};

########################

Log paths

########################
log {
source(s_net);
source(s_sys);
destination(d_librenms);
};

Include all config files in /etc/syslog-ng/conf.d/

@include “/etc/syslog-ng/conf.d/*.conf”

And the option in config.php is there.

https://pastebin.com/f7W3e9y6
Config looks like mine with syslog-ng

Im not sure at this point.

Can you validate your install, please.

Validation is fine other than the daily check which I’ve disabled and ownership for the weathermap plugin.

Can you post it?

Updates…

Updates Warning
Warning: Your install is over 24 hours out of date, last update: Tue, 19 Dec 2017 09:32:31 +0000
Make sure your daily.sh cron is running and run ./daily.sh by hand to see if there are any errors.

And to save the huge output…it’s everything under the second directory listed…

Fail: We have found some files that are owned by a different user than librenms, this will stop you updating automatically and / or rrd files being updated causing graphs to fail.
chown -R librenms:librenms /opt/librenms

Files
/opt/librenms/mysqltuner.pl
/opt/librenms/html/plugins/Weathermap

The wethermap issue is not affecting anything as those are working just fine.

You do know that daily.sh dose more than just updates?

Also when you validate install we really need to see this part too, helps with troubleshooting.

====================================

Component Version
LibreNMS 1.33-257-g7b36d65
DB Schema 226
PHP 7.0.25
MySQL 5.5.56-MariaDB
RRDTool 1.4.8
SNMP NET-SNMP 5.7.2

====================================

Fair point…

I’ve run the daily script manually with no errors…

====================================
Component | Version
--------- | -------
LibreNMS  | 1.33-245-gd6d4a3a
DB Schema | 224
PHP       | 7.0.22-0ubuntu0.16.04.1
MySQL     | 10.0.31-MariaDB-0ubuntu0.16.04.2
RRDTool   | 1.5.5
SNMP      | NET-SNMP 5.7.3
====================================

EDIT…and now I’ve run it after recommenting out the relevant config in config.php…

====================================
Component | Version
--------- | -------
LibreNMS  | 1.33-257-g7b36d65
DB Schema | 226
PHP       | 7.0.22-0ubuntu0.16.04.1
MySQL     | 10.0.31-MariaDB-0ubuntu0.16.04.2
RRDTool   | 1.5.5
SNMP      | NET-SNMP 5.7.3
====================================

So this doesn’t become another of the myriad of dead threads, any other suggestions?

Perhaps there is another Syslog service running? like Rsyslog?

Not in this case. This was a new VM from scratch. While present, it’s not running nor has it ever been configured.

And automagicly, it started to work. Something happened on Feb 2nd where LibreNMS just started populating syslog data from these devices. Can’t say I know what that is as yet.