LibreNMS into Kafka into Prometheus

Plugins
1

Hello to all,

Hope this is the right place. If you think it should be posted in an other topic, please let me know.

I would like to share my findings and experience with transporting LibreNMS exported-data with Kafka so that to funnel these data into Prometheus.

My use case is to gather local LibreNMS instances (for reasons) into a big Prometheus (+ grafana) central data gathering platform (and I can’t use distributed pollers … again reasons.

Any feedback is welcome.

This is really a first try. VRL language of Vector is a bit tricky to use.

I will try to post added content.

Htam.

1 Like
2

I’m currently doing this using:
LibreNMS → InfluxDB exporter → VictoriaMetrics → Grafana

It works well. VictoriaMetrics is very fast, storage efficient, and easy to use. It’s compatible with many types of ingestion (including InfluxDB formats) and exporting (including Prometheus formats).

Good luck on your project!

3

Thank you. Looks interesting.

I really need to pass through kafka because pipelining and need some sort of inversing connections/flows for some security constrains.

The next step is to able to pass through a network data diode (UDP unidirectional only).

I see some other talking about VictoriaMetrics. How is that different from Prometheus ?

Sincerely yours, Mathieu.

4

VictoriaMetrics is a high-performance, very storage and memory efficient, and highly compatible TSDB. You can ingest and query using Prometheus protocols (in addition to many others).

They have a vmagent that is very small but can collect metrics (push and pull), modify them if needed, and forward to VictoriaMetrics (or Prometheus) TSDB. vmagent can also cache metrics to disk so nothing gets lots when the network or TSDB becomes unavailable.

Perhaps this could work for your needs?

1 Like