COMPLETED: Feedback request: Emitting alert data to ELK

Feature Requests
1

Hello, I am planning to write an alert plugin that will emit alert data to ELK implementations, and wanted feedback from the community.

Mainly, would it be better to emit as a Syslog or as a direct Elasticsearch insertion?

Syslog Pros:

  • Allows people to use this for more than ELK stack
  • Allows people to do post-processing in Logstash

Syslog Cons:

  • More challenging format for transmitting alert attributes - will need to establish a parsable format, and people will need to write logstash parsing code.
  • People might prefer LibreNMS as a syslog destination than as a syslog emitter

Elasticsearch Pros:

  • Can submit alert data “normally”, with explicit key:value pairs
  • Allows people to use this if they only have ES and not the full ELK stack

Elasticsearch Cons:

  • Will need a rigid structure, all flexibility will be in the plugin design which will never be as full-featured as logstash
  • Leaves out the people who use syslogs without ELK

Please give feedback ASAP! Thanks!

2

Sounds like you just want to write a transport that deals with the alerting to either (two transports)

I think it’s a great idea.

3

Thanks!

Yeah, I figured this would eventually become “why not both?!”, so it’s more of a “what should come first?”.

4

Which ever you want imho :slight_smile:

5

Going to submit a PR soon, I went with the syslog transport. Elasticsearch has a library dependency that I don’t want to get into right now.

6

So you would be taking alert data from an alert that has fired and shipping that alert info into an ES cluster? I like this a lot better than syslog.

I saw another one of your replies about ES library dependencies and I don’t understand why that would come into play here. ES document format is nothing more than a JSON data and creating a new document in ES is just an HTTP POST or PUT. If you can just take the alert data and serialize into JSON and send it along, I think that would be better than taking an alert and putting it into an unstructured syslog message and shipping it to another syslog collector.

7

I agree, and will be looking into it more. the library comment was just in reference to the fact that a quick google didn’t bring back any php interactions with ES other than with the elasticsearch-php library.

I have not experimented with the actual POST/PUT insertion of a new document, it has always been abstracted for me. If you could share an example curl call and perhaps the field structure you would be interested in using it would help quite a bit.

8
1 Like
9

Both features now merged, thanks @laf

10

No thank you @psb :slight_smile: