Multi-line Event Parsing Using Grok

[ScottieJ]

Turns out that the Grok Pattern "GREEDYDATA" not that all that greedy...hopefully this will save someone some time.

I  needed to write a parser extension for a multi line Windows event formatted in XML. I not so quickly discovered that Grok patterns match to the end of a line and my logs could have multiple lines. I initially and incorrectly assumed that the "GREEDYDATA" pattern would grab all the event data to put into a message field. It would not and my error conditions kept triggering. My solution was a custom regex pattern like the example parser snippet below. 

filter{
  grok {
    match => {
      # "message" => ".*?<Event %{GREEDYDATA:xml}</Event>"
      "message" => "(?P<xmlmessage><Event (.|\\n|\\r)*</Event>)"
    }
    on_error => "is_not_xml"
  }