Clarification on drop function syntax in a parser

[JaredBloomberg]

Hello,

I was wondering if anyone could add some clarity to the documentation around drop function in the Parser syntax reference?

Per the docs "This function is used to drop all messages that reach this filter logic." which is pretty straightforward but how does this work in the context of a parser extension? I understand that the default parser will process the log data first and then pass to the extension, assuming one exists. 

If I add a drop{} to my extension does that mean that it drops all parsing of the log altogether (including whatever the default parser has already done) or just what the extension would have done while the previous parsing from the default parser is still maintained?

[rfhart]

Hi JaredBloomberg,

No the drop filter will only drop events that match it's filter logic. For instance, if you had the following:

if [domain] == "-" {
  drop {}
}

This means that any event where the domain field is a dash/hyphen/minus, will be dropped, but the any event where the domain is not a dash/hyphen/minus will not be dropped. You can also find more information about drop here. 

[Dimarsky · Accepted solution]

Jared, 

The drop filter does exactly what you understand it to do, it stops the parsing of a log and goes on to the next log. 

With respect to what happens when you have a parser followed by a parser extension... 

If a drop filter is hit in the initial parser the log gets dropped and no further parsing occurs. You will see the raw log in the Chronicle SIEM UI and it will show up as an unparsed log (there will be no UDM associated with it). If there was a parser extension configured for this logtype the parser extension will not execute.

If the initial parser completes and generates a UDM event and there is a parser extension configured, the parser extension runs, and then the parser extension logic hits a drop filter then the UDM generated by the initial parser will still exist and be stored along side the raw log. 

I use the term initial parser here to make it clear that the same behavior happens with both a default parser or with a custom parser if you have one in place. 

[JaredBloomberg]

I think where I am confused is what happens to an event after it is dropped e.g. if an event is successfully processed by the default parser but then encounters a drop filter in the subsequent parser extension.

Do all parsed fields (from both the default and extension parsers) get dropped at that point and you are just left with a raw log?
Or is all the parsing that has been done by the default parser still maintained but no additional parsed fields are added by the extension and the system simply moves on to process the next event?

[rfhart]

My apologies. If there is a match to the drop filter the logic, the event parsing will stop. You will still have the raw event.

[JaredBloomberg]

@rfhart @Dimarsky thank you both for chiming in, I think it is clear to me now.

Much appreciated!