I am working on a Logstash parser for CEF-formatted syslogs, which includes both short and long formats. The long format contains a status field, whereas the short format does not.
I extract CEF event attributes using the following Logstash filter:
match => {
"message" => [
"%{abc} CEF: (xxxxxxxxxxx)\\|%{GREEDYDATA:cef_event_attributes}"
]
}
overwrite => ["message"]
}
kv {
source => "cef_event_attributes"
field_split => "|"
value_split => "="
target => "cef_fields"
}
Issue: The cef_fields.status field exists only in the long log format. However, when applying conditional logic to process this field, Logstash throws the following error:
"generic::invalid_argument: pipeline failed: filter conditional (13) failed: failed to evaluate expression: generic::invalid_argument: 'cef_fields.status' not found in state data"
Current Conditional Logic:
if [cef_fields][status] != "" { # Error occurs here if 'status' does not exist
if [cef_fields][status] in ["xxxxxx", "xxxxxxxx"] {
mutate {
replace => { "xxxxxx" => "xxxxxx" }
}
}
else if [cef_fields][status] == "xxxxxxx" {
mutate {
replace => { "xxxxxxxx.xxxxxxx" => "xxxxxx" }
}
}
else {
mutate {
replace => { "xxxxxxxx.xxxxxx" => "xxxxxxxxxxxxx" }
}
}
}
Excepted Behaviour: If cef_fields.status exists, apply the parsing logic. If it does not exist, the pipeline should continue without errors.
How can I correctly check for the existence of cef_fields.status before applying conditions to avoid the "not found in state data" error?
Solved! Go to Solution.
I see that you added the initialization snippet, but you should put it at the very beginning of the parser, just below the "filter {" line.
Where you put it in your pasted code, it just overwrites the parsing done by grok / kv, this is why you have unexpected behavior.