Concatinating field from nested json by extending the parser

[swanand]

Hi All,  Need to implement this urgently - tried all options including ruby / split etc but none of that works within google chronicle parser extension. 

I have a simple requirement to concatenate field value from recurring field as per below. Please suggest how this can be implemented in parser extension.  Input and expected output is as per below.. Thanks in advance.

Input:

{
"folders": [
{"resourceFolderDisplayName": "L1"},
{"resourceFolderDisplayName": "L2"},
{"resourceFolderDisplayName": "L3"}
]
}

Expected Output:

UDM.METADATA.RESOURCEPATH = L1/L2/L3

[AbdElHafez]

Is the number of items in the folders list fixed as 3 items or is it variable ? and are the keys names "resourceFolderDisplayName" fixed ?

[AbdElHafez · Accepted solution]

This is a valid version, I could not find a resourcepath under metadata so I put in metadata.description

filter {

    
json {
  source => "message"
  array_function => "split_columns"
}
statedump {
    "label" => "1"}

        mutate {replace => {"temp"=>""}} #Init the concat token

for index1,item1 in folders {
        mutate {convert => {"index1"=>"string"}}
        mutate {replace => {"label1"=>""}} #Init the collector token

        mutate {replace => {"obj.val"=>"%{item1.resourceFolderDisplayName}"}}
        
        mutate {replace => {"label1"=>"%{obj.val}"}}

        mutate {merge => {"labels"=>"label1"}} #Init the collector token
        
        
        # Concat
        if [index1]=='0' {mutate {replace => {"temp"=>"%{label1}"}}} 
        else {mutate {replace => {"temp"=>"%{temp}|%{label1}"}}} 

        statedump {
            "label" => "2"}
}


    mutate {
    replace => {
    "event.idm.read_only_udm.metadata.product_name" => "Upstream"
    "event.idm.read_only_udm.metadata.vendor_name" => "Upstream"
    "event.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT"
    "event.idm.read_only_udm.metadata.description" => "%{temp}" #Concat in Temp
    }
    }

mutate {
    merge => {
        "@output" => "event"
    }
    }


    statedump {
        "label" => "3"}

}

[swanand]

This worked like a charm... Thanks a ton. 

I came across one more issue - when trying to map the concatenation  result to the UDM field (which is mapped to metadata.description in the code you shared). Seems array's are overwritten and not appended in the extension - and I was not able to add this in any of the existing array fields.

The parser i m extending is GCP SCC threat and since i can't use any array fields, ended up using this to store hierarchy folders for GCP project: 

    "event.idm.read_only_udm.src.process.file.full_path" => "%{temp}" #Concat in Temp

Any thoughts on whether this is the right field or should i consider using any other please ?

Thanks again !

[AbdElHafez]

Happy to know it worked.
The concatenation token "temp" is a string field so it must be assigned to any string field, if you want to add/concatenate to an existing string field that is populated by the parser already, then you would to add the code block to the parser and you will have to maintain it with updates, as the token values do not persist between the parser and the parser extension unfortunately.

Same goes for arrays, but you would need to properly format "temp" as an array or add a key value to it and append it to the labels in the main parser block.