Entity mapping in parsers

[vishnu_manu]

Has anyone tried to map entities in the parser ? (Ex. entity.user.user_id, entity.url, entity.hostname, etc...).

What is the possible solution to parse IP, Hostname, URL, etc.. as entities ?

[mikewilusz]

Yes, you can map entity fields in a parsers, along with UDM. A common parser that does this is the Okta user context parser if you're looking for an example. 

-mike

[vishnu_manu]

Hello,

I tried the similar syntax shown above,

my code is,

        if message =~ "PanOSUserIdentifiedBySource" {

        grok {

            match => {

                "message" => ".*PanOSUserIdentifiedBySource=%{DATA:tuid} .*"

            }

            on_error => "gili_gili"

        }

        mutate {

            replace => {

                "event.idm.entity.entity.user.userid" => "%{tuid}"

            }

        }

    }

I received an error

I've also tried by declaring product_name, vendor_name, metadata.entity_type. Still error is seen.

[AbdElHafez]

@vishnu_manu your grok statement is missing the overwrite to assign the captured token to the variable tuid , plus I think you mistakenly repeated entity in the replace statement.
I would also recommend using statedump after each problematic block to see the error tags or what values are being tokenized.
You could start focusing on tokenizing the variables first before writing the UDM event, and you could try using simpler examples first.

[mikewilusz]

You can use this example from GitHub to see a full example, along with the required metadata fields being set: https://github.com/goog-cmmartin/thatsiemguy/blob/main/misp/chronicle_parser/misp_ioc.conf

-mike

[vishnu_manu]

Mike,

Thanks for the github repo.

Lets say I bring in logs from an endpoint management tool, where the logs will be from an API call which returns the entities of a host (hostname, username, mac, IP, agents installed, etc...).

By using the above github repo as reference, I will parse all the entities which is required.

Will this help me in enriching other missing entities in other log sources ? for example I have a firewall log, where only the hostname and username is present in the log. Will it enrich the remaining entites associated to the host and username like mac and its associated IP ?