Windows Parsers and Asset Enrichment

[lukas-lr]

Hi everyone,

Does anyone know how the asset enrichment with the parsers for Windows events is supposed to work? I feel like a lot of parser extensions is required to get consistent IDs.

We are ingesting Windows events as WINEVTLOG and AD asset data as WINDOWS_AD.

• The parser for WINDOWS_AD sets the asset_id to the SamAccountName, i.e. ending with a $ sign. hostname is set to just the name of the host (not the full FQDN).
• The parser for WINEVTLOG (at least often) sets the asset_id to the hostname, prepended with "ASSET_ID: " (this cannot be changed even with a parser extension, since there is a regex validating this). The hostname is set to the full FQDN.

To me this feels very inconsistent and makes it hard to enrich events based on the hostname (everything works fine with IPs). User IDs have similar issues, where sometimes domains are included and sometimes not.

I would be thankful for hints how to handle these issues. Are the recommendations on how to set asset IDs?

[cmmartin_google]

As `Hostname` is a singular value in the UDM Entity Graph, i.e., not a repeated field, you can't create an Asset context record with multiple hostnames.  One alternative is creating an Asset context record for each hostname, i.e., duplicate the record just to have the hostname or FQDN match, but that's not out of the box and requires additional work on your export script from AD.

As you say, an option of using Parser extensions is effective  to fix the hostname issue, e.g.,

...
            grok {
                match => {
                    "Hostname" => [
                        "^(?P<_Hostname>[^$|^\\.]+)"
                    ]
                }
                overwrite => ["_Hostname"]
                on_error => "grok_error_on_hostname"
            }

            if ![grok_error_on_hostname] {
                mutate {
                    lowercase => [ "_Hostname"]
                }
                mutate {
                    replace => {
                        "event.idm.read_only_udm.principal.hostname" => "%{_Hostname}"
                    }
                }
            }
...

The above you apply to any Windows Event log integration, e.g., WINEVTLOG, WINDOWS_SYSMON, WINDOWS_DEFENDER_AV, and then it'll only normalize the hostname and drop the domain part.

One caveat, *if* the parser puts the `Hostname` field into target.hostname or src.hostname this logic fails.  

It does require a more thorough approach to then apply different logic for the override based upon those specific (but few) event IDs where the value is not normalized into principal.

[lukas-lr]

Thank you for the reply and the code example. I'll think about which way to normalize it works better for us