Rule detection search

[Tonio]

Hello everyone!

Looking at the detections for a specific rule, it gives you back only a specific timeframe, like last months or so.

I cannot set any other specific time window for the search, nor filters on other fields.

A couple of examples:

-  I see a rule has been triggered like 100 times in January (3 months ago), I see notghing at all.

- I am checking a draft rule with more than 10.000 detection, need to focus on a specific day. I can't, because that day is outside the first 10.000 shown in the panel

Is it just me missing some peculiar button or function? Shouldn't be possible to "search" on the detections as we do on the events?

A

[chrisd2]

Hello @Tonio ,

You can search through your detections by creating a dummy Native Dashboard with a dummy Table. Then you can query the Alerts and Detections by using the detection.* UDM fields (e.g. focus on some specific days and perform aggregations based on match fields or outcome fields). A must have to perform fine-tuning !

Hopefully it will be soon available via the classic UDM Search UI, it is a bit annoying to go through Native Dashboards menus to do it.

Reference docs : https://cloud.google.com/chronicle/docs/reference/sample-yaral-for-native-dashboard#detections

[cmorris]

The detection data source is not available in search today, but this might be a good fit for the native dashboards.

What info are you trying to return from the detections?

Something like this could be a starting point:

detection.detection.rule_name = "mitre_test"

match:
    detection.id

outcome:
    $users = array_distinct(detection.collection_elements.references.event.principal.user.userid)
    $hostnames = array_distinct(detection.collection_elements.references.event.principal.hostname)

Change the rule name to the rule you are interested in. The outcome section can be used to pull info from the detection. In the example above, I'm pulling a list of users and hostnames, that may or may not be relevant for you.

[Tonio]

Hi @cmorris! Thanks for the reply!

We do not have the native dashboard active yet, is it available upon request?

To reply to your question: in this particular case, I was just trying to report on which log sources have been more useful, so something between the lines of:

- retrieve the metadata.enrichment_labels.log_type and metadata.log_types 

- pivoting and counting on these fields

Definitely seems something that the native dashboard can work with!

But in general, i feel quite frustrating if I cannot visualize detections before a specific date, or when there were more than 10k results. I suppose I will have to cope with doing specific retrohunt each time?

[chrisd2]

Hello @Tonio ,

Using the detection.* UDM fields in Native Dashboards Tables, you can filter your alerts/Detections based on their timestamp 🙂

Docs :

- https://cloud.google.com/chronicle/docs/reference/sample-yaral-for-native-dashboard#detections

[cmorris]

+1 to @chrisd2 - you can filter with the Native Dashboards.

We can definitely get counts on these fields (via Outcome section) and group by log types as well (via Match section). We have curated (prebuilt) detection dashboards in Native Dashboards that may have some of what you are looking for already as well.

You can submit a support request to see if Native Dashboards can be enabled. There are a few prereqs - BYOP, BYOID or GAIA Auth authentication, and IAM authorization.

[cmorris · Accepted solution]

An example:

$rule_name = detection.detection.rule_name

match:
    $rule_name

outcome:
    $log_types = array_distinct(detection.collection_elements.references.event.metadata.log_type)
    $count = count(detection.id)

order:
    $count desc

[Tonio]

That's great guys, I will submit the request for the native dashvboard right away. Looking forward to test this out!

[Tonio]

Dear all, 

We get the Native Dashboard enabled last week, and I took some time to play with it.

Thanks for all the suggestions, this is the final version of the table I got, just few steps away from @cmorris 's code:

$tot = group(detection.collection_elements.references.event.metadata.enrichment_labels.log_types,detection.collection_elements.references.event.metadata.log_type)
$tot != ""

match:
    $tot

outcome:
    $rule_counting = count_distinct(detection.detection.rule_name)
    $event_count = count(detection.id)


order:
    $rule_counting desc

Here I just pivoted on both the rules and the detections. 

The  <$tot != ""> part is there since I found there always was a minor number of results with no label, which I acutally fail to understand, but has a negligible quantity, so for the moment I will keep it like it.

Here the results, in both tabular and donuts way (which comes with nice percentages values for free!):

There are a lot of fields to use, for each of the sources listed here. Following the links to each source I found more ore less a list of the fields available, but not for the "detection" source.

I tried digging a little bit with some trial and error in the query generator of the dashborad, but the rabbit hole runs quite deep and stopped to the 3rd level, you can find it in this file. Is there anywehere an exhaustive list?

Thanks!

[cmorris]

Detection data source is here - https://cloud.google.com/chronicle/docs/reference/rest/v1alpha/Collection. You can click through the different levels, for example to get to event information, you may use something like detection.collection_elements.references.event.principal.hostname

[Tonio]

@cmorris that's perfect! Thanks very much!

May I suggest to add this (and the respective references for the other sources) in the table here? That would clarify a lot!

Thanks

[jstoner]

We have asked for this as it will be super useful for other use cases beyond dashboards. It is in the backlog but totally agree that something like a field list would be much easier to consume.