LinkedIn
Twitter
Today’s blog will continue exploring methods to build composite rules. This time we will take a look at using rules you’ve created (custom) and rules that Google has created (Curated Detections) and use them together as we build a composite rule.
The detections that are created by both custom and curated detections end up within the same collections endpoint that we’ve discussed previously, so everything we’ve discussed to date applies. The wrinkle this time is establishing which curated detection rules or rule sets should be used. Let’s start with this query, executed in dashboards and see what information we have in the detections dataset as it pertains to curated detections.
This search is returning detections generated from curated rules for the time window in the chart or dashboard. We know these are Google created rules because the rule_id starts with the string ur_, where custom rules start with ru_.
From there, we are aggregating by the rule name and outputting a series of outcome variables. Some of these are new, like $rule_set_display_name and $rule_set_category_display_name, which are unique to curated detections, and others like $alert_type and $rule_type we’ve previously talked about that describe detections, as well as $rule_label_technique which contains values from the meta section of the rule that the detection is generated from.
$rule_name = detection.detection.rule_name
detection.detection.rule_id = /^ur_/
match:
$rule_name
outcome:
$detection_time = timestamp.get_timestamp(max(detection.created_time.seconds))
$severity = array_distinct(detection.detection.severity)
$alert_type = array_distinct(detection.detection.alert_state)
$rule_type = array_distinct(detection.detection.rule_type)
$rule_label_technique = array_distinct(detection.detection.rule_labels["technique"])
$rule_set_display_name = array_distinct(detection.detection.rule_set_display_name) //curated detections
$rule_set_category_display_name = array_distinct(detection.detection.ruleset_category_display_name) //curated detections
limit: 10
The two columns on the far right in the dashboard below are the ones I’d like to call your attention to. The rule set display name and category are groupings of rule packs that the curated rules are bundled into. We can see the first detection is part of the Mandiant Frontline Threats which is part of the Windows Threats category. Other packs and categories include Active Breach indicators that are part of Applied Threat Intelligence, as well as CDIR Security Command Center (SCC) packs that are part of Cloud Threats.
This query did not include variables and values found in the match and outcome sections of the curated detection, though if you know the names of the variables, it certainly could. More on that in a moment.
At this point, I may want to dig into the Mandiant Frontline Threat rule that I’ve seen in my detection chart. To view the specific detection, you could click on the rule name in the dashboard view or you could navigate to the Rules & Detections page in Google Security Operations (SecOps) and under Curated Detections select the dashboard and then click on the rule of interest.
Either way, we can see the detections and alerts that this rule triggered. If we click on a detection, the alert viewer appears on the right side of the page. Clicking on the Outcome and Match Variables subtab will display the match and outcome variables that are part of this rule.
Reviewing the detection, we can see the name of the variables used. Take note of these variables and the values contained in them. These variable names will be used to join our custom detections to the curated detections.
A little bit of foreshadowing, my match variable is hostname as well, so this will make things easier. If I had chosen to use the outcome variable or built my composite rule based on the user, I would need to take into account these variable names when building the composite rule.
If you want to add these variables and additional fields to the detection view as you explore the data, you can do this by clicking on the columns button and selecting the Detections or Events table type in the drop down and then selecting fields of interest. We will do this again when we test our rule, so consider this a warm-up 😀.
It’s highly likely that your naming convention for rules will differ from curated rules so identifying this prior to writing a composite rule will make this process much simpler if the rule is joining on a common match or outcome variable.
With that inspection of the curated detections, we can start moving forward writing the composite rule. In the image below, we have our funnel. The items on the right in blue pertain to the custom rules we’ve developed and covered previously. In short, we are looking for custom rules that are of type single event, not alerting and have a type label in the meta section with the value of producer. On the left are the curated detections. We only want the detections that come from the Mandiant Frontline Threats package.
Based on our inspection of the curated detections, we know that the match variable name aligns with our producer rule, so that makes things a bit easier. However, I don’t want to just alert when I have one custom rule and one Mandiant Frontline Threat. Instead, I want to build some thresholds around the different types of rules into my composite detection.
In the events section, we have two event variables, one for the custom rules and one for the curated. The custom rule section uses the rule id starting with ru_ to focus just on rules I’ve written. From there, the alert state and rule type are defined as specified previously. The type label associates this rule as a producer and is in the meta section. Finally, we want to join these rules by the common match variable of hostname.
In the curated detection section of the rule, we are specifying the name of the rule pack and the variable that we want to join the custom to the curated detections. Because we did our homework, we know that the curated detection in this case is using that same match variable of hostname, so we can specify that here and join the two detections together.
rule composite_custom_rules_with_mandiant_frontline_curated_detection {
meta:
author = "Google Cloud Security"
description = "Detects when four or more custom, single event, non-alerting producer rules and two or more Mandiant Frontline rule pack rules trigger by hostname"
severity = "High"
priority = "High"
type = "composite"
events:
$detect_prod.detection.detection.rule_id = /^ru_/
$detect_prod.detection.detection.alert_state = "NOT_ALERTING"
$detect_prod.detection.detection.rule_type = "SINGLE_EVENT"
$detect_prod.detection.detection.rule_labels["type"] = "producer"
$detect_prod.detection.detection.detection_fields["hostname"] = $hostname
$detect_frontline.detection.detection.rule_set_display_name = "Mandiant Frontline Threats"
$detect_frontline.detection.detection.detection_fields["hostname"] = $hostname
match:
$hostname over 4h
outcome:
$risk_score = 60
$custom_rule_distinct_count = count_distinct($detect_prod.detection.detection.rule_id)
$frontline_rule_distinct_count = count_distinct($detect_frontline.detection.detection.rule_id)
$rules_triggered = arrays.concat(array_distinct($detect_prod.detection.detection.rule_name), array_distinct($detect_frontline.detection.detection.rule_name))
condition:
$detect_prod and $detect_frontline and $custom_rule_distinct_count > 3 and $frontline_rule_distinct_count > 1
}
The match variable in the composite rule is the hostname that we joined these disparate rules on. We are rolling together custom and composite rules that align with the criteria in the events section over a four hour window. If this is met, outcome variables are calculated.
The outcome variables I want to draw your attention to are the ones named $custom_rule_distinct_count and $frontline_rule_distinct_count. Here we are generating distinct counts of the custom and curated rules that triggered, respectively. We are going to use these in the condition section of the rule and state that we need to have more than 3 distinct custom rules and more than 1 distinct Mandiant Frontline Threat rule for this composite rule to trigger.
When we test the rule, we have a detection that impacts win-server. It is created based on the seven unique custom rules and two Mandiant Frontline Threats rules triggered during our four hour match window. Notice in the top right corner, we’ve added columns with the outcome variables including the rules triggered, which is expanded to show all of the associated rules contributing to this detection.
It’s also worth mentioning that we have both detections and alerts within this composite rule test results. Keep in mind that when we built our composite rule, we specifically said that the custom rules should not alert, but we did not say this about the curated detections. There isn’t a right or wrong way to go about this, perhaps the curated detections should always trigger alerts and then can be still considered in a broader composite rule, but it also serves as a word of caution that depending on how you configure your curated detections, you will want to take their alert state into consideration when building composite rules.
Today we continued to build on the previous concepts of composite rules and added curated detections to custom detections to build a new composite rule. Remember to do your homework to identify the match and outcome variables in the curated detections to create joins with custom rules because the naming conventions may not be the same. Otherwise, the concepts of composite rules with curated detections are nearly identical.