Reference List multiple columns ?

[rahul7514]

Hi 

Can reference list have multiple columns ? Lets say i am trying to create a reference list that has 2 columns one is IOC ip and other is its confidence score ? 
Has anyone used it , if yes please share a use case where this is used ?

[AymanC]

Hi @rahul7514,
The following Medium Blog post by thatsiemguy will be of interest - https://medium.com/@thatsiemguy/automating-chronicle-siem-reference-list-using-chronicle-soar-78f2e7...

$location = strings.concat($event.principal.location.country_or_region, "|", $event.principal.location.state)    
not $location in %workspace_superuserlogin_exclusions

Kind Regards,

Ayman

[citreno]

I believe there's a new feature coming for the multi column use case. Perhaps you can inquire with Google about joining the preview of Data Tables, I'm not sure when it's supposed to be released.

That said I always prefer ingesting IOCs in the entity graph instead of hosting them in Reference lists, you get way more features that way. There's native parsers for things like MISP, anomaly, recorded future and crowdstrike, but if you have a custom feed/list you can use something like CSV_IOC_CUSTOM which has a very basic csv parser. Parsed IOCs show up in the UI, allow for retroactive lookups easier etc and just have better native support.

[raybrian]

You are correct that reference lists in Chronicle currently only support a single column of values. You cannot directly create a reference list with two columns like "IOC IP" and "Confidence Score." This is a limitation of the current reference list functionality.

Workarounds and Alternatives

However, there are ways to achieve a similar effect or work around this limitation:

1. Multiple Reference Lists: You could create two separate reference lists: one for "IOC IPs" and another for "Confidence Scores." Then, in your Yara-L rule, you would need to:

   • Check if the IP address is present in the "IOC IPs" list.
   • If a match is found, use additional logic (e.g., array indexing, a lookup table, or a separate data source) to retrieve the corresponding confidence score from the "Confidence Scores" list. This would require maintaining a consistent order or mapping between the two lists.

2. UDM Entities: For more complex multi-column data, UDM entities are a more suitable solution. You can create an entity type to represent your IOC information, with attributes for "IP" and "Confidence Score." Then, you can ingest your IOC data as entities and use entity correlation within your Yara-L rules to match and retrieve the confidence score.

Use Case Example

Let's imagine you want to create a rule that detects connections to known malicious IP addresses but only triggers an alert if the confidence score of the IOC is above a certain threshold.

Using UDM Entities:

1. Define an entity type:

   entity_type: {
     name: "MALICIOUS_IP"
     attributes: {
       ip_address: { type: "STRING" }
       confidence_score: { type: "INT" }
     }
   }
   

2. Ingest your IOC data as entities, e.g.:

   entity: {
     entity_type: "MALICIOUS_IP"
     ip_address: "192.168.1.10"
     confidence_score: 90
   }
   

3. Write your Yara-L rule:

   rule high_confidence_ioc {
     events:
       $e.principal.ip = $ip_address
       $ioc.graph.metadata.entity_type = "MALICIOUS_IP"
       $ioc.graph.entity.ip_address = $ip_address
   
     condition:
       $e and $ioc and $ioc.graph.entity.confidence_score > 80
   }
   

   This rule correlates network events with "MALICIOUS_IP" entities, retrieving the confidence score and triggering an alert only if the score exceeds 80.

Key Considerations:

• Data Structure and Complexity: Choose the approach that best fits the complexity of your data and the logic of your rules. For simple lists, multiple reference lists might suffice. For structured data with multiple attributes, UDM entities are more appropriate.
• Performance: Optimize your data storage and rule logic to minimize latency.
• Maintenance: Consider the ease of updating and maintaining your IOC data. UDM entities and reference lists provide built-in mechanisms for management within Chronicle.

While multi-column reference lists are not directly supported in Chronicle at present, you can employ alternative strategies using UDM entities or multiple reference lists with custom logic to achieve similar results. Evaluate the trade-offs between these options based on your specific use case and data requirements.