Google Threat Intelligence: Step 3.1 - Analysis | ...

GCSCommunity · ‎09-11-2024

This section of Google Threat Intelligence Onboarding is going to cover the administration of Google TI’s IOC Investigation capability.

Google Threat Intelligence has a vast repository containing millions of files (malware, clean software, documents, etc.), urls, ips, domains and its associated behavior and metadata collected from various sources. This wealth of data provides a valuable resource for threat hunters and security researchers to discover new threats, track threat evolution and develop detection rules. IOC Investigation provides a suite of tools and features to help you navigate and analyze its corpus.

Everyday, users around the world upload millions of files, urls, ips and domains for checking, this artifacts are analyzed, metadata and relationships extracted, and top security vendors give their verdict on its maliciousness. This information is gathered into reports that show all the information related to them, where users can assess it and act in consequence.

Prerequisites

Access to the Homepage and its features, requires the user to have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).

Actions

IOC Analysis

Google Threat Intelligence’s IOC Investigation allows users to check on individual files, URLs, IPs, or domains and get a report on them. More advanced searches are called Intelligence, where you use search modifiers to filter on metadata, relationships, behavior, content and much more.

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).

Steps

On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation.
The IOC Investigation dropdown menu will appear. Select Check with VirusTotal from the dropdown menu and the IOC Analysis page will appear. This is the same as the platform’s main page.
Users can search for:
1. Files
2. Hashes
3. Domains
4. IPs
5. URLs
6. Malware Failies
7. Threat Actors
8. CVEs
9. Other Malicious Observations
If the user selects Choose File, to load a File, a popup will appear called Choose a File.
Users will see two options, to a Public or Private option:
1. Public files will be visible to others in the VirusTotal Community, for collaboration purposes.
2. Private files will remain restricted for the user only.
To choose the Public option, click Check with our Intelligence.
To choose the Private option, click Private Scanning.

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/get-started-ioc-investigation

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s). Steps On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation. The IOC Investigation dropdown menu will appear. Select Check with VirusTotal from the dropdown menu and the IOC Analysis page will appear. This is the same as the platform’s main page. Users can search for: Files Hashes Domains IPs URLs Malware Failies Threat Actors CVEs Other Malicious Observations If the user selects Choose File, to load a File, a popup will appear called Choose a File. Users will see two options, to a Public or Private option: Public files will be visible to others in the VirusTotal Community, for collaboration purposes. Private files will remain restricted for the user only. To choose the Public option, click Check with our Intelligence. To choose the Private option, click Private Scanning. Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/get-started-ioc-investigation

Perform IOC Searches

Google Threat Intelligence’s IOC Investigation allows users to check on individual files, URLs, IPs, or domains and get a report on them. More advanced searches are called Intelligence, where you use search modifiers to filter on metadata, relationships, behavior, content and much more.

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).

Steps

If the user choose to conduct an advanced IOC search, they will select File Search Modifiers, by clicking on an icon in the Search bar of the IOC Analysis page.
A popup box will appear with multiple options:
1. Files
2. URLs
3. Domains
4. IPs
5. MultiSearch
6. Collection
7. Examples
These options have several sub-options below, specific to each option tab.
When all relevant options are selected, users will click the Search Button.
If users select a search, as an example, of entity:file p:10+ engines: ransomware, users will see a File Report results page.
In the File Report results page, users will see the relevant results, with five dropdown options in the top-right corner of the page:
1. Sort By
2. Filter By
3. Export
4. Tools
5. Help Menu
In the list of File Report results, with relevant information below each Entity Name, for this example these will include:
1. Match Content
2. Related Network Locations
3. Signature
4. Tags
To the right of the Entity Name, are additional details in the File Report results, per this example, these includes:
1. GTI Score
2. Detections
3. Size
4. First Seen
5. Submitters
6. Type (shown as an icon)

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/get-started-ioc-investigation#ioc-analysis

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s). Steps If the user choose to conduct an advanced IOC search, they will select File Search Modifiers, by clicking on an icon in the Search bar of the IOC Analysis page. A popup box will appear with multiple options: Files URLs Domains IPs MultiSearch Collection Examples These options have several sub-options below, specific to each option tab. When all relevant options are selected, users will click the Search Button. If users select a search, as an example, of entity:file p:10+ engines: ransomware, users will see a File Report results page. In the File Report results page, users will see the relevant results, with five dropdown options in the top-right corner of the page: Sort By Filter By Export Tools Help Menu In the list of File Report results, with relevant information below each Entity Name, for this example these will include: Match Content Related Network Locations Signature Tags To the right of the Entity Name, are additional details in the File Report results, per this example, these includes: GTI Score Detections Size First Seen Submitters Type (shown as an icon) Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/get-started-ioc-investigation#ioc-analysis

IOC Stream Subscription

Google Threat Intelligence’s IOC Investigation has an IOC Stream view. This view allows users to digest the incoming VT flux into relevant threat feeds that you can study here or easily export to improve detection in your security technologies.

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).
Users must subscribe to Collections and Threat Actors in the Threat Landscape page, to receive matches and reference information.

Steps

On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation.
The IOC Investigation dropdown menu will appear. Select IOC Stream option from the dropdown menu and the IOC Stream view page will appear.
In the IOC Stream page, users will see five tab options available:
1. Aggregated
2. Files
3. Domains
4. IP Addresses
Each of these options will have a Search Bar at the top of the page, along with filtering options, below the Search bar.
On the IOC Stream view page, the Aggregated tab will display all Feeds combined and a Manage Sources link.
Users will be able to select by different Source Type, for their results, under a dropdown menu below the Search Query bar:
1. Collection
2. Hunting Ruleset
3. Retrohunt Job
4. Threat Actor
The Manage Sources link is on the top right side of page.
Users will click on the Managed Sources link to see the three Source options:
1. Collections
2. Threat Actors
3. Hunting Rulesets
To search for a Managed Source, users will enter the Search Query options by:
1. Name
2. Description
3. Unspecified options
Once a Search Query is entered in the bar, users will hit enter
The Search Query results will appear in a list below the Search Query bar.

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/ioc-stream
Additional Documentation: https://gtidocs.virustotal.com/docs/sources-subscriptions
Additional Documentation: https://gtidocs.virustotal.com/docs/ioc-stream#feeds

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s). Users must subscribe to Collections and Threat Actors in the Threat Landscape page, to receive matches and reference information. Steps On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation. The IOC Investigation dropdown menu will appear. Select IOC Stream option from the dropdown menu and the IOC Stream view page will appear. In the IOC Stream page, users will see five tab options available: Aggregated Files Domains IP Addresses Each of these options will have a Search Bar at the top of the page, along with filtering options, below the Search bar. On the IOC Stream view page, the Aggregated tab will display all Feeds combined and a Manage Sources link. Users will be able to select by different Source Type, for their results, under a dropdown menu below the Search Query bar: Collection Hunting Ruleset Retrohunt Job Threat Actor The Manage Sources link is on the top right side of page. Users will click on the Managed Sources link to see the three Source options: Collections Threat Actors Hunting Rulesets To search for a Managed Source, users will enter the Search Query options by: Name Description Unspecified options Once a Search Query is entered in the bar, users will hit enter The Search Query results will appear in a list below the Search Query bar. Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/ioc-stream Additional Documentation: https://gtidocs.virustotal.com/docs/sources-subscriptions Additional Documentation: https://gtidocs.virustotal.com/docs/ioc-stream#feeds

Live Hunt

Google Threat Intelligence’s IOC Investigation has a Livehunt option that allows users to hook into the stream of files analyzed by Google Threat Intelligence (Google TI) and get notified whenever a new one of them matches a certain rule written in the YARA language.

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s)

Steps

On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation.
The IOC Investigation dropdown menu will appear. Select Livehunt option from the dropdown menu and the Livehunt page will appear.
In the Livehunt page, users will see one tab named Rulesets. Below Rulesets tab is a Search bar that allows users to enter a Search Query by:
1. Ruleset
2. Rule Name
Other Filtering options are available below the Ruleset Search bar, to filter query results by:
1. Rulesets
2. Entities
3. Enabled / Disabled
4. Deleted Rulesets
  - API Documentation
  - Writing YARA Rules of Livehunt
On the top right of the Livehunt page is an option titled Crowd-Sourced YARA Hub. It is a dashboard that provides access to YARA rules published online by the community of security professionals.
The Crowd-Sourced YARA Hub, serves a central hub for YARA rules that allows users to quickly and easily import community-created rules in the users’ own hunting workflows.
Users can enter a Search Query in the Search bar and search by:
1. Rule Name
2. Description
3. Other Metadata
Users can filter search results by:
1. Author
2. Modification Date
3. Threat Category
4. Ascending/ Descending of Modification Date results
Users will see a dropdown tab for Help, to Learn More About Crowd-sourced YARA Hub.
Below the Rulesets Search bar, users will see an option on the left-side titled New Livehunt Ruleset tab.
Click on the New Livehunt Ruleset tab, a dropdown menu will appear listing four options:
1. New Ruleset to Get Files
2. New Ruleset to Get URLs
3. New Ruleset to Get Domains
4. New Ruleset to Get IPs
Select one of the options and an New Untitled YARA Ruleset creation page will appear.
The Untitled YARA Ruleset creation page will have a Template section on the left-side of the page.
In the center of the page is a YARA Ruleset Text Editor.
Users will enter a name for their new YARA Ruleset replacing Untitled YARA ruleset: (ex; My First Rule/ Ransomware Rule).
In the right-side of the page is a Settings section, which allows users to set:
1. Notification Limits
2. Notify by Email
3. An option to Run as Retrohunt
At the bottom of the page is the Testing section, which users will see Testing option tabs:
1. Test
2. Test Results
3. Problems
To the right of those tabs is the Run Test button. Click the Run Test button to test the YARA Ruleset.

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/livehunt
Additional Documentation: https://gtidocs.virustotal.com/docs/file-hunting-writing-yara-rules-for-livehunt

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s) Steps On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation. The IOC Investigation dropdown menu will appear. Select Livehunt option from the dropdown menu and the Livehunt page will appear. In the Livehunt page, users will see one tab named Rulesets. Below Rulesets tab is a Search bar that allows users to enter a Search Query by: Ruleset Rule Name Other Filtering options are available below the Ruleset Search bar, to filter query results by: Rulesets Entities Enabled / Disabled Deleted Rulesets API Documentation Writing YARA Rules of Livehunt On the top right of the Livehunt page is an option titled Crowd-Sourced YARA Hub. It is a dashboard that provides access to YARA rules published online by the community of security professionals. The Crowd-Sourced YARA Hub, serves a central hub for YARA rules that allows users to quickly and easily import community-created rules in the users’ own hunting workflows. Users can enter a Search Query in the Search bar and search by: Rule Name Description Other Metadata Users can filter search results by: Author Modification Date Threat Category Ascending/ Descending of Modification Date results Users will see a dropdown tab for Help, to Learn More About Crowd-sourced YARA Hub. Below the Rulesets Search bar, users will see an option on the left-side titled New Livehunt Ruleset tab. Click on the New Livehunt Ruleset tab, a dropdown menu will appear listing four options: New Ruleset to Get Files New Ruleset to Get URLs New Ruleset to Get Domains New Ruleset to Get IPs Select one of the options and an New Untitled YARA Ruleset creation page will appear. The Untitled YARA Ruleset creation page will have a Template section on the left-side of the page. In the center of the page is a YARA Ruleset Text Editor. Users will enter a name for their new YARA Ruleset replacing Untitled YARA ruleset: (ex; My First Rule/ Ransomware Rule). In the right-side of the page is a Settings section, which allows users to set: Notification Limits Notify by Email An option to Run as Retrohunt At the bottom of the page is the Testing section, which users will see Testing option tabs: Test Test Results Problems To the right of those tabs is the Run Test button. Click the Run Test button to test the YARA Ruleset. Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/livehunt Additional Documentation: https://gtidocs.virustotal.com/docs/file-hunting-writing-yara-rules-for-livehunt

Retro Hunt

Google Threat Intelligence’s IOC Investigation has a Retrohunt option that allows users to match a YARA rule back in time against our historical collection of files. Users can scan the files sent to Google Threat Intelligence in the past 12 months with the user’s own YARA rules.

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).
The 12 months limit applies to users of Hunting Pro, for standard users the limit is 3 months.

Steps

On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation.
The IOC Investigation dropdown menu will appear. Select Retrohunt option from the dropdown menu and the Retrohunt page will appear.
In the Retrohunt page, first time users will see a Create Your First Retrohunt Job tab at the center of the page.
In the New Retrohunt Job page, users will see a similar page to the one displayed when creating a new YARA Livehun Ruleset.
Users will enter a name for their new New Retrohunt Job (ex; My Retrohunt Job).
One difference between the YARA Livehunt Ruleset creation page and the New Retrohunt Job page is, instead of a Settings section, users will see a Corpus section, with options to select a:
1. Time Range
2. Notify
In the Notify text box, users will enter an email address to receive notifications.

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/retrohunt
Additional Documentation: https://gtidocs.virustotal.com/docs/retrohunt#creating-a-retrohunt-job

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s). The 12 months limit applies to users of Hunting Pro, for standard users the limit is 3 months. Steps On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation. The IOC Investigation dropdown menu will appear. Select Retrohunt option from the dropdown menu and the Retrohunt page will appear. In the Retrohunt page, first time users will see a Create Your First Retrohunt Job tab at the center of the page. In the New Retrohunt Job page, users will see a similar page to the one displayed when creating a new YARA Livehun Ruleset. Users will enter a name for their new New Retrohunt Job (ex; My Retrohunt Job). One difference between the YARA Livehunt Ruleset creation page and the New Retrohunt Job page is, instead of a Settings section, users will see a Corpus section, with options to select a: Time Range Notify In the Notify text box, users will enter an email address to receive notifications. Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/retrohunt Additional Documentation: https://gtidocs.virustotal.com/docs/retrohunt#creating-a-retrohunt-job

Diff

Google Threat Intelligence’s IOC Investigation has a Diff option. DIFF helps threat analysts in creating YARA rules by automating the identification optimal patterns to detect groups of files (malware families, threat campaigns, threat actor toolset).

Prerequisites

Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s).

Steps

On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation.
The IOC Investigation dropdown menu will appear. Select Diff option from the dropdown menu and the Diff page will appear.
There is a second entry point to access the Diff, via the Enterprise File Search bar at the top of the platform page. If a Filename Search Query is selected, a Results page will appear with a Tools tab on the right-side of the screen will appear. Select Tools and a dropdown menu will appear with an option to select Send to Diff.
In the Diff page, first time users will see there were no previous sessions created and users will click the New Diff Session button in the center of the page.
A new popup window will appear titled Create New Diff Session.
Users will see three text entry options:
1. Session Description
2. Write/ Paste File Hashes for Including
3. Write/ Paste File Hashes for Excluding
The text box that says Session Description, allows users to name their DIFF Session/ Job name.
The Write/ Paste File Hashes for Including text box allows users to specify the hashes whose common and optimal detection patterns should be identified.
Write/ Paste File Hashes for Excluding allows users to to provide an exclusion list of hashes, no subportions of those files will ever be identified as detection patterns. For certain less common file types the exclusion list is mandatory.
When entries are complete users will select the Create box in the bottom right corner.
Selecting the Create button will launch a Diff Job session. This process will take under a minute to conclude.
After the Diff Job session is complete, a list of binary patterns will be produced.
Users will see results on the page, with Filter option tabs in the top right corner, which include:
1. View
2. Filter By
3. Tools
Users can select the DIFF Patterns that are relevant to the user, by checking the box next to the DIFF Pattern(s).
Users can select the Tools dropdown and select Create a Hunting YARA Rule.
After selecting Create a Hunting YARA Rule, users will select to either:
1. Create a Livehunt Ruleset
2. Create a Retrohunt Job
Users will see results on the page, with dropdown option tabs in the top right corner.
Users will have the option to copy the Diff patterns.
Users can click on the Search icon next to each binary pattern in order to trigger an n-gram content search for it. This allows users to understand the kind of files that match each pattern and whether they are prone to false positives.

Relevant Documentation Links

All Steps: https://gtidocs.virustotal.com/docs/diff

Prerequisites Access requires users have access, and a valid authentication. Users must be provided access and authentication from the organization’s administrator(s). Steps On the main page of Google Threat Intelligence Platform, go to the Left Navbar to select IOC Investigation. The IOC Investigation dropdown menu will appear. Select Diff option from the dropdown menu and the Diff page will appear. There is a second entry point to access the Diff, via the Enterprise File Search bar at the top of the platform page. If a Filename Search Query is selected, a Results page will appear with a Tools tab on the right-side of the screen will appear. Select Tools and a dropdown menu will appear with an option to select Send to Diff. In the Diff page, first time users will see there were no previous sessions created and users will click the New Diff Session button in the center of the page. A new popup window will appear titled Create New Diff Session. Users will see three text entry options: Session Description Write/ Paste File Hashes for Including Write/ Paste File Hashes for Excluding The text box that says Session Description, allows users to name their DIFF Session/ Job name. The Write/ Paste File Hashes for Including text box allows users to specify the hashes whose common and optimal detection patterns should be identified. Write/ Paste File Hashes for Excluding allows users to to provide an exclusion list of hashes, no subportions of those files will ever be identified as detection patterns. For certain less common file types the exclusion list is mandatory. When entries are complete users will select the Create box in the bottom right corner. Selecting the Create button will launch a Diff Job session. This process will take under a minute to conclude. After the Diff Job session is complete, a list of binary patterns will be produced. Users will see results on the page, with Filter option tabs in the top right corner, which include: View Filter By Tools Users can select the DIFF Patterns that are relevant to the user, by checking the box next to the DIFF Pattern(s). Users can select the Tools dropdown and select Create a Hunting YARA Rule. After selecting Create a Hunting YARA Rule, users will select to either: Create a Livehunt Ruleset Create a Retrohunt Job Users will see results on the page, with dropdown option tabs in the top right corner. Users will have the option to copy the Diff patterns. Users can click on the Search icon next to each binary pattern in order to trigger an n-gram content search for it. This allows users to understand the kind of files that match each pattern and whether they are prone to false positives. Relevant Documentation Links All Steps: https://gtidocs.virustotal.com/docs/diff

Next Step: Google Threat Intelligence: Step 3.2 - Analysis | Private Scanning

Previous Step: Google Threat Intelligence: Step 3 - Analysis Overview

New SecOps Webinar May 14th! Learn about Gemini's generative AI within Google SecOps