Needhelp with yara rule to find the users who are accessing the website which is hosted in AppEngine

[NikhilBattula]

Hi everyone,

I'm currently working on setting up some security monitoring for my Google App Engine-hosted website, and I'm looking to create a YARA rule in chronicle to detect users who are accessing the website. I'd like to monitor this activity for security and compliance purposes.

I tried various udm fields (target.) to find the users who accessing the website, But could'nt be able to find the correct UDM field.Could someone please provide guidance on how I can create an effective YARA rule to achieve this? Specifically, I'm looking to detect HTTPS/HTTP requests or any other relevant activity indicating user access to the website.

Any advice, examples, or resources would be greatly appreciated.

Thanks in advance!
Nikhil

[Ben_T]

Hi NikhilBattula,

Here are some sample UDM searches applicable to Google App Engine default logging to help get you started following by a YARA sample:

• Event Type: HTTP_SERVER_LOG (or whatever your equivalent log source is)
• App Engine Domain: We used a placeholder "your-app-engine-domain.com"
• HTTP Methods: GET and POST
• Success Status Codes: 200 to 399

Relevant UDM Fields

Here are the most vital UDM fields you'd likely use for searching:

• target.domain: The domain name the request was made to.
• http_event.request.method: HTTP method (GET, POST, etc.)
• http_event.response.status_code: The HTTP status code of the response
• principal.ip: Client IP address making the request
• principal.geoip.country: (If available) Country of origin based on IP address

Example UDM Searches

1. Basic Access to Your Domain:

   event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" 
   

2. POST Requests Only:

   event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.request.method = "POST"
   

3. Successful Requests from a Specific Country:

   event_type = "HTTP_SERVER_LOG" AND target.domain = "your-app-engine-domain.com" AND http_event.response.status_code >= 200 AND http_event.response.status_code < 400 AND principal.geoip.country = "US" 
   

Important Notes:

• Log Content: These searches assume your web server logs are being ingested into Chronicle and parsed to populate the correct UDM fields.
• UDM Search Syntax: Remember that Chronicle supports basic operators like =, !=, >, <, AND, OR, etc. See the official docs for details (https://cloud.google.com/chronicle/docs/investigation/udm-search)
• Refinement: UDM searches can get quite complex. Add more fields to filter, use regular expressions for sophisticated pattern matching, and leverage Chronicle's built-in functions if needed.

Pro Tip:

Start with simple searches, examine the results, then add more conditions iteratively to fine-tune your queries. Chronicle's search interface will offer autocomplete suggestions based on the available UDM fields in your ingested data.

Focus Points for the YARA Rule

• IP Addresses: Client IP addresses will be the primary way to track 'users' in this scenario. Understand that a single IP address can represent multiple users over time (DHCP, shared devices, etc.).
• Request Patterns: Target HTTP methods (GET, POST), requested URLs, user-agents, and potentially referrers. These help distinguish regular access from potentially malicious probes.
• Geolocation: Consider IP-based geolocation to detect accesses from unusual regions.
• Behavioral Baselines (If Possible): This depends on your logging depth. You might track regular access times/patterns per IP to detect anomalies.

##Example YARA-L rule

rule detect_website_activity {
meta:
author = "Your Name"
description = "Detects accesses to Google App Engine Website"
yara_version = "YL2.0"
rule_version = "1.0"

events:
$http_event.metadata.event_type = "HTTP_SERVER_LOG" // Adjust if your log type is different

condition:
$http_event.target.domain == "your-app-engine-domain.com" and
($http_event.request.method == "GET" or $http_event.request.method == "POST") and
$http_event.response.status_code >= 200 and
$http_event.response.status_code < 400 // Focus on successful requests
}

Explanation

• meta: Descriptive information about the rule
• events: Specifies the type of log events this rule applies to. Adjust 'HTTP_SERVER_LOG' if needed.
• condition: The heart of the detection. We look for:
  • Traffic to your App Engine domain
  • GET or POST requests (common for browsing and logins)
  • Status codes indicating successful page loads/actions

Important Notes:

1. Customize: Replace "your-app-engine-domain.com" with your actual domain. Add more specific URL rules if you like (e.g., targeting a login page).
2. Log Availability: This rule ASSUMES you are feeding appropriate web server logs into Chronicle. If not, you must first set up that data flow.
3. Refinement: This is a starting point. Enhance it over time by adding:
   • Geolocation checks
   • User-agent pattern matching (unusual browsers, etc.)
   • Anomaly detection (if you can baseline normal behavior)

[NikhilBattula]

Hi Ben,

Thank you for guiding me and giving me an example yara rule. 

I tried to find the event type for app engine logs (I didn't see any UDM events (metadata.event_type) for App engine) and tried using target.url UDM Field also. but i culdn't be able to find it.I can be able to see only GCP_CLOUDAUDIT logs in chronicle. But i want to create rules (And alerts ) in chronicle based on (request_logs) type.googleapis.com/google.appengine.logging.v1.RequestLog logs (which we can see in logs explorer) of the app engine.

I am new to chronicle and using direct ingestion to send app engine logs into Chronicle, could you please share or guide me on how to send the app engine request logs to chronicle instance?

Thanks,
Nikhil

[ion_]

Hi Nikhil

It looks like theres the following default log type and parser:

GCP_APP_ENGINE

I'd try to get the app engine logs ingested through that parser as a first step. How are you currently ingesting your data? 

EDIT for clarity: When you say direct ingestion could you clarify please