I've noticed while troubleshooting our log ingestion metrics that despite SIEM logs being successfully ingested, my query to extract log types from the last 90 days returns empty results. I'm trying to understand why the last_heartbeat_time field is returning null values.
Current Query
SELECT log_type, MAX(last_heartbeat_time) AS last_heartbeat_time
FROM `datalake.ingestion_metrics` WHERE last_heartbeat_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 90 DAY)
GROUP BY log_type
ORDER BY log_type
The Issue
When running this query, I get no results - last_heartbeat_time is null and no log types are retrieved, even though I can verify that logs are being ingested in our SIEM system.
What I've Verified
- Logs are definitely being ingested into our SIEM system
- The issue appears specific to the last_heartbeat_time field or the time-based filtering
Has anyone encountered this discrepancy before? What could be causing the last_heartbeat_time field to be null despite active log ingestion? Are there any known issues with timestamp fields in this context?
LinkedIn
Twitter
