Log Latency Dashboard

[AJMSecOps]

Has anyone built any SIEM Dashboards in SecOps to monitor for latency? I'm specifically interested in finding a solution to monitoring the difference between event_timestamps and ingestion_timestamps by log type. I'd want to visualize the average difference, per log type, based on whatever timeframe I search on.

[amithpatil · Accepted solution]

If you're using native dashboards ,  You can write a query something similar to below to achieve this : 

match:
    metadata.log_type
outcome:
    $event_count = count_distinct(metadata.id)
    $event_latency = avg(metadata.ingested_timestamp.seconds - metadata.event_timestamp.seconds)

[cmorris · Accepted solution]

Please try this:

metadata.log_type != ""
$logType = metadata.log_type

match:
  $logType

outcome:
  $eventTotal = count_distinct(metadata.id)
  $deltaSec = math.round((sum(metadata.ingested_timestamp.seconds) - sum(metadata.event_timestamp.seconds))/$eventTotal,0)
  $deltaMin =  math.round($deltaSec/60,2)

order:
  $deltaMin desc

[amithpatil · Accepted solution]

If you're using native dashboards ,  You can write a query something similar to below to achieve this : 

match:
    metadata.log_type
outcome:
    $event_count = count_distinct(metadata.id)
    $event_latency = avg(metadata.ingested_timestamp.seconds - metadata.event_timestamp.seconds)

[cmorris · Accepted solution]

Please try this:

metadata.log_type != ""
$logType = metadata.log_type

match:
  $logType

outcome:
  $eventTotal = count_distinct(metadata.id)
  $deltaSec = math.round((sum(metadata.ingested_timestamp.seconds) - sum(metadata.event_timestamp.seconds))/$eventTotal,0)
  $deltaMin =  math.round($deltaSec/60,2)

order:
  $deltaMin desc

[cmmartin_google · Accepted solution]

See https://medium.com/@thatsiemguy/fix-rfc3164-timestamps-with-bindplane-for-enterprise-fb96dd16d015 for some examples too

[AJMSecOps]

Very helpful, so thanks to all who posted here.