LinkedIn
Twitter
We’ve previously looked at different functions that can be helpful when working with timestamps. For instance, timestamp.get_timestamp can provide various formats for dates and times to be displayed in while others, like timestamp.truncate or timestamp.as_unix_seconds, can be used to modify the time value for specific use cases.
timestamp.diff
Today, we are going to look at another timestamp function. timestamp.diff provides a method to calculate the difference between two timestamps and output a value in a time unit, like minutes, hours, or days, to name a few.
I feel like I need to preface this by stating that calculating the difference between two timestamps does not require the use of the timestamp.diff function. After all, we can use mathematical operations to calculate a difference between two timestamps. The value that the timestamp.diff function brings is the capability to specify a time unit and return a value based on the difference within that time unit of the timestamp. There will be times (pardon the pun) when mathematical operations to calculate the difference are warranted and other times when we want to show the difference in the time unit. Like many of the functions we have discussed recently, there isn’t a single “right” way to do things, it depends upon what you are looking to accomplish. You have a selection of tools to choose from for the job at hand.
timestamp.time_diff takes two arguments with an option for a third. The first two arguments are the timestamp values with the end date being the first argument and the start date being the second. So, if I have two time fields like metadata.event_timestamp.seconds and metadata.ingested_timestamp.seconds, the event timestamp is a smaller value than the ingested timestamp because the event occurred before it was ingested, hence the event timestamp is a lower epoch time value and would be the second argument and the ingest timestamp would be the first.
However, sometimes we don’t know which timestamp field goes first in our function or perhaps you mix up your timestamp values. Either way, we can leverage the math.abs function to prevent the result from being a negative value. To apply the absolute value with the difference function, the functions working together would look like this: math.abs(timestamp.time_diff(x,y,"time unit")).
The third argument in the timestamp.diff function is optional, but really, the beauty of the function is defining a time unit in this argument, so why exclude it? If you don’t provide this argument, the results will return in seconds, which is fine, but then again, you could calculate the difference in seconds by just using the subtraction operator.
Let’s use a search to illustrate how this function could be applied. For this example, we will use Office 365 logs. These logs have some level of variability between the time that an event occurs and when the data is ingested due to how Microsoft makes the data in their API available. Because of this, we want to understand what kind of lag may occur between event generation and ingestion.
To see this difference, we’ve created the following search, but we are going to look at the difference in a few different ways, including with different time units, which is why we have a number of outcome variables in our search.
metadata.product_name = "Office 365"
outcome:
$date_difference = metadata.ingested_timestamp.seconds - metadata.event_timestamp.seconds
$event_date = timestamp.get_timestamp(metadata.event_timestamp.seconds)
$ingested_date = timestamp.get_timestamp(metadata.ingested_timestamp.seconds)
$diff_seconds = timestamp.diff(metadata.ingested_timestamp.seconds, metadata.event_timestamp.seconds, "SECOND")
$diff_minutes = timestamp.diff(metadata.ingested_timestamp.seconds, metadata.event_timestamp.seconds, "MINUTE")
$diff_hours = timestamp.diff(metadata.ingested_timestamp.seconds, metadata.event_timestamp.seconds, "HOUR")
order:
$date_difference desc
The first outcome variable, $date_difference, is calculating the difference by using the subtraction operation. To clearly see the times that are being used in our functions, we’ve outputted to the $event_date and $ingested_date outcome variables these time values formatted using the timestamp.get_timestamp function which makes those dates nice and readable. The last three outcome variables all use the timestamp.diff function, each with a different time unit argument.
Looking at the results above, we can see that the date_difference and diff_seconds columns have the same values. This makes sense since the subtraction operation and the timestamp.diff function with the seconds argument are essentially interchangeable. The diff_minutes column also looks pretty straightforward, however, the diff_hours looks a bit, well, off. After all, in the first row, 96 minutes isn’t 2 hours, and in the bottom two rows, 36 in one row is associated with 0 diff_hours and with 1 diff_hours in the other. What is going on?
The answer goes back to the earlier statement around the purpose of this function which is that it is calculating the difference based on the specified time unit. If we look at those columns of nicely formatted times, we can see in the first box that while there is a difference of 90 minutes, the hour time units for the event and ingest times are 2 hours units apart. Similarly in the bottom two rows, while both are 36 minutes apart, one occurs within the same hour and the other spans 1900 and 2000 hours, so the hour time unit difference in that case is 1.
Like other functions in YARA-L, timestamp.diff can be used in both searches and rules. While we only used a search today, the same concepts apply for rules. Remember, the big difference between calculating difference using the subtraction operator and this function is that the function is outputting a difference based on the time unit specified. While we showed examples including hours and days, the function can also be used for other units like week, month, quarter and year.
I hope this timestamp function and the others we have recently discussed will help you in your searches and rule writing as you conduct threat hunts, investigations and development of detections within Google Security Operations!