Finding Unparsed events

[srijankafle]

Hi,

What is the best way to field unparsed events ingested in SecOps SIEM without it being a pain. I have so far been searching for raw log with regex and using Procedural Filtering for individual log source until I collect few hundreds and then try to work on parser extension. This has been painful and a lot of manual task and chances of missing unparsed event is high for us (considering the volume of data we ingest and search result capping the returned events to x result (10,000 default)

I believe there should be a way for searching for just unparsed logs so that we have better visibility of what we are missing out on. 

[deeshu]

You can now run raw searches through udm search page. 

[chrisproudley]

Hello @srijankafle, the raw scan is your best approach. A few pointers that might help narrow things down a little:

• .* is a nice broad wildcard to leverage
• A dropdown of log sources (types) can be selected to narrow the search to a specific type of data, e.g.

• A time boundary is always applied to raw search, could this be narrowed further from whatever you're using?

If the above doesn't get you further ahead, you might find some more answers here: https://cloud.google.com/chronicle/docs/investigation/search-raw-log or feel free to follow up here with further questions.

[srijankafle]

Hi @chrisproudley ,

We are currently using the same method. However in an MSSP environment where there are thousands of events ingested every few seconds, finding the unparsed log is like searching for a needle in a haystack.

We have reached where a point where we have to reduce the timeframe to only include few minutes of logs (as the maximum result is capped) and identify unparsed logs and repeat this until we assume there are no other logs missing.
We are searching for a way to search just for unparsed log. As a engineer this is a very crucial part of the task that we would need and I do not see this being discussed anywhere else.