Migrate Off That Old SIEM Already!

[herrald]

If you are like us, you may be surprised that, in 2024, traditional security information and event management (SIEM) systems are still the backbone of most security operations centers (SOC). SIEMs are used for collecting and analyzing security data from across your organization to help you identify and respond to threats quickly and effectively. But if you're still using an outdated SIEM, you're putting your organization at risk.

Legacy SIEMs are often slow, cumbersome, and difficult to use. They may be unable to keep up with the latest threats or support the latest features and capabilities. They may not offer the flexibility to support your organization's specific needs. They may not be suited to the multi-cloud strategy that is the reality for most organizations today, or they may be poorly positioned to take advantage of developments in artificial intelligence (AI).

But before we jump straight to “replace the sucker!”, we need to recognize that not all SIEM deployment failures are due to the product itself, and simply replacing a product may not make the organization better. Process (and occasionally people) breakdowns also play a critical role in success.

Identifying shortcomings in your current SIEM is much easier than selecting the best replacement and executing a successful migration. That's where this blog comes in. The authors have seen hundreds of SIEM migrations as practitioners, analysts, and vendors over multiple decades. So, let's take stock of the top SIEM migration tips for 2024. We'll divide this list into categories and sprinkle in lessons we've learned from the trenches throughout.

Selecting a New SIEM

How do you know when you need a new SIEM? Start by asking yourself and your team some key questions to help uncover each offering's strengths and weaknesses. We recommend quickly identifying each SIEM's "superpowers" and planning how your organization can take maximum advantage of those benefits. For example:

Is the SIEM offered by a primary cloud service provider (CSP) that can provide world-scale infrastructure at wholesale prices? Tip: Our experience shows that SIEM providers who operate in clouds they don’t own have difficulty overcoming the inescapable "margin stacking" that comes with such models. This question is inextricably linked to cost. 

Does the SIEM vendor have a continuous stream of frontline threat intelligence to drive out-of-the-box detection of new and emerging threats? Tip: These golden sources typically arise from top-tier incident response practices, the operation of massive consumer IaaS or SaaS cloud offerings, or massive install bases of security software products or operating systems. 

Does the SIEM offer an extensive library of supported parsers and detection content? Tip: Some SIEM vendors rely almost exclusively on their user community or technical alliance partners to create parsers for popular data feeds. While the existence of a thriving user community is essential, over-reliance on the user community to provide fundamental capabilities like parsing is a problem. Parsers for common data sources should be created, maintained, and supported directly by the SIEM vendor. Take the same approach when looking at detection content. Community rules are essential, but you should expect your vendor to create and maintain a solid library of core detections that are supported and improved regularly.

Does the SIEM incorporate AI, and is it positioned to continue innovating? Tip: The whole story about AI's role in SIEM has yet to be discovered by any vendor or customer. However, leading SIEMs already have tangible AI-driven features shipping today. These features include natural language processing for expressing searches, automated case summarization, and recommended response actions. Most customers and industry observers consider features like threat detection and predictive adversary analysis to be the "holy grail" of AI-driven SIEM capabilities. No SIEM reliably offers these features today, but as you choose a new SIEM in 2024, consider what kind of vendors are investing the resources necessary to make progress on these big problems.

SIEM Migration

So you've decided to make the move. Your approach to migration is critical in ensuring you maintain required capabilities and start extracting value from the new platform as soon as possible. It comes down to prioritization. A typical trade off is recognizing that, on the one hand, a SIEM migration represents an opportunity to modernize your entire investigation, detection, and response approach. However, many SIEM migrations fail because organizations try to "boil the ocean." So here are our best tips for planning and executing your successful SIEM migration.

Here are our favorite SIEM migration tips:

Define your migration goals. This sounds obvious, but your SIEM migration is a lengthy process, so the need to define your desired outcomes (e.g., faster threat detection, easier compliance reporting, improved visibility, reduced analyst toil, while also lowering cost) is strongly correlated with success.

Use the migration as an opportunity to clean house. This is a good time to clean up your detection rules and log sources and only migrate the ones that you actually use. It's also a good time to review your alert triage and tuning processes and make sure that they're up to date.

Don't migrate every log source. Moving to a new SIEM is a great opportunity to decide what logs you need, be it for compliance or security reasons. Many organizations accumulate a vast amount of log data over time, and not all of it is necessarily valuable or relevant. By taking the time to evaluate your log sources before you migrate them, you can streamline your SIEM and focus on the data that is most important to your security and compliance needs.

Don't migrate all content. Migrating all of your existing detection content, rules, alerts, dashboards, and visualizations to a new SIEM is a lot of work and it's not always necessary. Take the time to evaluate your current detection coverage and only migrate the rules that you need for your new environment.

Detection content is recreated, not migrated. There is a process of rebuilding the detection content (rules, alerts, dashboards, models, etc.)from scratch and using your old content as inspiration. In some happy future where Sigma rules, this may change, but today, there is no way to “run a converter” on rules from vendor 1 to convert to rules for vendor 2 (with some exceptions).

Prepare to spend more time on this than you think is likely. SIEM migrations are complex and they can take a long time to complete. Be prepared to allocate plenty of time and resources for the migration project. 

Develop a realistic migration timeline. This includes accounting for data transfer, testing, tuning, training and potential overlaps where you may need to run both systems in parallel. A well-defined migration plan will help you to identify and mitigate risks, and ensure that the migration is completed successfully. The plan should include a detailed timeline, a list of tasks, resources, and a budget. Recognize that major projects like a SIEM migration must be broken into phases.

Testing. We recommend adopting the practice of testing your SIEM and detection content by regularly injecting data that will test your detections, check parsing, and validate data flow from detection to case to response playbook. A SIEM migration is the perfect time to adopt a rigorous detection engineering program that includes testing like this. 

Prepare to run both old and new tools for some time. Avoid a disruptive "rip and replace" approach. A phased migration, where you move log sources or use cases gradually, helps control the process and reduces risk. Also, think twice about re-ingesting data from your old SIEM into the new. This approach sounds good in the abstract but rarely pays off.

Enable your teams. Your SIEM migration will fail if your analysts can’t use the new system. A good migration plan will include deep enablement for your teams. Think about training engineers on data onboarding and parsing, training analysts on case management/investigation/triage, threat hunters on anomaly detection/search, and detection engineers on rule-writing. 

Get help! If you are lucky (or maybe unlucky?) as a practitioner or leader, you will have perhaps gone through one or two SIEM migrations in your career. Why not seek help from specialists who have done it dozens or hundreds of times? Professional services teams from the vendor and/or consulting teams from qualified services partners are a great choice. SIEM migrations are largely human-centered efforts.

Additional reading:

• “How Chronicle SIEM Can Help Augment Your SOC Stack” paper
• “Future of the SOC: Evolution or Optimization — Choose Your Path” paper
• Detection Engineering Weekly Newsletter 
• detect.fyi - Practitioner-centric tips on detection engineering