AI-Powered Efficiency: Cutting SOC Workloads by 30% in 30 Days

AI has gone from theory to production faster than any technology in recent memory. In many areas of expertise, we’re seeing demos turn into usable products within months – for example, customer support, sales outreach, software engineering, etc. However, cybersecurity hasn’t been as quick to deploy LLMs on large operational workloads.

‍

Automating Security Operations at Doppel

At Doppel, we automated 30% of our security operations workload with AI in less than 30 days. Before I get into how, let’s go over what exactly got automated.

Every day Doppel ingests over 10 million:

• Websites
• Social media accounts
• Mobile apps
• and more

Traditional machine learning filters out obvious false positives, but the most time-consuming work has always been the nuanced decisions: Does this site merit a takedown? On which platforms? Under which policies? These are not straightforward decisions, and getting them wrong can mean missing a threat or disrupting legitimate activity.

The Challenge of Sophisticated Decision-Making Automations and the Solution

Automating this sophisticated decision-making at the scale we operate at is particularly challenging. These decisions aren’t just about spotting basic patterns – they require judgment. Analysts must interpret unstructured data like screenshots, time-series activity, and customer-specific policies to choose the correct response and explain their rationale. Even for humans, getting these consistently correct requires a high degree of knowledge and training. Again, this is not an environment that’s tolerant of errors.

To solve this problem, we built a security-oriented AI agent that can work alongside our security operations team. Using OpenAI’s latest o1 model (showcased at OpenAI DevDay), we trained the AI to not only meet but exceed human-level benchmarks Specifically, o1’s false positive rate was the lowest that we measured when compared to other models and even many of our analysts. To train the model, we mirrored many of the techniques we use to onboard and upskill our own analysts: cybersecurity instruction manuals, case studies, and thousands of past decisions to help the model develop intuition. For every decision it made, we ensured it had access to the same data a human agent would consider.

Ensuring Seamless Human Collaboration

A critical part of the training process was teaching the AI to recognize when to escalate a decision to a human expert. Our engineers spent countless hours reviewing the model’s justifications and fine-tuning its prompts to ensure this escalation process worked seamlessly. OpenAI’s innovation, combined with Doppel’s expertise in cybersecurity, has enabled us to reengineer the SOC in record time. The result is an AI agent that can reliably offload a significant portion of the decision-making workload from our overburdened security operations team. For our customers, this means faster response times: the AI has reduced our MTTR (mean time to respond), helping threats come down faster than ever.

While it’s still early, we’re very excited about the productivity gains we’ve been able to achieve in such a short amount of time. Looking into the future, we believe this is just the beginning of AI’s transformation of the security operations center. And our engineering team is committed to exploring, and productionizing, the frontier.