Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps
arXiv preprint · April 2026 · lead author, with Igor Kozlov and Ambuj Kumar
It's the first public benchmark for open-ended threat hunting by AI agents. Agents get raw Windows event logs recorded from real attack techniques, with no alert and no guiding questions, and have to find the malicious events themselves. Passing takes 50% recall on every ATT&CK tactic, and no model has passed yet. The design is what makes hunting checkable: no one can prove a log is clean, but anyone can check whether an agent found the attacks known to be in it.