Independent tests of AI security agents and SOC decision models. DFIR results, evidence checks and practical findings. Follow for new benchmark results.secbenchmarks.comJoined September 2026
Our first Gemini 4 Argon Benchmark and it performed incredibly well!
Agentic SOC investigation harness. Run the models against different cases. Track key performance.
If you're interested in Decision Models like Jev and Clef we have some of that too! secbenchmarks.com/blog/2026-10-1…
Each model started with one confirmed part of an attack and assessed connected processes, files, computers and accounts.
The score rewards finding related items and penalizes unrelated additions. The supplied starting point earns no credit.
Small pilot; no unseen test set.
We tested 3 AI decision models on a DFIR task: finding other activity connected to a confirmed attack.
Cloudflare's Clef, Cloudflare's Clef Flash and Jev each ran 8 cases, 3 times.
Here's how they stacked up:
secbenchmarks.com/blog/2026-10-0…
🔎 GPT-6 Astra retrieved sufficient evidence in 36/36 basic DFIR cases.
25/36 ended with a correct decision and accepted proof.
Finding the records doesn’t guarantee a supported conclusion.
Preliminary, one pass per case.
secbenchmarks.com/blog/2026-09-3…
📊 Our first AI security benchmark results are live.
In 36 basic DFIR cases, Claude Opus 5.5 got 33 verdicts right. 12 also met our proof requirements.
Why we score the answer and its evidence separately:
secbenchmarks.com/blog/2026-09-3…
🧭 18/18 verdicts. 12/18 next steps.
Jev 1.13's preliminary SOC results show why “right answer” and “right action” need separate scores.
Our new analysis covers six models and the limits of a six-case test:
secbenchmarks.com/blog/2026-09-3…
@DFIRTraining The timing comparison is useful. Were the web-activity summaries also checked for correct artifact citations and missed domains? We’d like to see speed, factual accuracy and evidence coverage measured separately.
9K Followers 5K FollowingFounder & CEO of AI Research & Engineering Lab | Weekly @twimlai GenAI meetup | @Cohere_Labs Community Lead & Researcher | Prev @TCS @Infobeans | No dupes ever
993 Followers 4K FollowingAuthor, Passionate about Distributed Systems. Currently AI @Google India, co founded @adalyz_ Opinions here are my own ! He/Him
119K Followers 514 FollowingMITRE ATT&CK® - A knowledge base for describing the behavior of adversaries. Replying/Following/Re-tweeting ≠ endorsement. @ https://t.co/wt46ArkZVt
303K Followers 76 FollowingPart of @CISAgov, we respond to major incidents, analyze threats, and exchange critical cybersecurity information with partners around the world.
79K Followers 762 FollowingEnd-to-end Cybersecurity consulting team leading the industry, supporting organizations, and giving back. #Hacktheplanet
Blogs, news, webinars, and tools!
18K Followers 643 FollowingHacker, trainer, and guitarist | Black Hills InfoSec #RedTeam | @BreakForge Training | Produces music to hack to at @N0BANDW1DTH
39K Followers 3K FollowingPartner / Threat Intelligence at Microsoft. Previously, Director of Incident Response & Intel Research at Mandiant. Former Chief Technical Analyst at CISA
112K Followers 104 FollowingThe world's leading Digital Forensics and Incident Response provider. This feed updates you on latest DFIR news, events, and training.