Devin Security Swarm replays known vulnerabilities
Cognition reports evaluating Devin Security Swarm on fifty advisory-linked vulnerabilities, with agents investigating code, composing attack paths and reproducing findings in sandboxes.
cognition.com
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Evaluate the authorized advisory cases using isolated sandboxes. Trace each suspected attack path, attempt reproduction, separate confirmed and unconfirmed findings and propose patches for security review.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source author
- The Cognition Team
- Original account
- Read the source ↗
- Published
- 1 Jul 2026
- Source link checked
- 15 Oct 2026
- Reported result
- Cognition reports evaluating Devin Security Swarm on fifty advisory-linked vulnerabilities, with agents investigating code, composing attack paths and reproducing findings in sandboxes.
- Human involvement
- Cognition controls benchmark selection and reports results. Security teams review proposed remediation; no customer breach-prevention claim is inferred.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Cognition supplies fifty advisory-linked vulnerabilities for evaluation.
- Parallel Devin agents investigate code segments and combine findings into attack paths.
- Findings are reproduced in isolated sandboxes to check runtime exploitability.
Cognition evaluated Devin Security Swarm on fifty real-world vulnerabilities associated with published GitHub Security Advisories. This is a vendor-run benchmark, not evidence that the agent prevented a customer breach. The account reports seventy-two percent recall for Devin in that evaluation.
The described workflow divides a codebase among parallel agents. Each investigates behavior across files rather than relying on a scanner alert alone. Devin combines findings into attack paths and tries to reproduce them in an isolated runtime. Confirmed findings are returned with reproduction steps; the product can then prepare remediation pull requests for human review.
Cognition identifies three benchmark findings that other evaluated tools missed: a PHP sandbox bypass, metadata argument injection and a Spring Kafka deserialization issue. These provide specific reported outputs, while the underlying methodology and advisory selection determine what the percentages mean.
The source also describes scheduling and threat-model profiles as product capabilities. Those should not be confused with separately verified customer deployments. The useful bounded account is the advisory-based evaluation and runtime-validation sequence. Its comparative cost and recall claims remain Cognition's measurements, and a reviewer must still assess any proposed patch before it is merged.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: cognition.com · The Cognition Team
Frequently asked questions
Was this an independent security benchmark?
No. Cognition reports its own evaluation on fifty advisory-linked vulnerabilities.
What follows a suspected code finding?
The described workflow attempts sandbox reproduction before presenting it as exploitable.
Does a proposed patch merge automatically?
The source says remediation pull requests are opened for review.
More details
Remaining steps
- The vendor reports recall and specific found issues; remediation PRs remain subject to human review.
About this account
Security Swarm investigated code in parallel and composed and reproduced attack paths; Cognition reports its recall on fifty advisory-linked vulnerabilities.
Firsthand account opened and read. Outcome is source-reported, not independently reproduced. Suggested prompt written by Agents In Use. Vendor-run benchmark and product workflow; no independent comparative performance verification.
Credit: The Cognition Team
Source published 1 Jul 2026
Source checked 15 Oct 2026
Tools and topics
Devin · Security Swarm · Sandbox runtime
Devin · Security Swarm · Sandbox runtime
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.