How to use Codex to generate vulnerability proofs
Codex traced Java call paths to vulnerable dependencies and generated executable proofs. Build feedback and human validation checked what the tests showed.
arxiv.org
Original prompt
Shared by the source author. See source notes for context.
You’re in the project workspace. Find every call path whose final hop is the sink method {sink_method}.Reach people exploring AI agents
See placements →What happened
Eighty-four generated tests triggered vulnerability behavior on the curated dataset. Submitted proposed findings had not received maintainer feedback at publication.
Reported by Shravya Kanchi, Xiaoyan Zang, Ying Zhang, Danfeng Yao and Na Meng — arXiv research paperHow the agent did it
- Codex traced public application entry points to vulnerable dependency APIs.
- It generated JUnit proofs from call paths and exemplar tests and revised them using execution feedback.
- The researchers checked paths and behavior before presenting evidence for developer review.
Where a person stepped in
The researchers curated vulnerable application-library pairs, manually checked paths and test behavior and supplied results for developer review.
Researchers at Virginia Tech used Codex in PoVSmith to determine whether vulnerable Java dependencies could actually be exploited through application code. A library advisory alone did not show which public application method could reach the affected API or supply the required input.
Codex first traced call paths from public methods to a specified vulnerable sink. The pipeline then supplied the path, project context and an exemplar library test so the agent could generate an application-level JUnit proof. Build and execution feedback drove revisions instead of accepting a plausible test that did not compile.
The instructions kept real call-path methods unmocked and required test inputs to be constructed within the test. A separate assessment stage inspected the code and logs, with outputs passed to developer review. The researchers also manually checked the identified paths and generated behavior.
Across the reported curated dataset, eighty-four generated tests demonstrated vulnerability-triggering behavior. Other tests did not, so generation and compilation were not treated as synonymous with a successful proof. The authors submitted proposed findings but had not received maintainer feedback at publication. This supports a measured research workflow for executable dependency-risk evidence, not a claim that every listed dependency vulnerability is exploitable in every application.
Prompt
Original prompt
Source: arXiv research paper · Shravya Kanchi, Xiaoyan Zang, Ying Zhang, Danfeng Yao and Na Meng
Frequently asked questions
Why not rely only on a library advisory?
It does not show whether the application's public methods can reach the vulnerable behavior.
Could call-path methods be mocked?
No. The instructions preserved the real methods whose behavior the proof needed to exercise.
Were submitted findings accepted?
At publication, the paper says proposed PRs and CVE requests had not received feedback.
More details
About this account
Codex traced public application entry points to vulnerable dependency APIs. It generated JUnit proofs from call paths and exemplar tests and revised them using execution feedback.
Original paper HTML and PDF opened. Original prompt excerpt is from the paper's explicitly simplified call-path template. Results are on the curated dataset; six proposed PRs/CVE requests had not received feedback at publication. Original prompt excerpt.
Credit: Shravya Kanchi, Xiaoyan Zang, Ying Zhang, Danfeng Yao and Na Meng — arXiv research paper
Source published 5 May 2026
Source checked 14 Oct 2026
Tools and topics
Codex · PoVSmith · Java · JUnit · Build logs
Dependency reachability · Executable evidence
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.