How to use Codex to generate vulnerability proofs

Researchers at Virginia Tech used Codex in PoVSmith to determine whether vulnerable Java dependencies could actually be exploited through application code. A library advisory alone did not show which public application method could reach the affected API or supply the required input.

Codex first traced call paths from public methods to a specified vulnerable sink. The pipeline then supplied the path, project context and an exemplar library test so the agent could generate an application-level JUnit proof. Build and execution feedback drove revisions instead of accepting a plausible test that did not compile.

The instructions kept real call-path methods unmocked and required test inputs to be constructed within the test. A separate assessment stage inspected the code and logs, with outputs passed to developer review. The researchers also manually checked the identified paths and generated behavior.

Across the reported curated dataset, eighty-four generated tests demonstrated vulnerability-triggering behavior. Other tests did not, so generation and compilation were not treated as synonymous with a successful proof. The authors submitted proposed findings but had not received maintainer feedback at publication. This supports a measured research workflow for executable dependency-risk evidence, not a claim that every listed dependency vulnerability is exploitable in every application.

Prompt
Original prompt

Source: arXiv research paper · Shravya Kanchi, Xiaoyan Zang, Ying Zhang, Danfeng Yao and Na Meng