How to use Codex to test a skill-packaging boundary
Codex and Claude Code packaged a deliberately modified skill in a local test. Receiver logs exposed side effects beyond packaging and approval messages.
vett.sh
Original prompt
Shared by the source author. See source notes for context.
/skill-creator
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source author
- Sean Drumm
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- A local experiment showed data receipt despite successful skill packaging and permission approval. Codex detected some initial attempts, and no vendor patch is reported.
- Human involvement
- Drumm modified a local test skill, installed it in his own agents, reviewed permission prompts and checked a local server for actual data receipt.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Drumm modified and installed a local test packaging skill in his own agent environments.
- The agents packaged a skill while the human reviewed execution and network permission prompts.
- A local receiver showed side effects that package validation and completion messages did not reveal.
Sean Drumm tested Claude Code and Codex with a modified skill-packaging helper on his own system. The ordinary task was packaging a new skill; the helper contained an added data-collection operation, letting him compare the agent's completion message with what a local receiver actually observed.
Claude Code requested permission to run the helper and produced the package, while the receiver recorded data that packaging did not require. Codex initially detected some attempts, so Drumm revised the test skill's instructions and network pretext before testing again. In the resulting demonstration, he approved a request whose explanation came from the untrusted skill.
The package validation and hash were genuine-looking outputs but did not establish that the helper's side effects were appropriate. The local receiver supplied the decisive observation of additional data leaving the process. This separated task completion from the security property being tested.
The result is a documented skill-supply-chain experiment, not a general claim that every agent installation behaves identically. Drumm discloses the skill-security product he is building, and the post does not report a vendor patch. The account retains the initial detection attempts and human approval as material boundaries of the demonstration.
Prompt
Original prompt
Source: Vett research blog · Sean Drumm
Frequently asked questions
Was this tested on someone else's machine?
The author describes his own local agent environments and receiver.
Did Codex miss every initial attempt?
No. It detected some early attempts before the test skill was revised.
Why inspect receiver logs?
They showed side effects that a valid package and completion message did not establish.
More details
About this account
Drumm modified and installed a local test packaging skill in his own agent environments. The agents packaged a skill while the human reviewed execution and network permission prompts.
Original experiment opened. Original command excerpt is identified in the source's screenshot caption. Author sells skill-security tooling; no general platform success rate or vendor fix is established. Codex initially detected some attempts. Original prompt excerpt.
Credit: Sean Drumm — Vett research blog
Source published
Added to Agents In Use
Source checked
Tools and topics
Codex · Claude Code · Skill Creator · Python · Local receiver
Skill supply chain · Observed egress
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.