How to build a binary fuzzer
An author built a binary fuzzer with models from several vendors, custom CI/CD and pytest-based TDD, emphasizing that the resulting fuzzer found FFmpeg bugs.
news.ycombinator.comSuggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Help develop [binary fuzzer] using [reference fuzzers and papers]. Propose information-theory techniques for extracting input structure, implement changes under [agent rules], and use [pytest suite and CI checks] to validate each change.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source poster / publisher
- dclavijo
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- The author reports that the resulting fuzzer found FFmpeg bugs, including an earlier linked issue, and describes a tradeoff of speed for edge novelty.
- Human involvement
- The author invested two months, wrote most agent rules by hand and maintained CI/CD and a pytest testing workflow.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Develop a fuzzer with AI models from several vendors.
- Implement information-theory algorithms and incorporate ideas from other fuzzers and papers.
- Set up custom CI/CD and impactguard with mostly handwritten agent rules.
The author describes spending two months and making more than 1,100 commits while developing a fuzzer with AI assistance. They used models from several vendors rather than relying only on Claude. Their stated direction was to implement information-theory algorithms, extract statistically important structure from the binary being fuzzed, and incorporate features from other fuzzers and papers.
The development process included a custom CI/CD setup and the author's impactguard tool, which they say helped them a couple of times. They also maintained agent.md rules, writing 70–80% of those rules by hand. A pytest test suite and adoption of test-driven development formed another part of the workflow; the author says models seemed to introduce fewer bugs after that change.
They emphasize that the fuzzer, rather than an LLM directly, found the bugs. The post reports FFmpeg bug findings and links an earlier issue. The author characterizes the tool as trading speed for edge novelty, without supplying comparative measurements here.
Who it's for: Developers building testing infrastructure who can maintain explicit rules, automated checks and a sustained development process.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Hacker News · dclavijo
Frequently asked questions
Did an LLM directly discover the FFmpeg bugs?
The author explicitly says the fuzzer found the bugs. AI models assisted development of the fuzzer.
What development checks did the author describe?
They describe custom CI/CD, their impactguard tool, mostly handwritten agent.md rules, a pytest test suite and adoption of test-driven development.
More details
Remaining steps
- Adopt pytest-based test-driven development.
- Use the resulting fuzzer to find FFmpeg bugs.
About this account
Several AI models assisted a two-month development effort; the source does not assign individual tasks or execution permissions to specific agents.
Platform: Hacker News. Attribution: https://news.ycombinator.com/item?id=49477038. Prompt: Suggested (written by Agents In Use, not the author's). Sourced by the Agents In Use pipeline.
Credit: dclavijo on Hacker News
Source published
Added to Agents In Use
Source checked
Tools and topics
Multiple AI models · impactguard · pytest · agent.md · Custom CI/CD
Fuzzing · FFmpeg · TDD · CI/CD
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.