How to use Claude to fix integration test failures
The author and a coworker report fixing over 200 failing integration tests for parity with the old code in one week, followed by another code review before launch.
news.ycombinator.com
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Run [integration test suite] for [replacement system]. Investigate [assigned failing tests] against [old-system behavior] and implement fixes. Re-run the suite and report exactly which failures changed before preparing [pull request].
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source author
- saguntum
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- The author and a coworker report fixing over 200 failing integration tests for parity with the old code in one week, followed by another code review before launch.
- Human involvement
- The author reviewed the resulting code again before launch when more well-rested.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Ran the integration test suite for the rearchitecture.
- Passed the list of failing tests to Claude.
- Created a PR when changes made more tests pass.
Saguntum describes using Claude during a major internal-system rearchitecture aimed at scalability and maintainability under tight latency constraints. The new implementation needed to remain correct relative to the old system. After returning from an international vacation jet lagged, the author had difficulty focusing on coding and tried a test-driven agent loop instead.
The pipeline started by running an integration test suite. The author passed a list of failing tests to Claude and made a pull request when the resulting changes increased the number of passing tests. They automated as much of this loop as they could at the time. The post does not specify the automation tools or quote the prompts used.
A coworker participated, and the engineers divided failing tests between themselves. As fixes succeeded, they subdivided the remaining failures again. The author reports that together they fixed over 200 failing integration tests for parity with the old implementation in one week.
Passing more tests was the condition for making a PR, rather than the only described oversight. Before launch, the author reviewed the code again after becoming more rested. They describe the technique as extremely effective.
Who it's for: engineers migrating a system with integration tests that capture compatibility requirements.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Hacker News · saguntum
Frequently asked questions
What triggered a pull request?
The author made a PR if Claude’s changes caused more integration tests to pass.
Was the generated code reviewed before launch?
Yes. The author says they reviewed the code again before launch after recovering enough to be more well-rested.
More details
Remaining steps
- Redistributed remaining failures between engineers after successful fixes.
- Reviewed the code again before launch.
About this account
Claude worked on supplied failing tests within a partly automated loop; engineers used passing-test improvements to decide when to create PRs.
Platform: Hacker News. Attribution: https://news.ycombinator.com/item?id=48431821. Prompt: Suggested (written by Agents In Use, not the author's). Sourced by the Agents In Use pipeline.
Credit: saguntum on Hacker News
Source published
Added to Agents In Use
Source checked
Tools and topics
Claude · Integration test suite · Pull requests
Integration tests · Migration · TDD
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.