How to use Claude to draft API conformance tests
A developer directed Claude to inventory AEP requirements and draft conformance tests in about a day. The result reports compliance and unimplemented specification areas, but was described as a first draft.
news.ycombinator.com
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Read [API specification] and inventory every MUST, SHOULD, and MAY with its relevant context. Use that inventory to draft [conformance test suite] for [API implementation], producing a report that distinguishes compliance failures from unimplemented specification areas.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source poster / publisher
- thegagne
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- The author reported a satisfactory first draft after about a day, including a report identifying compliance and unimplemented specification areas.
- Human involvement
- The author reported guiding Claude quite a bit and explicitly characterized the result as a first draft.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- Have Claude inventory every MUST, SHOULD, and MAY in the AEP specification.
- Direct Claude to write conformance tests from the requirements inventory.
- Guide the work toward the desired first draft and its compliance report.
Thegagne used Claude to write conformance tests for the AEP specification. The starting instruction was to inventory every MUST, SHOULD, and MAY in the specification. After that requirements inventory, the author let Claude proceed with the test-writing work.
The process took about a day, according to the post. It was not entirely hands-off: the developer said they provided substantial guidance to get the result they wanted. They described the output as a first draft they were happy with, rather than a finished or independently validated conformance authority.
The resulting project generates a report that identifies both whether an API follows the specification and which parts it does not implement. The author acknowledged an existing AEP end-to-end validator that does similar work, but characterized this project as more thorough.
Writing the tests also helped the developer learn the specification. They expected the result to help refine their dotnet AEP server and aepbase, although the post did not report those later improvements as completed.
Who it's for: API developers who want a guided first draft of specification-based tests and implementation-gap reporting.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Hacker News · thegagne
Frequently asked questions
What did the developer ask Claude to do first?
They first had Claude inventory every MUST, SHOULD, and MAY in the AEP specification before proceeding with conformance-test generation.
What does the generated report show?
According to the author, it shows whether an API follows the specification and which parts of the specification the API does not implement.
Was the result considered final?
No. The author called it a first draft and said they had guided Claude substantially to obtain the desired result.
More details
About this account
Claude generated the requirements inventory and tests, with substantial developer guidance throughout the roughly day-long effort.
Platform: Hacker News. Attribution: https://news.ycombinator.com/item?id=48885388. Prompt: Suggested (written by Agents In Use, not the author's). Sourced by the Agents In Use pipeline.
Credit: thegagne on Hacker News
Source published
Added to Agents In Use
Source checked
Tools and topics
Claude · AEP specification · aep-conformance-test
API conformance · Test generation · Specifications
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.