Claude Code tests literature-grounded training ideas

Paper Lantern connects Claude Code to its research-paper MCP server for an autoresearch experiment. Two runs start with the same small GPT model and conditions. Each explores one hundred experiments with a five-minute training budget per trial and at most three hyperparameter-tuning attempts per idea.

In the literature-assisted run, the agent searches for candidate approaches, examines implementation details and tries ideas against the model. The team reports considering 520 papers and directly testing ideas from twenty-five. One useful example is batch-size reduction: paper-backed guidance supplies the learning-rate adjustment needed when halving the batch, rather than treating the batch setting in isolation.

Not every researched technique fits. The source identifies Dynamic Tanh and SeeDNorm as incompatible with the architecture. When the best configurations are trained for two hours, the team reports validation-loss values of 0.4624 without the paper server and 0.4475 with it. This is a vendor-run comparison on a roughly seven-million-parameter model, not independent evidence of a universal improvement or larger-model scaling. The practical workflow is to turn literature into testable implementation choices, keep failed trials visible and use measured validation results to decide which configuration survives.

Prompt
Original prompt

Source: Blog · Paper Lantern team