Codex optimizes a Rust template engine

Max Woolf used agent-driven benchmark iteration to improve Rust software. One concrete example in his account is a template engine built with Codex. After initial optimization passes, he asked it to compare the implementation against askama, minijinja and tera across different input sizes and realistic workloads.

The workflow used Criterion benchmarks to establish performance, make a library change and measure again. Woolf's constraints forbade editing benchmark code to manufacture a win and sought comparable workloads rather than a special advantage for his implementation. His broader process also included review agents and refactoring with regression checks.

For the template engine, Woolf reports roughly twice the performance of minijinja and tera on most benchmarks. It still lost to askama, whose compile-time approach differed. He then instructed Codex to add a compile-time path and reports that the comparison improved. This sequence is useful because an unfavorable measurement changed the implementation strategy.

The results remain the author's observations. The project was still being prepared for release, the full implementation was not available for independent inspection in the account, and Woolf explicitly acknowledges the possibility of subtle benchmark gaming. Neither the target in a prompt nor a screenshot establishes universal performance superiority.

Prompt
Original prompt

Source: Blog · Max Woolf