Agents In Use
What’s new todayLast update
18 additions

Codex builds a rate limiter for an over-crawled site

When crawlers hammered datasette.io, Willison used Codex to build a configurable IP rate-limiting plugin. His release note includes the configuration used on the live site.

Added simonwillison.netAgent: Codex
TRY A BOUNDED VERSION

Prompt to try

Editorial adaptation, not the source author’s exact prompt. Replace bracketed inputs and review tool permissions first.

Inspect sanitized request logs for [service]. Propose route-specific rate limits and exemptions, then implement them on a branch with checks for normal traffic, bursts and proxy IP handling. Produce configuration and rollback instructions. Stop before enabling limits in production.

What happened

Willison published the plugin and says the included configuration is running on datasette.io; no measured traffic reduction is given.

Reported by Simon Willison

How the agent did it

  1. Identify the areas receiving excessive crawler requests.
  2. Delegate implementation of configurable per-IP request limits to Codex.
  3. Configure exempt paths and rule-specific windows for the deployed plugin.

Where a person stepped in

Willison selected the production configuration. The note does not document a separate approval or review protocol.

A person approved each step before the agent continued.

More details

About this account

Codex implemented the plugin from the operational goal; the short release account does not expose every intermediate decision.

Firsthand release note links code and production configuration. Intermediate testing and effectiveness are not documented here.

Credit: Simon Willison

Source published 14 May 2026

Source checked 14 Oct 2026

Tools and topics

Codex · GPT-5.5 · Datasette

Operations · Checkpoints

Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.

Related use cases