How to use Claude to build a Pi inference engine

The author spent three months building geistlib, an LLM inference engine written from scratch in C, with heavy pair-programming help from Claude Code. It runs Microsoft's BitNet 2B fully offline on a 4 GB Raspberry Pi 5. Engine and model are packaged in a single executable of approximately 1.2 GB, while the runtime itself is under 1 MB.

The implementation specializes in BitNet's ternary weights. On ARM64, the author says the main inference loop replaces much multiplication with additions and maps work to the SDOT instruction. Embedded weights are demand-paged, with memory usage described as comparable to mapping a separate model file.

The author reports 15–18 tokens per second on their Pi, compared with 9.3 for Microsoft's bitnet.cpp, and energy use of 0.44 joules per token. They published recordings, methodology and measurements and provided a make bench command that follows a frozen protocol, waits for thermal conditions and compares installed engines. The post also preserves a confidently incorrect model answer about the Moon, acknowledging the limits of the small model. These remain the author's measurements. Who it's for: C and ARM developers exploring local inference who want agent-assisted implementation accompanied by reproducible benchmarks.

Prompt
Suggested prompt (written by Agents In Use, not the author's)

Source: Hacker News · geisten