How to test local context memory with Hermes Agent
John_Miracleworker ran two local Hermes context engines through the same 340-turn workload. The reported comparison found 41 of 44 delayed-recall probes correct versus 39 of 44 for stock, but the alternative took roughly 3.3 times the wall time.
reddit.com
Suggested prompt
Suggested prompt written by Agents In Use, not the source author. Replace bracketed inputs and review tool permissions first.
Run [local context engine] and [baseline] through the same deterministic [turn count]-turn task. Plant facts and test delayed recall at fixed intervals. Record correct answers, context size, prefill cost, wall time and timeouts. Keep the model and hardware identical, and report limitations alongside any improvement.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source author
- u/John_Miracleworker
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- The author reports 41/44 delayed-recall answers correct versus 39/44 for stock, and a 91.8k versus 112.6k maximum wire context. The alternative cost about 2.8x prefill and 3.3x wall time. Both 340-turn runs completed. This was one self-authored benchmark run per engine on one machine.
- Human involvement
- The author built the engine and workload, configured the machine, analysed results and implemented the recovery fix. The source does not claim that Hermes independently designed the benchmark or diagnosed and patched itself.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- The author configured local Hermes with Qwen3.8-27B Q4 and two context engines.
- They generated a 340-turn deterministic workload with planted facts and 44 delayed-recall probes.
- They ran each context engine to completion on the same model lane and compared recall, context footprint and timing.
Reddit author John_Miracleworker tested a local Hermes Agent setup with two context engines. Their alternative, CLM + Context Map, keeps a small active working set and an explicit memory map that can restore material verbatim. They compared it with the stock hermes-lcm compressor on one machine using a local Qwen3.8-27B Q4 model.
The author generated a deterministic 340-turn workload containing planted facts, noise, large text dumps and 44 recall probes. Each probe appeared 34 turns after its fact was introduced. Both engines completed the same workload on the same model lane. The alternative answered 41 probes correctly, compared with 39 for stock, and reached a reported maximum wire context of 91.8k tokens rather than 112.6k.
That improvement carried a cost. The author reports roughly 2.8 times the prefill time and 3.3 times the wall time. They also traced expensive recovery episodes to repeated full-body prefills. Keeping the rendered recovery presentation stable limited each episode to at most one bounded prefill, validated over the final 178 turns.
These are the author's benchmark results, not an independently reproduced comparison. There was one run per engine, a self-authored workload and two outstanding upstream patches for full tool delivery on stock Hermes. Who it's for: local-agent builders testing memory changes who need to measure recall, context size and runtime together rather than report only the winning metric.
Prompt
Suggested prompt (written by Agents In Use, not the author's)
Source: Reddit · u/John_Miracleworker
Frequently asked questions
Did the alternative context engine run faster?
No. The author reports about 2.8 times the prefill time and 3.3 times the wall time despite the smaller context footprint and higher recall result.
How strong is the comparison evidence?
Both engines completed the same deterministic 340-turn workload, but there was only one run per engine on one machine. The author wrote the workload and benchmark, and these results have not been independently reproduced here.
More details
Remaining steps
- They traced repeated full-body recovery prefills and stabilised the rendered recovery presentation.
- They checked the recovery change across the final 178 turns and reported remaining tool-delivery patch requirements.
About this account
Hermes ran the local multi-turn workload with each context engine. The alternative engine looked up and restored mapped context; the author reports that automatic guards absorbed two unattended model runaways and allowed completion.
Firsthand benchmark account by u/John_Miracleworker: https://www.reddit.com/r/hermesagent/comments/1x6u1i3/340_turns_two_context_engines_one_local_27b_clm/. Public RSS evidence recovered on 16 October 2026 passed the repaired body eligibility policy at 0.92. Measurements are the author's, not independently reproduced. Suggested prompt by Agents In Use, not the author's original prompt. Original rejection remains in the evidence ledger.
Credit: u/John_Miracleworker on Reddit
Source published
Added to Agents In Use
Source checked
Tools and topics
Hermes Agent · Qwen3.8-27B Q4 · CLM + Context Map · hermes-lcm · deterministic benchmark
local agents · context memory · benchmarking · delayed recall
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.