How to test thinking rules with mini-swe-agent
A user compared the same model and mini-swe-agent setup with and without nine reasoning instructions on Terminal-Bench 2.0. They report a higher task score with the instructions, with explicit single-run limitations.
reddit.comOriginal prompt
Shared by the source author. See source notes for context.
Finish one approach before switching. Pick the most promising approach and carry it to a conclusion.
Reach people exploring AI agents
See placements →Evidence at a glance
Sourced account- Source poster / publisher
- u/PilgrimofHaqq2
- Original account
- Read the source ↗
- Published
- Source link checked
- Reported result
- The author reports 59.6% with the rules and 52.8% without them across 89 Terminal-Bench 2.0 tasks. Three tasks could not start and counted as failures. These are single-attempt, author-reported scores, not independently reproduced leaderboard results.
- Human involvement
- The author chose the model, agent, thinking level and test conditions and reported failures. The experiment used one attempt per task and does not establish a general improvement across agents or models.
A linked account supports attribution. It does not mean we independently reproduced its result.
Report an issue with this case
Open correction emailOpens your email app with the case link included. Nothing is sent automatically.
How the agent did it
- The author used an agent-assisted research and synthesis process to develop nine thinking rules.
- They ran GLM-5.3-Flash in mini-swe-agent on the same Terminal-Bench 2.0 tasks with and without the rules.
- They used High thinking because Max thinking had exceeded task time limits in their setup.
A LocalLLaMA poster tested whether nine thinking instructions helped an agent on longer tasks rather than only short questions. They kept the model, GLM-5.3-Flash, and agent, mini-swe-agent, the same and ran Terminal-Bench 2.0 tasks with and without the rules.
The instructions address behaviour such as finishing an approach, reopening conclusions only for concrete evidence, and checking against outside facts rather than repeatedly reconsidering an answer. The quoted prompt here is a short exact excerpt, not the complete nine-rule set. An author comment says an agent had helped synthesise and test the rules after research into agent efficiency.
The reported scores are 59.6% with the rules and 52.8% without them. The denominator includes all 89 tasks, including three that would not start on the author's machine and counted as failures. The author used High thinking because Max thinking took too long in their setup.
These are single-attempt results. The source explicitly warns that leaderboard scores average multiple attempts, so its comparison with other published systems is only approximate. No independent rerun or general cross-model improvement is established.
Who it's for: people evaluating instruction changes in a terminal agent who want a concrete paired test design and a candid account of startup failures, rather than treating one reported score as a universal recommendation.
Prompt
Original prompt
Source: Reddit · u/PilgrimofHaqq2
Frequently asked questions
Which model and agent were compared?
The author reports using GLM-5.3-Flash with mini-swe-agent in both conditions, with the nine thinking rules present in one condition and absent in the other.
Can the scores be treated as official leaderboard results?
No. The author reports single-attempt scores, includes three startup failures among 89 tasks, and warns that leaderboard averages over multiple attempts are not directly comparable.
More details
Remaining steps
- They counted all 89 tasks, including three startup failures, and reported the two single-attempt scores with a leaderboard-comparability warning.
About this account
mini-swe-agent attempted long-horizon terminal benchmark tasks with GLM-5.3-Flash under two instruction conditions. The post reports task scores rather than detailed per-task execution logs.
Firsthand account by u/PilgrimofHaqq2: https://www.reddit.com/r/LocalLLaMA/comments/1x8q308/followup_tested_my_9_thinking_rules_on/. Full public RSS post and returned comments captured on 19 October 2026. Body eligibility passed at 0.8. Results are author-reported and not independently reproduced. Original prompt: Exact contiguous excerpt of rule 2, not the entire nine-rule instruction set.
Credit: u/PilgrimofHaqq2 on Reddit
Source published
Added to Agents In Use
Source checked
Tools and topics
mini-swe-agent · GLM-5.3-Flash · Terminal-Bench 2.0
agent evaluation · reasoning instructions · terminal tasks
Editorial completeness: 100/100. Measures documented fields, not agent performance or verified outcomes.