LLM setups · harness · model · thinking
The best setup
is the one measured
on your hardware.
Public leaderboards rank models on machines you don't own, through agents you don't run. m-bench measures serving config, thinking mode, and harness — then names a winner for this box.
Judging typed decisions instead? System One benchmark v3 — 200 decision questions →
Setups
Which harness + model should I use?
Pick the hardware that matches yours. Each card is a measured run — score, thinking mode, and a link to the raw report.
Harness
Same weights. Four harnesses. 23 points apart.
Half of what you run is not the model — it is the coding agent's system prompt, tool schemas, and how much context it resends.
Your daily setup against a clean one.
A live run keeps your skills, MCP servers and project context switched on; an isolated run strips them. Both measured in the same campaign. Live runs are one sample, so read these gaps as a direction, not a settled margin.
Thinking mode
Both modes. Different products.
Reasoning-trained models collapse without their thinking block. Non-reasoning models burn thousands of tokens with it. The answer flips by workload.
The same switch helps one model and wrecks another.
Ledger
Every LLM-suite run, filterable.
The all, agentic-hard and agentic-all runs. Click a column to sort, click a model to open its report. Green marks the best score per suite in view. Noise floor ≈ 8 points at two samples. System One and phishing runs live on their own page.
Campaign log
What the measurements answered.
Each campaign asks one question and ends in a decision — not a summary.
Suites
Hidden tests. Proven passable.
Every task ships a reference solution. If ./bench validate is not 28/28, no score means anything.
Reproduce it
Measure your own box.
Point at any OpenAI-compatible endpoint. Run both thinking modes. Rank the harness. Apply the winner.
pip install -e . && ./bench validate