LangChain took a coding agent into the top five on Terminal-Bench 2.0 without touching the model. The base model stayed gpt-5.2-codex the entire time; the score climbed from 52.8% to 66.5% purely by rebuilding the scaffolding around it. That result names the lever most teams still underweight. The harness, the system layer that feeds a model its context and turns its output into action, now moves agent performance about as much as a model upgrade does. If you evaluate or buy agents by the model on the label, you are measuring the wrong variable.
Key takeaways
- LangChain reported a jump from 52.8% to 66.5% on Terminal-Bench 2.0 with the model (gpt-5.2-codex) held fixed, climbing into the top five from outside the top thirty.
- Harness-Bench measured a 23.8-point spread between the best and worst harness across the same eight models: 76.2% for NanoBot against 52.4% for OpenClaw.
- Weaker models swing more across harnesses while stronger models stay stable, so harness quality matters most exactly where the model budget is tight.
- A reasoning schedule of xhigh, high, xhigh scored 66.5% against 53.9% for constant maximum reasoning, because uniform high effort blew the task time budget.
- Independent work warns the gains can overfit: one study found evolved harnesses added 5-plus points on training tasks but only 0.6 on held-out tasks.
- Researchers now argue agent capability should be reported at the model-harness configuration level, not attributed to the base model alone.
What Is an Agent Harness, and What Does It Actually Control?
An agent harness is the system layer that conditions each model call and turns the model's output into actions inside a real workspace, per the definition in Harness-Bench. The model proposes; the harness decides what the model sees, what it is allowed to do, and what happens when it stalls.
Concretely, the harness owns the prompt templates, the tool definitions and how tools get invoked, context assembly and eviction, workspace and file access, permissions, budget and turn caps, and recovery from failure. A bare shell tool paired with a short prompt is one harness. Middleware that injects the directory tree, forces a verification pass, and interrupts repetitive edits is a very different one, running the same model underneath.
The practical consequence: two teams can run identical weights and ship agents with different completion rates, costs, and audit trails. The model is a single component inside a larger system, and the rest of that system is now where a lot of the engineering lives.
How Much Can the Harness Alone Move Agent Performance?
Enough to reorder a leaderboard, and the range is now measured rather than asserted. LangChain's climb from 52.8% to 66.5% happened with the model frozen. Harness-Bench put a number on the general effect: across 5,194 execution trajectories spanning 106 realistic tasks, six configurable harnesses, and eight model backends, the best harness averaged 76.2% and the worst 52.4%, a 23.8-point gap. Completion rates ran from 60.0% to 81.6% depending on the harness.
Trajectory length did not explain it. The top-scoring harness reached its result while spending fewer tokens than several harnesses that scored lower, so this is not a story about letting the agent grind for longer. It is about how well the scaffolding organizes context, tools, and recovery.
| Study / benchmark | Model held fixed | Harness or config change | Result |
|---|---|---|---|
| LangChain deepagents-cli, Terminal-Bench 2.0 | gpt-5.2-codex | Baseline harness vs verification, context, and loop-detection middleware | 52.8% to 66.5% |
| LangChain reasoning schedule, Terminal-Bench 2.0 | gpt-5.2-codex | Constant xhigh reasoning vs an xhigh, high, xhigh schedule | 53.9% to 66.5% |
| Harness-Bench, 106 tasks | Eight models, averaged | Best harness (NanoBot) vs worst (OpenClaw) | 76.2% vs 52.4% |
Which Harness Techniques Produced LangChain's Gains?
Four changes, all outside the model, carried most of the improvement. None of them required new weights, and each targets a specific failure LangChain saw in the traces.
Forced self-verification. Their most common failure pattern was an agent that wrote a solution, re-read its own code, decided it looked fine, and stopped. A PreCompletionChecklistMiddleware turns verification into a deterministic gate the agent has to pass, not a polite suggestion in the system prompt.
Local context injection. A LocalContextMiddleware injects the directory structure and available tooling into context, so the agent stops spending turns rediscovering its own environment before it can do any work.
Loop detection. A LoopDetectionMiddleware counts edits to the same file and, after a threshold, interrupts with a prompt to reconsider the approach. That kills the doom loop where an agent rewrites the same file into a corner.
A reasoning schedule, not a fixed dial. The xhigh, high, xhigh sandwich scored 66.5%, while pinning the model at constant xhigh scored 53.9%. Uniform maximum effort ran the task past its time limit. The lesson a senior engineer takes from this: reasoning effort is a budget to spend across a task, not a slider to leave at maximum. I covered that trade-off in more depth in tuning reasoning effort.
Why Report Capability at the Model-Harness Level, Not the Model?
Because the same model posts very different numbers depending on the harness, so a bare model name does not predict what you will get in production. Harness-Bench states it plainly: agent capability should be reported at the model-harness configuration level rather than attributed to the base model alone.
The variance is not uniform across models. Stronger models showed lower cross-harness variance, staying relatively stable no matter the scaffolding, while weaker models swung much more. Harness quality therefore pays off most where you are trying to run a cheaper or smaller model, which is exactly where teams reach for it to control cost.
For anyone buying or scoping agents, that reframes the demo. A vendor showing a polished result on a frontier model can be masking a thin harness, and a cheaper model wrapped in a strong harness may beat it on your tasks and your budget. This is the same logic that makes small-model routing viable; a good harness is often what lets you drop down a tier. See when to use small language models and, for how to measure any of this honestly, measuring agent reliability beyond pass@1.
When Does Harness Engineering Stop Paying Off?
When the gains are memorized fixes for one benchmark rather than transferable design. A 2026 study on evaluating harness evolution found evolved harnesses gained 5-plus points on the tasks they were tuned against but only 0.6 points on held-out tasks, evidence of severe overfitting. The authors describe harness edits that "memorize fixes rather than distill strategies," encoding a particular failure as a prompt rule that does not generalize.
Two further caveats sharpen the point. When search and evaluation share a benchmark, automated harness evolution did not consistently beat plain test-time scaling, such as parallel sampling and sequential refinement, at equal compute, so some reported wins are really extra search wearing a harness costume. And Terminal-Bench itself may be insensitive to harness design on many tasks, because a shell tool plus a basic prompt already solves most of the solvable ones, leaving model capability as the real ceiling.
This does not contradict LangChain's result. Hand-built middleware aimed at failure modes an engineer actually observed is a different activity from an automated loop searching for prompt edits that flatter a leaderboard. The distinction is the thing to watch: measure whether a harness change transfers to tasks it was never tuned on before you trust the number.
How Should You Build and Evaluate a Harness in Production?
Treat the harness as a versioned component with its own evaluation, decoupled from the model. That single discipline resolves most of the confusion above and survives the next model swap.
- Evaluate on held-out tasks that mirror your real workload, and record the model and harness together as the unit under test. A score without a harness attached is not reproducible.
- Add deterministic gates for the failure you actually see in traces, verification passes and checklists, rather than generic encouragement buried in a system prompt.
- Instrument loops and cap turns and tokens per branch, so a stalled run bails before it drains the budget instead of after.
- Force structured returns from any subagent, so an orchestrator aggregates typed results instead of parsing prose. That constraint is what keeps supervisor-style multi-agent systems auditable.
- Schedule reasoning effort against the task's time budget instead of pinning it at maximum, as the sandwich result showed.
- Re-run the harness evaluation on every model change, because the two interact and a harness tuned for one model can regress on another.
The stakes here rise with time horizon. METR's measurements show the length of tasks agents can complete roughly doubling every seven months, which means longer autonomous runs, more context to manage, more loops to catch, and more recovery to engineer. Model progress keeps raising the ceiling. The harness decides how much of that ceiling your system actually reaches, and it is the part you control. For the context-management half of that work, see context engineering for long-horizon agents.