Every prospective client asks which model we use. It is a reasonable question, asked in good faith, and it is almost never the question that determines whether their agent works. Earlier this month, a group of researchers ran the controlled version of this argument: they held the task set fixed, varied the model, then held the model fixed and varied the software around it. On a hundred-task subset, the variance introduced by the surrounding software was 7.8 times larger than the variance introduced by changing the model.1 Not larger by a margin. Larger by most of an order of magnitude.
That surrounding software is the harness, and it is the subject of this essay. I wrote about the loop — what happens across turns, how an agent plans, observes, corrects and knows when to stop. This is the other axis. The harness is what the model can see and touch inside a single turn, and it is almost entirely under your control in a way the model's weights never will be.
A narrower definition than the literature's
I should be honest that I am taking a slice of a contested term. The paper I just cited defines the harness broadly — "the software layer between the model and the task that constructs the context the model sees, mediates its tool calls, validates its outputs, and decides when to retry, escalate, or stop" — and decomposes it into seven layers spanning execution, tooling, context, scheduling, observability, verification and governance.1 That is a good definition for a benchmark methodology paper. It is too big to be a discipline you can practise on a Tuesday.
So I am splitting it. Retry, escalate and stop are decisions across turns; that is the loop, and I have written about it separately. What is left — the part I want to name and treat as its own craft — is the surface. Three things, and only three:
What the agent can touch. The tools you expose, their boundaries, their names.
What the agent can see. What those tools hand back, in what shape, at what length.
What the agent is told when it is wrong. The error, which is the only part of your harness that speaks to the model in the middle of a failure.
That is the whole of it. Every harness decision I have found worth arguing about lands in one of those three buckets, and the reason the split is useful is that each one fails differently and each one is fixed differently.
The numbers in that chart are worth sitting with. On SWE-bench Pro, one model — Claude Opus 4.5 — scores 45.9% under one scaffold and 55.4% under another.2 Same weights, same tasks, 9.5 points apart. On Terminal-Bench, infrastructure changes alone moved a fixed model 13.7 points. Independent monitoring of SWE-bench Verified reports 11 to 15 points of scaffold-only variation.2 If you have ever read a leaderboard and concluded something about a model, you were probably reading a fact about somebody's harness.
Surface one: what it can touch
The received wisdom about tools is that you should have few of them, because too many make the agent stupid. You have read this. I have repeated it. The number that travels with it — accuracy collapsing from 43% to 2% as a catalogue grows from four tools to fifty-one — is quoted constantly.
It appears to be wrong, or at least badly misread. A chance-corrected study published in May went looking for that collapse and did not find it. Correcting for the fact that guessing randomly among fifty-one tools is mechanically harder than guessing among four — which is most of what the naive number measures — the researchers found graceful degradation rather than a cliff.3 Agents do get worse as the catalogue grows. They do not fall off a table.
What the same study found instead is more useful, because it points at something you can change. The variable that mattered was not how many tools existed but how many were put in front of the model for a given decision. Selecting the shortlist adaptively — two or three candidates when the task is obvious, more when it is genuinely ambiguous — beat a fixed shortlist of five: 93.1% tool-selection accuracy against 87.1%, with the gap widening to 76.8% against 60.9% on medium-difficulty queries.3 Fewer, better-chosen candidates. Not a smaller catalogue.
That reframes the design problem. You are not budgeting tools, you are budgeting distractors — and a distractor is only a distractor relative to a specific decision. This is why the advice to "just have fewer tools" is unsatisfying in practice: it asks you to permanently amputate capability to solve a problem that is local to one turn.
The complementary move, which Anthropic's engineering team argues from production experience, is to stop wrapping your API and start building workflows. Their examples are concrete and slightly humbling to read if you have ever shipped a tool catalogue that was really just a REST surface with a JSON schema stapled on: list_users, list_events and create_event collapse into a single schedule_event. read_logs becomes search_logs, which returns only matching lines. Three customer-lookup calls become one get_customer_context.4 Each consolidation removes a decision the model would otherwise have to get right, and every decision you remove is a decision that cannot go wrong.
The naming matters more than it should. Namespacing tools by prefix so related capabilities cluster, and refining the descriptions themselves, is reported as one of the highest-leverage interventions available — a claim supported by the fact that careful description work contributed to a state-of-the-art SWE-bench result.4 Tool descriptions are prompt engineering. They are usually written by whoever wrote the endpoint, in the register of internal API documentation, and then never touched again.
[ PLACEHOLDER: a tool you removed or consolidated on a shipped Gigabit agent, and what it fixed. The concrete "we had three tools, we shipped one, the failure class disappeared" story — this is the passage that separates the essay from every generic version of it. ]
Surface two: what it can see
If tools are the verbs, returns are the nouns, and returns are where I see the most waste. The instinct — a reasonable engineering instinct — is that a tool should return everything it knows, because information is good and the consumer can ignore what it does not need. That instinct is inherited from writing functions for deterministic callers, and it is close to backwards here.
The cheap framing is cost: verbose returns burn tokens, tokens are money. True and boring. The framing that actually matters is reliability. Long context degrades models — not in some models, in every model tested. Chroma's evaluation of eighteen frontier systems found performance falling as input length grew across all of them, on tasks as simple as retrieving a single fact, with degradation accelerating when the target was semantically distant from surrounding text and when plausible distractors were present.5 The finding that stuck with me is that a shuffled haystack outperformed a coherent one, which is a deeply strange result and a warning against reasoning about context from intuition.
Put those together and a verbose tool return is not a cost problem with a reliability side-effect. It is a reliability problem that also happens to cost money. Every unnecessary field you hand back is a distractor competing for attention with the thing that mattered, and it stays in the context degrading every subsequent turn — you are not paying once, you are paying for the rest of the session.
The practical moves are unglamorous. Pagination, filtering, range selection, truncation with defaults that are sensible rather than generous. Semantic identifiers rather than UUIDs, so the model can reason about what it is holding instead of pattern-matching hex. And letting the caller choose verbosity: a single response-format flag offering a detailed or concise variant of the same payload cut a representative response from 206 tokens to 72 — about a third of the original — with no loss of the information the agent actually needed.4
The discipline I would state as a rule: a tool should return what the agent needs to decide what to do next, and nothing else. Not what the endpoint knows. Not what a human debugging it would want. What the next decision requires.
[ PLACEHOLDER: an observation format you changed that fixed a whole class of failure — the shape of what a tool handed back, not its logic. If the honest answer is that you have mostly fixed these by truncating, say that; the unglamorous version is more useful than a clever one. ]
Surface three: what it is told when it is wrong
The third surface is the one almost nobody designs, and it is the one I would look at first in an agent that is failing intermittently.
An error message is the only component of your harness that addresses the model in the middle of a failure. Everything else — tool definitions, system prompt, return schemas — is written before anything has gone wrong. The error is the live channel. And in most systems it is a stack trace, or an HTTP status, or a database driver's opinion about a constraint violation: text written for a human engineer reading logs on Monday, handed instead to a non-deterministic system trying to decide what to do in the next four hundred milliseconds.
The contrast Anthropic's team draws is between errors that are opaque and errors that are actionable — text that steers the model toward a correct input format rather than merely reporting that the previous one was rejected.4 It sounds obvious written down. It is nearly universal in the other direction in production code, because error strings are the least-reviewed text in any codebase.
This is also where the harness hands off to the loop, and where the two disciplines have to be designed together. A good error turns a failed turn into a corrected one — the agent reads it, adjusts, retries, succeeds, and the loop never has to escalate. A bad error produces a retry that fails identically, then another, until whatever budget you set runs out and a human gets paged. The same loop policy looks robust or broken depending entirely on the quality of the text you handed the model at the moment of failure. If you are tuning retry limits to stop an agent spinning, there is a good chance the actual defect is upstream, in a message written by someone who never imagined a model would read it.
The discipline
Three surfaces is a taxonomy, not a practice. The practice is smaller and duller than the taxonomy suggests, and it comes down to three habits.
Change one surface at a time. This is the whole reason the split earns its keep. If you widen a tool catalogue, tighten a return format and rewrite your error strings in the same week — which is exactly what a motivated engineer does after reading an essay like this — you will observe a delta and learn nothing about its cause. Harness work is empirical or it is decoration.
Version the harness like you version the model. The benchmark paper's recommendation is that agent results should ship with a "harness card" — a structured disclosure of the surrounding configuration — and that leaderboard comparisons without one should be treated as incomplete and potentially misleading.6 That is aimed at researchers, but the operator's translation is direct: if you cannot say which harness version produced a result, you cannot attribute a regression, and you will spend a Thursday blaming a model release for a change one of your own engineers made on Tuesday.
Rule out the harness before you blame the model. This is the habit the 7.8× figure earns. When an agent underperforms, the available explanations are ranked by ego rather than by likelihood: the model is not smart enough, the task is too hard, the user prompted it badly. The evidence says the surrounding software is the larger source of variance by a wide margin, and the surrounding software is also the only one of those you can change today. "Try the bigger model" is not a diagnosis. It is what you do after the diagnosis fails.
[ PLACEHOLDER: whether Gigabit actually versions harness configuration, and how. If the honest answer is that this is a practice you are arguing for rather than one you have fully implemented, say exactly that — the argument survives it, and the credibility is worth more than the claim. ]
What this does not cover
Two deliberate omissions, both of which get their own treatment.
The first is the loop — everything temporal, which I have written about separately and which is the other half of any working agent. The second is adversarial: everything above assumes the content flowing across these three surfaces is merely unhelpful rather than hostile. The moment a tool return can contain text written by someone who wants your agent to misbehave, the return surface stops being a formatting question and becomes a security boundary. That is a different essay, and I have written it: prompt injection.
What I would take from this one is narrow and, I think, actionable. The model is a component you rent and cannot modify. The harness is a component you own and can change this afternoon. The measured evidence says the second explains more of your outcomes than the first, and the industry spends most of its attention on the first because that is where the announcements are.
The honest caveat: the numbers in this essay come from benchmark research, and benchmarks are not production. A 9.5-point spread on SWE-bench Pro tells you harness choice is consequential; it does not tell you that your harness has 9.5 points in it, and I would be suspicious of anyone — including me — who implied otherwise. What I can say is that the direction has matched my own experience consistently enough that I now treat "which model" as the last question in a debugging session rather than the first. I am also aware that "harness engineering" is a term the literature is still settling, that my three-surface split is a working simplification rather than a standard, and that a better one may well replace it. I will update this here if it does.