When an AI agent fails at a task, most people’s first reaction is: the model isn’t strong enough. Switching to a larger model, waiting for the next generation, or allocating more inference budget—all are intuitive fixes. Yet over the past two years, a growing body of research and engineering practice has repeatedly pointed to another, often overlooked variable: keep the model entirely unchanged, and only modify the structure surrounding it—the ‘running structure’—and task performance can differ dramatically.
This surrounding structure has a name: Agent Harness. It broadly handles three things: how the model and tools are invoked, how the environment is managed, and how a task persists across multiple steps. Different teams use the term with slightly varying scopes, but they share a key point: it resides outside the model itself, yet determines how much of the model’s capability actually materializes.
## Everything Outside the Loop Is Harness
In a December 2024 article, Anthropic offered a plain definition of an agent: an agent is, essentially, “an LLM inside a loop, using tools based on environmental feedback.” Reading that sentence backward clarifies the boundary of harness—the loop interior is the model; everything outside the loop is harness: what interface the model receives, which tools are provided, how the workflow is organized, and how state is handed off across sessions.
The same article distinguishes workflow (orchestrating LLMs and tools via predefined code paths) from agent (where the LLM dynamically directs its own process and tool usage), and repeatedly advises: start with the simplest viable solution, and add complexity only when necessary—some scenarios shouldn’t involve agentic systems at all. Frameworks help you get started quickly, but as you move toward production, don’t hesitate to strip away abstraction layers.
A common early misconception about harness is treating it as mere “glue code”—a packaging exercise done at the end. Four layers of evidence below show it is instead a first-class engineering variable—one that can be measured and improved: the first two layers directly shift numbers on public benchmarks; the latter two alter failure modes and handoff mechanisms.
## Interface Layer: The Model Is a New Kind of User
The May 2024 SWE-agent paper (arXiv:2405.15793) advanced an initially uncommon stance: language model agents constitute “a new class of end user, with their own needs and capabilities,” and software interfaces should be designed specifically for them—the authors call this layer the agent-computer interface (ACI). This paper itself is an empirical study in interface design: adopting a purpose-built ACI significantly enhanced the agent’s ability to create and modify code files, navigate entire repositories, and run tests and programs—achieving pass@1 scores of 12.5% on SWE-bench and 87.7% on HumanEvalFix, both state-of-the-art results at the time, far surpassing prior non-interactive LM baselines.
What matters is not the scores themselves, but the question the paper asks: How does interface design affect agent performance? Only the interface layer—the model’s point of contact with software—was changed.
## Process Layer: One Greedy Step Isn’t Enough—Add Search
The July 2024 paper “Tree Search for Language Model Agents” (arXiv:2407.01476) tackles another weakness: LMs are primarily optimized for text understanding and generation—not for multi-step reasoning, planning, or leveraging environmental feedback to perform real computer tasks. The authors’ approach adds best-first tree search *during* inference—not simulating in a sandbox, but exploring multiple paths in the actual environment’s state space, evaluating outcomes, and backtracking.
The result is direct: on VisualWebArena, adding search to a GPT-4o agent increased its success rate by 39.7% relative to the identical agent without search—reaching 26.4%, the best result at the time; on WebArena, the relative gain was 28.0%, reaching 19.2%. Performance also scaled further with increased test-time compute. Not a single model weight was altered; the tools remained identical; only an extra layer of process structure was added—and performance shifted.
## Tool Layer: More Tools ≠ Better Performance
Anthropic’s September 2025 article on tool design acknowledges a new reality: with protocols like MCP, agents can now access hundreds of tools—so the problem shifts to maximizing tool effectiveness. Their answer: more tools aren’t always better. The most common mistake is wrapping existing, human-designed API endpoints directly for agent use—agents have different affordances (i.e., what operations feel natural to them) than human users, and their context windows are limited.
The article gives concrete examples: for a contacts app, instead of offering list_contacts (forcing the agent to paginate manually), provide search_contacts and message_contact directly; for a calendar tool, merge the three discrete actions—“find free time,” “schedule,” and “create event”—into a single schedule_event call. Another example involved text correction: after deploying a tool, they observed the model consistently appending "2025" to search queries, biasing results and lowering performance—no code change was needed; rewriting the tool description alone corrected the behavior.
## Temporal Layer: Handoffs for Amnesiac Shift Workers
The hardest harness problems arise along the time dimension. Anthropic’s November 2025 long-task article offers a painful analogy: long-running agents operate in discrete sessions, each new session having zero memory of prior events—like a software project staffed by shift workers, where every incoming engineer suffers amnesia. Their general-purpose harness (Claude Agent SDK) includes context management features like compaction, but the article bluntly states: compaction doesn’t always transmit fully clear instructions to the next agent; even cutting-edge models, given only a high-level prompt like “build a claude.ai clone,” fail to produce production-grade web apps when executed across successive windows.
They observed two failure patterns: first, attempting too much at once—trying to “go all-in” in one session—causing context exhaustion mid-implementation, leaving half-finished artifacts with no documentation, forcing the next session to guess and waste time recovering basic functionality; second, late in the project, the agent surveys existing progress and declares the task complete—even though critical pieces remain unfinished.
The fix isn’t a stronger model—it’s structure. The first session is assigned to a dedicated initializer agent that lays groundwork: an init.sh startup script, a claude-progress.txt progress log, an initial git commit, and—crucially—a feature list decomposing the requirement (e.g., “a claude.ai clone” becomes >200 individual features, all initially marked failing). Thereafter, each coding session advances exactly one feature, ending only when the code is in a clean, merge-ready state for the main branch. The list uses JSON—not Markdown—because models struggle less with unintended modifications to JSON files; agents are permitted to edit only the passes field, and the prompt includes a firm warning: “Deleting or modifying tests is unacceptable.”
Testing also gets specialized treatment: agents often mark tasks complete after running unit tests or curl commands—even when the full end-to-end flow is broken. The fix is to equip them with browser automation tools (e.g., Puppeteer) and require them to “act like a human user” to complete *all* tests—including screenshot verification. The article’s key insight is simple: enable rapid understanding of current work status in a new window—via progress files plus git history. This approach draws inspiration from observing how high-performing software engineers work daily.
## Three Judgments
The following three judgments emerge from reading five sources side-by-side—not conclusions drawn from any single original piece.
First, model and harness stand in a multiplicative relationship, not additive (my judgment). An agent’s task capability approximates model × harness: the model sets the ceiling; the harness determines the realization rate. The interface and process layers yield directly measurable leverage; the tool and handoff layers each address empirically verified failure modes; and multiplication implies that if any factor approaches zero, overall performance collapses—no matter how powerful the model, poor runtime structure wastes it. For engineering teams, this is both bad and good news: bad, because failures can no longer be blamed solely on the model; good, because harness lies squarely within your control—changes manifest immediately in metrics, no need to wait for the next model release.
Second, the correct posture for harness engineering is “simplest-first” (standard interpretation). Experimental evidence and methodological advice converge: Anthropic repeatedly stresses, in its agent design article, starting with the simplest viable solution, adding complexity only when measurable gains justify it, and shedding abstraction layers en route to production. Good harness isn’t about maximal structural complexity—it’s about each structural element mapping to a validated failure mode: the feature list counters “premature completion”; JSON counters “unintended modification”; end-to-end testing counters “false completion”; progress files counter “shift-worker amnesia.” One structure, one hole—no more, no less.
Third, harness is becoming an independent engineering discipline (industry observation). Consider the timeline: December 2024 on agent design patterns; September 2025 on writing tools for agents; November 2025 on long-task harness; March 2026 on a follow-up to harness design; and in public knowledge directories, “Agent Harness” now appears as a standalone entry. When something acquires a dedicated name, dedicated literature, and a dedicated taxonomy, it signals recognition as a first-class object—just as “database” and “compiler” did in their time.
## Takeaway Checklist
Next time your agent falters, resist the urge to swap models—first, inspect all four layers: Is the interface designed *for the model*, or merely adapted from a human UI? Beyond single-step greedy execution, does the process include search, evaluation, and backtracking? Are tools designed around the agent’s operational habits—or just wrappers around existing APIs? For long tasks, what enables handoff between windows—progress files, git history, or pure guesswork?
These four questions apply equally to human organizations—the “shift-worker amnesia” analogy originated from engineering team management experience. If your team’s newcomers rely on oral tradition to get up to speed, the issue likely isn’t people—it’s harness.
Which part of your agent’s failures stemmed from the model—and which from the harness? Share your experiences in the comments.
Source: Anthropic, “Effective harnesses for long-running agents” https://www.anthropic.com/research/effective-harnesses-for-long-running-agents