Your AI Agent Doesn't Need a Better Model

On August 21, Nvidia published a result that should change how most businesses think about their AI agent budget. Researchers took Claude Opus 5 and ran it against ARC-AGI-3 — a benchmark of 2D games with no instructions, where the model has to work out how to play and win purely by interacting with the environment. Run raw, with no scaffolding around it, Opus 5 scored 30%. That was already the best score of any model tested. Then the researchers wrapped the exact same model in a custom "harness" — a layer that handled memory better and added a supervisor component to check and redirect the agent's reasoning — and reran the test. The score went to 100%.

Same model. Same weights. Same training. The only thing that changed was the software wrapped around it, and the result more than tripled. For anyone who has been told that the fix for an underperforming AI agent is to swap in a bigger, newer, more expensive model, this is a fairly direct rebuttal — and it happens to confirm something Wizeb has been telling clients for over a year: the model is rarely the bottleneck. The harness is.

What a "Harness" Actually Is

A harness is everything that sits between a raw language model and a working agent: how memory is managed across a long task, how tools and their outputs get fed back to the model, how errors get caught and corrected instead of silently compounding, what a supervisor or checker component is allowed to override, and how the whole system knows when a task is actually finished versus just looks finished. None of that is the model. All of it determines whether the model succeeds.

• Memory management — how the agent tracks what it has already tried, what worked, and what state the task is in, especially over long or multi-step work where context would otherwise get lost or overwritten
• Supervision — a checking layer that catches the agent going down a wrong path early, rather than letting a bad decision three steps in compound into a completely wrong outcome ten steps later
• Tool and feedback loops — how cleanly the results of an action get back to the model in a form it can actually reason over, instead of raw, noisy output it has to re-interpret every time
• Task-completion criteria — an explicit definition of "done" that the system checks against, rather than trusting the model's own claim that it finished

None of these are exotic. They're engineering discipline — the same kind of discipline that separates a fragile script from production software. The difference is that with AI agents, that discipline determines not just reliability but raw capability. Nvidia's result shows a 3.3x jump in task success from harness quality alone, on a benchmark specifically designed to be resistant to memorization and pattern-matching.

Why Businesses Keep Reaching for the Wrong Fix

When an AI agent underperforms in production — it misses steps, loses track of a multi-part task, or confidently reports success on something it didn't actually finish — the instinctive response is to blame the model. Swap GPT for Claude, or the reverse. Upgrade to the newest release. Pay for the bigger context window. These are the levers that are visible from the outside, and vendors are happy to sell an upgrade as the fix.

The harness, by contrast, is invisible unless someone built it deliberately. Most off-the-shelf agent tools and quick internal builds ship with a thin, generic harness — enough to demo well, not enough to hold up over a long or ambiguous task. When that thin harness is the actual constraint, no model swap fixes it, because the model was never the limiting factor. Nvidia's benchmark makes this concrete: the same model went from the best-in-class score at 30% to a perfect score at 100% with zero changes to the model itself.

The Center of Gravity Has Shifted

For the last two years, the dominant AI narrative has been about model capability — which lab shipped the smartest model this quarter, whose benchmark scores are highest, which context window is longest. That race still matters, but it's no longer where most of the achievable performance gain sits for a business deploying agents on real work. Long-horizon, multi-step, real-world tasks — the kind businesses actually need agents for, not benchmark trivia — are exactly where harness quality dominates model choice, because they're where memory, error-correction, and task tracking get stress-tested hardest.

A Realistic Scenario

A Wizeb client, a mid-size logistics brokerage, had built an internal agent to handle carrier rate negotiations — a multi-step task involving pulling historical rates, checking current capacity, drafting a counter-offer, and tracking the back-and-forth across several email exchanges. It ran on a strong, current model and still failed constantly: it would lose track of what had already been offered, occasionally repeat a rejected rate, and report a negotiation as "complete" when a carrier had actually gone quiet mid-thread. The team's first instinct, understandably, was to try a different model. It made almost no difference. The problem wasn't reasoning quality — it was that nothing in the system tracked negotiation state across exchanges, flagged a stalled thread as unresolved rather than done, or caught an offer repeating itself before it went out. We rebuilt the harness around the same underlying model: explicit state tracking per negotiation thread, a supervisor check before any offer went out that compared it against everything already sent, and a completion check that required an explicit carrier confirmation rather than just silence. Failure rate on the task dropped by roughly 80%, with no model change at all.

How to Check Whether This Is Your Problem

1. 1Look at your last five agent failures and ask whether the agent reasoned incorrectly, or whether it lost track of state, missed feedback it should have caught, or falsely reported completion — those are harness failures, not model failures
2. 2Check whether anything in your system verifies a task is actually done, or whether "done" just means the agent stopped generating output
3. 3Check whether errors get caught and corrected mid-task, or whether they silently compound until the final output is wrong in a way nobody flagged early
4. 4Before paying for a model upgrade to fix a reliability problem, run the same failing task with a stronger completion check and better memory handling around the current model — if that closes most of the gap, the model was never the constraint
5. 5If you're evaluating vendors or off-the-shelf agent tools, ask specifically what their harness does for memory, supervision, and completion checking — not just which model they call

How Wizeb Approaches This

When we build an AI agent for a client, model selection is one of the smaller decisions in the project — often the last one we lock in, not the first. Most of the engineering effort goes into the harness: how the agent tracks state across a long task, what checks a supervisor layer runs before an action goes out, how failures get caught and corrected instead of silently compounding, and what "done" actually means for that specific task. That's the difference between an agent that demos well and one that holds up on real, messy, multi-step work in production. If an agent you're running — or considering — keeps underperforming and the instinct has been to blame the model, it's worth having someone look at the harness first. Start at wizeb.com/services/ai-agents.

Three Questions Before Your Next Model Upgrade

1. 1When your agent fails, is it actually a reasoning failure — or a memory, verification, or completion-tracking failure wearing a reasoning failure's clothes?
2. 2Does anything in your system independently verify a task is finished, or does the agent get to grade its own homework?
3. 3If Nvidia can take the same model from 30% to 100% with harness changes alone, how confident are you that your last model upgrade was the fix — versus a more expensive way of not fixing the actual problem?