buy the car, not the engine every week someone posts “Claude destroyed GPT” or “Gemini is catching up.” Grok 4.20 just launched with four agents arguing with each other. DeepSeek V4 is imminent. it’s sports for nerds, and a distraction from the question that matters: what gets me a smart model with the tools it needs to do real work. engines are not cars think of AI as a car. the model (GPT, Claude, Gemini, Grok, DeepSeek) is the engine. the harness is the rest of the car — steering, brakes, fuel system, navigation, trunk. Latent Patterns defines an agent harness as “the orchestration layer that constructs context, executes tool calls, enforces guardrails, and decides when each loop iteration should continue or stop.” if the model is the reasoning engine, the harness is the operating system that makes the engine useful, safe, and repeatable. they break it into five concerns: instruction layering, action mediation, loop control, policy enforcement, and memory strategy. in practice, most reliability problems blamed on “the model” are harness design problems. Latent Patterns five concerns same engine, completely different car the Lotus Evora and the Toyota Camry share the same 3.5L V6. Toyota tunes it to 301hp for commuting. Lotus supercharges it to 400hp in a mid-engine track weapon. same engine. one hauls groceries, the other races. what changed? everything around the engine. this is happening in AI right now and it’s not subtle. Lotus Evora and the Toyota Camry share the same 3.5L V6 Gemini 3 Pro powers both Google Sheets and NotebookLM. in Sheets, it hits a 350-cell ceiling, can’t see your full spreadsheet, and has no undo. in NotebookLM, the same model uploads your entire document library, cites every claim back to its source, and generates audio overviews. one’s a formula helper in a cage. the other’s a research analyst. Gemini 3 Pro hits a 350-cell ceiling, can’t see your full spreadsheet, and has no undo cites every claim back to its source GPT-5 powers both Copilot in Excel and ChatGPT. enterprise users report Copilot fails simple column sums and feels “night and day” slower than ChatGPT — despite using the same underlying model. ChatGPT gets file uploads, web search, custom GPTs, memory, and a model picker. one’s in a straitjacket. the other’s a full workbench. GPT-5 report Copilot fails simple column sums and feels “night and day” slower than ChatGPT Claude Sonnet 4 powers both GitHub Copilot and Claude Code. in Copilot it gets ~128K context (vs 1M native), a hidden system prompt, and no thinking control. in Claude Code it gets repo-wide reasoning, explicit thinking budgets, full MCP tool use, and your own custom instructions. one’s on a leash. the other’s unleashed. Claude Sonnet 4 ~128K context (vs 1M native), a hidden system prompt, and no thinking control or as Latent Patterns puts it: “two tools can use the same model and produce dramatically different outcomes because their harnesses differ in context assembly, policy checks, and loop control semantics.” Evangelos Pappas tested this empirically: frontier models scored 24% pass@1 on real professional tasks in the APEX-Agents benchmark. the failures were overwhelmingly orchestration problems, not knowledge gaps. the engine knew the answer. the car couldn’t get there. Evangelos Pappas tested this empirically even OpenAI agrees. their “harness engineering” write-up describes building a million-line codebase with zero manually-written code. the bottleneck was never the model. it was the environment. “early progress was slower than we expected, not because Codex was incapable, but because the environment was underspecified.” when something failed, the fix was almost never “try harder.” it was: what tool, guardrail, or context is missing from the harness? “harness engineering” write-up the convergence problem every engine got dramatically more powerful. but they all got powerful at the same time. take GPQA Diamond — 198 PhD-level science questions where human experts score about 65%. in November 2023, GPT-4 scored 39% — barely above a coin flip. one engine, mediocre. GPQA Diamond 65% GPT-4 scored 39% by mid-2024, Claude 3 Opus hit ~56%, GPT-4o managed ~51%, Gemini 1.5 Pro was in the mix. four engines, all below human experts, 30+ point spread. today? Gemini 3 Pro scores 91.9%, GPT-5.2 hits 92.4%, Claude Opus 4.5 reaches 87%. six engines, all above human experts, clustered within five points. the engines went from 39% to 92%. incredible. but the gap collapsed. Gemini 3 Pro scores 91.9% GPT-5.2 hits 92.4% 87% above the small engines? GPT-5 mini, Haiku 4.5, Gemini 3 Flash, Phi-4, Mistral 7B — beat where frontier models were 18 months ago. run on your phone, cost pennies. Gartner predicts 3x more small task-specific models than general-purpose LLMs by 2027. beat where frontier models were 18 months ago Gartner predicts six companies make great V8s. a dozen more make great four-cylinders. the engine is a solved problem. what this means for you if you’re picking or building a car, you make different decisions depending on what you need. do you want a workhorse? a beater? do you plan to drive on rugged terrain? freeways all the way? the same holds true when you pick or build “AI products”. the harness is where your taste and decision making live. every decision is a trade-off, and the right trade-off depends entirely on what you’re trying to do. depth vs speed: do you let the model think for 30 seconds and return a thorough answer, or force a 2-second response that’s 80% as good? a legal research tool and a customer service bot need opposite answers to this question. same engine, opposite harness.
context vs cost: do you stuff the full conversation history into every call, or summarize aggressively and risk losing nuance? a therapy app and a code assistant make different bets here.
autonomy vs control: does the AI act on its own or wait for approval? a scheduling agent should book the meeting. a financial advisor should not execute the trade. depth vs speed: do you let the model think for 30 seconds and return a thorough answer, or force a 2-second response that’s 80% as good? a legal research tool and a customer service bot need opposite answers to this question. same engine, opposite harness. depth vs speed: context vs cost: do you stuff the full conversation history into every call, or summarize aggressively and risk losing nuance? a therapy app and a code assistant make different bets here. context vs cost: autonomy vs control: does the AI act on its own or wait for approval? a scheduling agent should book the meeting. a financial advisor should not execute the trade. autonomy vs control: these are the same trade-offs car designers make. speed vs comfort. luxury vs mainstream. track suspension vs grocery-run ride quality. nobody asks “which engine does a Cayenne use?” because the engine isn’t the only thing that makes it a Cayenne. it’s every decision made around the engine to serve a specific driver. make decisions that are engine-swappable: route hard questions to the V8, simple ones to the golf cart engine. know that your moat is the trade-offs you chose and why. make decisions that are engine-swappable: know that your moat is the trade-offs you chose and why. if you’re picking tools: stop asking “which model does it use?” start asking: what can it read? what can it do with my files? does it remember me? how long can it focus? how does it handle mistakes? those are harness questions. that’s why the same model feels magic in one app and useless in another. the analogy goes further than you think once you stop arguing about engines, the design space explodes. you start asking better questions. maybe you don’t need a faster car. maybe you need a shorter route. (that’s context engineering: the same engine covers more ground when you stop feeding it a 4,000-word system prompt and start giving it a map.)
maybe you don’t need a car at all. maybe you need a fleet of bicycles. (that’s small model routing: twenty Haiku calls that each cost a fraction of a cent, instead of one Opus call that takes 30 seconds and costs a dollar.)
maybe the problem isn’t the vehicle. maybe it’s the road. (that’s your data infrastructure: the smartest model in the world can’t reason about customers who haven’t converted yet if nobody’s piping that data into the context window.)
and maybe you’ve been optimizing the car when you should’ve been building a boat. (that’s the real question: not “how do I make AI better at this task?” but “is this even the right task for AI?”) maybe you don’t need a faster car. maybe you need a shorter route. (that’s context engineering: the same engine covers more ground when you stop feeding it a 4,000-word system prompt and start giving it a map.) maybe you don’t need a car at all. maybe you need a fleet of bicycles. (that’s small model routing: twenty Haiku calls that each cost a fraction of a cent, instead of one Opus call that takes 30 seconds and costs a dollar.) maybe the problem isn’t the vehicle. maybe it’s the road. (that’s your data infrastructure: the smartest model in the world can’t reason about customers who haven’t converted yet if nobody’s piping that data into the context window.) and maybe you’ve been optimizing the car when you should’ve been building a boat. (that’s the real question: not “how do I make AI better at this task?” but “is this even the right task for AI?”) the engine debate is comfortable because it has a leaderboard. it’s measurable. it updates every week. but the hard problems, the ones where AI actually transforms a business, are all harness problems, road problems, route problems. they don’t have benchmarks. they require taste. the engine matters less every quarter. the rest of the vehicle, the route, and the terrain is what determines whether you arrive. the engine matters less every quarter. the rest of the vehicle, the route, and the terrain is what determines whether you arrive.

The is an opinion piece based on the author’s POV and does not necessarily reflect the views of HackerNoon.

As AI Models Converge, System Design Becomes the Differentiator

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Beep Beep Bop Bop: How to Deploy Multiple AI Agents Using Local LLMs

How Will Software Engineers Lose Their Jobs Within the Next 5 Years?

🎬 Introducing MetaGPT: Unleashing the Power of AI Agents for Complex Tasks

We Asked AI to Improve This Article

How to Build an Agent With an OpenAI Assistant in Python - Part 1: Conversational

How to Build an Agent With an OpenAI Assistant in Python - Part 2: Function Calling / Tools

Beep Beep Bop Bop: How to Deploy Multiple AI Agents Using Local LLMs

How Will Software Engineers Lose Their Jobs Within the Next 5 Years?

🎬 Introducing MetaGPT: Unleashing the Power of AI Agents for Complex Tasks

We Asked AI to Improve This Article

How to Build an Agent With an OpenAI Assistant in Python - Part 1: Conversational

How to Build an Agent With an OpenAI Assistant in Python - Part 2: Function Calling / Tools

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps