What is a Harness?
From test runners to AI agents — the system that wraps and controls a target
The word Harness originally means "horse tack" — the equipment that wraps around a horse to control it. Same meaning in software.
Test Harness — The Original
A Test Harness wraps the System Under Test (SUT) to execute it. It bundles the test runner, mock data, log collection, and result evaluation.
JUnit, RSpec, pytest — all Test Harnesses. Not the test code itself, but the "environment" that runs tests and collects results.
Tools like Keploy are also Test Harnesses. They record app network traffic at the kernel level via eBPF and auto-generate tests from recordings. Creating a test environment by "wrapping" the app without modifying its code.
AI Harness — The Extension
In AI/LLM context, it refers to the entire system connecting models to real tasks: file reading, code modification, test execution, error analysis, and retry loops.
The key insight: 'Harness design matters more than target (model/app) performance.' Same Codex with a simple harness (prompt → code → done) vs a good harness (repo search → patch → test → error analysis → retry loop) shows vastly different SWE-bench scores.
Common Harness Patterns
Regardless of domain, harness structure is similar.
Wrapping — control the target externally without modifying it. Keploy wraps apps with eBPF, Claude Code wraps models with prompts and tools.
Isolation — block external dependencies, run in controlled environments. Test harnesses mock DBs, AI harnesses sandbox code execution.
Observation — record and analyze target I/O. Test harnesses track test results, AI harnesses track model behavior.
Iteration — re-execute based on results. Failed test → fix code → rerun. Failed agent → try different approach.
How It Works
Wrapping — control target externally without modification (wrapping apps with eBPF, or models with prompts)
Isolation — block external dependencies, run in controlled environments (DB mocks, sandboxes)
Observation — record and analyze target I/O (test results, model behavior tracking)
Iteration — re-execute based on results (test fail → fix → rerun, agent fail → retry)
Pros
- ✓ Improving Harness design dramatically boosts performance without changing models
- ✓ Harness is the key differentiator of AI coding tools like Cursor, Claude Code, Devin
- ✓ In 2026, Harness Engineering is more important than Model Engineering in the AI industry
Cons
- ✗ More complex Harness means harder debugging and maintenance
- ✗ Model updates may cause compatibility issues with the Harness