01 · Architecture overview

Harness²: how agents tune their own harness

Google Research's Harness² (paper, code) improves an agent without touching the model. It rewrites the harness (everything around the model) task by task, and remembers what worked.

A harness is the system around a model that lets it act as an agent: prompts, tools, memory, skills, orchestration. Harnessing is optimizing that system.

The eight parts it can edit

This is the README's count of eight. In the code the harness is the c1..c8 component tree, and the two lists don't map one to one: c2 (context policy) is owned by the agent framework and is not editable, c6 covers both guardrails and skills, and on Codex c1, c3, c4 and the guardrails all share one AGENTS.md. The c8 slot is TypeScript plugins on OpenCode and .codex/hooks.json lifecycle hooks on Codex. See 04 · The harness tree.

Two levels of recursion

1. Task level: Propose → Probe → Compose

For the task in front of it:

1. Propose. A harness editor suggests edits: invariant ones (good for any task) and domain ones (good for this kind of task).
2. Probe. Each candidate harness actually runs the same task.
3. Compose. A composer compares how the runs behaved and merges the edits that helped into one refined harness.

No ground-truth answers are needed: the signal is the contrast between executions.

2. Harnessing level: an experience library

After each task, a reflector looks at what each edit actually did and saves the adopted harness and the lesson behind it to a per-domain library (INDEX.md, the full harness tree, and an EXPERIENCE.md with why it was picked, what each edit did and a transferable cue/lesson/evidence). An entry is written only when the reflection yields a complete lesson. Each recursion mode keeps its own library, and before every task the library is trimmed to the tasks that came earlier in the domain's stream. The next task in that domain starts from that experience, so refinement gets better over time.

Wider or deeper

Either way one task costs exactly $4k$ rollouts and $3k$ improver sessions per mode, and the code checks it. The command-line harness2 runs only parallel unless you pass --modes parallel,sequential.

Results

One Harness² step ($k = 1$) improves GLM-5 and Gemini 3.5 Flash over their base harnesses:

A larger recursion budget gives further gains.

Inside the code: one task, end to end

core/schedule.py → run_task drives everything for one task:

sequenceDiagram
  participant S as Scheduler (schedule.py)
  participant X as Executor (OpenCode / Codex)
  participant C as Evidence tree (context.py)
  participant E as Editors (proposer.py)
  participant P as Selection (selection.py)
  participant L as Library (library.py)
  S->>X: k shared base rollouts
  X-->>C: full trajectories + deliverables
  L-->>C: earlier tasks in this domain, full harnesses
  C->>E: INV and DOM editors read the evidence
  E->>X: k + k candidate harnesses (diffs) run the task
  X-->>E: behaviour of each candidate
  E->>X: k compositions, then k final rollouts
  X-->>P: anonymized outputs of the k finals only
  P->>P: blind pick, then reflect (why + lesson)
  P->>L: adopted harness + EXPERIENCE.md

The evidence tree is the only way in

A model never sees evidence through a prompt summary. context.py writes a folder and the editor reads it with file access:

Tripwires refuse any context that contains benchmark answer-key paths or the grader's own verdicts, so the method can't learn from the test. In an improvement stage a tripped wire means the stage falls back without running a session. The tripwires are a safety net: the real defence is that adapters only ever read agent-authored bytes.

Edits are diffs

The harness is the c1..c8 component tree (tree.py), stored as files on the framework's editable surface. The editor edits a copy in place, so a proposal is only what it changes: files it doesn't touch are inherited, and deleting a file removes that component. An invalid proposal gets one retry with the error text, then the stage falls back to the harness it started from. Every tree is validated (allowed paths, size caps, hooks schema), skills get auto-trigger lines, and a diversity score (pairwise line-Jaccard) shows when candidates collapse into the same body.

A fixed budget, checked

ledger.py checks an identity per task and mode: R = 4k rollouts ($k$ shared base + $2k$ candidates + $k$ finals) and P = 3k improver sessions ($k$ INV + $k$ DOM + $k$ compositions). The bases run once but are charged to both modes' ledgers: at $k = 2$ with both modes that is 14 distinct rollouts, 16 as the ledgers count them. Selection and reflection sessions are outside the ledger, and retries don't add entries. If the counts don't land exactly there (or a label repeats), that mode fails with a BudgetError; the other mode still runs.

Choose blind, measure after

Selection picks among the anonymized outputs of the mode's k finals (never a base) and never sees a score. If the selector fails, a neutral fallback (the alphabetically first final) is recorded, but the selection stays failed: it is not reflected, not added to the library and not judged. Judging (judge.py) happens only afterwards, in a separate command, on the chosen run. Nothing in the method's run path imports the judge; only core/report.py, the harness2-judge CLI and the tests do. A judge failure is excluded from the denominator, never counted as zero.

Three axes kept apart

spec.py separates method (recursion, prompts, selection, library), substrate (OpenCode or Codex) and benchmark (a BenchSpec value + an adapter). Implement the adapter and the whole method works on a new benchmark.

How the code is laid out

harness2/
  core/        recursion.py (Propose–Probe–Compose), proposer.py (editor sessions),
               selection.py (pick + reflect), library.py (experience library R)
  substrate/   opencode.py, codex.py  (the agent that executes; OpenCode by default)
  benchmarks/  jobbench.py, lab.py, workbuddy.py
  core/        + schedule.py (sweep → domain → task), context.py (evidence tree),
               tree.py (c1..c8), ledger.py (budget), judge.py (after the choice), spec.py
  cli/         harness2, harness2-setup, -check, -judge, -report, -demo, -bridge

Try it

uv venv --python 3.12 .venv && source .venv/bin/activate
uv pip install -e '.[render]'
harness2-demo --output runs/demo --k 2       # no model credentials needed

For a real (paid) run, write a .env by hand (the .env.example the docs mention is not in the repo at this commit), run harness2-setup lab and harness2-check --bench lab, then harness2 --bench lab --k 1 --mode parallel --only-domains antitrust-competition --max-tasks 1. --max-tasks is per domain, so without --only-domains it runs one task in every domain. Add --substrate cx to execute with Codex. Judge and report afterwards with harness2-judge and harness2-report using the same flags.

Why it matters

For anyone building on Claude Code, Codex or OpenCode: your CLAUDE.md/AGENTS.md, skills and subagents are a harness, and Harness² treats them as something an agent can tune itself, with evidence, per domain.

Known limits worth knowing

• Costs are partly invisible. RunView.cost_usd and wall_s are never set, so rollout cost is recorded as 0, and the report's spend line covers improver, selector and reflection sessions only, not executor or judge cost.
• The anonymisation seed depends on the task, k and mode, not on the sweep's --seed.
• The executor's network is not restricted by the code. Block Hugging Face and GitHub at the network level before evaluating (see 08).

Go deeper

• 00 · Start here: reading order, depth standard and the glossary.
• 02 · Code structure: packages, the typed seams between method, substrate and benchmark, import rules.
• 03 · One task, end to end: the CLI, stream order, run_task, parallel vs sequential, resume.
• 04 · The harness tree: c1..c8 on both frameworks, omitted-is-inherited, validation, diversity.
• 05 · Evidence and safety: the evidence tree, leak tripwires, improver sessions, caching.
• 06 · Budget, selection and judging: the ledger, blind selection, reflection, judge and report.
• 07 · Executors and benchmarks: OpenCode and Codex, the three adapters, vendored runners, adding your own.
• 08 · Setup and running: install, configuration, the seven commands, costs, troubleshooting.