Octonote AI
Make your own →

00 · Start here

Sunil Tiwari· Oct 27, 2026 · 9 min

00 · Start here

A chapter-by-chapter guide to google-research/harness2 (Harness²: recursive agent harnessing). Pinned to commit 1cfe254, so every code link points at exactly the code described.

Reading order

#ChapterWhat you'll understandMain diagrams
01Architecture overviewThe method and the code on one pageArchitecture (canvas)
02Code structurePackages, core types, seamsClass, package
03One task, end to endThe run lifecycleSequence, flow (canvas)
04The harness treeThe c1..c8 components and edit rulesClass, state
05Evidence and safetyWhat editors see; leak tripwiresFlow, sequence
06Budget, selection and judgingFixed spend, blind choice, measurementState, flow
07Executors and benchmarksPlug-in points and adding your ownComponent, class
08Setup and runningInstall to judged reportDeployment (canvas), flow

Depth standard (every chapter)

  • Cover everything in scope: every public class, function and constant in the chapter's files, with what it does, its inputs and outputs, and why it exists.
  • Link to the code at the line: blob/<pinned commit>/path#Lstart-Lend for each class and function named.
  • Clickable diagrams: every class or component box links to its definition in the code.
  • Show real code: short excerpts for the key logic, with line links.
  • Explain the why: design choices, invariants and failure modes, not just the what.
  • Worked example: trace one concrete run or input through the chapter.
  • No guessing: say "not covered by the code" rather than inventing behaviour.

Glossary

50 terms used across the guide. The last column links the chapters that explain each one.

TermMeaningChapter
HarnessThe system around a fixed model that makes it an agent (prompts, tools, memory, skills, orchestration); in the code, a tree of text files on one framework's editable surface.01, 04
HarnessingOptimising the harness; in Harness² the outer level of recursion, which carries adopted harnesses and lessons from task to task.01
SubstrateThe agent framework a solver rollout runs under: OpenCode (oc) or Codex CLI (cx). Owns the surface, staging and trajectory format.02, 07
ExecutorThe agent that actually does the task (the solver on a substrate); its runs are the rollouts.07
RolloutOne execution of one harness on one task; core sees it only as a RunView. Dead rollouts are recorded, never raised.03
Base rolloutsThe k runs of the unmodified harness (base1..basek), rolled once in the agg namespace and shared by both modes.03
Improver sessionOne model session that edits a harness (INV, DOM, AGG or COMPOSE); also called editor or proposer session. Run by the Proposer class.05
Channel (ChannelSpec)How improver and selector models are reached: opencode (default) or claude_cli. Independent of the substrate.02, 05
INV / DOM editsThe two proposal dimensions: domain-invariant edits (inv_i) and domain-specific edits (dom_i).03
CompositionA stage that merges earlier candidates into one harness: agg1..aggk in parallel mode, c1..ck in sequential mode.03
ProbeRunning each candidate harness on the same task so its behaviour can be compared; the contrast between runs is the signal.01, 03
StageOne improver step including its single retry; recorded as a Stage and as one ledger session.03, 04
Stage fallbackAfter two failed attempts (or a transport error) a stage inherits the harness it started from; it is still rolled and still counted.04
KEEP_BASEA valid proposal that changes nothing (kept_base=True); a legitimate answer when the evidence argues for no change.04
Parallel mode (agg)Width: 2k independent proposals from the same base evidence, then k independent compositions over all of them.03
Sequential mode (seq)Depth: k rounds; each round edits the current harness (INV + DOM) and composes c_t, which the next round builds on.03
kThe scale: number of bases, finals and sequential rounds; sets the budget R = 4k, P = 3k.03, 06
FinalsThe mode's k compositions (agg1..aggk or c1..ck); the only candidates selection may choose from.06
R / PPer task and per mode: R = 4k rollouts (k bases + 2k candidates + k finals), P = 3k improver sessions (k INV + k DOM + k compositions).06
LedgerPer-mode record of every rollout and session; assert_identity raises BudgetError unless R = 4k, P = 3k and no label repeats.06
Evidence treeThe folder context.build writes for a session (task, source, trajectories, deliverables, harness, pool, library): the only way evidence reaches a model.05
MISSION.mdThe mission prompt written into an OpenCode session workspace; a short pointer to context/, not the evidence itself.05
TripwireA check that raises when answer-key paths (leak_globs) or grader-verdict markers would enter model context or be stored; three of them.05
leak_globsPer-benchmark answer-key path patterns: filtered out of source copies, rejected anywhere else.05, 07
Names-only modeProposers see deliverable file names but not contents (proposer_deliverables="names", WorkBuddy code).05
HarvestCopying the edited context/harness/ out of the throwaway session workspace; how a proposal comes back.05
Experience libraryPer-mode, per-domain store of adopted harnesses plus lessons (INDEX.md), read before each task and trimmed to earlier tasks.03, 06
EXPERIENCE.mdA library entry's lesson: why it was chosen, what each edit did, and a transferable cue / lesson / evidence / confidence.06
Blind selectionThe selector picks one of the k finals from anonymised deliverables (ids A, B… from a SHA-1-seeded shuffle of task, k and mode), with no labels, harnesses or scores.06
Trivial selectionAt k = 1 the single final is delivered without a selector session.06
Selector fallbackAfter a selector failure, the alphabetically first final is recorded with fallback=True, but the selection stays ok=False: not reflected, not stored, not judged.03, 06
ReflectionThe deliberately non-blind experience session after adoption; distils edit verdicts and at most one lesson for the library.06
JudgeThe benchmark grader, reached only through adapter.score from core/judge.py, in a separate command after selection; a failure is None, never zero.06, 07
Namespace (ns())<bench>-<substrate>-<mode><k>[-s<seed>][-<tag>]; with task and label it forms the run id <ns>/<task>/<label>, the cache key.02, 03
Stream orderThe seeded, per-domain order of tasks; domains run concurrently, tasks within a domain strictly in order.03
DomainThe grouping unit (practice area, profession, category or role); one library stream per domain and mode.03, 07
summary.jsonThe per-task record (modes, selections, status), kept in the agg namespace and merged under a file lock.03
ResumeRerun the same command: tasks whose summary is ok with intact artifacts are skipped; caches replay the rest. config.json blocks a changed experiment.03, 08
BenchSpecFrozen value describing one (benchmark, substrate) cell: install mode, prompts, channels, leak globs, limits.02, 07
BenchmarkAdapterThe protocol a benchmark implements (tasks, execute, trajectory, deliverables, base harness, validate_tree, score).02, 07
BenchPromptsThe benchmark's wording for the shared prompts; may not mention benchmark, evaluation, grader, rubric or score.02, 07
RunViewWhat the method knows about one rollout (label, run id, dir, ok, delivered); never carries a score, and its cost fields are never set.02, 07
SurfaceSpecData describing one framework's editable surface (always-on files, caps, skill dir, plugins or hooks); all tree logic reads it.02, 04
c1..c8The component slots: c1 system prompt, c2 context policy (not editable), c3 memory, c4 tool guidance, c5 subagents, c6 guardrails + skills, c7 scripts, c8 plugins/hooks.04
Omitted-is-inheritedThe edit contract: files a proposal doesn't touch are inherited unchanged; deleting a file removes the component.04
AutotriggerAutomatic repair that appends a trigger line for any skill not named in the always-on text, within the cap.04
DiversityMean pairwise line-Jaccard and distinct count over the parallel finals; makes collapse visible but nothing acts on it.04
Install / LauncherA staged harness with its source fingerprint / the benchmark-supplied body a substrate calls to run it.02, 07
Run markThe .adapt_complete marker plus harness fingerprint that makes execute idempotent by run id.07
PreflightOffline readiness checks (harness2-check, repeated by harness2 before spending); never proves a model is reachable.08

Known gaps and surprises in the code

Findings the chapters flagged as dead code, unused settings or behaviour the docs don't describe, at commit 1cfe254.

  • RunView.cost_usd and wall_s are never set, so rollout cost is 0 everywhere and the report's spend omits executor and judge cost. (03, 06, 07)
  • A selector fallback stays ok=False although the docstring says it "falls back to a delivered base"; the base branch of _fallback and run_selection's pool_kind="mixed" default can't fire in this release. (02, 06)
  • The anonymisation seed uses task, k and mode, not the sweep's --seed. (06)
  • The harness2 CLI defaults to --modes parallel (the Python run_sweep defaults to both) and passes no tag or task filter; harness2-judge has no --tag either. (03, 06)
  • .env.example is referenced by the README, SETUP and MANIFEST.in but is not in the tree. (08)
  • BenchSpec.domain_word is set by every adapter but never read; max_chars is documented as a BenchSpec field but is a literal in each adapter. (02)
  • The import rules (core never imports benchmarks or the judge) are conventions: no linter or test enforces them. (02)
  • The selector-wording hooks deliverable_assets_note / _line / _reference_only have no implementer. (02, 07)
  • Threaded base rollouts are marked ok=True without checking the run directory exists, unlike _roll and the batched path. (03)
  • ModeResult.errors is never appended to in recursion.py. (03)
  • No per-task wall-clock or cost cap exists in the scheduler. (03)
  • Diversity is computed but nothing reads it, and there is no collapse threshold. (04)
  • OpenCode plugins (.ts) get no content check, while Codex hooks get a full schema check; Codex subagents are gated off on all three benchmarks. (04)
  • No caller passes a system prompt, so the SYSTEM_ARG_MAX branch is unused; the Claude CLI channel has no directory sandbox. (05)
  • Report denominators are graded rows only and are not reconciled across modes; base_judged.json has no consumer. (06, 08)
  • JobBench passes the profession slug as the domain, so the domain lens rarely applies. (07)
  • Exported but unused: SCOREABLE, sweep_orphans, codex.final_message, config.env_int; CodexSubstrate.preflight ignores require_dist. (07, 08)
  • The code never restricts the executor's network, and some rubrics are public; block Hugging Face and GitHub before evaluating. (08)
Was this useful?
Written bySunil TiwariMore from Harness² · repo guide →
Make a page like this. Free.Notes and diagrams side by side, written by you or by Claude Code and Codex. Publish it in one click.
Make your own, free