RRSI: Regularized Recursive Self-Improvement of Agent Harnesses

Peng Xia, Rujun Han, Zifeng Wang, Yanfei Chen, Yufan Zhang, Yoonho Lee, Chengsong Huang, Han Yu, Zhongying CuiZhu, Yifei Ming, Huaxiu Yao, Burak Gokturk, Tomas Pfister, Chen-Yu Lee — Google Cloud AI Research

arXiv:2609.24972

Charts below re-plot numbers reported in the paper (Figures 1 & 3, Tables 1–3). Policy frozen at Claude Opus 4.8; the harness is evolved on one suite per domain and then evaluated unchanged on held-out and out-of-distribution (OOD) benchmarks.

Figure 1aEvolve-set gain vs. out-of-distribution gain

Agentic workspace instance. Points far to the upper-left are the goal: small evolve-set gain, large transfer. Points left of the dashed line at 0 fall below the unevolved harness H₀.

Evolve-set gain (Harvey LAB) OOD gain (mean of JobBench, GDPval, APEX-Agents)

Reading it: Meta-Harness has the biggest evolve-set gain (+3.6) but keeps only +0.9 OOD. AHE and TTHE are negative OOD (−0.5 and −1.7) — their evolution actively hurt tasks they were never scored on. RRSI has the smallest evolve gain (+1.1) and the largest transfer (+3.9), the exact trade the regularizers are designed to make.

Figure 3Main results in all three domains (Δ vs. H₀)

Evolve split In-distribution held-out OOD held-out (never scored during search)

Every held-out split improves and none regresses. Frontier-Eng is measured in medal points: +4.3 is a 24.3% relative gain. The engineering-design domain matters because its tasks are graded by frozen deterministic simulators, not a judge model — so the transfer can't be explained by a harness learning to write the way a judge rewards.

EfficiencyPolicy tokens per trial

Regularization is cheaper, not more expensive: RRSI spends 2.42M tokens/trial against 3.80M for unregularized evolution (≈36% less), and only 1.6× the unevolved baseline. Unregularized selection spends most of its accepted edits on noise and on context rather than on mechanism — it buys score with compute, and the compute doesn't transfer.

Table 1Comparison with prior harness evolution methods

Method In-distribution Out-of-distribution OOD avg.
Harvey LAB (evolve) Harvey LAB (ID held-out) JobBench GDPval APEX-Agents

OOD avg. is computed as the mean of JobBench, GDPval and APEX-Agents (the paper reports only the H₀ and RRSI averages explicitly, 39.7 and 43.6; the baseline averages shown for the others are recomputed from the table's own columns). Same H₀, same evolve split, same candidate budget for every row.

Table 2Ablation: is it the regularizers or the score?

Variant Harvey LAB (evolve) Harvey LAB (ID held-out) OOD avg. Tokens/trial (M) ↓

The strongest single signal in the paper: evolve-set score and transfer move in opposite directions. Dropping both regularizer groups yields the highest evolve score of any arm (92.8) and an OOD average within a point of the unevolved harness. Dropping only the acceptance constraints raises the evolve score by 1.0 pt, costs 2.6 pts OOD, and inflates token cost by half.

Table 3Robustness to the backbone policy (coding)

PolicyBenchmarkH₀RRSIΔ

Evolution is run independently per policy on Terminal-Bench 2.1 only; the resulting harness is then evaluated unchanged on SWE-bench Verified. Gemini 3.5 Flash gains more on the evolve split (+14.1 — the paper's headline number) while the stronger Claude Opus 4.8 sits closer to both ceilings and gains less.

Figure 2What is actually regularized

Proposal side — how search capacity is spent

  • L₀-style annealed edit budget: b_t = ⌈b_min + (b_max−b_min)·½(1+cos(πt/T))⌉. Early rounds bundle coordinated edits to find mechanisms; late rounds become sparse and attributable.
  • Evidence-aware credit assignment: every candidate's component, hypothesis, diff, score/cost delta and accept/reject outcome are recorded; falsified hypotheses stay negative evidence.
  • Structured exploration: when progress over w rounds stays inside the noise band δ, part of the budget is reserved for components never yet exercised.

Selection side — what becomes permanent state

  • Leakage screening: a critic rejects diffs encoding task/entity names, task-specific values, answers, or inert machinery — before evaluation, so a leaking candidate never banks an inflated score.
  • Stability-aware acceptance: Ŝ(H′) ≥ S⋆ − δ, with δ estimated from repeated runs of the unchanged base harness. Blocks slow downhill walks of individually noise-sized regressions.
  • L₂-style complexity-aware acceptance: for ΔS > δ, require ΔC ≤ β₀ + β₁ΔS — extra tokens must be justified by measured improvement.
  • L₁-style structural pruning: components with no strictly positive measured gain over a window are handed back to the proposer as deletion targets.