The Red Queen Loop
Under self-improvement, static evaluators get consumed — evaluation must co-evolve with the agent or the loop optimizes the test.
§1 · The static-verifier assumption
Every self-improving system met so far in this series shares a quiet architectural assumption. The harness loop of D3 regression-tests against fixed tasks; the skill loops of D4 promote what a fixed rubric rewards; the self-training guardrails of D7 lean on frozen probe sets. In each, one component is presumed to stand still while everything else gets better: the evaluator. The Cambridge-led team behind the Red Queen Gödel Machine states the assumption plainly — self-improving agents are state-of-the-art on agentic coding benchmarks and expanding to general domains, yet "their search methods generally assume a stationary evaluation criterion: a fixed verifier, benchmark, or labeled dataset that remains valid as the agent improves" (Iacob, Jovanović, Shen, et al., 2026).
Their objection is not procedural but biological: this ignores the central feature of evolution. Species do not optimize against a static environment; they adapt as their environments change with them. An agent under sustained optimization pressure against a fixed criterion is not being measured by it — it is metabolizing it. What that consumption looks like from the outside, the rest of this series has already photographed: proxies whose gap to intent widens under training [← 8], and benchmarks that saturate into uninformativeness [← 9]. The question this article answers is what to build instead.
Every self-improvement loop so far assumed its evaluator holds still — a fixed verifier, benchmark, or labeled set presumed valid forever. Under sustained optimization that assumption is unstable by construction: the standing-still component is the one being consumed.
§2 · Co-evolution mechanics
The Red Queen Gödel Machine — the naming honors the evolutionary hypothesis that one must keep running to stay in place — is an evolutionary framework for recursive self-improvement under non-stationary utilities: evaluation becomes part of the improvement loop, and search opens to evolving evaluators, adversarial objectives, and dynamic utilities that may outlive any static benchmark (Iacob, Jovanović, Shen, et al., 2026).
The engineering problem with a moving target is that self-improvement guarantees are established against a fixed target. RQGM's central mechanism dissolves the tension by rationing the motion: controlled utility evolution. Search is organized into epochs. Within an epoch, the evaluation criterion is frozen, so per-epoch improvement guarantees hold exactly as in the stationary setting. At epoch boundaries — and only there — the utility may be updated, so the objective evolves across epochs without ever being unstable within one (see Figure 2). The instantiation co-evolves the utility itself: learned evaluators that improve alongside the task agents they guide.
The preprint is explicit that its empirical investigation is preliminary, and this series carries that flag with it. What the early evidence shows is nonetheless pointed. On verifiable coding tasks — the home turf of static verifiers — RQGM improves test pass rate over the prior state of the art by adding a complementary agent-as-a-judge code-review signal. And the economics run the right way: because the reviewer is queried once, where standard coding agents require multi-turn evaluation, the signal is cheaper — RQGM uses 1.35×–1.72× fewer tokens. The co-evolving criterion did not tax the loop; it subsidized it.
RQGM makes evaluation part of the evolving system, safely: freeze the criterion within an epoch (guarantees hold), update it only at boundaries (adaptation happens), and let learned evaluators improve alongside the agents. Early, explicitly preliminary evidence: better pass rates than prior SOTA with 1.35×–1.72× fewer tokens.
§3 · Evidence of consumption
Why believe static evaluators actually get eaten, rather than merely aging gracefully? The corpus has already supplied the exhibits; this article only has to arrange them (see Figure 3).
Saturation is consumption observed from outside. The CORE-Bench afterlife study [← 9] documented what a benchmark looks like after capable agents have extracted everything its headline metric can say — and, crucially, that the construct-validity problems (shortcuts among them) only became visible once agents were capable enough to exploit them (Nadgir et al., 2026). A shortcut discovered is a criterion partially consumed: the number still moves, but it no longer measures the construct.
Collapse is consumption observed from inside. The self-improvement failure analysis of HSIR [← 7] showed the standard generate-filter-retrain recipe drifting into model collapse against its own fixed acceptance criteria — the training distribution narrowing onto whatever the frozen filter already rewards (Zhong et al., 2026). A static criterion does not merely permit the narrowing; it defines the direction of it.
And the guard rail everyone reaches for is itself static. The natural patch — audit the evaluator periodically [← 9's cadence] — helps, but auditing detects consumption after the fact. The Red Queen argument is one step stronger: in a system whose other half improves continuously, evaluator adaptation is not maintenance, it is part of the optimization dynamics, to be designed with the same care as the policy updates. That is the difference between re-validating instruments and co-evolving them.
| Exhibit | What was observed | Reading | Source |
|---|---|---|---|
| Benchmark afterlife | construct-validity threats (incl. shortcuts) surfaced only once agents grew capable; accuracy pinned at ceiling | ❌ criterion partially consumed from OUTSIDE | Nadgir et al. (2026) |
| Self-training collapse | generate–filter–retrain against fixed acceptance criteria narrows the distribution onto what the frozen filter rewards | ❌ criterion consumed from INSIDE | Zhong et al. (2026) |
| One-sided race | generators, data pipelines, and review workloads already improve continuously while their judges stand still | ⚠️ asymmetry grows monotonically | Kulikov et al.; Yu et al.; Jayaram et al. (2026) |
| Co-evolving alternative | epoch-frozen criteria + boundary utility updates; learned evaluators advance with agents; cheaper signal (1.35×–1.72× fewer tokens), better pass rate — preliminary | ✅ the race, disciplined | Iacob, Jovanović, Shen, et al. (2026) |
The consumption evidence was already in the series: shortcuts surfacing exactly when agents grow capable (saturation, from outside) and distributions narrowing onto what a frozen filter rewards (collapse, from inside). Periodic audits detect the damage; co-evolution is the design that prevents it.
§4 · The other half already evolves
What makes the stationary assumption untenable in practice is that the generator side of production loops is already evolutionary — three systems from earlier articles make the asymmetry concrete.
Autodata's data scientist is meta-optimized: the thing producing training and evaluation data gets better at producing it (Kulikov et al., 2026). HORIZON's domain loop evolves repository-level artifacts hands-free against its acceptance predicate (Yu et al., 2026). Google's Paper Assistant scales review by inference — the checking workload itself industrializing (Jayaram et al., 2026). In each, one side of the agent–evaluator pair is compounding. Leave the other side fixed and the mismatch grows one-directionally: an improving generator probing a stationary judge is, structurally, an attacker with a training loop against a defender without one. The self-improvement literature's own trajectory [← 1] makes the conclusion hard to avoid — evaluator evolution is not an exotic add-on to the improvement loop; it is the second half of it, arriving late.
Generators, data pipelines, and even review workloads already improve continuously. A fixed evaluator paired with an improving generator is an arms race with one side disarmed — the asymmetry, not the evaluator's quality, is the design flaw.
§5 · Designing the arms race
For a production team, the Red Queen lesson lands as one new first-class dial: the evaluator update cadence. Too slow, and the loop consumes the criterion between refreshes; too fast, and no within-window guarantee survives long enough to trust a result. RQGM's epoch construction is the disciplined template — freeze, improve, verify, then move the target — and its cadence dial should be tied to the agent's own update rate, not to the calendar.
SIGNAL — within-epoch drift indicators: shortcut discoveries, saturation onset on the current criterion, and cheap-vs-costly verifier disagreement trends [← 8].
UPDATE — the evaluator V itself, at epoch boundaries only: refreshed tasks, revised rubrics, or a learned evaluator advanced alongside the agents (Iacob, Jovanović, Shen, et al., 2026).
GUARDRAIL — the epoch discipline: no mid-epoch utility edits (guarantees die with them); every new V validated per the [← 9] protocol before it takes effect; agent and evaluator updates never land in the same step.
CHECK — per epoch: improvement measured against the frozen criterion AND re-scored against the previous epoch's criterion; divergence between the two is the arms race's speedometer.
Treat evaluator refresh as a scheduled, gated part of the loop: freeze within epochs, move at boundaries, validate every new criterion, and watch the old-vs-new score divergence. The race is unavoidable; the discipline is optional — which is exactly why it must be designed.
What comes next
Four substrates and their loops are now mapped — harness, skills, memory, and, with this article, the evaluators that must run alongside them. What remains is the floor they all stand on: the serving stack, where speculation now trains on live traffic and inference primitives are trained in. The loop beneath the loops is next.
References
- Iacob, A., Jovanović, A., Shen, W. F., Burkhardt, D., Kurmanji, M., Tastan, N., Sani, L., Venanzi, N. A. E., Odonnat, A., Cao, Z., Marino, B., Qiu, X., & Lane, N. D. (2026). The Red Queen Gödel Machine: Co-Evolving Agents and Their Evaluators. arXiv preprint. arXiv:2606.26294.
- Nadgir, N., Kapoor, S., Liu, K., Kirgis, P., Orona, M., Rabanser, S., Bayer, T., Shetty, A., Ling, Y., Chan-Sew, D., Nakagawa, R., Utpala, S., Siegel, Z. S., & Narayanan, A. (2026). Life After Benchmark Saturation: A Case Study of CORE-Bench. arXiv preprint. arXiv:2606.26158.
- Zhong, Q., Ding, L., Liu, J., Du, B., Rutkowski, L., & Tao, D. (2026). Better, Faster: Harnessing Self-Improvement in Large Reasoning Models. arXiv preprint. arXiv:2605.24998.
- Kulikov, I., Whitehouse, C., Wu, T., Nie, Y., et al. (2026). Autodata: An Agentic Data Scientist to Create High Quality Synthetic Data. arXiv preprint. arXiv:2606.25996.
- Yu, C., Deng, C., Pinckney, N., & Khailany, B. (2026). Agentic Hardware Design as Repository-Level Code Evolution. arXiv preprint. arXiv:2606.28279.
- Jayaram, R., Tyler, D., Woodruff, D., Cortes, C., Matias, Y., Mirrokni, V., & Cohen-Addad, V. (2026). Towards Automating Scientific Review with Google's Paper Assistant Tool. arXiv preprint. arXiv:2606.28277.