Automation Ends Where Agency Begins
The agency/automation line is a definable engineering boundary — independence of decision, not tool-calling — and where you draw it fixes architecture, evaluation, and what to fear.
§1 · The most abused word in the industry
The same architecture ships twice. One vendor's page calls it an AI agent — autonomous, goal-driven, a colleague. The other's calls it workflow automation — reliable, bounded, auditable. Same model, same tools, same loop; the only variable is the marketing meeting. The word "agent" now stretches across everything from a templated retrieval pipeline to systems whose sales pitch is that nobody needs to supervise them — and the practitioner shelf reflects the stretch: vendor primers walk newcomers from "what is an AI agent" to "what is an AI agent system" in a dozen pages, while Haggai Roitman's book-length Hitchhiker's Guide to Agentic AI spans foundations to systems precisely because the term no longer picks out one thing (Roitman, 2026).
This matters beyond taxonomy hygiene, and the paper this article is built on says why. Xing, Deng and Hou — writing from MBZUAI and CMU — argue that it has become essential to clarify where automation ends and agency begins (the question in this article's title is theirs), "both for building capable systems and for understanding whether and what to fear" (Xing, Deng & Hou, 2026). Both halves of that sentence are engineering claims. Build-side: the two readings of the same machine imply different architectures, different failure modes, different loops. Fear-side: existential worry about "machine agency" and dismissal of "just automation" are both category errors when applied to the wrong half.
"Agent" has stretched until it no longer distinguishes a templated pipeline from a self-directed system — and the ambiguity is not cosmetic. Where the automation/agency line falls determines what you build, how you evaluate it, and which fears are even coherent.
§2 · The Cartesian line
The Critique of Agent Model gives the line philosophical footing and then — usefully for this series — makes it structural. Drawing on Descartes' grounding of agency in independent thought, and on the long science-fiction record of imagined autonomous beings, the authors survey today's agent landscape and analyze architectures along five dimensions: goal, identity, decision-making, self-regulation, and learning (Xing, Deng & Hou, 2026).
Their claim is the sharpest definitional move in this collection: genuine agency requires those five structures to be internalized within the system itself, rather than assembled through external scaffolding. That yields a two-word vocabulary this article will lean on. Agentic systems — the overwhelming majority of what ships today — locate their competence in engineered workflows: the goals arrive in prompts, the decision policy lives in the harness, the learning happens in an external loop somebody else runs. Agentive systems would hold those structures endogenously — goals, identity, decision-making, self-regulation, and learning arising within the system, including its social behavior. The distinction is not a capability threshold; it is a question about where the structures live (see Figure 2).
The line has content: five dimensions — goal, identity, decision-making, self-regulation, learning — and one question per dimension: scaffolded or internalized? Engineered-workflow competence is agentic; endogenous structure would be agentive. Capability does not decide the column; architecture does.
§3 · An interface spec, not a philosophy seminar
Turn the five dimensions sideways and they become an audit any architect can run on any deployed system — three ownership questions per component. Who sets the goals? If every objective enters through a prompt or a ticket, goal-ownership is external, whatever the marketing says. Who owns the state? An identity that resets with the context window is a costume, not a self — and the governed persistent state of [← 5] is precisely the engineering that would make this row real. Who verifies the outcomes? If acceptance lives in an external evaluator — as this series has argued it should [← 8] — then self-regulation is, by construction, on the scaffold side of the line.
Run that audit honestly across the systems of this series and nearly everything lands left of the line — including, instructively, the most capable ones. The failure literature agrees from the other direction: the Oxford synthesis of agent limitations [← 9] catalogued its six failure clusters — tool errors, planning failures, long-horizon degradation, coordination breakdowns, safety failures, measurement problems — across systems that are, in this vocabulary, elaborate automations whose "agency" is an attribution made by their operators. The audit's value is not gatekeeping the word; it is that each row names an engineering program. Internalized goals, persistent identity, owned decision policy, self-regulation, endogenous learning — the substrate articles of this series are, row by row, the literature of moving structures across the line deliberately.
The five dimensions collapse into three auditable ownership questions — who sets goals, who owns state, who verifies outcomes — and today's most capable systems still answer "the operator" to all three. That is not an insult; it is a roadmap. Each row is one of this series' improvement programs.
§4 · The ceilings of orchestrated automation
If most shipped "agents" are compositions of automations, then their limits should be the limits of composition — and the collection's most elegant single-author paper measures exactly that. Josef Chen studies multi-model systems — routing, voting, cascades, fusion, mixture-of-agents — and shows that for any policy whose output is one of the member models' answers, accuracy cannot exceed 1 − β, where β is the rate at which every model is wrong on the same query (Chen, 2026). The field, meanwhile, reports average pairwise error correlation ρ — and Chen establishes mathematically that ρ cannot identify β: error laws with identical marginals and identical pairwise correlations can still differ in the quantity that actually sets the ceiling.
The empirical payload spans 67 frontier models from 21 providers — GPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, Grok-4.3, DeepSeek V4, Qwen3.7-Max, Kimi K2.7 among them. A correctly calibrated single-factor model still underprices the all-wrong tail by about 2.5× on open-ended mathematics (90% CI 1.7–3.4, k = 17), and under the full 67-model Gaussian copula the measured β is 0.052 against a predicted 0.023 — the models fail together far more often than pairwise statistics suggest. The practical gift is a Clopper–Pearson bound that turns one graded, held-out query set into what Chen calls a "$0 certificate" on the largest gain any such policy could deliver — before anyone trains a router. And the conceptual gift belongs to this article: a committee of automations shares fates. Whatever crosses the agency line, it will not be a voting bracket over members that are all wrong together 5.2% of the time.
The capability edge makes the same point from above. MirrorCode's strongest model rebuilds entire programs — a 16,000-line toolkit among them — from behavior alone at 56% across the benchmark [← 8]. Staggering capability; and in the five-dimension audit it is pure left column: externally specified goal, externally verified acceptance, no persistent identity between tasks. Capability and agency are different axes, and 2026's systems are climbing exactly one of them.
Composed automations inherit a measurable ceiling: accuracy ≤ 1 − β, β is what pairwise correlation cannot see, and across 67 frontier models the all-wrong tail runs ~2.5× above single-factor predictions (β = 0.052 vs 0.023). Meanwhile the capability frontier — rebuilding 16,000-line programs — sits entirely on the automation side of the line. The axes are independent.
§5 · What to fear, what to build
The fear-side conclusion falls out cleanly. For the agentic systems we actually ship, the rational worries are automation worries at scale: the reward pathologies of [← 7], the consumed evaluators of [← 10], the co-failure tail measured above — engineering risks, auditable with this series' instruments. The speculative "machine agency" fear attaches to the agentive column, and the five-dimension analysis gives that debate something it has chronically lacked: a checklist of what would actually have to be true. A system whose goals, identity, decision policy, self-regulation, and learning are all internalized is a different object with a different risk profile — and, per Figure 2, one we can watch for row by row rather than argue about in the abstract.
The build-side conclusion is this series' closing argument, and D1's opening one. Every loop in these twelve articles — harnesses revising themselves, skills compounding, memory training, evaluators co-evolving — is a structure migrating from scaffold toward system, one row of the audit at a time, with a human gate at the crossing. Draw the line explicitly in your own architecture: label each of the five rows scaffolded or internalized, attach a Loop Card to every row you intend to move, and re-run the audit each quarter. Agency, if it arrives in production systems, should arrive the way everything else in this series arrives — measured, gated, and on purpose.
SIGNAL — the five-row ownership audit (goal / identity / decision-making / self-regulation / learning: scaffolded or internalized?), plus the β-certificate for any multi-model composition (Chen, 2026).
UPDATE — the architecture docs and the roadmap: which rows are intended to cross the line, via which substrate loop [← 3][← 4][← 5][← 10].
GUARDRAIL — no row crosses implicitly: a structure moving from scaffold to system is a design review with a human gate, not a side effect of capability growth.
CHECK — quarterly: re-run the audit; any row whose answer changed without a review is the finding.
Fear the automation risks you can measure; watch the agency checklist you can now audit. And build deliberately: five rows, each scaffolded or internalized, each crossing gated. The improvement loop is how structures move across the line — the audit is how you notice.
What comes next
With the line drawn, the series' substrate work begins where automation is already rewriting itself: the harness — model-specific, human-authored, and, as of this June, increasingly authored by the agent it serves. Next: Harnesses That Build Themselves.
References
- Xing, E., Deng, M., & Hou, J. (2026). Critique of Agent Model. arXiv preprint. arXiv:2606.23991.
- Chen, J. (2026). When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models. arXiv preprint. arXiv:2606.27288.
- Roitman, H. (2026). The Hitchhiker's Guide to Agentic AI: From Foundations to Systems. arXiv preprint, v1.2.2. arXiv:2606.24937.
- A Compact Guide to AI Agents (2026). Vendor practitioner ebook (attributed as a practitioner source; no listed authors).
- Albayaydh, W., Zhao, R., & Flechais, I. (2026). Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents. arXiv preprint. arXiv:2607.05775.
- Adamczewski, T., Owen, D., Rein, D., Brand, F., Edkins, G., Hart, A., & O'Connell, D. (2026). MirrorCode: AI Can Rebuild Entire Programs from Behavior Alone. Preprint, Epoch AI.