The World Model Turn
Next-state prediction is consolidating perception, prediction, and control — world models are becoming agent infrastructure rather than a research aesthetic.
§1 · From aesthetic to infrastructure
In one week of late June 2026 — the 24th to the 30th — three labs on three continents published three different answers to the same bet. The Qwen Team shipped language world models built to simulate agentic environments. The Beijing Academy of Artificial Intelligence introduced a general world foundation model organized around a single objective. And an NYU group made latent world models adapt while planning. None of these are driving demos or robotics showreels; all three aim squarely at the substrate this series cares about — agents that perceive, predict, and act in one loop.
SCOPE, stated up front: this article covers the 2026 agent-facing wave in this collection, by design. The world-model lineage — its classic simulators and latent-imagination ancestors — is deliberately not re-treated here; the big-world formalism lives in the published series [← A3][← A4], and the ten-year latent-world-model bet has its own essay [← C2]. What is new, and what this article documents, is the moment world models stopped being a research aesthetic and started shipping as agent infrastructure (the wave is audited in Figure 3).
Within a single late-June week, BAAI, Qwen, and NYU converged on the same bet from three directions — a general world foundation model, language world models for agent environments, and test-time-adaptive latent planners. The convergence, not any single system, is the story.
§2 · Next-state prediction as the unifier
The clearest statement of the consolidation is Orca's objective. Instead of optimizing isolated next-token, next-frame, or next-action prediction — one head per modality, one research community per head — the Orca Team centers everything on Next-State-Prediction: a unified state-transition modeling route toward understanding, predicting, and acting upon the world (Orca Team, 2026). The system learns a unified world latent space from multimodal signals and exposes it through multimodal readout interfaces (see Figure 2).
Two design choices make Orca more than a slogan. First, its learning is split into two complementary paradigms with unusually honest names: unconscious learning captures dense, natural state transitions from continuous video; conscious learning models sparse, meaningful transitions supervised by language-described events and VQA. Second, the scale is stated: a pre-training inventory of 125K hours of video and 160M event annotations. After pre-training, the claim is tested the only way a latent space can be — through downstream readouts: text generation, image prediction, and an embodied readout, three different windows onto one learned state.
Orca replaces three isolated prediction objectives with one — Next-State-Prediction over a unified latent — trained by dense unconscious learning from 125K hours of video and sparse conscious learning from 160M event annotations, then read out through text, image, and embodied interfaces.
§3 · Language world models, built for agents
The Qwen Team's entry answers the question this series has been circling since D1: what does a world model do for a digital agent? Their answer — simulate the agent's environment. Qwen-AgentWorld-35B-A3B and its 397B-A17B sibling are, per the team, the first language world models capable of simulating agentic environments across 7 domains through long chain-of-thought reasoning, trained on more than 10M real-environment interaction trajectories (Qwen Team, 2026). The pipeline is a three-stage arc that reads like this series in miniature: continued pre-training injects general world-modeling from state-transition dynamics, supervised fine-tuning activates next-state-prediction reasoning, and reinforcement learning sharpens simulation fidelity under hybrid rubric-and-rule rewards.
Why an agent stack wants this is not mysterious: a faithful environment simulator is training infrastructure — rollouts without the tool-world's cost — and the missing piece of D7's exhaust economy. But the imagined-rollout bargain has terms, and the corpus's theory entry prices them. Timor et al. — a Weizmann–NYU–Columbia collaboration including LeCun and Harel — quantify how errors in learned dynamics and learned reward models propagate into returns when policies train purely in imagination (Timor et al., 2026). Extending classic error analysis to MDPs with learned rewards, they derive the optimal sample allocation between dynamics and reward data under power-law scaling assumptions, and identify low Lipschitz constants — of dynamics, reward, and policy — as the representation property that tightens the whole bound. Imagination is a budget line, and now it has an allocation formula.
Language world models now simulate agent environments outright — 7 domains, 10M+ real interaction trajectories, a CPT→SFT→RL pipeline aimed at simulation fidelity. And the theory wing prices the bargain: return error under imagined training decomposes into dynamics-vs-reward terms with an optimal data allocation between them.
§4 · Latent adaptation: the frozen-model problem
The third convergence attacks the assumption that a world model, once trained, holds still — the moving-target theme of [← 10] in miniature, one substrate down. AdaJEPA, from Ying Wang, Oumayma Bounou, Yann LeCun and Mengye Ren at NYU, starts from the operational failure: latent world models are typically frozen at test time, so when their predictions drift — precisely under distribution shift — planning fails (Wang, Y., et al., 2026). Their fix runs inside the model-predictive-control loop: plan, execute the first action chunk, treat the observed next-state transition as a self-supervised adaptation signal, update the model, replan. No new demonstrations, no offline retraining — and the reported efficiency is striking: planning success substantially improves with as few as one gradient step per MPC replanning step. The world model becomes a live participant in its own deployment, recalibrated by every transition it mispredicts.
Beneath the planners sits an architectural question: why don't transformers build good state representations by default? Microsoft Research's NextLat paper — Jayden Teoh with John Langford's group — gives the sharp answer: attention is a memory that grows with sequence length and permits ad-hoc lookups, so the architecture has no inherent incentive to compress history into compact latents with consistent transition rules, and generalization suffers for it (Teoh et al., 2026). Their Next-Latent Prediction adds a self-supervised auxiliary: train the transformer's latent to be predictive of the next latent given the next token. The theoretical payoff is exactly the object this article is about — those latents converge toward belief states, the compressed summary of history sufficient to predict the future — injecting a recurrent inductive bias while leaving architecture, parallel training, and inference untouched. World-model structure, retrofitted into the workhorse architecture as a loss term.
The frozen-model problem is falling from both ends: AdaJEPA recalibrates a latent planner inside the MPC loop at one gradient step per replan, and NextLat shows a one-line auxiliary objective pulls transformer latents toward belief states — compact world-model state without changing the architecture.
§5 · The generative-simulator wing
The wave's most visible artifacts — high-fidelity generative simulators — sit deliberately at this article's edge, as domain evidence rather than spine. Wayve's GAIA-2 is a latent diffusion world model for autonomous driving: controllable multi-camera video generation conditioned on structured inputs — ego-vehicle dynamics, agent configurations, environmental factors, road semantics — with spatiotemporal consistency across UK, US, and German driving scenes, and hooks for external latent embeddings from a driving model (Russell, Hu, Bertoni, et al., 2026). It demonstrates what the simulator wing can already deliver when a vertical owns its data.
And Adobe's Self Forcing contributes the wing's most transferable training lesson. Autoregressive video diffusion suffers exposure bias — trained on ground-truth context, deployed on its own imperfect outputs. Self Forcing closes the gap by conditioning each frame's generation on previously self-generated outputs during training itself, via autoregressive rollout with KV caching, supervised by a holistic video-level loss rather than frame-wise objectives, made affordable with few-step diffusion and stochastic gradient truncation (Huang, X., et al., 2026). Readers of [← 7] will recognize the shape instantly: train under the distribution you will deploy under. The exhaust lesson, rediscovered independently by the simulator wing — which is roughly what you would expect if the loop framing of this series is carving the field at its joints.
The generative simulators supply the wave's spectacle — and its most portable lesson: GAIA-2 shows vertical-scale controllable simulation, while Self Forcing's fix for exposure bias (train on your own rollouts, score the whole sequence) is D7's train-under-deployment-distribution principle arriving from an entirely different community.
| System | Substrate / domain | Training signal | Status per pulled evidence |
|---|---|---|---|
| Orca (BAAI) | general world foundation model | 125K h video (unconscious) + 160M event annotations (conscious) | ⚠️ latent evaluated via text / image / embodied readouts; downstream numbers not in pulled pages |
| Qwen-AgentWorld 35B-A3B / 397B-A17B | agentic-environment simulation, 7 domains | 10M+ real interaction trajectories; CPT→SFT→RL (rubric+rule rewards) | ⚠️ first language WMs for agent envs (team's claim); eval suite name truncated in pull |
| AdaJEPA (NYU) | latent planner, goal-reaching MPC | observed transitions as self-supervised test-time signal | ✅ substantially improved planning success at one gradient step per replan |
| NextLat (MSR) | transformer latents → belief states | auxiliary next-latent prediction | ✅ theoretical convergence to belief states; architecture unchanged |
| Training in Imagination (Weizmann/NYU) | theory: imagined-rollout error | — | ✅ optimal dynamics-vs-reward sample allocation; Lipschitz desideratum |
| GAIA-2 (Wayve) · Self Forcing (Adobe) | generative simulators (driving · video) | structured conditioning · self-generated rollouts in training | ⚠️ domain evidence; the transferable lesson is train-under-deployment-distribution |
SIGNAL — observed next-state transitions wherever the agent acts: the same stream AdaJEPA adapts on and Qwen-AgentWorld trains simulation fidelity against.
UPDATE — the world model, on two cadences: test-time recalibration inside the planning loop (one gradient step per replan), and periodic fidelity training on accumulated trajectories.
GUARDRAIL — imagination stays budgeted: dynamics-vs-reward data allocated per the return-error analysis [Timor et al., 2026], and policies trained in simulation validated against true-environment probes before promotion [← 7's exhaust discipline].
CHECK — monthly: prediction error on held-out real transitions per domain, and the sim-vs-real gap on a frozen task set; a simulator whose fidelity metric improves while the sim-to-real gap widens is training the agent for a world that isn't there.
What comes next
A world model predicts the consequences of actions; training on those predictions is literally training on the system's own exhaust — which is why every claim in this article leaned on the pathology report that precedes it in reading order. The failure modes of learning from self-generated experience — collapse, addiction, hacking, and the guardrails that catch them — are treated in the next article.
References
- Orca Team, Beijing Academy of Artificial Intelligence (2026). Orca: The World is in Your Mind. arXiv preprint. arXiv:2606.30534.
- Qwen Team (2026). Qwen-AgentWorld: Language World Models for General Agents. arXiv preprint. arXiv:2606.24597.
- Wang, Y., Bounou, O., LeCun, Y., & Ren, M. (2026). AdaJEPA: An Adaptive Latent World Model. arXiv preprint. arXiv:2606.32026.
- Teoh, J., Tomar, M., Ahn, K., Hu, E. S., Pearce, T., Sharma, P., Krishnamurthy, A., Islam, R., Lamb, A., & Langford, J. (2026). Next-Latent Prediction Transformers Learn Compact World Models. arXiv preprint. arXiv:2511.05963.
- Timor, N., Shwartz-Ziv, R., Goldblum, M., LeCun, Y., & Harel, D. (2026). On Training in Imagination. arXiv preprint. arXiv:2605.06732.
- Russell, L., Hu, A., Bertoni, L., Fedoseev, G., Shotton, J., Arani, E., & Corrado, G. (2026). GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving. arXiv preprint. arXiv:2503.20523.
- Huang, X., Li, Z., He, G., Zhou, M., & Shechtman, E. (2026). Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion. arXiv preprint. arXiv:2506.08009.