World Models

A world model is a learned model of how an environment responds to actions. Given what an agent has observed and an action it might take, the model predicts what the agent will observe next, and usually the reward it will receive. An agent with such a model can try out actions in the model, “in imagination”, before it acts in the real environment, and can use the imagined experience to plan or to train a policy. In computer vision the observations are usually images or video, so a world model is an action-conditioned form of video prediction. The model may predict pixels directly, or it may predict a compact latent state from which pixels and rewards can be decoded.

The term covers several lines of work that grew apart and are now meeting again:

• latent dynamics models trained alongside reinforcement learning agents, from Ha and Schmidhuber’s World Models (2018) to PlaNet and the Dreamer agents;
• generative interactive environments such as Genie, learned from video without action labels;
• large video generators presented as simulators of the physical world;
• driving and robot models that predict the future of a scene under a planned manoeuvre.

Video Prediction owns the predictors themselves, their losses, benchmarks and metrics, and its Action-Conditioned Prediction section describes visual model-predictive control for robot pushing. Embodied AI covers the agents, simulators and tasks in which world models are used. Video generation surveys the generators that some authors now present as world simulators. A small example at the end fits two action-conditioned models of a sprite world in closed form and plans with them.

Definition

Ha and Schmidhuber’s World Models article frames the idea through human mental models: an internal model of the world, built from limited senses, on which decisions are based (Ha & Schmidhuber, 2018). In their agent, the world model is the part that compresses observations and predicts their future, trained without access to rewards, while a separate controller chooses actions. Genie’s authors give a working definition by contrast with video models. In their comparison table, world models are trained on video together with actions and are controllable frame by frame, while video models are trained on video and text and are controllable only at the level of the whole video (Bruce et al., 2024, Table 1). Drive-WM’s authors describe a world model as one that predicts future states from current states and ego actions (Wang et al., 2024, §1).

These definitions share three parts:

• A state. The model keeps a representation of what it has seen, either the recent frames themselves or a latent vector that summarizes the history.
• A transition conditioned on actions. Given the state and an action, it predicts the next state, or a distribution over next states.
• A use for decisions. The predictions serve an agent: to choose actions by searching over imagined futures, or to train a policy on imagined experience.

A video generator conditioned only on text or on past frames lacks the second part, and an action-conditioned predictor that is never used to decide anything lacks the third. Where to draw the line is not settled. Generators that take driving commands or latent actions as inputs sit between the two, and several papers below call them world models.

“World model” has an older meaning in robotics: the symbolic or geometric description of the surroundings that a planner works on, as in the modular robots described in robot perception. This article uses the term in the learned, predictive sense.

Intuition

A planner that can simulate its environment can compare actions before committing to one. When the simulator is given, as in board games or a physics engine, planning is a search over action sequences. A world model replaces the given simulator with a learned one, which the agent builds from its own experience or from recorded video. The appeal, as PlaNet’s authors put it, is that planning with a learned model can be more data-efficient than model-free reinforcement learning, can improve with more computation spent on search, and can transfer to new tasks in the same environment, because the dynamics do not depend on the task (Hafner et al., 2019, §1).

The difficulty is that the agent now relies on predictions it cannot check while it plans. PlaNet’s authors list the key problems: model inaccuracies, errors that accumulate over multi-step predictions, failure to capture several possible futures, and overconfident predictions outside the training distribution (§1). Two consequences recur through this article.

• Compounding error. A model rolled out open loop, feeding its own predictions back as inputs, carries its first error into every later step. Video prediction describes the same effect in pixel space.
• Exploitation. A planner or policy optimized inside the model searches for whatever scores best there. If the model is wrong in some region, the optimizer may find and use that error. Ha and Schmidhuber’s agent learned to move so that, in some rollouts, the monsters in its model never fired, an exploit that does not exist in the real game (see Latent Dynamics Models for Control).

Much of the design of world models is a response to these two problems: predicting in a compact latent space rather than in pixels, modelling uncertainty, replanning after every real step, and keeping the agent’s experience close to the data the model was trained on.

Formal Definition

Most work fits a partially observable Markov decision process, as PlaNet’s authors state it (Hafner et al., 2019, §2). There are hidden states $s_t$, image observations $o_t$, actions $a_t$ and rewards $r_t$, with

$$s_t \sim p(s_t \mid s_{t-1}, a_{t-1}), \qquad o_t \sim p(o_t \mid s_t), \qquad r_t \sim p(r_t \mid s_t).$$

The agent’s goal is a policy $p(a_t \mid o_{\le t}, a_{<t})$ that maximizes the expected sum of rewards. A world model learns approximations of the transition, observation and reward functions from recorded episodes, together with an encoder $q(s_t \mid o_{\le t}, a_{<t})$ that infers a belief about the current state from the history (§2–3).

PlaNet’s recurrent state-space model (RSSM) splits the state into a deterministic part $h_t$ and a stochastic part $s_t$ (§3):

$$h_t = f(h_{t-1}, s_{t-1}, a_{t-1}), \quad s_t \sim p(s_t \mid h_t), \quad o_t \sim p(o_t \mid h_t, s_t), \quad r_t \sim p(r_t \mid h_t, s_t),$$

with $f$ a recurrent network. It is trained by maximizing a variational bound on the likelihood of the observed sequence, the sequence form of the evidence lower bound used to train variational autoencoders:

$$\ln p(o_{1:T} \mid a_{1:T}) \ge \sum_{t=1}^{T} \Big( \mathbb{E}_{q(s_t \mid o_{\le t}, a_{<t})}\big[\ln p(o_t \mid s_t)\big] - \mathbb{E}\big[ D_{\mathrm{KL}}\big(q(s_t \mid o_{\le t}, a_{<t}) \,\Vert\, p(s_t \mid s_{t-1}, a_{t-1})\big)\big] \Big),$$

written for observations only; the reward term follows by analogy (§3, Eq. 3). The first term is a reconstruction loss. The second keeps the model’s one-step prediction of the state close to the state the encoder infers once the next image has been seen.

Two ways of using the model follow.

• Planning. At each real step, search over action sequences $a_{t:t+H}$ for the one whose imagined rollout has the highest predicted return, execute its first action and replan. This is model-predictive control (MPC). PlaNet searches with the cross-entropy method: sample candidate sequences from a Gaussian, evaluate them in the model, refit the Gaussian to the best ones and repeat (§2). The simplest version, random shooting, samples sequences once and keeps the best.
• Learning in imagination. Train a policy, and usually a value function, on trajectories generated by the model rather than by the environment, as Ha and Schmidhuber’s controller in its “dream” and the Dreamer agents do.

The objective an agent optimizes in imagination is the model’s prediction of the return, not the return itself. The gap between the two is where compounding error and exploitation enter.

Latent Dynamics Models for Control

World Models (Ha and Schmidhuber, 2018)

Ha and Schmidhuber’s agent has three parts, which they call V, M and C (Ha & Schmidhuber, 2018). The interactive article on worldmodels.github.io was read for this article; the NeurIPS paper of the same year, which the article asks readers to cite, was not.

• V, vision. A convolutional VAE compresses each $64 \times 64$ frame into a latent vector $z_t$, of 32 dimensions for car racing and 64 for Doom.
• M, memory. An LSTM with a mixture-density output predicts a distribution over the next latent, $P(z_{t+1} \mid a_t, z_t, h_t)$, as a mixture of five Gaussians. It had 256 hidden units for car racing and 512 for the Doom task.
• C, controller. A single linear layer maps $[z_t, h_t]$ to the action. Its few parameters are found by the evolution strategy CMA-ES, which needs only the total reward of each rollout.

V and M are trained on 10,000 rollouts of a random policy, without rewards; only C sees the reward. On the CarRacing-v0 task, which is counted as solved at an average above 900 over 100 random tracks, the full agent scored 906 ± 21. A controller that saw only $z_t$, without M’s hidden state, scored 632 ± 251, and 788 ± 141 with an added hidden layer. The authors read this as showing that M’s prediction of the future makes the controller’s job easier.

In the VizDoom “Take Cover” task they went further and trained the controller entirely inside M, decoding its imagined latents to frames only for display. For this M also predicts whether the agent dies. The best agent trained in the dream survived 1,092 ± 556 time steps over 100 trials in the real game, where 750 counts as solving the task.

Their account of what went wrong first is the clearest early description of exploitation. The controller sees M’s hidden state, which they compare to having access to the game engine’s memory. In early experiments it found a way of moving that, in some rollouts, kept the dream’s monsters from firing at all. The fix was M’s temperature $\tau$, which scales the randomness of sampling from the mixture. Training the controller in a noisier dream made exploits harder to find (their table):

Temperature $\tau$	Score in the dream	Score in the real game
0.10	2086 ± 140	193 ± 58
0.50	2060 ± 277	196 ± 50
1.00	1145 ± 690	868 ± 511
1.15	918 ± 546	1092 ± 556
1.30	732 ± 269	753 ± 139

At low temperature the dream’s monsters fail to fire, an effect the authors attribute to mode collapse in the mixture. The controller then scores near the maximum in the dream and below a random policy (210 ± 108) in reality. Too much randomness makes the dream too hard to learn in. The authors also note that a VAE trained without the task may encode irrelevant details and miss relevant ones, and propose an iterative procedure in which the agent explores, retrains M on new data and repeats.

PlaNet

PlaNet (Hafner et al., 2019) learns a model from pixels and plans in its latent space, with no policy network at all. It collects an episode with the current model every few update steps, starting from a few random seed episodes. It plans with the cross-entropy method over a horizon of 12 steps, with 1,000 candidates, 10 iterations and the top 100 kept, scoring each candidate by the rewards the model predicts along a single sampled trajectory. Because the reward is predicted from the latent state, planning never generates an image (§2; supplement, Appendix A).

The paper’s experiments isolate the parts of the design:

• Stochastic and deterministic paths. On six DeepMind Control tasks from $64 \times 64$ images, purely deterministic (GRU) and purely stochastic state-space models were both worse than the RSSM. The authors report that the agent does not learn without the stochastic component (§5, Fig. 4). Their figure caption adds that purely deterministic transitions cannot capture several futures and make it easy for the planner to exploit inaccuracies (Fig. 2).
• Multi-step training. The standard bound trains only one-step predictions. Latent overshooting adds KL terms between multi-step predictions and the encoder’s beliefs, in latent space, to train predictions of every distance up to a horizon (§4). The authors found it helps several model types, although the final RSSM agent does not need it (§4).
• Search. A version that picks the best of 1,000 random sequences at each step, without refining them, did worse than iterative CEM on all tasks (§5, Fig. 5).
• Online data. Collecting data by planning, rather than with random actions, helped on all tasks and was necessary on three (§5).

Table 1 compares PlaNet after 1,000 episodes with D4PG after 100,000 and with CEM planning in the true simulator, which the authors take as an estimated upper bound. PlaNet scored 821, 832, 662, 700, 930 and 951 on cartpole swing-up, reacher, cheetah, finger, cup and walker. CEM with the true simulator scored 850, 964, 656, 825, 993 and 994, so the learned model scored between about 3% and 15% below planning with the true dynamics on five tasks, and slightly above it on cheetah (our arithmetic from Table 1).

The Dreamer line

The Dreamer agents keep PlaNet’s RSSM but replace online planning with an actor and a critic trained on imagined rollouts. The first two papers, Dream to Control (Hafner et al., 2019) and DreamerV2 (Hafner et al., 2020), were not read for this article. They are described here through DayDreamer, a paper with Hafner among its authors that applies the algorithm to physical robots and restates it (Wu et al., 2022, §2):

• World model. An encoder fuses all sensor inputs, camera images and joint readings alike, into discrete codes. A dynamics network predicts the next codes from the recurrent state and the action, without seeing the intermediate inputs. A decoder reconstructs the inputs and a reward network predicts rewards. All parts are trained jointly. The authors note that predicting future representations rather than images reduces accumulating errors and allows large batches.
• Behaviour. An actor $\pi(a_t \mid s_t)$ and a critic $v(s_t)$ are trained only on rollouts imagined in latent space, with batches of 16K on one GPU and an imagination horizon of $H = 16$. The critic regresses $\lambda$-returns, $V^\lambda_t = r_t + \gamma\big((1-\lambda)\, v(s_{t+1}) + \lambda V^\lambda_{t+1}\big)$, so that rewards beyond the horizon count. The actor maximizes them, with gradients through the model for continuous actions and REINFORCE for discrete ones.
• No feedback into the model. The actor’s and critic’s gradients do not change the world model, which, the authors write, would make its predictions incorrect and overly optimistic.

With one set of hyperparameters, built on the DreamerV2 code, DayDreamer learned directly in the real world, without simulators (§3):

• A Unitree A1 quadruped learned to roll off its back, stand up and walk in about an hour, and adapted to being pushed within 10 minutes more. SAC learned only to roll over in the same budget.
• On visual pick-and-place from sparse rewards, a UR5 arm reached 2.5 objects per minute within 8 hours, approaching human teleoperation, while Rainbow DQN and PPO learned only to grasp and drop objects in the same bin. A low-cost XArm reached 3.1 objects per minute in 10 hours.
• A Sphero robot learned to navigate to a goal from images within 2 hours. Here the model-free DrQv2 performed similarly.

DreamerV3 was published in Nature in 2025; that site is not used for this article, and the paper was not read. The README of Hafner’s reimplementation describes a world model that encodes sensory inputs into categorical representations and predicts future representations and rewards given actions, used to train an actor–critic policy on imagined trajectories. The README states that one fixed set of hyperparameters works across a wide range of domains, and that larger models increased both final performance and data efficiency (danijar/dreamerv3 README at commit e01491f, as of November 2026). Drive-WM’s authors summarize DreamerV3 as learning to obtain diamonds in Minecraft from scratch with sparse rewards (Wang et al., 2024, §2.2).

Task-oriented latent models

Not every latent model reconstructs its inputs. TD-MPC (Hansen, Wang & Su, 2022) learns a “task-oriented” latent dynamics model together with reward and value functions by temporal-difference learning, trained with a latent consistency loss rather than the reconstruction objective of PlaNet and Dreamer, against which the authors ablate it (§4, §5). It plans with MPPI, a sampling-based MPC method, over a short horizon of 5 and adds a learned terminal value for the return beyond it (§3, §5). The authors compare with MPC using the ground-truth simulator, which, in their account, performs well on locomotion tasks but fails on tasks with sparse rewards (§5). A short horizon with a value function avoids long rollouts, so it limits compounding error, at the price of trusting a learned value.

Generative Interactive Environments

The models above are trained on an agent’s own experience, with actions recorded. Genie (Bruce et al., 2024) learns a controllable environment from Internet videos that have no action labels. Its authors present it as the first generative interactive environment trained without supervision from unlabelled Internet videos (abstract). Given an image, which may be a photograph, a sketch or the output of a text-to-image model, it generates a playable world one frame at a time.

Genie has three components, all built on a spatiotemporal transformer whose blocks interleave spatial attention within a frame and causal temporal attention across frames (§2):

• A video tokenizer, a VQ-VAE with spatiotemporal attention that turns frames into discrete tokens, with a codebook of 1,024 codes (§2.1, §3).
• A latent action model. An encoder sees the past frames and the next frame and outputs a latent action; a lightweight decoder must reconstruct the next frame from the past frames and that action alone. A VQ codebook limits the actions to $|A| = 8$ discrete codes, so that the action carries only the most meaningful change between frames and a person can play with it. At inference only the codebook is kept and the user chooses the action (§2.1).
• A dynamics model, a decoder-only MaskGIT transformer that predicts the next frame’s tokens from past tokens and latent actions, trained with a masked cross-entropy loss (§2.1).

The main model was trained on 30,000 hours of 2D platformer gameplay, filtered from 55 million 16-second clips at 10 frames per second and $160 \times 90$ resolution. Its dynamics model has 10.1 billion parameters, 10.7 billion with the tokenizer and action model (§3); the abstract and introduction give 11 billion. The authors found that each of the eight latent actions kept a consistent meaning across different starting images, so learning them is “akin to learning the buttons on a new controller” (§2.2).

To measure controllability the authors introduced $\Delta_t\text{PSNR}$: the PSNR of a frame generated with latent actions inferred from the real video, minus the PSNR of a frame generated with random latent actions, both against the real frame, at $t = 4$ (§3). A large value means the actions change the outcome. In the ablations of Table 2, a latent action model fed pixels gave higher $\Delta_t\text{PSNR}$ than one fed tokens, on platformers (1.91 against 1.33) and on robot videos (2.07 against 1.65).

Two further results bear on agents. A model trained on action-free videos from the RT-1 robot datasets learned consistent latent actions for arm motion and learned to simulate deformable objects such as bags of chips (§3.2). The paper gives this model’s size as 1.3 billion parameters in one paragraph and 2.5 billion in the next, so the text is inconsistent. And a frozen latent action model labelled expert videos of an unseen game, CoinRun. A policy trained on those labels, with a small set of real actions to map latent actions to real ones, matched an oracle behavioural cloning model given as few as 200 expert samples (§3.3). The authors list the limits: the model can hallucinate unrealistic futures, remembers only 16 frames, and runs at about one frame per second (§5).

Video Generators as Simulators

Large video generators can be prompted with a scene and continue it plausibly. Kang et al. (2025, §1) note that this has sparked new interest in building world models, citing OpenAI’s Sora report (Brooks et al., 2024), which frames video generators as world simulators in its title. The report was not read for this article, because its host is not used; the Video Generation article summarizes it, including the physical failures it lists. Earlier, Video LDM’s authors trained their latent video diffusion model on real driving videos and described it as having great potential as a simulation engine for autonomous driving. They generated different plausible futures from the same initial frame, and started scenarios from a scene laid out with car bounding boxes (Blattmann et al., 2023, §1, §4.1.2, Fig. 7).

Plausible continuation is not the same as correct simulation, and Kang et al. (2025) tested the difference under controlled conditions. They rendered 2D videos of simple shapes governed by classical mechanics, such as uniform motion, elastic collisions and parabolic motion, and trained diffusion transformers with a frozen video VAE to continue the videos from their first frames (§2). Because the simulator is known, they could recover each object’s position and velocity from the generated pixels and compare them with the true laws. Their findings:

• In distribution, error fell with scale. On uniform motion the velocity error went from 0.022 for the smallest model with 30,000 videos to 0.012 for the largest with 3 million, against 0.010 for the ground-truth videos passed through the same measurement (§3.2).
• Out of distribution, with velocities outside the training range, errors were an order of magnitude higher and did not improve with data or model size. For the largest model with 3 million videos, the out-of-distribution velocity error was 0.427 against 0.012 in distribution (§3.2).
• In combinations of objects and interactions not seen together in training, raising the number of training combinations from 6 to 60 cut the share of the largest model’s unseen-combination videos that human raters judged physically abnormal from 67% to 10% (§4.2, Table 2).
• The mechanism, in the authors’ analysis, is “case-based” generalization: the models imitate the closest training example rather than abstracting a rule, and when matching they weigh colour over size, size over velocity and velocity over shape (abstract).

The authors conclude that scaling alone is insufficient for video generation models to uncover fundamental physical laws (abstract). For world models, this means a generator’s visual quality, as measured by the metrics in the Video Generation article, says little about whether its dynamics are right outside its data.

Driving World Models

Driving offers large amounts of video with the ego vehicle’s motion recorded, and a reason to predict: a planner that can foresee the outcome of a manoeuvre can reject unsafe ones. The papers below were read for this article, except GAIA-1. Genie’s authors describe GAIA-1 (Hu et al., 2023) and UniSim (Yang et al., 2023), which were not read, as scaled world models for driving and robotic manipulation that need both text and action labels (Bruce et al., 2024, §4).

• GenAD (Yang et al., 2024) is a latent diffusion video predictor trained on OpenDV-2K: 2,059 hours of driving video paired with text, 1,747 hours of it from YouTube and 312 from seven public datasets, covering more than 40 countries (§2). For the YouTube videos the authors labelled driving commands with a video classifier and described frames with BLIP-2; for the public datasets they refined the metadata with GPT and derived commands from each clip’s logged trajectory (§2.2). The authors report zero-shot generalization to unseen driving datasets, and adapt the model into an action-conditioned predictor by fine-tuning it with future ego trajectories, and into a planner with a lightweight head that maps its features to a trajectory (abstract, §3.3).
• Drive-WM (Wang et al., 2024) generates consistent multiview videos around the car with a latent video diffusion model, factorizing the views so that some are generated conditioned on their neighbours (§1, §3). It plans by imagination: for candidate trajectories sampled from a planner it generates the futures, scores them with a reward computed from perception results on the generated frames, combining a map reward (distance from the curb, staying near the lane centre) and an object reward (distance from other road users), keeps the best and repeats, building a planning tree (§4). On nuScenes, choosing among three driving commands this way performed close to using the ground-truth command (§5.4, Table 3). The authors write that the combined reward outperforms each part, particularly on collisions; in their Table 4 it equals the object reward alone in average L2 error (0.80 m) and is slightly lower in average collision rate (0.26% against 0.27%).
• OccWorld (Zheng et al., 2024) predicts not images but 3D semantic occupancy, the grid representation of occupancy prediction. A VQ-VAE turns the occupancy of each frame into scene tokens, and a GPT-like spatial–temporal transformer predicts the next scene tokens together with ego-motion tokens, which a decoder turns into the planned trajectory. The authors argue that occupancy is more expressive than boxes, cheaper to obtain from sparse LiDAR, and usable with cameras or LiDAR. On nuScenes, with ground-truth 3D occupancy as input and 2 s of history, it forecast 3 s of occupancy with an average IoU of 26.63 and mIoU of 17.14 over 1–3 s (Table 1; the introduction gives 17.13), and planned with an average L2 error of 1.17 m and a collision rate of 0.60% (Table 2), without instance or map supervision (§1, §4).

These models make prediction and planning one step, as the planners of end-to-end autonomous driving do, whereas the perception, prediction and planning stack of autonomous driving perception keeps them separate. The task of forecasting other agents’ trajectories from tracked positions is described in motion forecasting.

Robotics World Models

Visual model-predictive control (Ebert et al., 2017) plans pushing actions with a video predictor that moves pixels, searching with the cross-entropy method and replanning at every step (§3.1); the Video Prediction article describes it.

World models offer robots several things, which DayDreamer’s authors list: planning and behaviour learning from small amounts of real interaction, dynamics knowledge reusable across tasks, representations that fuse several sensors, and generalization from offline data (Wu et al., 2022, §1). They contrast learning in the real world with learning in simulators, which can fail to capture the complexity of the real world and may not adapt to changes in it (abstract). A learned model is also a form of simulator, with its own reality gap. Sim-to-real transfer describes the gap for engineered simulators, and the policies that world models train are the subject of visuomotor policy learning. Vision–language–action models, in contrast, map images and instructions to actions directly, without an explicit model of the dynamics.

Properties

What is predicted. Ha and Schmidhuber’s M predicts latents, and pixels are decoded only to look at them. PlaNet and Dreamer predict latents, rewards and, as a training signal, reconstructions, and plan or imagine without decoding images. TD-MPC drops reconstruction altogether. Genie, GenAD and Drive-WM predict frames, in a token or latent space. OccWorld predicts occupancy. Predicting less detail costs less and leaves fewer places for error to grow. Pixels keep the predictions inspectable, and allow rewards that read the generated image, as in Drive-WM.

Uncertainty. A deterministic model that must predict several possible futures averages them, as video prediction explains for pixels. Both Ha and Schmidhuber and PlaNet’s authors found stochastic transitions important for control, and both connect it to exploitation: a deterministic model is easier for the agent to exploit (Ha & Schmidhuber, 2018; Hafner et al., 2019, Fig. 2).

Actions. A world model needs actions in its training data. Agents record their own; Internet video has none. Genie infers discrete latent actions instead; GenAD and Drive-WM use the recorded ego trajectory or driving commands.

Data distribution. A model is accurate where its data are. Ha and Schmidhuber’s model was trained on random play; PlaNet showed that collecting data with the current planner helped on every task (Hafner et al., 2019, Fig. 5). An agent that leaves its data distribution is where Ha and Schmidhuber’s exploit came from: as they explain, the policy visits states where the model is wrong because those states lie away from the training distribution.

Horizon. Every method limits how far it trusts the model: PlaNet plans 12 steps ahead, DayDreamer imagines 16 steps and adds a critic, TD-MPC plans 5 steps with a terminal value, and all replan or retrain as real data arrive.

Examples

Example: two closed-form models of a sprite world

The script builds a small world whose true dynamics are known, fits two action-conditioned models to its recorded transitions in closed form, without gradient descent, and asks the two questions this article has raised: does open-loop error grow with the horizon, and does planning inside the model do as well as planning in the true environment?

The world. Each observation is a $12 \times 12$ grayscale image: background 0, a static wall bar in row 6 across columns 2 to 9 (value 0.5), and a $2 \times 2$ sprite (value 1). Rows count downward and columns rightward. The sprite’s position is the (row, column) of its top-left cell, and a position is valid when no sprite cell lies on the wall, which leaves 103 positions. There are five actions: stay, up, down, left and right, one pixel per step. A move that would leave the image or overlap the wall is blocked, and the sprite stays where it is, so the world is deterministic. The sprite can pass the wall only at the left or right edge.

The data. 200 episodes of 50 uniformly random actions from random valid starts give 10,000 transitions, as Ha and Schmidhuber trained their model on random play.

The models. Both predict the next image from the current image and action.

• Least squares fits one linear map per action, $x' = W_a [x; 1]$, from the 144 pixels plus a constant to the 144 next pixels. np.linalg.lstsq returns the minimum-norm solution, which is deterministic.
• Counts keeps a table of observed images: for each (image, action) pair it predicts the next image most often observed after it. A pair never observed predicts the image unchanged.

The measures.

• Open-loop error. From 500 random starts, 16 random actions are applied. Each model is fed its own previous prediction, and its frames are compared with the true ones at horizons 1 to 16. The script reports the per-pixel mean squared error and the distance, in pixels, between the predicted and true sprite centroids. A centroid is read from a frame as the intensity-weighted mean pixel position of the frame minus the static wall, keeping positive values. Pixel indices serve as coordinates, so a true sprite at $(r, c)$ has its centroid at $(r + 0.5, c + 0.5)$.
• Planning. Each of 100 tasks gives a start and a goal at least 6 moves apart along the shortest valid path. At every step, a random-shooting planner samples 200 sequences of 8 actions and rolls each out in its model from the current observation. The cost of a sequence is the sum over the 8 predicted frames of the centroid’s distance to the goal’s centroid, with 100 per step if a predicted frame has no sprite left. The planner executes the first action of the cheapest sequence in the true environment and replans, for at most 40 steps. A task succeeds if the sprite reaches the goal exactly. The same random candidates are drawn for every planner on a given task, so the planners differ only in their model. The third planner uses the true environment as its model, which is the reference, as CEM with the true simulator is in PlaNet’s Table 1.
• Optimism. For every decision the script also records the cost that the planner’s model predicted for the chosen sequence, and the cost of the same sequence executed in the true environment. Equal values mean the planner’s beliefs held up; a predicted cost below the true one means the planner chose a sequence whose good score came from the model’s error, the exploitation described above.

Everything comes from seeded generators. The script loops over seeds 0, 1 and 2, each seeding its own data, test rollouts and tasks, and prints the mean and range over the three. It uses NumPy 2.3.5 and ran in 27 to 33 s in four runs on an Apple laptop CPU. The set-up shown was fixed before any pilot run and is used here unchanged; before the run shown, the pilot’s code was rewritten in vectorized form and checked to give identical results on seed 0.

import numpy as np  # NumPy 2.3.5
from collections import deque

S, K = 12, 2                                   # image size, sprite size
WALL = np.zeros((S, S)); WALL[6, 2:10] = 0.5   # static wall bar
MOVES = np.array([[0, 0], [-1, 0], [1, 0], [0, -1], [0, 1]])  # stay, up, down, left, right
VALID = np.array([[not WALL[r:r + K, c:c + K].any() for c in range(S - K + 1)]
                  for r in range(S - K + 1)])  # top-left sprite cells that avoid the wall
YY, XX = np.mgrid[0:S, 0:S]


def render(pos):                               # positions (n, 2) -> images (n, 144)
    img = np.repeat(WALL[None], len(pos), 0)
    for dr in range(K):
        for dc in range(K):
            img[np.arange(len(pos)), pos[:, 0] + dr, pos[:, 1] + dc] = 1.0
    return img.reshape(len(pos), -1)


def step(pos, a):                              # true dynamics: blocked moves stay put
    new = pos + MOVES[a]
    ok = (new >= 0).all(1) & (new <= S - K).all(1)
    ok[ok] = VALID[new[ok, 0], new[ok, 1]]
    return np.where(ok[:, None], new, pos)


def centroid(x):                               # sprite centroid read from (predicted) frames
    s = np.clip(x.reshape(-1, S, S) - WALL, 0, None)
    m = s.sum((1, 2))
    w = np.maximum(m, 1e-12)
    return np.stack([(s * YY).sum((1, 2)) / w, (s * XX).sum((1, 2)) / w], 1), m


def random_valid(rng, n):
    cells = np.argwhere(VALID)
    return cells[rng.integers(len(cells), size=n)]


def collect(rng, episodes=200, length=50):     # random actions from random starts
    P, A, Q = [], [], []
    for _ in range(episodes):
        pos = random_valid(rng, 1)
        for _ in range(length):
            a = rng.integers(5, size=1)
            nxt = step(pos, a)
            P.append(pos[0]); A.append(a[0]); Q.append(nxt[0]); pos = nxt
    return np.array(P), np.array(A), np.array(Q)


class TrueEnv:                                 # the simulator itself, as a "model"
    def start(self, x):
        on = x.reshape(-1, S, S) == 1.0
        return np.stack([on.any(2).argmax(1), on.any(1).argmax(1)], 1)
    def step(self, p, a): return step(p, a)
    def image(self, p): return render(p)


class PixelLS:                                 # x' = W_a [x; 1]: a free weight for every pixel pair
    def __init__(self, X, A, Y):
        Xb = np.hstack([X, np.ones((len(X), 1))])
        self.W = [np.linalg.lstsq(Xb[A == a], Y[A == a], rcond=None)[0] for a in range(5)]
    def start(self, x): return x
    def step(self, x, a):
        xb, out = np.hstack([x, np.ones((len(x), 1))]), np.empty_like(x)
        for k in range(5):
            out[a == k] = xb[a == k] @ self.W[k]
        return out
    def image(self, x): return x


class Counts:                                  # table: observed (image, action) -> next image
    def __init__(self, X, A, Y):
        self.imgs, inv = np.unique(np.vstack([X, Y]), axis=0, return_inverse=True)
        self.ids = {img.tobytes(): i for i, img in enumerate(self.imgs)}
        i, j, n = inv[:len(X)], inv[len(X):], len(self.imgs)
        tally = np.zeros((n, 5, n), int)
        np.add.at(tally, (i, A, j), 1)
        self.seen = tally.sum(2) > 0
        self.T = np.where(self.seen, tally.argmax(2), np.arange(n)[:, None])  # unseen: stay
    def start(self, x):                        # an image never observed gets a new row that stays
        out = []
        for img in x:
            key = img.tobytes()
            if key not in self.ids:
                self.ids[key] = len(self.imgs)
                self.imgs = np.vstack([self.imgs, img])
                self.T = np.vstack([self.T, np.full(5, len(self.imgs) - 1)])
            out.append(self.ids[key])
        return np.array(out)
    def step(self, s, a): return self.T[s, a]
    def image(self, s): return self.imgs[s]


def rollout_cost(model, pos, seqs, goal):      # sum over the horizon of predicted distance to goal
    s = model.start(render(np.repeat(pos[None], len(seqs), 0)))
    cost = np.zeros(len(seqs))
    for h in range(seqs.shape[1]):
        s = model.step(s, seqs[:, h])
        c, m = centroid(model.image(s))
        cost += np.where(m > 1e-6, np.linalg.norm(c - (goal + 0.5), axis=1), 100.0)
    return cost


def plan_episode(model, start, goal, rng, N=200, H=8, max_steps=40):
    """Random-shooting MPC; also returns the chosen plans' predicted and true costs."""
    pos, pred, true = start.copy(), [], []
    for t in range(max_steps):
        if (pos == goal).all():
            break
        seqs = rng.integers(5, size=(N, H))
        cost = rollout_cost(model, pos, seqs, goal)
        best = seqs[np.argmin(cost)][None]
        pred.append(cost.min()); true.append(rollout_cost(TrueEnv(), pos, best, goal)[0])
        pos = step(pos[None], best[:, 0])[0]
    return bool((pos == goal).all()), pred, true


def geodesic(start):                           # breadth-first search over valid positions
    dist = -np.ones(VALID.shape, int); dist[tuple(start)] = 0
    q = deque([tuple(start)])
    while q:
        r, c = q.popleft()
        for a in range(1, 5):
            nr, nc = step(np.array([[r, c]]), np.array([a]))[0]
            if dist[nr, nc] < 0:
                dist[nr, nc] = dist[r, c] + 1; q.append((nr, nc))
    return dist


def make_tasks(rng, n=100, min_dist=6):
    out = []
    while len(out) < n:
        s, g = random_valid(rng, 2)
        if geodesic(s)[tuple(g)] >= min_dist:
            out.append((s, g))
    return out


def mmr(v):                                    # mean (min-max) over seeds
    v = np.array(v)
    return "%.4f (%.4f-%.4f)" % (v.mean(), v.min(), v.max()) if v.max() < 1 else "%.2f (%.2f-%.2f)" % (v.mean(), v.min(), v.max())


H_SHOW = [1, 2, 4, 8, 16]
names = ["true env", "counts", "least squares"]
cover, blocked = [], []
mse = {m: {h: [] for h in H_SHOW} for m in names[1:]}
cen = {m: {h: [] for h in H_SHOW} for m in names[1:]}
succ, pred, true = ({m: [] for m in names} for _ in range(3))
for seed in [0, 1, 2]:
    rng_data, rng_test, rng_plan = (np.random.default_rng(100 * seed + k) for k in range(3))
    P, A, Q = collect(rng_data)
    X, Y = render(P), render(Q)
    models = {"true env": TrueEnv(), "counts": Counts(X, A, Y), "least squares": PixelLS(X, A, Y)}
    cover.append(100 * models["counts"].seen.sum() / (5 * VALID.sum()))
    blocked.append(100 * ((A > 0) & (P == Q).all(1)).sum() / (A > 0).sum())

    # open-loop prediction: 500 random starts, 16 random actions, models fed their own output
    p = random_valid(rng_test, 500); acts = rng_test.integers(5, size=(500, 16))
    st = {m: models[m].start(render(p)) for m in names[1:]}
    for h in range(1, 17):
        p = step(p, acts[:, h - 1]); xt = render(p); ct, _ = centroid(xt)
        for m in names[1:]:
            st[m] = models[m].step(st[m], acts[:, h - 1])
            if h in H_SHOW:
                x = models[m].image(st[m]); c, _ = centroid(x)
                mse[m][h].append(((x - xt) ** 2).mean())
                cen[m][h].append(np.linalg.norm(c - ct, axis=1).mean())

    # planning by random shooting in each model, executed in the true environment
    tasks = make_tasks(rng_plan)
    task_seeds = rng_plan.integers(2**32, size=len(tasks))
    for m in names:
        ok, pc, tc = [], [], []
        for (s, g), sd in zip(tasks, task_seeds):
            o, p_, t_ = plan_episode(models[m], s, g, np.random.default_rng(sd))
            ok.append(o); pc += p_; tc += t_
        succ[m].append(100 * np.mean(ok)); pred[m].append(np.mean(pc)); true[m].append(np.mean(tc))

print("data: 10000 transitions per seed; (state, action) pairs seen %s%%; moves blocked %s%%"
      % (mmr(cover), mmr(blocked)))
print("\nopen-loop error, mean (min-max) over seeds 0-2")
print("%3s  %-27s %-27s %-23s %-23s" % ("h", "pixel MSE, least squares", "pixel MSE, counts", "centroid px, l. sq.", "centroid px, counts"))
for h in H_SHOW:
    print("%3d  %-27s %-27s %-23s %-23s" % (h, mmr(mse["least squares"][h]), mmr(mse["counts"][h]),
                                         mmr(cen["least squares"][h]), mmr(cen["counts"][h])))
print("\nrandom-shooting planning, 100 tasks per seed, mean (min-max) over seeds 0-2")
print("%-14s %-22s %-22s %-22s" % ("model", "success %", "predicted plan cost", "true plan cost"))
for m in names:
    print("%-14s %-22s %-22s %-22s" % (m, mmr(succ[m]), mmr(pred[m]), mmr(true[m])))

Output (two runs gave identical output):

data: 10000 transitions per seed; (state, action) pairs seen 100.00 (100.00-100.00)%; moves blocked 15.39 (14.30-16.26)%

open-loop error, mean (min-max) over seeds 0-2
  h  pixel MSE, least squares    pixel MSE, counts           centroid px, l. sq.     centroid px, counts
  1  0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)  0.0000 (0.0000-0.0000)
  2  0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)  0.0000 (0.0000-0.0000)
  4  0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)  0.0000 (0.0000-0.0000)
  8  0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)  0.0000 (0.0000-0.0000)
 16  0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)      0.0000 (0.0000-0.0000)  0.0000 (0.0000-0.0000)

random-shooting planning, 100 tasks per seed, mean (min-max) over seeds 0-2
model          success %              predicted plan cost    true plan cost
true env       81.00 (77.00-85.00)    27.72 (26.92-28.79)    27.72 (26.92-28.79)
counts         81.00 (77.00-85.00)    27.72 (26.92-28.79)    27.72 (26.92-28.79)
least squares  81.00 (77.00-85.00)    27.89 (27.08-28.82)    27.89 (27.08-28.82)

The results do not show compounding error or a planning gap. We planned the example expecting the least-squares model’s error to grow with the horizon and its planner to do worse than the true environment’s, and to be optimistic. As printed, neither happens: both models’ errors are zero to the printed precision at every horizon, and every planner’s predicted costs equal the true costs.

For the counts model this is expected from the data line. Ten thousand random transitions visited every one of the 515 (position, action) pairs, so its table holds the true next image for every image the world can produce, and it is the simulator.

For the least-squares model the reason lies in the size of the world, and it can be checked from the construction (our reasoning). Order the 103 valid positions by row and then column. The sprite at position $(r, c)$ lights pixel $(r, c)$, and no position later in that order lights it, because a later sprite lies either in a lower row or further right in the same row. The wall is zero on these pixels, so the matrix of images restricted to them is triangular with ones on its diagonal, and the 103 images, with or without the constant feature, are linearly independent. For each action, a linear map from the 145-dimensional $[x; 1]$ can then send each of the 103 images to its successor exactly, and least squares finds such a map with zero residual on the data. Because the data contain every image with every action (the data line), that map is exact on every image the world can produce. The linear model, with a free weight for every pair of pixels, has memorized a lookup table of this world. In our reading, a model that memorizes a world this small cannot show compounding error: it has no wrong prediction to compound, and its planner has no error to exploit.

The planners succeed on 81% of tasks on average, the same for all three, because the models agree with the simulator. The failures belong to the planner, not the model. In our reading, a cost that measures straight-line distance to the goal has local minima behind the wall, and an 8-step random search sometimes cannot see far enough to leave them. The least-squares planner’s mean plan cost differs slightly from the exact planners’, so it sometimes chose different sequences, but its predicted costs still matched the true ones. Our reading is that its predicted frames differ from the exact ones by floating-point rounding, which can reorder candidate sequences whose costs are otherwise tied.

What the example does show is the condition for the problems described earlier to appear: a model must be wrong somewhere the agent goes. Here the model had seen every situation, and its class could represent the world exactly. Real environments have far more states than a table can hold, and the models described in this article must predict situations they have not seen. The accumulating multi-step errors that PlaNet’s authors list, and the exploits that Ha and Schmidhuber’s controller found in states away from its model’s training data, arise there.

Evaluation

World models are judged at three levels, and good results at one need not carry over to another.

Prediction quality. Generated frames are compared with real ones by per-frame PSNR, SSIM and LPIPS, and by distribution-level measures such as FVD. Video Prediction and Video Generation describe these measures, their best-of-K protocols and their biases. Genie, for example, reports FVD on its robotics test split (Bruce et al., 2024, §3.2), and Drive-WM reports FID and FVD of its multiview videos on nuScenes (Wang et al., 2024, §5).

Controllability and physical correctness. A world model must respond to actions and get the dynamics right, which visual quality does not test. Genie’s $\Delta_t\text{PSNR}$ measures how much the latent actions change the generated frames (§3). Kang et al. (2025) recover object states from generated videos and compare them with the physical laws that produced the training data, separately within the training distribution, outside it, and in new combinations (§2.3). Human ratings of whether generated videos look physically plausible complement such measures (§4).

Usefulness for decisions. The end test is how well an agent does with the model:

• the return of an agent that plans or learns in the model, often compared with planning in the true simulator, as in PlaNet’s Table 1 and TD-MPC’s MPC:sim baseline;
• real-world success after training in the model, such as DayDreamer’s walking time and pick rates, and the gap between scores in the dream and in reality in Ha and Schmidhuber’s temperature table;
• for driving, the L2 distance between planned and logged trajectories and the collision rate on nuScenes, as OccWorld and Drive-WM report.

Rewards and success rates depend on the agent and the planner as well as on the model, so comparisons of models hold the rest fixed, as PlaNet’s ablations and the example above do.

Common Misconceptions

“A video generator is a world model.” A world model’s predictions are conditioned on actions and used to decide. A generator conditioned on text or past frames predicts what may happen, not what will happen if the agent acts. Genie’s authors draw this line (Bruce et al., 2024, Table 1), and generators gain action inputs, as Genie and the driving models do, before they are used as world models.

“Realistic video means correct dynamics.” Kang et al. (2025) found near-perfect in-distribution prediction alongside failure out of distribution, with models imitating similar training cases rather than applying the law. Visual quality is measured on data like the training data. An agent’s decisions often take it elsewhere.

“A model that fits its data will train a policy that works in reality.” Ha and Schmidhuber’s dream-trained controller scored highest in the dream and below a random policy in reality at low temperature. Optimization searches for whatever the model gets wrong, and the states it reaches lie outside the data the model was checked on.

“Zero training error means a good world model.” In the example, the least-squares model has zero error because it has memorized every state of a tiny world. Nothing in the fit makes the same model class right about a state it has never seen (our reasoning).

Open Problems

• Long horizons and memory. Genie remembers 16 frames, which makes long-horizon consistency hard (Bruce et al., 2024, §5). Latent agents bound their imagination horizon and lean on value functions beyond it (Wu et al., 2022; Hansen, Wang & Su, 2022).
• Physical correctness beyond the data. Scaling improved in-distribution and combinatorial prediction but not extrapolation in Kang et al.’s tests (2025). How to make generated dynamics obey laws in situations the data do not cover is open.
• Exploitation and uncertainty. Stochastic transitions, temperature and replanning reduce exploitation without removing it (Ha & Schmidhuber, 2018; Hafner et al., 2019). Knowing where a model is wrong, so that an agent can avoid or explore those regions, remains a central problem.
• Actions from video. Most video has no action labels. Genie’s latent actions are one answer, learned from eight codes in 2D games and robot video. Whether such actions scale to rich, continuous control is open.
• Speed. Genie runs at about one frame per second, too slow for interactive use, as its authors note (§5). Planning by generating video, as Drive-WM does, multiplies the cost by the number of candidates.
• Evaluation. Prediction metrics, controllability and physical tests, and downstream success measure different things, and the largest models are evaluated mainly by their developers. Driving planners are scored by their distance to the trajectory recorded in the log.
• Openness. Genie’s authors chose not to release its checkpoints, its training dataset or examples from it, citing responsible release (Bruce et al., 2024, Impact Statement). Generative world models trained on Internet video raise the data questions that Video Generation describes.

Where It Is Used

• Model-based reinforcement learning, in simulation and on real robots: Ha and Schmidhuber’s agents, PlaNet, the Dreamer line and TD-MPC, and the agents of embodied AI.
• Robot learning and manipulation: visual model-predictive control for pushing (Ebert et al., 2017), DayDreamer’s online learning, and action-free robot video in Genie, as alternatives or complements to the policies of visuomotor policy learning and to engineered simulators.
• Autonomous driving: video and occupancy world models for prediction, simulation and planning (GenAD, Drive-WM, OccWorld), next to the modular stack of autonomous driving perception.
• Interactive content and agent training environments: playable worlds generated from images (Genie), which their authors present as a possible source of environments for training generalist agents (Bruce et al., 2024, §3.3, §5).
• Simulation from video generators: driving-scene simulation with Video LDM, and the broader claim that video generators can serve as world simulators, which Kang et al. (2025) test.