Reading World Models Through 'LLMs Can't Jump'

I started reading about world models after coming across a question in Position: LLMs Can’t Jump. Tom Zahavy asks whether a modern AI, restricted to the scientific knowledge available to Einstein, could have invented General Relativity.

Zahavy focuses on a missing form of inference. Language models can reason and help with science. Induction and deduction leave out the move that creates a new explanatory premise.

The paper calls that move the “jump.” I used it to frame the rest of my reading: what would an AI need to form, test, and revise an explanation about the physical world?

The missing inference is abduction

The paper separates three kinds of inference. Deduction applies a known rule to a case. Induction observes cases and results, then infers a rule that fits them. Abduction starts with a surprising result and proposes a case or rule that could explain it.

The paper places the creative jump in abduction: proposing the explanation that the other two modes can test or apply.

In Zahavy’s account of General Relativity, Newtonian gravity worked well, and the observations that later supported Einstein’s theory were not yet available. An induction system looking for statistical error would have found little reason to replace the existing theory. A deduction system could derive consequences once someone supplied the right premises. It still needs those premises from somewhere else.

Einstein took a different route with his elevator thought experiment. Imagine an observer inside a sealed elevator accelerating through empty space. Released objects appear to fall toward the floor with the same acceleration. The observer experiences that situation and gravity in the same way. Einstein used the imagined situation to connect acceleration and gravity through the equivalence principle.

The paper describes this as manipulative abduction. The thinker changes a simulated situation, observes the consequence, then proposes an explanation. The simulation gives the hypothesis a physical result to explain.

A world model needs a loop

People use “world model” for several kinds of system. A video generator, a learned dynamics model, and a persistent 3D scene can all receive the label while producing very different outputs.

The common thread I found was a loop. An agent receives an observation, forms an internal state, considers an action, predicts its consequences, acts, and then receives a new observation. That result can support the prediction or expose an error in the model.

The loop matters more than a single prediction. An agent needs to intervene, compare the result with its expectation, and update what it believes.

To turn this loop into a synthetic laboratory, an agent needs sensory input and stable counterfactuals it can control. If it cannot choose an action, it cannot run the equivalent of cutting the elevator cable. If the simulation changes its physics just to produce a plausible frame, the result cannot test a new hypothesis.

World Labs’ functional taxonomy separates three outputs in this loop: renderers produce observations, simulators produce state, and planners produce actions. The categories overlap, but I still find the contracts useful. A system can serve one contract while leaving the others open.

Three routes toward a world model

I grouped the work I read into three routes. Each keeps a different part of the world available for prediction or control.

The three routes overlap, but their strongest evidence comes from different tasks.

Latent dynamics: keep what planning needs

The latent route compresses observations into a representation, then predicts how that representation changes. A model on this route can leave out texture or other details when they do not help the task. World Models and the Dreamer family use learned dynamics so an agent can train or plan inside compact imagined trajectories.

V-JEPA 2 connects this idea to video and robot control. Its action-free training stage learns from video. An action-conditioned stage then gives its predictor robot data so it can score candidate moves against a visual goal. Meta reports demonstrations on short robot tasks such as reaching, picking, and placing.

Of the three, this route makes the planning loop easiest for me to see. That compression also creates a risk. A representation may discard an object’s identity, a contact relationship, or uncertainty that becomes important during a future step. A planner may exploit errors in the learned model and choose actions that work in imagination but fail in the environment.

Generative video: make the future visible

Generative video systems produce the predicted observation itself. DeepMind says Genie 3 turns a text prompt into an environment that a user can navigate in real time. The Cosmos repository provides models and workflows for video generation, action-conditioned prediction, and physical AI.

They make counterfactual futures visible and let users act. Visual continuity can coexist with an inconsistent state. A few minutes of coherent video do not prove that object identity, collision, and force follow one causal model. A renderer can produce the expected falling apple because that image sequence is common, while lacking a representation of gravity that an agent can test.

Spatial models: keep a world to return to

Spatial models build persistent geometry that stays available across frames. World Labs says Marble can generate explorable 3D worlds from text, images, or video and export assets such as Gaussian splats and meshes.

That persistence helps with viewpoint changes, navigation, and downstream tools. Geometry covers structure, but a simulator or learned dynamics model still has to provide friction, deformation, material properties, and responses to action.

The synthetic laboratory is still a research proposal

Together, the three routes cover pieces of the paper’s synthetic laboratory: latent models support planning, video models provide observable futures, and spatial models preserve geometry. Putting them together still leaves the abductive jump as a research hypothesis.

To test physical explanations in such a system, I would look for at least four things:

1. The agent can intervene.
2. The model keeps its state consistent across time and hidden views.
3. The model reports uncertainty, and its errors show up in the results.
4. New evidence changes the hypothesis and guides the next action.

The examples I found are narrower: robots complete short manipulation tasks, generated worlds stay navigable for a few minutes, and 3D scenes remain explorable from new viewpoints. These results cover parts of the loop, but none shows an agent inventing a new physical principle from an unexplained observation.

The paper presents a theoretical framework and asks future studies to test it. Its case study concerns physical science, where sensory grounding has an obvious meaning. Mathematics and computer science may require different forms of simulation and different priors. Even in physics, an interactive world model could make experiments possible. The agent would need taste, goals, or prior beliefs to decide which strange result deserves an explanation.

What I took from the reading

What stayed with me from the reading was the loop, more than the label. I want to know whether a system can keep track of relevant state, choose an action, predict its consequence, and revise its model when the result differs. A convincing image can show that it renders an observation, but it does not tell me whether it can use that observation to update its model.

The “jump” also gives me a way to think about claims about automated science. Better induction can find patterns, and better deduction can extend a theory. Scientific invention asks a system to propose the premise worth testing. A world model might provide the laboratory for that step, but the evidence I read only covers pieces of the loop. The broader act of discovery remains a proposal.

I am still unsure what these systems can do in practice. Most of what I read described what a world model would need, rather than what happens when those pieces have to work together. An agent would need to choose an intervention, handle an unexpected result, and revise its model. That still feels closer to a research proposal than a demonstrated capability to me.

I probably need to test the idea with one of my own use cases. I would start by making the loop concrete: what can the agent observe, which actions can it take, what state should persist, and what result would show that its model is wrong? That would not show that the system can discover a new physical principle. It might show me whether this framework changes how I design or evaluate an agent.