How can an AI model understand and interact with an unfamiliar world?
The physical world is continuous and high-dimensional, and consists of information from different modalities. It is also dynamic and inherently changing all the time: an action may trigger a sequence of events and changes of the world states. As humans, we perceive the world through multiple senses, reason about the world in our brain, and take actions based on our understanding and past experience. This process robustly supports us in understanding and acting in the world, even in an unfamiliar environment.
But do existing AI models have such capability? We show that a similar idea also applies to them: a general-purpose multimodal model can observe the world through high-dimensional sensory inputs, reason about the world in a fuzzy but flexible manner, recall past experience through a lossless memory mechanism, and effectively take actions to interact with the world to achieve the goal.
To study this idea, we use ARC-AGI-3, a suite of interactive visual games in which an agent must discover the rules and goals of unfamiliar worlds through interaction. We introduce VISTA, a visual harness that gives a general-purpose multimodal model long-horizon vision. The agent directly observes the world through continuous high-dimensional sensory data, in this case raw PNG images, reasons about the world in its own way, recalls past observations in their original detail when needed, and takes actions to explore the world and achieve the goal.
Using Claude Opus 5.0 as the base model, VISTA completes all 25 public games, with a 100% win rate and a perfect 100 Relative Human Action Efficiency (RHAE) score. It is also efficient, using 56.0% fewer actions than first-time human players, and robust to different sensory inputs. We aim for a minimalist design, avoiding complex systems and task-specific engineering, instead focusing on a simple setup that supports the model to perceive and reason about the world in a natural way. While we use ARC-AGI-3 as a testbed, the same method extends naturally to other interactive environments. Our results demonstrate the potential of general-purpose multimodal models in understanding and acting in the physical world.
What is ARC-AGI-3?
ARC-AGI-3 extends the abstract reasoning of ARC-AGI-1 and ARC-AGI-2 into interactive visual games. Unlike ARC-AGI-1 and ARC-AGI-2, which present static input-output transformations, ARC-AGI-3 requires the agent to explore a dynamic world through interaction. Each game begins without instructions, known rules, or a stated goal. At each step, the agent observes the current state and chooses from the available actions. The resulting observation provides evidence about what the action did and how the game may work. A game contains a sequence of levels that share underlying mechanics while introducing new objects, layouts, or constraints. The agent must infer object roles, transition rules, hidden state, and possible goals from experience, then carry that understanding into later levels.
Performance evaluation combines completion with efficiency. Relative to first-time human players, an agent receives more credit when it solves levels using fewer actions. A completed level receives full credit when the agent uses no more actions than the first-time-human baseline; an unfinished level receives zero credit, while a completed but less action-efficient level receives partial credit. ARC-AGI-3 therefore measures how efficiently an agent learns and completes the games through interaction and past experience.
VISTA: A Visual Harness
VISTA uses three key designs: perception through natural high-dimensional visual input, reasoning through language, and memory management through lossless visual memory.
Visual perception
The agent perceives the world by directly observing the original sensory input, in this case by reading 2D PNG images. This allows it to observe the world as faithfully as possible to the original information. A raw image retains spatial information, allowing the agent to reason directly about objects and their spatial relations.
In our experiments, the agent receives a 512 by 512 PNG image of the current rendered state, a nearest-neighbor upscaling (8x) of the official 64 by 64 frame with one-pixel grid lines between cells. We note that the agent is never told that the world is a 64 by 64 grid; the 512 by 512 PNG images are all it receives. It can also request an enlarged view of any rectangular region of the board, and thus, in principle, has intact information access.
In this way, the model must leverage its visual priors during reasoning, and we indeed observe that it forms and discovers visual concepts. Our framework is not restricted to 2D images as observation representations, and generalizes to more realistic cases, such as a 3D observation, as we demonstrate later.
Comparing different observation representations
In ARC-AGI-3, a 2D image is a natural way to represent the world, but it does not have to be the only way. The same world can be given to the agent as sensory input with different numbers of dimensions: a 1D flattened textual grid, where each color is represented by an integer, or a 3D scene rendered from the same game.
These mappings are fully reversible and theoretically preserve the same information, but they do not necessarily support the same computation. We find that while the agent is still able to perform well with the textual grid, it is less efficient in terms of token usage. We also find that the agent is still able to act effectively in the 3D scene, a form that is closer to how humans perceive the real physical world.
| Representation | What the agent receives |
|---|---|
| Text grid 64 by 64 integers, one color is represented by one integer. |
|
| 2D image A 512 by 512 PNG, an image render of the same 64 by 64 integer grid. |
 |
| 3D render A 3D rendering of the same game. |
 |
Language reasoning
Observations are turned into hypotheses by natural language to support the agent’s decision making. In ARC-AGI-3, the agent needs to figure out the role of each object, the underlying mechanism of how the world works, the criteria for winning, and so on. We choose to let the agent reason in free-form language without particular constraints: it decides what to reason about, what language to use, and when and how long to reason. It can also take notes, revise its hypotheses as it gathers more evidence, and use them to guide its next action.
Comparing different reasoning processes
An alternative approach commonly used to tackle ARC-AGI is to reason with a code world model, where the agent abstracts the world into a set of symbolic programs. Program-based agents such as Schema, Tycho, and Retrodict turn the traces they collect while exploring the world into an executable reconstruction of the game: a state representation, transition rules, and often goal conditions. Because that reconstruction is executable, it can be checked against what the agent has already seen and searched over to choose an action.
We think much of the reasoning that leads to a good action can be carried out in a fuzzy manner. Natural language reasoning behaves more similarly to human beings and turns out to suffice for even these challenging ARC-AGI-3 games.
The table below compares the two approaches using the same rule from the LF52 game: its jumping mechanism, illustrated below. A faithful program has to name the board, the coordinates of every object, the conditions under which a move is legal, and the state update it produces. A descriptive sentence, in contrast, cannot be executed, but it is enough to convey the rule. Free-form language therefore gives the agent the maximum level of flexibility in the reasoning process.
1 · The boardGreen pegs sit on a grid of gray holes.
2 · SelectClicking a green selects it, and a landing hole appears two cells away.
3 · JumpThe green lands in the hole. Its source and the peg it crossed are now empty.| Program-based reasoning | Free-form language (ours) | |
|---|---|---|
| How the rule is written | An executable reconstruction of the game: state representation, transition rules, goal conditions. | Sentences, in whatever terms the model finds useful at the time. |
| Example rule | |
GUIDE.md for this game. |
| Size | Roughly 4,000 lines of Python for this game. | A page of notes. |
| What can be done with it | Explicit and executable. | Implicit and fuzzy. |
Lossless visual memory
As the game progresses, the agent accumulates a record of its observations. When a new event happens, the agent may need to return to earlier frames to check details or compare them with new evidence. In standard VLMs and LLMs, the core memory mechanism is the KV cache, which the model can only attend over implicitly: it is typically compressed, lossy, and of limited horizon. In that case, the visual states of past turns may be lost, or not effectively stored, in the agent’s reasoning process.
A core design of VISTA is to maintain the game states as an explicit visual
memory, which stores every frame returned by the environment, together with its
turn and frame index, in a lossless way. The agent can then use an inspect
tool to select earlier states, intermediate animation frames, or spatial regions
and view them again through the same visual input. Several views can be
requested together for read-only comparison. For small discrete details, read_pixels returns exact color
samples from a selected region. This visual memory ensures that the complete
pixel history remains accessible, and the agent decides which past frame or
region to bring back into view through this “explicit attention mechanism” at frame, region, or pixel level.
Comparing different memory mechanisms
Over long interactions, an agent must retain information from earlier observations. Different memory mechanisms preserve different parts of that experience.
| Memory | What is kept | What is lost | How it comes back |
|---|---|---|---|
| Context window | Recent turns, as tokens and activations in the KV cache. | Whatever falls outside the window. The context length and the compaction mechanism decide what goes, and older visual detail is compressed or lost. | Implicit attention over what still fits. |
| Program world model | Observations distilled into code: state, transitions, goal conditions. | Anything the reconstruction does not model, and the past visual frames themselves. | Executing the reconstruction. |
| Written text notes | The model’s own description of what happened. | Everything not written down. The model chooses what to drop. | Re-reading its own text. |
| Lossless visual memory | Every returned frame at full resolution, indexed by turn and frame. | Nothing. | inspect and read_pixels, on the model’s own decision. |
The key property of visual memory is that all visual information the agent has received about the world remains retrievable in its original form: any frame the game has returned, and any region within it, can be brought back at full resolution whenever the model decides it matters.
We also allow the model to take notes in a minimal file, GUIDE.md, that describes the core idea of the game. This gives it an additional way to keep information, but in a high-level and abstract form.
The Complete Pipeline
One agent plays each game from its first observation to completion. It begins with only the current visual state and available actions: no instructions, stated rules, or goal. On each turn it observes, reasons, and executes one game action. The same agent can revisit visual memory or maintain two notes whenever useful. When the model approaches its context limit, it writes a concise continuation state, then resumes from the current visual state in a fresh context. Its notes, visual memory, and action history remain available. Every game uses the same interface and short prompt.
Observe
A single action may produce a sequence of animation frames. Every returned frame enters the visual memory in order, and the final frame becomes the next current visual state, together with game status and level progress.
Looking back is available but optional. The model may act on the current frame
alone, or call inspect to bring an earlier state, an intermediate animation
frame, or an enlarged region back into view, and read_pixels to look at
details. Which past moment to re-examine, and whether to
re-examine anything at all, is the model’s decision rather than a fixed step in
the loop.
Reason, then act
The model uses the current visual state, past evidence, and its notes to
maintain a revisable understanding of the game. It reasons and plans in
free-form language. Before calling play, the prompt asks it to state the visual result it expects.
We ask the agent to write notes that build and use a compact, revisable model of
the game and its current state (see prompt below). The word compact encourages the
model to organize its observations into higher-level abstractions, in the spirit
of Occam’s razor; the word revisable allows for updates and corrections as new evidence
becomes available.
The notes externalize the agent’s understanding of the world. GUIDE.md holds
what may remain useful across levels, while WORKING.md is a scratchpad for the
current level. Together they give us a readable view of the abstractions the
agent uses when exploring the world.
The agent prompt
# Visual game task
Complete the game with as few game actions as possible.
Build and use a compact, revisable model of the game and its current state. Update it as new evidence changes what is supported.
Before each `play`, briefly state what you expect to see. Afterward, briefly state all visible changes, expected or not.
Keep concise, durable, revisable game understanding in `GUIDE.md`; use `WORKING.md` as a scratchpad when useful.VISTA Results on ARC-AGI-3
We instantiate VISTA with two general-purpose multimodal model backends: Opus 5.0 through the Claude Code CLI and GPT-5.6 Sol through the Codex CLI. Each backend is run independently on all 25 public ARC-AGI-3 games. The model receives PNG observations, public game status and progress, and the currently available actions. Within each backend, the same prompt, tools, model, and reasoning setting are used across games.
ARC-AGI-3 uses Relative Human Action Efficiency (RHAE): completed levels are scored by the squared ratio between a first-time human action baseline and the agent’s actions, later levels receive greater weight, and the final score is averaged across games. Only environment actions enter this count; internal reasoning and read-only inspection are free under the scoring protocol. See the official scoring methodology for the complete definition.
With Opus 5.0, VISTA completes all 183 levels across all 25 games. Its mean game score is 100.00, with 25 perfect game scores, using 7,542 game actions (56% fewer than humans). With GPT-5.6 Sol, VISTA also completes all 183 levels across the 25 public games. Its mean game score is 98.27, with 22 perfect game scores. The remaining 1.73 points are concentrated in a small number of levels where the model spends extra actions discovering a mechanism or recovering from an incorrect game model.
Select a model below to view its complete 25-game result table and level-by-level breakdown. Each available replay link opens the recorded trajectory, including game actions, public model output, visual inspections, and agent notes.
Claude Opus 5.0
Effortxhigh- Mean score
- 100.00
- Games completed
- 25 / 25
- Perfect games
- 25 / 25
- Actions · agent / human
- 7,542/ 17,135
| Progress | Status | Replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 100.00 | 270/ 748 Cumulative progress0.36× human Agent 270Human 748 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 449/ 651 Cumulative progress0.69× human Agent 449Human 651 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 92/ 171 Cumulative progress0.54× human Agent 92Human 171 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 235/ 789 Cumulative progress0.30× human Agent 235Human 789 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 635/ 1228 Cumulative progress0.52× human Agent 635Human 1,228 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 80/ 208 Cumulative progress0.38× human Agent 80Human 208 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 321/ 879 Cumulative progress0.37× human Agent 321Human 879 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 302/ 730 Cumulative progress0.41× human Agent 302Human 730 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 881/ 1339 Cumulative progress0.66× human Agent 881Human 1,339 | 10 / 10 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 111/ 388 Cumulative progress0.29× human Agent 111Human 388 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 517/ 776 Cumulative progress0.67× human Agent 517Human 776 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 256/ 1107 Cumulative progress0.23× human Agent 256Human 1,107 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 68/ 233 Cumulative progress0.29× human Agent 68Human 233 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 593/ 1255 Cumulative progress0.47× human Agent 593Human 1,255 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 251/ 638 Cumulative progress0.39× human Agent 251Human 638 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 124/ 213 Cumulative progress0.58× human Agent 124Human 213 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 176/ 350 Cumulative progress0.50× human Agent 176Human 350 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 525/ 1070 Cumulative progress0.49× human Agent 525Human 1,070 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 121/ 518 Cumulative progress0.23× human Agent 121Human 518 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 90/ 361 Cumulative progress0.25× human Agent 90Human 361 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 156/ 317 Cumulative progress0.49× human Agent 156Human 317 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 204/ 414 Cumulative progress0.49× human Agent 204Human 414 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 192/ 462 Cumulative progress0.42× human Agent 192Human 462 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 180/ 447 Cumulative progress0.40× human Agent 180Human 447 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 713/ 1843 Cumulative progress0.39× human Agent 713Human 1,843 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GPT-5.6 Sol
Effortmax- Mean score
- 98.27
- Games completed
- 25 / 25
- Perfect games
- 22 / 25
- Actions · agent / human
- 10,063/ 17,135
| Progress | Status | Replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 100.00 | 327/ 748 Cumulative progress0.44× human Agent 327Human 748 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 85.25 | 638/ 651 Cumulative progress0.98× human Agent 638Human 651 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 84/ 171 Cumulative progress0.49× human Agent 84Human 171 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 264/ 789 Cumulative progress0.33× human Agent 264Human 789 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 805/ 1228 Cumulative progress0.66× human Agent 805Human 1,228 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 75/ 208 Cumulative progress0.36× human Agent 75Human 208 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 376/ 879 Cumulative progress0.43× human Agent 376Human 879 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 395/ 730 Cumulative progress0.54× human Agent 395Human 730 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 982/ 1339 Cumulative progress0.73× human Agent 982Human 1,339 | 10 / 10 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 102/ 388 Cumulative progress0.26× human Agent 102Human 388 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 93.59 | 696/ 776 Cumulative progress0.90× human Agent 696Human 776 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 264/ 1107 Cumulative progress0.24× human Agent 264Human 1,107 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 128/ 233 Cumulative progress0.55× human Agent 128Human 233 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 684/ 1255 Cumulative progress0.55× human Agent 684Human 1,255 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 304/ 638 Cumulative progress0.48× human Agent 304Human 638 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 131/ 213 Cumulative progress0.62× human Agent 131Human 213 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 77.88 | 346/ 350 Cumulative progress0.99× human Agent 346Human 350 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 949/ 1070 Cumulative progress0.89× human Agent 949Human 1,070 | 8 / 8 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 239/ 518 Cumulative progress0.46× human Agent 239Human 518 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 129/ 361 Cumulative progress0.36× human Agent 129Human 361 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 191/ 317 Cumulative progress0.60× human Agent 191Human 317 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 180/ 414 Cumulative progress0.43× human Agent 180Human 414 | 6 / 6 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 238/ 462 Cumulative progress0.52× human Agent 238Human 462 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 300/ 447 Cumulative progress0.67× human Agent 300Human 447 | 7 / 7 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| 100.00 | 1236/ 1843 Cumulative progress0.67× human Agent 1,236Human 1,843 | 9 / 9 | WIN | View replay | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Level breakdown
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Actions are shown as agent / human reference.
System-level comparison
In the table below, we compare VISTA with state-of-the-art methods on the ARC-AGI Community Leaderboard, most of which are concurrent work released within the past few days.
| System | Program-based | Model | Reasoning effort | RHAE |
|---|---|---|---|---|
| Official minimal interface | No | GPT-5.6 Sol | max | 13.33 |
| Opus 5.0 | high | 30.16 | ||
| Schema | Yes | GPT-5.6 Sol | xhigh → max | 95.35 |
| Opus 4.8 → Fable 5 | max | 98.98 | ||
| ewma_sv_v1.6 | Yes | GPT-5.6 Sol | xhigh | 98.97 |
| Retrodict | Yes | GPT-5.6 Sol | max | 99.86 |
| Tycho | Yes | GPT-5.6 Sol | max | 100.00 |
| Opus 5.0 | xhigh | 100.00 | ||
| VISTA (ours) | No | GPT-5.6 Sol | max | 98.27 |
| Opus 5.0 | xhigh | 100.00 |
An arrow marks a fallback: Schema runs Opus 4.8 and Sol at xhigh first, reruns any game scoring below 80 with Fable 5 and Sol at max, and keeps the higher per-game score. Efforts for the other systems are taken from their published configurations and traces.
VISTA reaches a perfect 100.00 RHAE with Opus 5.0 and 98.27 with GPT-5.6 Sol. Notably, to our knowledge, it is the first system to reach a perfect or near-perfect score without program synthesis. As a reference, the official baselines from the ARC-AGI-3 organizers, which also do not use program synthesis, score 30.16 with Opus 5.0 and 13.33 with GPT-5.6 Sol.
Varying the observation representation
The results above use the original 2D renderings. To examine how observation representation affects interaction, we compare three ways of presenting the same underlying worlds: a 1D text grid serialized row by row, the original 2D image, and a 3D rendering. We asked VISTA to play S5I5 and CD82 from each representation alone. It inferred how both games worked and completed Level 1 in all three settings. These are independent trajectories rather than identical action sequences.
The underlying rules are unchanged: in S5I5 the agent must extend two colored structures toward their targets, and in CD82 it must transform a canvas to match a reference.
Together, these runs show that VISTA is robust to different observation representations: it acts effectively from a serialized symbolic observation, the native 2D game view, or a visually richer 3D scene. Its transfer to 3D scenarios opens up the broader potential of VISTA acting in more complex physical worlds, from richer games to embodied environments.
What This Shows and What’s Next
VISTA gives a general-purpose multimodal model raw observations, free-form language to reason in, and a lossless visual memory of its past experience. This allows the model to explore and interact with the world in a natural way. We show that a minimal visual harness is enough for the agent to discover the rules of 25 unfamiliar visual games on ARC-AGI-3, play them to a perfect score, and do so more efficiently than first-time human players.
We acknowledge that the model’s existing capabilities are a key factor in this success, and that the visual harness is a simple but effective way of eliciting them. In addition, the models we use were released after the public ARC-AGI-3 games, so we cannot rule out that these games were seen during training; the private set remains the real test of generalization.
Nevertheless, ARC-AGI-3 is a testbed, not the target. The design is not restricted to 2D visual games, and extends straightforwardly to any interactive environment. We are excited about the potential of VISTA to support more complex and realistic tasks, even in embodied environments closer to the physical world.