ARC-AGI-3: The Benchmark That Measures Agentic Intelligence in AI
ARC-AGI-3 tests whether an AI can explore an unknown environment without instructions. Humans score 100%, while GPT-6 Astra reaches 99.9% with an optimized harness.
In March 2026, the ARC Prize Foundation launched ARC-AGI-3, a benchmark that changes how artificial intelligence is measured. Forget static grids to complete: here, an AI must explore an unknown game without any instructions, figure out its rules on its own, and win. Humans succeed in 100% of cases. The best models in March 2026 were capped at 0.51%. In September 2026, OpenAI's GPT-6 Astra reached 99.9% in an optimized test configuration. Here is what that really means.
Where does ARC-AGI-3 come from?
ARC-AGI, the Abstraction and Reasoning Corpus for AGI, is a family of benchmarks created by François Chollet and continued by the ARC Prize Foundation. The principle is simple: measure a system's ability to adapt to new situations without specific prior training.
- ARC-AGI-1 (2019): input/output grids where the AI must infer the transformation rule. Reasoning models solved the benchmark in 2024.
- ARC-AGI-2 (2024): harder grids with more complex rules.
- ARC-AGI-3 (2026): a radical change. It is no longer a static grid, but an interactive environment where an agent must act, explore, and learn in real time.
The 2026 ARC Prize, with $2 million in rewards, includes two Kaggle competitions: one for ARC-AGI-3 and the final one for ARC-AGI-2, whose $1 million grand prize is guaranteed this year.
How does ARC-AGI-3 work?
ARC-AGI-3 is made up of hundreds of original game environments, each designed by human creators. There are no instructions, written rules, or stated objectives. The agent discovers everything through interaction.
The agent sees a game screen and takes actions. The environment reacts. It must:
- Explore: try actions to understand what happens.
- Model: build an internal representation of the game's rules.
- Set a goal: identify what counts as a win without being told.
- Plan and execute: chain actions to reach that goal while adapting to surprises.
These four abilities are the pillars of agentic intelligence according to the ARC Prize Foundation.
The RHAE metric: why efficiency matters more than success
ARC-AGI-3 does not only measure whether an agent wins or loses. It measures action efficiency relative to humans through the RHAE metric, or Relative Human Action Efficiency.
The calculation works as follows:
- For each level, record the median number of actions taken by humans.
- Count the number of actions taken by the AI.
- The level score is: (human actions / AI actions) squared.
- The final score is the average across all levels.
For example, if the median human solves a level in 10 actions and the AI takes 100, the score is (10/100)² = 0.01, or 1%.
A score of 100% means the AI is at least as efficient as the median human across the benchmark. Squaring the ratio makes the metric highly sensitive to weak performance: a modest difference in actions produces a much lower score.
That non-linearity is deliberate. It heavily penalizes agents that muddle through by trial and error instead of reasoning efficiently.
Why do humans score 100% while AI models were stuck?
Before inclusion, every ARC-AGI-3 environment was tested with at least 10 human participants. Only environments that at least two people could solve independently on their first attempt were retained, according to the technical paper. They are all easy for a human.
For an AI, the situation is different. In March 2026, the best model, GPT-5.6 Sol with the standard harness, did not exceed 0.51%, according to the ARC Prize Foundation's launch announcement.
Why such a gap?
- AI models are trained to follow explicit instructions. ARC-AGI-3 gives none.
- AI systems struggle to explore efficiently when they do not know what they are looking for.
- The standard harness resets private reasoning after each action.
- AI systems struggle to set intermediate goals for themselves.
GPT-6 Astra: the jump to 99.9%
On 3 September 2026, OpenAI announced GPT-6 Astra. On ARC-AGI-3, the results were the best ever observed:
| Configuration | Score | Estimated cost |
|---|---|---|
| Standard harness (max) | 62.7% | $26, 098 |
| Provider Adapter (high) | 99.9% | $18, 817 |
Source: ARC Prize Foundation, public analysis, arcprize.org/blog/astra, 3 September 2026.
Two points are essential:
- The Provider Adapter harness preserves the model's opaque reasoning state between actions and uses compaction to handle long histories. In July 2026, OpenAI showed that the same two settings tripled GPT-5.6 Sol's score, from 13.3% to 38.3% on the public set. With Astra, they make 99.9% possible.
- In this configuration, Astra exceeds human action efficiency: it uses fewer actions than the median human on 96% of levels, and 51.7% fewer actions on average.
Greg Kamradt of the ARC Prize Foundation said: “Astra surpassed our human action-efficiency baseline on 96% of levels, reaching human parity on the benchmark. It is the best model we have ever tested.”
Why 99.9% is remarkable
- Five months earlier, the best models scored 0.51%. The progress between March and September 2026 is substantial.
- 99.9% means Astra is as efficient as a human on nearly every level, whereas previous AI systems struggled to solve even one level.
- Astra develops its own internal language. ARC Prize researchers observed the model creating a compact algebraic notation to represent game mechanisms as logical rules. This kind of abstraction had not been seen at this scale, according to the ARC Prize Foundation.
The ARC Prize Foundation immediately adds an important warning: saturating the benchmark is not proof of AGI. It does not claim that Astra is AGI.
The caveats: what 99.9% does not prove
A benchmark is not proof of general intelligence. ARC-AGI-3 has limits that its creators acknowledge:
- Closed scope: the environments have deterministic mechanisms and fixed goals. They are not the open-ended real world.
- Limited scale: a few hundred environments is small compared with the diversity of real situations.
- Optimized harness: the 99.9% score depends on the Provider Adapter harness. With the Standard harness, under more comparable conditions, Astra scores 62.7%, impressive but not perfect.
- No guaranteed generalization: excelling at ARC-AGI-3 does not guarantee that a model can adapt to any novel situation.
- Cost: $19, 000 for one benchmark session is not trivial. Economic efficiency is not there yet.
As the ARC Prize Foundation puts it, ARC-AGI-3 has a limited scope and deterministic mechanisms. It does not represent the complexity and openness of the real world.
What does ARC-AGI-3 really measure?
ARC-AGI-3 measures a system's ability to learn through exploration and act efficiently in an unknown but structured environment.
It does not measure:
- The ability to use broad general knowledge, which the benchmark deliberately excludes.
- The ability to interact with humans.
- The ability to manage real-world uncertainty or ambiguity.
- Creativity or breakthrough innovation.
What makes ARC-AGI-3 a useful benchmark is not that it is AGI-proof, but that it resists direct optimization. It cannot be solved through simple memorization or ingestion of training data. Progress requires improving exploration and adaptation.
Why it matters for agentic AI
ARC-AGI-3 measures what separates a model that follows instructions from an agent that explores and learns on its own. That is the key ability required to deploy autonomous agents in professional environments where every situation is different.
An agent that can adapt without prior documentation, a step-by-step guide, or a huge prompt: that is what ARC-AGI-3 evaluates. It is also what products such as Atako aim to make available to businesses: agents that can navigate unexpected situations.
The road is still long. But the move from 0.51% to 99.9% in five months shows that the direction is right. The next step is to achieve similar results with a standard harness, at a cost that falls from thousands of dollars to a few dollars.
Sources and methodological notes
This article was written from the following sources, all checked on 11 September 2026:
- ARC Prize Foundation, Announcing ARC-AGI-3, 25 March 2026: https://arcprize.org/blog/arc-agi-3-launch
- ARC Prize Foundation, ARC-AGI-3 benchmark: https://arcprize.org/arc-agi/3
- ARC Prize Foundation, GPT-6 Astra on ARC-AGI-3, 3 September 2026: https://arcprize.org/blog/astra
- Technical paper: ARC-AGI-3, arXiv:2603.24621: https://doi.org/10.48550/arxiv.2603.24621
- OpenAI, How two settings tripled scores on ARC-AGI-3, 29 July 2026: https://openai.com/index/how-two-settings-tripled-our-arc-agi-3-scores/
- BenchLM leaderboard: https://benchlm.ai/benchmarks/arcagi3
Verified facts: model scores on ARC-AGI-3, the RHAE methodology, test conditions, launch dates, and public statements come from official publications.
Editorial analysis: the interpretation of the scores, implications for businesses, and the comparison with AGI.
Limitations: scores for different models were not all obtained under strictly identical conditions, notably Standard versus Provider Adapter. The 99.9% score is the Provider Adapter result, not the Standard result. Estimated session costs come from the ARC Prize Foundation.