✳ the wire · analysis

Gemini 4 Argon scores 97.5% on ARC-AGI-3 through a provider adapter, 49.8% on the standard harness

Gemini 4confirmedby ArtificialWatch

Get launch alerts like this, free →

Gemini 4 Argon scores 97.5% on ARC-AGI-3 through a provider adapter, 49.8% on the standard harness
Source imagery · verified against a primary source

ARC Prize has published verified scores for Gemini 4 Argon, and on ARC-AGI-3 there are two very different numbers. Run through what ARC Prize calls the provider adapter harness, Argon scores 97.5% at low effort. On ARC Prize's standard harness it scores 49.8%. Google is the second lab past 95% on ARC-AGI-3, after OpenAI.

THE SCORES

All figures are ARC Prize's semi-private sets, read from arcprize.org on October 15.

ARC-AGI-3, provider adapter harness: 97.5% at low effort, 97.3% at medium and 97.3% at high. The low-effort run cost about $4,770.

ARC-AGI-3, standard harness: 49.8% at low effort (about $5,820) and 46.3% at medium (about $6,180). There is no high-effort row.

ARC-AGI-2: 91.3% at high effort, $0.32 a task. ARC-AGI-1: 98.5% at high, $0.16 a task.

WHERE IT RANKS

On the adapter harness only OpenAI scores higher: GPT-6 Astra from low effort up (99.9% at high). On the standard harness Argon's 49.8% is fifth, behind GPT-6 Astra at max (62.7%), xhigh (59.3%) and high (54.8%) and GPT-6.1 Sol at max (52.7%). It got there for far less: the Astra max run cost about $26,100.

Anthropic's best ARC-AGI-3 entry is Claude Opus 5 at 30.2%. Opus 5.5 has no ARC-AGI-3 score on the board yet.

On ARC-AGI-2, Argon's 91.3% trails GPT-6 Astra at max (95.0%), GPT-6.1 Sol at max (94.2%) and Claude Opus 5.5 at high (93.3%, $0.41 a task).

READ THE FINE PRINT

The 97.5% headline is the adapter figure. ARC Prize lists the two harnesses as separate entries, and the standard one is the like-for-like number for most of the board.

Low effort beat medium and high on both harnesses.

ARC Prize marks these runs as partial dataset coverage: scores use the full task count as the denominator, and costs count only recorded usage, leaving out attempts that were not recorded.

WHAT WE DO NOT KNOW

Why the adapter harness roughly doubles Argon's score. We have not seen a write-up from ARC Prize or Google that explains the gap.

Source: ARC Prize leaderboard data and results page, read directly ↗ · Gemini 4 tracker · the bench index

sweeping every 60 seconds

Hear the minute Google’s next model drops — not the minute we write it up.

Claude Opus 5 went live at 16:51 UTC. The alert was in subscribers' inboxes at 16:52.

  • Free forever
  • No card
  • Unsubscribe in one click

← back to the wire