WeirdML v3 is out: GPT-6 Astra leads at 42.2%, and GPT-6.1 Terra edges Fable 5.1 at about 1/11 the cost

Håvard Tveit Ihle published WeirdML v3 on October 28. It is an agentic benchmark of 11 hand-made tasks in which a model has to explore unfamiliar data, build its own machine-learning or data-analysis pipeline and produce results with little data, an unspecified goal or very limited feedback. The first results file ranks 27 model configurations. GPT-6 Astra (xhigh) is first at 42.2%, then GPT-6.1 Sol (xhigh) at 38.3%, Claude Opus 5.5 (xhigh) at 31.2%, GPT-6.1 Terra (xhigh) at 28.1% and Claude Fable 5.1 (xhigh) at 26.0%.

THE COST

Average API cost per task run, from the same file: GPT-6.1 Terra $2.86, GPT-6.1 Sol $5.29, Claude Opus 5.5 $10.25, GPT-6 Astra $27.85 and Claude Fable 5.1 $31.93. That puts Terra one place above Fable 5.1 at about 1/11 of its cost per run, and one place below Opus 5.5 at about 1/4.

HOW IT IS SCORED

Each model's official score is 80% the normalized area under its best-so-far curve on a logarithmic token axis from 500,000 to 50 million tokens, plus 20% of its final value, so it rewards reaching a good result early as well as reaching it at all. That matters here: Fable 5.1 ends higher than Terra on the final value alone, 43.5% against 40.4%, but Terra gets there on fewer tokens, about 147,000 output tokens per run against Fable 5.1's 251,000.

OPEN WEIGHTS

The best open-weights entry is DeepSeek V4.1 Flash 1017 (high) at 9.2%, 14th of 27, ahead of Kimi K3 (high) at 7.5%.

WHAT IS SOURCED

Every score, interval and cost above is from WeirdML v3's published results file, generated at 15:49 UTC on October 28, read directly. Ihle built the benchmark at the Norwegian Defence Research Establishment, and his page says the API costs were paid mainly by Epoch AI, then METR and NDRE.

WHAT WE DO NOT KNOW

How Gemini 4 Carbon would place. Google announced it on October 28, but it is not on a public API, so it is not on the board. The Terra and Fable 5.1 gap is also narrow: their 95% intervals, 26.6% to 29.6% and 24.6% to 27.3%, overlap slightly.