Vals AI launches a science benchmark built from three agentic research tests

The new Vals Science Index averages biology-data analysis, experimental mechanism discovery and scientific workflows, while its proprietary composite and small score differences need careful interpretation.

2 min read

Laboratory bench with scientific equipment, used as an illustrative image for Vals AI's science benchmark.

Vals AI released the Vals Science Index on October 16, a new composite benchmark intended to compare how well AI agents carry out scientific work rather than answer science questions from memory. Its leaderboard puts GPT-6 Astra first at 65.09%, followed by GPT-6.1 Sol at 59.63% and Claude Sonnet 5.5 at 59.12%. Vals displays uncertainty values alongside the scores (±2.26, ±2.31 and ±2.31, respectively), so the small gaps should not be read as definitive evidence of superiority.

The index averages three benchmarks equally: BioMysteryBench, which tests investigations of real biological datasets; MysteryMechanism, where agents infer hidden mathematical relationships through bounded experiments; and Terminal-Bench-Science, which covers research workflows across five scientific domains. The tasks require agents to use tools, analyze evidence and return solutions, not just retrieve facts. Vals says science performance changes the ordering seen on its broader Vals Index, underscoring that benchmark choice affects which model appears strongest.

The result is a useful new comparison, but not a universal measure of scientific ability. The index is marked proprietary and combines evaluations created by different organizations. BioMysteryBench was developed by Anthropic; its task materials carry CC BY 4.0 with a restriction against model-training use, while underlying biological data remain subject to their original repositories’ terms. MysteryMechanism has 222 mechanisms, but its validation mechanisms are private. Vals says BioMysteryBench scores average three independent full-benchmark runs, and the composite leaderboard provides uncertainty figures, though the index page does not explain how those values are calculated.

The rankings and costs are Vals AI’s own evaluations, not an independent replication of the new index. Readers should compare component results and task limits, rather than interpret a few percentage points as a definitive gap in real-world research capability.

Laboratory bench with scientific equipment, illustrative image; not a Vals AI benchmark scene.

Illustrative photo by Adam Bezer on Unsplash, used under the Unsplash License.

Sources: Vals Science Index and methodology · BioMysteryBench evaluation and methodology · MysteryMechanism evaluation and methodology · Terminal-Bench-Science · Photo by Adam Bezer · Unsplash License

AI benchmarksscienceagent evaluationVals AI