screamingface
Leaderboard

FrontierScience

160 expert-authored frontier science problems (the FrontierScience test split from OpenAI), imported from inspect_evals: olympiad-style short answers and open research questions across physics, chemistry, and biology. Grading is the eval's own LLM judge β€” olympiad answers against the official grading prompt, research answers against a per-case rubric β€” with every judge call routed and metered through our gateway, so judge tokens count in the run's cost. Cases are served in a fixed seeded shuffle so a limited run spans formats and subjects. Benchmark score = mean judge grade over the cases run. Answer format: no fixed format. A judge reads the whole reply. An olympiad question scores 1 when the reply's answer matches the reference answer or is equivalent to it (an equal expression, a number within one-decimal rounding, an equivalent name, or equal once units are converted); a research question earns points out of 10 from its own rubric.

about The leaderboard's top row is the current state of the art. Every submission carries its url4 expression, a complete recipe for the run. Copy it with the Copy button in the Run Locally column to rerun, verify or build on any score here.
Loading…
😱 screamingface GitHub