screamingface
Leaderboard

O-NET M6

372 multiple-choice questions from Thailand's national grade-12 exam (O-NET M6): mathematics, science, social studies, English and Thai language, in Thai or English, imported from inspect_evals. Like inspect, the Benchmark keeps only answerable single-answer questions; it also leaves out 24 more whose numbered options inspect splits wrongly (an option written '1.x' or '1 x' is lost, a year inside one is split off), so the answer letter points past the last choice (6) or at another option's text (16), or an option is missing (2); and 1 more whose source gives two options the same text, one of them the key, so the right text under the other letter is marked wrong. It serves 372 of inspect's 397. Each prompt opens with the eval's language note and asks the model to think step by step before its final 'ANSWER: X' line. Serving order is pinned by a policy seed (inspect_evals shuffles per run). Benchmark score = plain accuracy over the cases run. Answer format: end the reply with a line like 'ANSWER: B', the letter of one option. A period after it still counts, as in 'ANSWER: B.', and so does a wrapper around the letter, as in 'ANSWER: $B$', 'ANSWER: (B)', 'ANSWER: **B**' or 'ANSWER: \boxed{B}', or a bold label, as in '**ANSWER:** B'. Words after the letter, as in 'ANSWER: B because it is red', or a reply with no 'ANSWER:' line, as in 'The answer is B', are read as no answer and score 0, even when the letter is right.

about The leaderboard's top row is the current state of the art. Every submission carries its url4 expression, a complete recipe for the run. Copy it with the Copy button in the Run Locally column to rerun, verify or build on any score here.
Loading…
😱 screamingface GitHub