2,376 grade-school science multiple-choice questions (the AI2 ARC Easy test split), imported from inspect_evals. The model answers with one lettered choice; grading is inspect's own choice scorer against the published key, so no judge tokens are spent. Benchmark score = plain accuracy over the cases run. Answer format: end the reply with a line like 'ANSWER: B', the letter of one option. A period after it still counts, as in 'ANSWER: B.', and so does a wrapper around the letter, as in 'ANSWER: $B$', 'ANSWER: (B)', 'ANSWER: **B**' or 'ANSWER: \boxed{B}', or a bold label, as in '**ANSWER:** B'. Words after the letter, as in 'ANSWER: B because it is red', or a reply with no 'ANSWER:' line, as in 'The answer is B', are read as no answer and score 0, even when the letter is right.