847 multiple-choice questions from college exams, quizzes and textbooks across 30 subjects in six disciplines (art, business, science, health, humanities, engineering): the multiple-choice part of MMMU's validation split, since the test split's answers are withheld. Imported from inspect_evals. Every question comes with one to five pictures (charts, diagrams, maps, photos, sheet music, chemical structures), so the Candidate must read images; the pictures are saved inside the Case at import and nothing is fetched at run time. The Candidate is shown the pictures, then inspect's multiple-choice instruction, the eval's own 'answer with the option's letter' line and the question with its options lettered in the dataset's order, and is graded by inspect's choice scorer against the correct letter, so no judge tokens are spent. Benchmark score = plain accuracy over the cases run. inspect_evals runs the model at temperature 0; this benchmark imposes no generation settings. Answer format: end the reply with a line like 'ANSWER: B', the letter of one option. A period after it still counts, as in 'ANSWER: B.', and so does a wrapper around the letter, as in 'ANSWER: $B$', 'ANSWER: (B)', 'ANSWER: **B**' or 'ANSWER: \boxed{B}', or a bold label, as in '**ANSWER:** B'. Words after the letter, as in 'ANSWER: B because it is red', or a reply with no 'ANSWER:' line, as in 'The answer is B', are read as no answer and score 0, even when the letter is right.