500 biomedical research questions, each answered yes, no or maybe from a PubMed abstract shown as context (the official PubMedQA test set), imported from inspect_evals. The eval loads all 1,000 labelled questions and keeps the 500 on its test list; the board keeps exactly the same 500, in dataset order like inspect. Graded by the eval's own choice scorer against the published answer, so no judge tokens are spent. Benchmark score = plain accuracy over the cases run. Answer format: end the reply with a line like 'ANSWER: B', the letter of one option. A period after it still counts, as in 'ANSWER: B.', and so does a wrapper around the letter, as in 'ANSWER: $B$', 'ANSWER: (B)', 'ANSWER: **B**' or 'ANSWER: \boxed{B}', or a bold label, as in '**ANSWER:** B'. Words after the letter, as in 'ANSWER: B because it is red', or a reply with no 'ANSWER:' line, as in 'The answer is B', are read as no answer and score 0, even when the letter is right.