screamingface
Leaderboard

MedMCQA

4,183 four-option multiple-choice questions from Indian medical entrance exams (AIIMS and NEET PG), across 21 subjects such as anatomy, pharmacology and surgery: the full validation split, since the test split's answers are withheld. Served as a local Task in inspect's shape: each question is shown with its options lettered A to D, in the dataset's order, under inspect's standard multiple-choice instruction, and graded by inspect's choice scorer against the letter of the correct option, so no judge tokens are spent. Benchmark score = plain accuracy over the cases run, comparable with papers that report on the validation (dev) split. Answer format: end the reply with a line like 'ANSWER: B', the letter of one option. A period after it still counts, as in 'ANSWER: B.', and so does a wrapper around the letter, as in 'ANSWER: $B$', 'ANSWER: (B)', 'ANSWER: **B**' or 'ANSWER: \boxed{B}', or a bold label, as in '**ANSWER:** B'. Words after the letter, as in 'ANSWER: B because it is red', or a reply with no 'ANSWER:' line, as in 'The answer is B', are read as no answer and score 0, even when the letter is right.

about The leaderboard's top row is the current state of the art. Every submission carries its url4 expression, a complete recipe for the run. Copy it with the Copy button in the Run Locally column to rerun, verify or build on any score here.
Loading…
😱 screamingface GitHub