screamingface
Leaderboard

MedXpertQA

2,450 expert-level medical multiple-choice questions (MedXpertQA Text). The model reasons step by step, then commits to one lettered choice; grading is an exact match against the published key, so no judge tokens are spent. Benchmark score = plain accuracy over every case run, reported beside the fraction actually answered β€” a case the model leaves unanswered, or that fails, counts as wrong. The two-turn exchange is applied at the candidate boundary, which reproduces the official protocol for a single model; for a fusion the exchange wraps the whole ensemble rather than each member, so those numbers are not comparable to a per-member implementation. Answer format: after its reasoning, the model is asked to finish 'the answer is' with one option letter. The grader reads that phrase first, so 'the answer is D. B would be wrong.' counts as D; without it, it takes the last bracketed letter, then the last option letter on its own, so 'D', '(D)', '**D**' and 'the answer is D' all count; a reply with no option letter, such as 'not sure', scores 0.

about The leaderboard's top row is the current state of the art. Every submission carries its url4 expression, a complete recipe for the run. Copy it with the Copy button in the Run Locally column to rerun, verify or build on any score here.
Loading…
😱 screamingface GitHub