screamingface
Leaderboard

GDPval Text Subset

The prose-only slice of GDPval's 220-task open gold set: 102 real professional work requests whose reference material and expected deliverable are documents rather than spreadsheets or slide decks. An AI judge grades the answer against the task's expert-written rubric, one criterion at a time; penalties subtract, so a case can score below zero. Benchmark score = the plain average of the case scores. Two deliberate differences from published GDPval numbers: submissions are plain text where 83 of these tasks expected a formatted document, so criteria checking the delivered file are excluded from scoring; and grading is per-criterion rubric judging rather than GDPval's official blinded expert pairwise comparison against a human professional's deliverable. Scores are therefore not comparable to the published GDPval leaderboard. Answer format: the deliverable is plain text. A judge checks it against each line of an expert-written points rubric, one line at a time, judging content only: a section, table or named field counts when its content is present and clearly identifiable. Met lines add their points and met penalty lines subtract.

about The leaderboard's top row is the current state of the art. Every submission carries its url4 expression, a complete recipe for the run. Copy it with the Copy button in the Run Locally column to rerun, verify or build on any score here.
Loading…
😱 screamingface GitHub