For a given cost, what's the best accuracy anyone can reach, and which combination of models gets there?
The best answer to a hard question will come from several models working together: one routes it, a few attempt it, one checks the work. Intelligence built this way has the properties of good engineering: every contribution is visible, any part can be swapped, and parts can come from anywhere, open-weight or closed. Capability that comes from many places is harder to capture, easier to check and improve.
The evidence is already here. At the same cost, compositions beat the best single model: cascades that match frontier accuracy at a fraction of the price, mixtures of open models that outscore closed ones, routers that keep the quality and cut the bill. But each method lives in its own code, on its own benchmark, against last year's models. Reproducing one means paying the inference bill again; comparing two means rebuilding both.
A growing community is pushing this work forward. For it to compound, it needs to:
screamingface is an open-source toolkit that makes exploring compositions cheap and community-driven. Ensembles, routers and cascades are written in one shared language and published so the next person starts where the last one finished.
Every row on the board is a recipe, one url4 line that holds the whole run: the models and their versions, the prompts, the seeds, the benchmark and its revision, the full cost. Calls are pinned and runs feed a shared cache, so replaying a published row is free and building on one costs only the calls that are new, reducing the cost of exploring new compositions.
Anyone can submit. Compose a method using screamingface, compile it to a url4 recipe, and publish it. Code and recipes are open under Apache-2.0, and you keep authorship.
Get started → Browse leaderboards → Research grant: Ensembles & Routing →
ScreamingFace is built in the open by a community of researchers and engineers, with the support of OpenMined, a non-profit foundation that has spent nearly a decade building open-source infrastructure for AI.