Multimodal & Grounded benchmark
ExtractBench Pro leaderboard
ExtractBench Pro: long-tail document extraction challenges (LlamaIndex). Every model the catalog carries a published ExtractBench Pro value for, ranked by that value.
CategoryMultimodal & Grounded
MeasurePerfect-document rate, value F1, word and page grounding F1 and cost, overall and per challenge
Tasks100 synthetic documents, 2,622 pages, 86 document types, six challenge groups
DifficultyThe best system completes 58 of 100 documents perfectly; several complete none
A harder companion to ExtractBench: 100 synthetic documents and 2,622 pages across 86 document types that test document conventions, house vocabulary, counting rules, drawn relations and charts, scored by perfect-document rate.
About ExtractBench Pro
What the benchmark measures and how its value is published, from the source’s own record.
What is ExtractBench Pro?
ExtractBench Pro (ExtractBench Pro: long-tail document extraction challenges (LlamaIndex)) is a multimodal & grounded benchmark published by ExtractBench Pro leaderboard and challenge descriptions. A harder companion to ExtractBench: 100 synthetic documents and 2,622 pages across 86 document types that test document conventions, house vocabulary, counting rules, drawn relations and charts, scored by perfect-document rate. Its evaluation set is described by the source as 100 synthetic documents, 2,622 pages, 86 document types, six challenge groups. The source rates its difficulty as the best system completes 58 of 100 documents perfectly; several complete none.
How is ExtractBench Pro scored?
Each model's result is published by ExtractBench Pro leaderboard and challenge descriptions as perfect-document rate, value F1, word and page grounding F1 and cost, overall and per challenge. ModelScale prints that value to 0 decimal places, exactly as published, and never converts or rescales it. This page ranks the 5 models that carry a published value, highest value first; a model the source never scored is left out rather than ranked at zero. The source does not state whether a higher value is the better result, so neither does this page.
ModelScale
