# GPQA leaderboard

Knowledge benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/knowledge-gpqa

Graduate-Level Google-Proof Q&A. Every model the catalog carries a published GPQA value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Knowledge |
| Measure | Multiple choice questions |
| Tasks | 448 questions |
| Difficulty | Graduate level |
| Published by | [GPQA: A Graduate-Level Google-Proof Q&A Benchmark](https://arxiv.org/abs/2311.12022) |

A challenging dataset of 448 multiple-choice questions written by domain experts in biology, physics, and chemistry. Designed to be difficult even for skilled non-experts with access to Google.

## GPQA ranking

150 models with a published GPQA value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Multiple choice questions | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [GPT-6.2 Astra](https://modelscale.dev/models/gpt-6-2-astra) | OpenAI | 97 | Observed 2026-11-21 · source benchlm:models |
| 2 | [GPT-6 Astra](https://modelscale.dev/models/gpt-6-astra) | OpenAI | 96 | Observed 2026-11-21 · source benchlm:models |
| 2 | [GPT-6.2 Sol](https://modelscale.dev/models/gpt-6-2-sol) | OpenAI | 96 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Sakana Fugu](https://modelscale.dev/models/sakana-fugu) | Sakana AI | 95.5 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Sakana Fugu-Ultra](https://modelscale.dev/models/sakana-fugu-ultra) | Sakana AI | 95.5 | Observed 2026-11-21 · source benchlm:models |
| 6 | [GPT-5.6 Sol](https://modelscale.dev/models/gpt-5-6-sol) | OpenAI | 94.6 | Observed 2026-11-21 · source benchlm:models |
| 7 | [Claude Opus 4.7](https://modelscale.dev/models/claude-opus-4-7) | Anthropic | 94.2 | Observed 2026-11-21 · source benchlm:models |
| 7 | [Claude Opus 4.7 (Adaptive)](https://modelscale.dev/models/claude-opus-4-7-max) | Anthropic | 94.2 | Last good 2026-11-18 · source benchlm:models |
| 7 | [Kimi K3.1 Pro](https://modelscale.dev/models/kimi-3-1-pro) | Moonshot AI | 94.2 | Observed 2026-11-21 · source benchlm:models |
| 10 | [Claude Mythos 5](https://modelscale.dev/models/claude-mythos-5) | Anthropic | 94.1 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Claude Opus 4.8](https://modelscale.dev/models/claude-opus-4-8) | Anthropic | 93.6 | Observed 2026-11-21 · source benchlm:models |
| 11 | [GPT-5.5](https://modelscale.dev/models/gpt-5-5) | OpenAI | 93.6 | Observed 2026-11-21 · source benchlm:models |
| 13 | [Kimi K3](https://modelscale.dev/models/kimi-3) | Moonshot AI | 93.5 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Inkling 2 Small](https://modelscale.dev/models/inkling-2-small) | Thinking Machines Lab | 93 | Observed 2026-11-21 · source benchlm:models |
| 15 | [GPT-5.6 Terra](https://modelscale.dev/models/gpt-5-6-terra) | OpenAI | 92.9 | Observed 2026-11-21 · source benchlm:models |
| 16 | [GPT-5.4](https://modelscale.dev/models/gpt-5-4) | OpenAI | 92.8 | Observed 2026-11-21 · source benchlm:models |
| 16 | [Ornith-1.5-397B](https://modelscale.dev/models/ornith-1-5-397b) | Ornith AI | 92.8 | Observed 2026-11-21 · source benchlm:models |
| 18 | [Qwen3.8 Max](https://modelscale.dev/models/qwen3-8-max) | Alibaba | 92.6 | Observed 2026-11-21 · source benchlm:models |
| 19 | [GPT-5.2](https://modelscale.dev/models/gpt-5-2) | OpenAI | 92.4 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Qwen3.7 Max](https://modelscale.dev/models/qwen3-7-max) | Alibaba | 92.4 | Observed 2026-11-21 · source benchlm:models |
| 21 | [GPT-5.6 Luna](https://modelscale.dev/models/gpt-5-6-luna) | OpenAI | 92.3 | Observed 2026-11-21 · source benchlm:models |
| 21 | [Hy4 preview](https://modelscale.dev/models/hy4-preview) | Tencent | 92.3 | Observed 2026-11-21 · source benchlm:models |
| 23 | [Gemini 3.5 Flash](https://modelscale.dev/models/gemini-3-5-flash) | Google | 92.2 | Observed 2026-11-21 · source benchlm:models |
| 24 | [Qwen3.8-Flash-Next](https://modelscale.dev/models/qwen3-8-flash-next) | Alibaba | 91.7 | Observed 2026-11-21 · source benchlm:models |
| 25 | [Claude Opus 4.6](https://modelscale.dev/models/claude-opus-4-6) | Anthropic | 91.3 | Observed 2026-11-21 · source benchlm:models |
| 26 | [GLM-5.2](https://modelscale.dev/models/glm-5-2) | Z.AI | 91.2 | Observed 2026-11-21 · source benchlm:models |
| 27 | [Qwen3.8-Omni-Flash](https://modelscale.dev/models/qwen3-8-omni-flash) | Alibaba | 91 | Observed 2026-11-21 · source benchlm:models |
| 28 | [DeepSeek V4.1 Flash 1017](https://modelscale.dev/models/deepseek-v4-1-flash) | DeepSeek | 90.9 | Observed 2026-11-21 · source benchlm:models |
| 29 | [Kimi K2.6](https://modelscale.dev/models/kimi-2-6) | Moonshot AI | 90.5 | Observed 2026-11-21 · source benchlm:models |
| 30 | [Qwen3.6 Plus](https://modelscale.dev/models/qwen3-6-plus) | Alibaba | 90.4 | Observed 2026-11-21 · source benchlm:models |
| 31 | [Qwen3.7 Plus](https://modelscale.dev/models/qwen3-7-plus) | Alibaba | 90.3 | Observed 2026-11-21 · source benchlm:models |
| 32 | [DeepSeek V4 Pro](https://modelscale.dev/models/deepseek-v4-pro) | DeepSeek | 90.1 | Observed 2026-11-21 · source benchlm:models |
| 32 | [DeepSeek V4 Pro 0813](https://modelscale.dev/models/deepseek-v4-pro-max) | DeepSeek | 90.1 | Last good 2026-11-18 · source benchlm:models |
| 32 | [Grok 4.3](https://modelscale.dev/models/grok-4-3) | xAI | 90.1 | Observed 2026-11-21 · source benchlm:models |
| 35 | [Claude Sonnet 4.6](https://modelscale.dev/models/claude-sonnet-4-6) | Anthropic | 89.9 | Observed 2026-11-21 · source benchlm:models |
| 35 | [Interfaze Beta](https://modelscale.dev/models/interfaze-beta) | Interfaze | 89.9 | Observed 2026-11-21 · source benchlm:models |
| 37 | [Inkling-Small](https://modelscale.dev/models/inkling-small) | Thinking Machines Lab | 89.5 | Observed 2026-11-21 · source benchlm:models |
| 38 | [Ornith-1.5-35B-A3B](https://modelscale.dev/models/ornith-1-5-35b-a3b) | Ornith AI | 89.2 | Observed 2026-11-21 · source benchlm:models |
| 38 | [Qwen3.8-27B](https://modelscale.dev/models/qwen3-8-27b) | Alibaba | 89.2 | Observed 2026-11-21 · source benchlm:models |
| 40 | [Qwen3.5 397B](https://modelscale.dev/models/qwen3-5-397b) | Alibaba | 88.4 | Observed 2026-11-21 · source benchlm:models |
| 41 | [DeepSeek V4 Flash 0731](https://modelscale.dev/models/deepseek-v4-flash-max) | DeepSeek | 88.1 | Observed 2026-11-21 · source benchlm:models |
| 42 | [GPT-5.4 mini](https://modelscale.dev/models/gpt-5-4-mini) | OpenAI | 88 | Observed 2026-11-21 · source benchlm:models |
| 43 | [Inkling](https://modelscale.dev/models/inkling) | Thinking Machines Lab | 87.9 | Observed 2026-11-21 · source benchlm:models |
| 44 | [Qwen3.6-27B](https://modelscale.dev/models/qwen3-6-27b) | Alibaba | 87.8 | Observed 2026-11-21 · source benchlm:models |
| 45 | [Kimi K2.5](https://modelscale.dev/models/kimi-k2-5) | Moonshot AI | 87.6 | Observed 2026-11-21 · source benchlm:models |
| 45 | [Kimi K2.5 (Reasoning)](https://modelscale.dev/models/kimi-k2-5-reasoning) | Moonshot AI | 87.6 | Last good 2026-11-18 · source benchlm:models |
| 47 | [Hy3 Preview](https://modelscale.dev/models/hy3-preview) | Tencent | 87.2 | Observed 2026-11-21 · source benchlm:models |
| 48 | [Claude Opus 4.5](https://modelscale.dev/models/claude-opus-4-5) | Anthropic | 87 | Observed 2026-11-21 · source benchlm:models |
| 48 | [Nemotron 3 Ultra](https://modelscale.dev/models/nemotron-3-ultra-500b) | NVIDIA | 87 | Observed 2026-11-21 · source benchlm:models |
| 50 | [Inkling 2 Mini](https://modelscale.dev/models/inkling-2-mini) | Thinking Machines Lab | 86.6 | Observed 2026-11-21 · source benchlm:models |
| 50 | [Qwen3.5-122B-A10B](https://modelscale.dev/models/qwen3-5-122b-a10b) | Alibaba | 86.6 | Observed 2026-11-21 · source benchlm:models |
| 52 | [Ornith-1.5-9B](https://modelscale.dev/models/ornith-1-5-9b) | Ornith AI | 86.4 | Observed 2026-11-21 · source benchlm:models |
| 53 | [GLM-5](https://modelscale.dev/models/glm-5) | Z.AI | 86 | Observed 2026-11-21 · source benchlm:models |
| 53 | [Qwen3.6-35B-A3B](https://modelscale.dev/models/qwen3-6-35b-a3b) | Alibaba | 86 | Observed 2026-11-21 · source benchlm:models |
| 55 | [Ternary Bonsai 2 27B](https://modelscale.dev/models/ternary-bonsai-2-27b) | Prism ML | 85.76 | Observed 2026-11-21 · source benchlm:models |
| 56 | [GLM-4.7](https://modelscale.dev/models/glm-4-7) | Z.AI | 85.7 | Observed 2026-11-21 · source benchlm:models |
| 57 | [Qwen3.5-27B](https://modelscale.dev/models/qwen3-5-27b) | Alibaba | 85.5 | Observed 2026-11-21 · source benchlm:models |
| 58 | [Ling 3.0 Flash](https://modelscale.dev/models/ling-3-0-flash) | InclusionAI | 84.97 | Observed 2026-11-21 · source benchlm:models |
| 59 | [Gemma 4 31B](https://modelscale.dev/models/gemma-4-31b) | Google | 84.3 | Observed 2026-11-21 · source benchlm:models |
| 60 | [MAI-Thinking-1](https://modelscale.dev/models/mai-thinking-1) | Microsoft | 84.2 | Observed 2026-11-21 · source benchlm:models |
| 60 | [Qwen3.5-35B-A3B](https://modelscale.dev/models/qwen3-5-35b-a3b) | Alibaba | 84.2 | Observed 2026-11-21 · source benchlm:models |
| 62 | [Ling 3.0 Flash FP8](https://modelscale.dev/models/ling-3-0-flash-fp8) | InclusionAI | 84 | Observed 2026-11-21 · source benchlm:models |
| 63 | [MiMo-V2-Flash](https://modelscale.dev/models/mimo-v2-flash) | Xiaomi | 83.7 | Observed 2026-11-21 · source benchlm:models |
| 64 | [Claude Sonnet 4.5](https://modelscale.dev/models/claude-sonnet-4-5) | Anthropic | 83.4 | Observed 2026-11-21 · source benchlm:models |
| 65 | [Gemini 2.5 Pro](https://modelscale.dev/models/gemini-2-5-pro) | Google | 83 | Observed 2026-11-21 · source benchlm:models |
| 66 | [GPT-5.4 nano](https://modelscale.dev/models/gpt-5-4-nano) | OpenAI | 82.8 | Observed 2026-11-21 · source benchlm:models |
| 67 | [Gemma 4 26B A4B](https://modelscale.dev/models/gemma-4-26b-a4b) | Google | 82.3 | Observed 2026-11-21 · source benchlm:models |
| 68 | [Qwen3 235B 2507 (Reasoning)](https://modelscale.dev/models/qwen3-235b-2507-reasoning) | Alibaba | 81.1 | Last good 2026-11-18 · source benchlm:models |
| 69 | [DeepSeek-R1-0528](https://modelscale.dev/models/deepseek-r1-0528) | DeepSeek | 81 | Observed 2026-11-21 · source benchlm:models |
| 70 | [Grok 3 Mini](https://modelscale.dev/models/grok-3-mini) | xAI | 80.3 | Observed 2026-11-21 · source benchlm:models |
| 71 | [DeepSeek V3.1 (Reasoning)](https://modelscale.dev/models/deepseek-v3-1-reasoning) | DeepSeek | 80.1 | Last good 2026-11-18 · source benchlm:models |
| 71 | [GPT-OSS 120B](https://modelscale.dev/models/gpt-oss-120b) | OpenAI | 80.1 | Observed 2026-11-21 · source benchlm:models |
| 73 | [Claude 4 Opus Thinking](https://modelscale.dev/models/claude-4-opus-thinking) | Anthropic | 79.6 | Observed 2026-11-21 · source benchlm:models |
| 74 | [Nemotron 3 Super 100B](https://modelscale.dev/models/nemotron-3-super-100b) | NVIDIA | 79.23 | Observed 2026-11-21 · source benchlm:models |
| 75 | [o1-pro](https://modelscale.dev/models/o1-pro) | OpenAI | 79 | Observed 2026-11-21 · source benchlm:models |
| 76 | [Gemma 4 12B](https://modelscale.dev/models/gemma-4-12b) | Google | 78.8 | Observed 2026-11-21 · source benchlm:models |
| 77 | [Claude 3.7 Sonnet Thinking](https://modelscale.dev/models/claude-3-7-sonnet-thinking) | Anthropic | 78.2 | Observed 2026-11-21 · source benchlm:models |
| 78 | [o4-mini](https://modelscale.dev/models/o4-mini) | OpenAI | 77.6 | Observed 2026-11-21 · source benchlm:models |
| 79 | [Qwen3 235B 2507](https://modelscale.dev/models/qwen3-235b-2507) | Alibaba | 77.5 | Observed 2026-11-21 · source benchlm:models |
| 80 | [o3-mini](https://modelscale.dev/models/o3-mini) | OpenAI | 77.2 | Observed 2026-11-21 · source benchlm:models |
| 81 | [Nemotron Ultra 253B](https://modelscale.dev/models/nemotron-ultra-253b) | NVIDIA | 76.01 | Observed 2026-11-21 · source benchlm:models |
| 82 | [o1](https://modelscale.dev/models/o1) | OpenAI | 75.7 | Observed 2026-11-21 · source benchlm:models |
| 83 | [Nemotron 3.5 Lightning 30B A3B NVFP4](https://modelscale.dev/models/nemotron-3-5-lightning-30b-a3b-nvfp4) | NVIDIA | 75.57 | Observed 2026-11-21 · source benchlm:models |
| 84 | [Grok 3 [Beta]](https://modelscale.dev/models/grok-3-beta) | xAI | 75.4 | Observed 2026-11-21 · source benchlm:models |
| 85 | [GLM-4.7-Flash](https://modelscale.dev/models/glm-4-7-flash) | Z.AI | 75.2 | Observed 2026-11-21 · source benchlm:models |
| 86 | [Kimi K2](https://modelscale.dev/models/kimi-k2) | Moonshot AI | 75.1 | Observed 2026-11-21 · source benchlm:models |
| 87 | [Claude 4 Opus](https://modelscale.dev/models/claude-4-opus) | Anthropic | 74.9 | Observed 2026-11-21 · source benchlm:models |
| 87 | [DeepSeek V3.1](https://modelscale.dev/models/deepseek-v3-1) | DeepSeek | 74.9 | Observed 2026-11-21 · source benchlm:models |
| 89 | [LFM2.5-24B-A2B](https://modelscale.dev/models/lfm2-5-24b-a2b) | LiquidAI | 74.42 | Observed 2026-11-21 · source benchlm:models |
| 90 | [Nemotron 3 Nano 30B](https://modelscale.dev/models/nemotron-3-nano-30b) | NVIDIA | 73 | Observed 2026-11-21 · source benchlm:models |
| 91 | [Nemotron 3 Nano Omni 30B A3B](https://modelscale.dev/models/nemotron-3-nano-omni-30b-a3b) | NVIDIA | 72.2 | Observed 2026-11-21 · source benchlm:models |
| 92 | [DeepSeek-R1](https://modelscale.dev/models/deepseek-r1) | DeepSeek | 71.5 | Observed 2026-11-21 · source benchlm:models |
| 92 | [GPT-OSS 20B](https://modelscale.dev/models/gpt-oss-20b) | OpenAI | 71.5 | Observed 2026-11-21 · source benchlm:models |
| 94 | [Ministral 3 14B (Reasoning)](https://modelscale.dev/models/ministral-3-14b-reasoning) | Mistral | 71.2 | Last good 2026-11-18 · source benchlm:models |
| 95 | [Qwen3 235B-A22B (Reasoning)](https://modelscale.dev/models/qwen3-235b-a22b-reasoning) | Alibaba | 71.1 | Observed 2026-11-21 · source benchlm:models |
| 96 | [ZAYA1-8B](https://modelscale.dev/models/zaya1-8b) | Zyphra | 71 | Observed 2026-11-21 · source benchlm:models |
| 97 | [Magistral Medium](https://modelscale.dev/models/magistral-medium) | Mistral | 70.8 | Observed 2026-11-21 · source benchlm:models |
| 98 | [Spark-X2.6-4B](https://modelscale.dev/models/spark-x2-6-4b) | SparkLLM | 69.9 | Observed 2026-11-21 · source benchlm:models |
| 99 | [GPT-4.5](https://modelscale.dev/models/gpt-4-5) | OpenAI | 69.5 | Observed 2026-11-21 · source benchlm:models |
| 100 | [Phi-4-reasoning-plus](https://modelscale.dev/models/phi-4-reasoning-plus) | Microsoft | 68.9 | Observed 2026-11-21 · source benchlm:models |
| 101 | [Qwen3 32B (Reasoning)](https://modelscale.dev/models/qwen3-32b-reasoning) | Alibaba | 68.4 | Observed 2026-11-21 · source benchlm:models |
| 102 | [Magistral Small](https://modelscale.dev/models/magistral-small) | Mistral | 68.2 | Observed 2026-11-21 · source benchlm:models |
| 103 | [Claude 3.7 Sonnet](https://modelscale.dev/models/claude-3-7-sonnet) | Anthropic | 68 | Observed 2026-11-21 · source benchlm:models |
| 104 | [Granite 4.2 30B](https://modelscale.dev/models/granite-4-2-30b) | IBM | 66.41 | Observed 2026-11-21 · source benchlm:models |
| 105 | [GPT-4.1](https://modelscale.dev/models/gpt-4-1) | OpenAI | 66.3 | Observed 2026-11-21 · source benchlm:models |
| 106 | [Phi-4-reasoning](https://modelscale.dev/models/phi-4-reasoning) | Microsoft | 65.8 | Observed 2026-11-21 · source benchlm:models |
| 107 | [QwQ-32B](https://modelscale.dev/models/qwq-32b) | Alibaba | 65.6 | Observed 2026-11-21 · source benchlm:models |
| 108 | [DeepSeek R1 Distill Llama 70B](https://modelscale.dev/models/deepseek-r1-distill-llama-70b) | DeepSeek | 65.2 | Observed 2026-11-21 · source benchlm:models |
| 108 | [Gemini 2.0 Flash](https://modelscale.dev/models/gemini-2-0-flash) | Google | 65.2 | Observed 2026-11-21 · source benchlm:models |
| 110 | [Gemini 2.5 Flash-Lite](https://modelscale.dev/models/gemini-2-5-flash-lite) | Google | 64.6 | Observed 2026-11-21 · source benchlm:models |
| 111 | [GPT-4.1 mini](https://modelscale.dev/models/gpt-4-1-mini) | OpenAI | 64.2 | Observed 2026-11-21 · source benchlm:models |
| 112 | [Granite 4.2 8B](https://modelscale.dev/models/granite-4-2-8b) | IBM | 64.14 | Observed 2026-11-21 · source benchlm:models |
| 113 | [LFM3-1.7B](https://modelscale.dev/models/lfm3-1-7b) | LiquidAI | 63.37 | Observed 2026-11-21 · source benchlm:models |
| 114 | [Qwen3 235B-A22B](https://modelscale.dev/models/qwen3-235b-a22b) | Alibaba | 62.9 | Observed 2026-11-21 · source benchlm:models |
| 115 | [DeepSeek-R1-0528-Qwen3-8B](https://modelscale.dev/models/deepseek-r1-0528-qwen3-8b) | DeepSeek | 61.1 | Observed 2026-11-21 · source benchlm:models |
| 116 | [Qwen2.5-Max](https://modelscale.dev/models/qwen2-5-max) | Alibaba | 60.1 | Observed 2026-11-21 · source benchlm:models |
| 117 | [o1-mini](https://modelscale.dev/models/o1-mini) | OpenAI | 60 | Observed 2026-11-21 · source benchlm:models |
| 118 | [Claude 3.5 Sonnet](https://modelscale.dev/models/claude-3-5-sonnet) | Anthropic | 59.4 | Observed 2026-11-21 · source benchlm:models |
| 119 | [DeepSeek R1 Distill Qwen 14B](https://modelscale.dev/models/deepseek-r1-distill-qwen-14b) | DeepSeek | 59.1 | Observed 2026-11-21 · source benchlm:models |
| 119 | [DeepSeek V3](https://modelscale.dev/models/deepseek-v3) | DeepSeek | 59.1 | Observed 2026-11-21 · source benchlm:models |
| 121 | [Ling 2.6 Flash](https://modelscale.dev/models/ling-2-6-flash) | InclusionAI | 59 | Observed 2026-11-21 · source benchlm:models |
| 122 | [Gemma 4 E4B](https://modelscale.dev/models/gemma-4-e4b) | Google | 58.6 | Observed 2026-11-21 · source benchlm:models |
| 123 | [Mellum2-12B-A2.5B-Thinking](https://modelscale.dev/models/mellum2-12b-a2-5b-thinking) | JetBrains | 57.6 | Observed 2026-11-21 · source benchlm:models |
| 124 | [ZAYA1-74B-Preview](https://modelscale.dev/models/zaya1-74b-preview) | Zyphra | 57.3 | Observed 2026-11-21 · source benchlm:models |
| 125 | [Phi-4](https://modelscale.dev/models/phi-4) | Microsoft | 56.1 | Observed 2026-11-21 · source benchlm:models |
| 126 | [Grok 2](https://modelscale.dev/models/grok-2) | xAI | 56 | Observed 2026-11-21 · source benchlm:models |
| 127 | [Granite 4.2 3B](https://modelscale.dev/models/granite-4-2-3b) | IBM | 54.8 | Observed 2026-11-21 · source benchlm:models |
| 128 | [Qwen3 32B](https://modelscale.dev/models/qwen3-32b) | Alibaba | 54.6 | Observed 2026-11-21 · source benchlm:models |
| 129 | [MiniMax-Text-01](https://modelscale.dev/models/minimax-text-01) | MiniMax | 54.4 | Observed 2026-11-21 · source benchlm:models |
| 130 | [Grok 2 mini](https://modelscale.dev/models/grok-2-mini) | xAI | 51 | Observed 2026-11-21 · source benchlm:models |
| 131 | [Gemini 2.0 Flash-Lite](https://modelscale.dev/models/gemini-2-0-flash-lite) | Google | 50.5 | Observed 2026-11-21 · source benchlm:models |
| 131 | [Llama 3.3 70B](https://modelscale.dev/models/llama-3-3-70b) | Meta | 50.5 | Observed 2026-11-21 · source benchlm:models |
| 133 | [GPT-4.1 nano](https://modelscale.dev/models/gpt-4-1-nano) | OpenAI | 50.3 | Observed 2026-11-21 · source benchlm:models |
| 134 | [Gemini 1.5 Flash](https://modelscale.dev/models/gemini-1-5-flash) | Google | 50 | Observed 2026-11-21 · source benchlm:models |
| 135 | [Llama 3.1 70B](https://modelscale.dev/models/llama-3-1-70b) | Meta | 46.7 | Observed 2026-11-21 · source benchlm:models |
| 136 | [Qwen2.5-VL-32B](https://modelscale.dev/models/qwen2-5-vl-32b) | Alibaba | 46 | Observed 2026-11-21 · source benchlm:models |
| 137 | [Mistral Small 3.1](https://modelscale.dev/models/mistral-small-3-1) | Mistral | 45.96 | Observed 2026-11-21 · source benchlm:models |
| 138 | [Gemma 4 E2B](https://modelscale.dev/models/gemma-4-e2b) | Google | 43.4 | Observed 2026-11-21 · source benchlm:models |
| 138 | [Soofi S 30B-A3B](https://modelscale.dev/models/soofi-s-30b-a3b) | Soofi Project | 43.4 | Observed 2026-11-21 · source benchlm:models |
| 140 | [Claude 3.5 Haiku](https://modelscale.dev/models/claude-3-5-haiku) | Anthropic | 41.6 | Observed 2026-11-21 · source benchlm:models |
| 141 | [Mellum2-12B-A2.5B-Instruct](https://modelscale.dev/models/mellum2-12b-a2-5b-instruct) | JetBrains | 40.9 | Observed 2026-11-21 · source benchlm:models |
| 142 | [LFM2.5-350M](https://modelscale.dev/models/lfm2-5-350m) | LiquidAI | 30.64 | Observed 2026-11-21 · source benchlm:models |
| 143 | [Llama 3.1 8B](https://modelscale.dev/models/llama-3-1-8b) | Meta | 30.4 | Observed 2026-11-21 · source benchlm:models |
| 143 | [Phi-3.5-mini](https://modelscale.dev/models/phi-3-5-mini) | Microsoft | 30.4 | Observed 2026-11-21 · source benchlm:models |
| 145 | [Granite-4.0-1B](https://modelscale.dev/models/granite-4-0-1b) | IBM | 29.91 | Observed 2026-11-21 · source benchlm:models |
| 146 | [Granite-4.0-H-1B](https://modelscale.dev/models/granite-4-0-h-1b) | IBM | 29.69 | Observed 2026-11-21 · source benchlm:models |
| 147 | [Granite-4.0-H-350M](https://modelscale.dev/models/granite-4-0-h-350m) | IBM | 26.12 | Observed 2026-11-21 · source benchlm:models |
| 148 | [LFM2.5-VL-450M](https://modelscale.dev/models/lfm2-5-vl-450m) | LiquidAI | 25.66 | Observed 2026-11-21 · source benchlm:models |
| 149 | [LFM2.5-230M](https://modelscale.dev/models/lfm2-5-230m) | LiquidAI | 25.41 | Observed 2026-11-21 · source benchlm:models |
| 150 | [Granite-4.0-350M](https://modelscale.dev/models/granite-4-0-350m) | IBM | 24.11 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
