# τ²-bench results leaderboard

Agentic benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/agentic-tau2-bench

τ²-Bench Tool-Agent-User Evaluation. Every model the catalog carries a published τ²-bench results value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Agentic |
| Measure | Published domain success or pass^k results |
| Tasks | Airline, retail, and telecom customer-service task sets |
| Difficulty | Dual-control customer-service workflows |
| Published by | [τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment](https://arxiv.org/abs/2506.07982) |

This route is a sourced ledger for published τ²-bench results. Most current rows come from Artificial Analysis's telecom implementation, while named provider rows can use telecom, airline, retail, or aggregate setups.

## τ²-bench results ranking

154 models with a published τ²-bench results value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Published domain success or pass^k results | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [GLM-5.2](https://modelscale.dev/models/glm-5-2) | Z.AI | 99.1 | Observed 2026-11-21 · source benchlm:models |
| 2 | [GPT-5.4](https://modelscale.dev/models/gpt-5-4) | OpenAI | 98.9 | Observed 2026-11-21 · source benchlm:models |
| 3 | [Claude Fable 5](https://modelscale.dev/models/claude-fable-5) | Anthropic | 98.5 | Observed 2026-11-21 · source benchlm:models |
| 3 | [GLM-5-Turbo](https://modelscale.dev/models/glm-5-turbo) | Z.AI | 98.5 | Observed 2026-11-21 · source benchlm:models |
| 3 | [GLM-5V-Turbo](https://modelscale.dev/models/glm-5v-turbo) | Z.AI | 98.5 | Observed 2026-11-21 · source benchlm:models |
| 3 | [Step 3.7 Flash](https://modelscale.dev/models/step-3-7-flash) | StepFun | 98.5 | Observed 2026-11-21 · source benchlm:models |
| 7 | [GLM-5](https://modelscale.dev/models/glm-5) | Z.AI | 98.2 | Observed 2026-11-21 · source benchlm:models |
| 8 | [GPT-5.5](https://modelscale.dev/models/gpt-5-5) | OpenAI | 98 | Observed 2026-11-21 · source benchlm:models |
| 9 | [GLM-5.1](https://modelscale.dev/models/glm-5-1) | Z.AI | 97.7 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Grok 4.3](https://modelscale.dev/models/grok-4-3) | xAI | 97.7 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Qwen3.6 Plus](https://modelscale.dev/models/qwen3-6-plus) | Alibaba | 97.7 | Observed 2026-11-21 · source benchlm:models |
| 12 | [DeepSeek V4 Pro](https://modelscale.dev/models/deepseek-v4-pro) | DeepSeek | 96.2 | Observed 2026-11-21 · source benchlm:models |
| 12 | [DeepSeek V4 Pro 0813](https://modelscale.dev/models/deepseek-v4-pro-max) | DeepSeek | 96.2 | Last good 2026-11-18 · source benchlm:models |
| 14 | [GLM-4.7](https://modelscale.dev/models/glm-4-7) | Z.AI | 95.9 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Kimi K2.5](https://modelscale.dev/models/kimi-k2-5) | Moonshot AI | 95.9 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Kimi K2.5 (Reasoning)](https://modelscale.dev/models/kimi-k2-5-reasoning) | Moonshot AI | 95.9 | Last good 2026-11-18 · source benchlm:models |
| 14 | [Kimi K2.6](https://modelscale.dev/models/kimi-2-6) | Moonshot AI | 95.9 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Qwen 3.6 Max (preview)](https://modelscale.dev/models/qwen3-6-max-preview) | Alibaba | 95.9 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Gemini 3.1 Pro](https://modelscale.dev/models/gemini-3-1-pro) | Google | 95.6 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Qwen3.5 397B](https://modelscale.dev/models/qwen3-5-397b) | Alibaba | 95.6 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Qwen3.5 397B (Reasoning)](https://modelscale.dev/models/qwen3-5-397b-reasoning) | Alibaba | 95.6 | Last good 2026-11-18 · source benchlm:models |
| 22 | [Gemini 3.5 Flash](https://modelscale.dev/models/gemini-3-5-flash) | Google | 95.3 | Observed 2026-11-21 · source benchlm:models |
| 22 | [Qwen3.6-35B-A3B](https://modelscale.dev/models/qwen3-6-35b-a3b) | Alibaba | 95.3 | Observed 2026-11-21 · source benchlm:models |
| 24 | [MiMo-V2-Pro](https://modelscale.dev/models/mimo-v2-pro) | Xiaomi | 95 | Observed 2026-11-21 · source benchlm:models |
| 25 | [Qwen3.7 Max](https://modelscale.dev/models/qwen3-7-max) | Alibaba | 94.7 | Observed 2026-11-21 · source benchlm:models |
| 26 | [Claude Opus 4.8](https://modelscale.dev/models/claude-opus-4-8) | Anthropic | 94.4 | Observed 2026-11-21 · source benchlm:models |
| 27 | [MiMo-V2.5-Pro](https://modelscale.dev/models/mimo-v2-5-pro) | Xiaomi | 94.2 | Observed 2026-11-21 · source benchlm:models |
| 27 | [Mistral Medium 3.5 128B](https://modelscale.dev/models/mistral-medium-3-5-128b) | Mistral | 94.2 | Observed 2026-11-21 · source benchlm:models |
| 27 | [Qwen3.6-27B](https://modelscale.dev/models/qwen3-6-27b) | Alibaba | 94.2 | Observed 2026-11-21 · source benchlm:models |
| 30 | [Qwen3.5-27B](https://modelscale.dev/models/qwen3-5-27b) | Alibaba | 93.9 | Observed 2026-11-21 · source benchlm:models |
| 31 | [Qwen3.5-122B-A10B](https://modelscale.dev/models/qwen3-5-122b-a10b) | Alibaba | 93.6 | Observed 2026-11-21 · source benchlm:models |
| 32 | [GPT-5.4 mini](https://modelscale.dev/models/gpt-5-4-mini) | OpenAI | 93.4 | Observed 2026-11-21 · source benchlm:models |
| 33 | [Grok 4.1 Fast (Reasoning)](https://modelscale.dev/models/grok-4-1-fast-reasoning) | xAI | 93.3 | Observed 2026-11-21 · source benchlm:models |
| 34 | [Qwen3.7 Plus](https://modelscale.dev/models/qwen3-7-plus) | Alibaba | 93 | Observed 2026-11-21 · source benchlm:models |
| 35 | [GPT-5.4 nano](https://modelscale.dev/models/gpt-5-4-nano) | OpenAI | 92.5 | Observed 2026-11-21 · source benchlm:models |
| 36 | [Claude Opus 4.6 (Adaptive)](https://modelscale.dev/models/claude-opus-4-6-thinking) | Anthropic | 92.1 | Observed 2026-11-21 · source benchlm:models |
| 36 | [GPT-5.2-Codex](https://modelscale.dev/models/gpt-5-2-codex) | OpenAI | 92.1 | Observed 2026-11-21 · source benchlm:models |
| 36 | [Qwen3.5-4B](https://modelscale.dev/models/qwen3-5-4b) | Alibaba | 92.1 | Observed 2026-11-21 · source benchlm:models |
| 39 | [Muse Spark](https://modelscale.dev/models/muse-spark) | Meta | 91.5 | Observed 2026-11-21 · source benchlm:models |
| 40 | [MiMo-V2-Omni](https://modelscale.dev/models/mimo-v2-omni) | Xiaomi | 91.2 | Observed 2026-11-21 · source benchlm:models |
| 41 | [Kimi K2.7 Code](https://modelscale.dev/models/kimi-k2-7-code) | Moonshot AI | 90.1 | Observed 2026-11-21 · source benchlm:models |
| 41 | [Trinity-Large-Preview](https://modelscale.dev/models/trinity-large-preview) | Arcee AI | 90.1 | Observed 2026-11-21 · source benchlm:models |
| 41 | [Trinity-Large-Thinking](https://modelscale.dev/models/trinity-large-thinking) | Arcee AI | 90.1 | Observed 2026-11-21 · source benchlm:models |
| 44 | [Claude Opus 4.5 Thinking](https://modelscale.dev/models/claude-opus-4-5-thinking) | Anthropic | 89.5 | Observed 2026-11-21 · source benchlm:models |
| 45 | [Qwen3.5-35B-A3B](https://modelscale.dev/models/qwen3-5-35b-a3b) | Alibaba | 89.2 | Observed 2026-11-21 · source benchlm:models |
| 46 | [MiniMax M3](https://modelscale.dev/models/minimax-m3) | MiniMax | 88.9 | Observed 2026-11-21 · source benchlm:models |
| 47 | [Claude Opus 4.7](https://modelscale.dev/models/claude-opus-4-7) | Anthropic | 88.6 | Observed 2026-11-21 · source benchlm:models |
| 47 | [Claude Opus 4.7 (Adaptive)](https://modelscale.dev/models/claude-opus-4-7-max) | Anthropic | 88.6 | Last good 2026-11-18 · source benchlm:models |
| 49 | [LFM2.5-8B-A1B](https://modelscale.dev/models/lfm2-5-8b-a1b) | LiquidAI | 88.07 | Observed 2026-11-21 · source benchlm:models |
| 50 | [Gemini 3 Pro](https://modelscale.dev/models/gemini-3-pro) | Google | 87.1 | Observed 2026-11-21 · source benchlm:models |
| 51 | [Spark-X2.6-4B](https://modelscale.dev/models/spark-x2-6-4b) | SparkLLM | 86.9 | Observed 2026-11-21 · source benchlm:models |
| 52 | [GPT-5 (medium)](https://modelscale.dev/models/gpt-5-medium) | OpenAI | 86.5 | Last good 2026-11-18 · source benchlm:models |
| 53 | [Claude Opus 4.5](https://modelscale.dev/models/claude-opus-4-5) | Anthropic | 86.3 | Observed 2026-11-21 · source benchlm:models |
| 53 | [GPT-5.6 Terra](https://modelscale.dev/models/gpt-5-6-terra) | OpenAI | 86.3 | Observed 2026-11-21 · source benchlm:models |
| 53 | [Solar Pro 3](https://modelscale.dev/models/solar-pro-3) | Upstage | 86.3 | Observed 2026-11-21 · source benchlm:models |
| 56 | [GPT-5.3 Codex](https://modelscale.dev/models/gpt-5-3-codex) | OpenAI | 86 | Observed 2026-11-21 · source benchlm:models |
| 56 | [Ling 2.6 Flash](https://modelscale.dev/models/ling-2-6-flash) | InclusionAI | 86 | Observed 2026-11-21 · source benchlm:models |
| 58 | [GPT-5.6 Sol](https://modelscale.dev/models/gpt-5-6-sol) | OpenAI | 85.1 | Observed 2026-11-21 · source benchlm:models |
| 59 | [Command A+](https://modelscale.dev/models/command-a-plus) | Cohere | 85 | Observed 2026-11-21 · source benchlm:models |
| 60 | [Claude Opus 4.6](https://modelscale.dev/models/claude-opus-4-6) | Anthropic | 84.8 | Observed 2026-11-21 · source benchlm:models |
| 60 | [GPT-5](https://modelscale.dev/models/gpt-5) | OpenAI | 84.8 | Observed 2026-11-21 · source benchlm:models |
| 60 | [GPT-5 (high)](https://modelscale.dev/models/gpt-5-high) | OpenAI | 84.8 | Last good 2026-11-18 · source benchlm:models |
| 60 | [GPT-5.2](https://modelscale.dev/models/gpt-5-2) | OpenAI | 84.8 | Observed 2026-11-21 · source benchlm:models |
| 60 | [MiniMax M2.7](https://modelscale.dev/models/minimax-m2-7) | MiniMax | 84.8 | Observed 2026-11-21 · source benchlm:models |
| 65 | [MiMo-V2-Flash](https://modelscale.dev/models/mimo-v2-flash) | Xiaomi | 83.9 | Observed 2026-11-21 · source benchlm:models |
| 66 | [Nemotron 3 Ultra](https://modelscale.dev/models/nemotron-3-ultra-500b) | NVIDIA | 83.3 | Observed 2026-11-21 · source benchlm:models |
| 67 | [GPT-5.1-Codex](https://modelscale.dev/models/gpt-5-1-codex) | OpenAI | 83 | Observed 2026-11-21 · source benchlm:models |
| 67 | [GPT-5.1-Codex-Max](https://modelscale.dev/models/gpt-5-1-codex-max) | OpenAI | 83 | Observed 2026-11-21 · source benchlm:models |
| 69 | [GPT-5.1](https://modelscale.dev/models/gpt-5-1) | OpenAI | 81.9 | Observed 2026-11-21 · source benchlm:models |
| 70 | [o3](https://modelscale.dev/models/o3) | OpenAI | 80.7 | Observed 2026-11-21 · source benchlm:models |
| 71 | [LLaDA2.2-flash](https://modelscale.dev/models/llada2-2-flash) | InclusionAI | 80.33 | Observed 2026-11-21 · source benchlm:models |
| 72 | [Ternary Bonsai 2 27B](https://modelscale.dev/models/ternary-bonsai-2-27b) | Prism ML | 80.22 | Observed 2026-11-21 · source benchlm:models |
| 73 | [Claude Sonnet 4.6](https://modelscale.dev/models/claude-sonnet-4-6) | Anthropic | 79.5 | Observed 2026-11-21 · source benchlm:models |
| 74 | [DeepSeek V3.2](https://modelscale.dev/models/deepseek-v3-2) | DeepSeek | 78.9 | Observed 2026-11-21 · source benchlm:models |
| 75 | [Agents-A1-4B](https://modelscale.dev/models/agents-a1-4b) | InternScience | 78.2 | Observed 2026-11-21 · source benchlm:models |
| 76 | [GLM-4.6](https://modelscale.dev/models/glm-4-6) | Z.AI | 76.9 | Observed 2026-11-21 · source benchlm:models |
| 77 | [Grok Code Fast 1](https://modelscale.dev/models/grok-code-fast-1) | xAI | 75.7 | Observed 2026-11-21 · source benchlm:models |
| 78 | [Grok 4](https://modelscale.dev/models/grok-4) | xAI | 74.9 | Observed 2026-11-21 · source benchlm:models |
| 79 | [K-Exaone](https://modelscale.dev/models/k-exaone) | LG AI Research | 74.3 | Observed 2026-11-21 · source benchlm:models |
| 79 | [Qwen3 Max](https://modelscale.dev/models/qwen3-max) | Alibaba | 74.3 | Observed 2026-11-21 · source benchlm:models |
| 81 | [Claude 4 Opus Thinking](https://modelscale.dev/models/claude-4-opus-thinking) | Anthropic | 73.4 | Observed 2026-11-21 · source benchlm:models |
| 82 | [Claude 4.1 Opus Thinking](https://modelscale.dev/models/claude-4-1-opus-thinking) | Anthropic | 71.4 | Last good 2026-11-18 · source benchlm:models |
| 83 | [Nemotron 3 Super 100B](https://modelscale.dev/models/nemotron-3-super-100b) | NVIDIA | 67.8 | Last good 2026-10-08 · source benchlm:models |
| 84 | [GPT-OSS 120B](https://modelscale.dev/models/gpt-oss-120b) | OpenAI | 65.8 | Observed 2026-11-21 · source benchlm:models |
| 84 | [Grok 4 Fast (Reasoning)](https://modelscale.dev/models/grok-4-fast-reasoning) | xAI | 65.8 | Observed 2026-11-21 · source benchlm:models |
| 86 | [Grok 4.1 Fast](https://modelscale.dev/models/grok-4-1-fast) | xAI | 63.7 | Observed 2026-11-21 · source benchlm:models |
| 87 | [o1](https://modelscale.dev/models/o1) | OpenAI | 62.6 | Observed 2026-11-21 · source benchlm:models |
| 88 | [Kimi K2](https://modelscale.dev/models/kimi-k2) | Moonshot AI | 61.1 | Observed 2026-11-21 · source benchlm:models |
| 89 | [GPT-OSS 20B](https://modelscale.dev/models/gpt-oss-20b) | OpenAI | 60.2 | Observed 2026-11-21 · source benchlm:models |
| 90 | [Gemma 4 31B](https://modelscale.dev/models/gemma-4-31b) | Google | 59.9 | Observed 2026-11-21 · source benchlm:models |
| 91 | [LLaDA2.2-mini](https://modelscale.dev/models/llada2-2-mini) | InclusionAI | 57.5 | Observed 2026-11-21 · source benchlm:models |
| 92 | [o4-mini](https://modelscale.dev/models/o4-mini) | OpenAI | 55.6 | Observed 2026-11-21 · source benchlm:models |
| 93 | [Nemotron 3.5 Bolt E4B BF16](https://modelscale.dev/models/nemotron-3-5-bolt-e4b-bf16) | NVIDIA | 55.1 | Observed 2026-11-21 · source benchlm:models |
| 94 | [Claude 3.7 Sonnet Thinking](https://modelscale.dev/models/claude-3-7-sonnet-thinking) | Anthropic | 54.7 | Observed 2026-11-21 · source benchlm:models |
| 95 | [Gemini 2.5 Pro](https://modelscale.dev/models/gemini-2-5-pro) | Google | 54.1 | Observed 2026-11-21 · source benchlm:models |
| 96 | [GPT-4.1 mini](https://modelscale.dev/models/gpt-4-1-mini) | OpenAI | 52.9 | Observed 2026-11-21 · source benchlm:models |
| 97 | [Nemotron 3.5 Bolt E4B](https://modelscale.dev/models/nemotron-3-5-bolt-e4b) | NVIDIA | 52.32 | Observed 2026-11-21 · source benchlm:models |
| 98 | [Claude 4 Sonnet](https://modelscale.dev/models/claude-4-sonnet) | Anthropic | 52.3 | Observed 2026-11-21 · source benchlm:models |
| 99 | [Claude 3.7 Sonnet](https://modelscale.dev/models/claude-3-7-sonnet) | Anthropic | 50 | Observed 2026-11-21 · source benchlm:models |
| 100 | [DeepSeek V3 0324](https://modelscale.dev/models/deepseek-v3-0324) | DeepSeek | 47.1 | Observed 2026-11-21 · source benchlm:models |
| 100 | [GPT-4.1](https://modelscale.dev/models/gpt-4-1) | OpenAI | 47.1 | Observed 2026-11-21 · source benchlm:models |
| 102 | [Sarvam 105B](https://modelscale.dev/models/sarvam-105b) | Sarvam | 46.8 | Observed 2026-11-21 · source benchlm:models |
| 103 | [GLM-4.5-Air](https://modelscale.dev/models/glm-4-5-air) | Z.AI | 46.5 | Observed 2026-11-21 · source benchlm:models |
| 104 | [Nemotron 3 Nano Omni 30B A3B](https://modelscale.dev/models/nemotron-3-nano-omni-30b-a3b) | NVIDIA | 45.3 | Observed 2026-11-21 · source benchlm:models |
| 105 | [Gemma 4 26B A4B](https://modelscale.dev/models/gemma-4-26b-a4b) | Google | 43.6 | Observed 2026-11-21 · source benchlm:models |
| 106 | [Gemini 3 Flash](https://modelscale.dev/models/gemini-3-flash) | Google | 43.3 | Observed 2026-11-21 · source benchlm:models |
| 107 | [Mistral Small 4](https://modelscale.dev/models/mistral-small-4) | Mistral | 41.2 | Observed 2026-11-21 · source benchlm:models |
| 107 | [Mistral Small 4 (Reasoning)](https://modelscale.dev/models/mistral-small-4-reasoning) | Mistral | 41.2 | Last good 2026-11-18 · source benchlm:models |
| 109 | [DeepSeek V3.1 (Reasoning)](https://modelscale.dev/models/deepseek-v3-1-reasoning) | DeepSeek | 37.4 | Last good 2026-11-18 · source benchlm:models |
| 109 | [North Mini Code](https://modelscale.dev/models/north-mini-code-1-0) | Cohere | 37.4 | Observed 2026-11-21 · source benchlm:models |
| 111 | [DeepSeek-R1](https://modelscale.dev/models/deepseek-r1) | DeepSeek | 36.5 | Observed 2026-11-21 · source benchlm:models |
| 112 | [Gemma 4 12B](https://modelscale.dev/models/gemma-4-12b) | Google | 36.3 | Observed 2026-11-21 · source benchlm:models |
| 113 | [DeepSeek V3.1](https://modelscale.dev/models/deepseek-v3-1) | DeepSeek | 34.8 | Observed 2026-11-21 · source benchlm:models |
| 114 | [Sarvam 30B](https://modelscale.dev/models/sarvam-30b) | Sarvam | 34.5 | Observed 2026-11-21 · source benchlm:models |
| 115 | [Solar Pro 2](https://modelscale.dev/models/solar-pro-2) | Upstage | 31.9 | Observed 2026-11-21 · source benchlm:models |
| 116 | [Mistral Large 2](https://modelscale.dev/models/mistral-large-2) | Mistral | 30.7 | Observed 2026-11-21 · source benchlm:models |
| 117 | [Gemini 2.0 Flash](https://modelscale.dev/models/gemini-2-0-flash) | Google | 29.5 | Observed 2026-11-21 · source benchlm:models |
| 118 | [o3-mini](https://modelscale.dev/models/o3-mini) | OpenAI | 28.7 | Observed 2026-11-21 · source benchlm:models |
| 119 | [Devstral Small](https://modelscale.dev/models/devstral-small) | Mistral | 28.4 | Observed 2026-11-21 · source benchlm:models |
| 120 | [Llama 3.3 70B](https://modelscale.dev/models/llama-3-3-70b) | Meta | 26.6 | Observed 2026-11-21 · source benchlm:models |
| 120 | [Magistral Small](https://modelscale.dev/models/magistral-small) | Mistral | 26.6 | Observed 2026-11-21 · source benchlm:models |
| 120 | [Ultravox v0.6 Llama 3.3 70B](https://modelscale.dev/models/ultravox-v0-6-llama-3-3-70b) | Fixie AI | 26.6 | Observed 2026-11-21 · source benchlm:models |
| 123 | [Nemotron 3 Nano 30B](https://modelscale.dev/models/nemotron-3-nano-30b) | NVIDIA | 25.4 | Observed 2026-11-21 · source benchlm:models |
| 124 | [GPT-4o](https://modelscale.dev/models/gpt-4o) | OpenAI | 25.1 | Observed 2026-11-21 · source benchlm:models |
| 124 | [Mistral Small 3.1](https://modelscale.dev/models/mistral-small-3-1) | Mistral | 25.1 | Observed 2026-11-21 · source benchlm:models |
| 126 | [Claude 3.5 Haiku](https://modelscale.dev/models/claude-3-5-haiku) | Anthropic | 24.6 | Observed 2026-11-21 · source benchlm:models |
| 126 | [Mistral Large 3](https://modelscale.dev/models/mistral-large-3) | Mistral | 24.6 | Observed 2026-11-21 · source benchlm:models |
| 128 | [Mistral Medium 3](https://modelscale.dev/models/mistral-medium-3) | Mistral | 24.3 | Observed 2026-11-21 · source benchlm:models |
| 129 | [Magistral Medium](https://modelscale.dev/models/magistral-medium) | Mistral | 23.1 | Observed 2026-11-21 · source benchlm:models |
| 130 | [DeepSeek V3](https://modelscale.dev/models/deepseek-v3) | DeepSeek | 22.8 | Observed 2026-11-21 · source benchlm:models |
| 130 | [Granite-4.0-1B](https://modelscale.dev/models/granite-4-0-1b) | IBM | 22.8 | Last good 2026-09-18 · source benchlm:models |
| 132 | [Llama 3.1 70B](https://modelscale.dev/models/llama-3-1-70b) | Meta | 21.6 | Observed 2026-11-21 · source benchlm:models |
| 133 | [Qwen3-Omni-30B-A3B-Thinking](https://modelscale.dev/models/qwen3-omni-30b-a3b-thinking) | Alibaba | 21.3 | Observed 2026-11-21 · source benchlm:models |
| 134 | [Claude 3 Haiku](https://modelscale.dev/models/claude-3-haiku) | Anthropic | 21.1 | Observed 2026-11-21 · source benchlm:models |
| 135 | [Gemma 4 E2B](https://modelscale.dev/models/gemma-4-e2b) | Google | 20.8 | Observed 2026-11-21 · source benchlm:models |
| 135 | [Gemma 4 E4B](https://modelscale.dev/models/gemma-4-e4b) | Google | 20.8 | Observed 2026-11-21 · source benchlm:models |
| 137 | [Exaone 4.0 1.2B](https://modelscale.dev/models/exaone-4-0-1-2b) | LG AI Research | 20.5 | Observed 2026-11-21 · source benchlm:models |
| 138 | [Granite-4.0-H-1B](https://modelscale.dev/models/granite-4-0-h-1b) | IBM | 19.6 | Observed 2026-11-21 · source benchlm:models |
| 139 | [Gemini 2.5 Flash-Lite](https://modelscale.dev/models/gemini-2-5-flash-lite) | Google | 19 | Observed 2026-11-21 · source benchlm:models |
| 139 | [Llama 3.1 405B](https://modelscale.dev/models/llama-3-1-405b) | Meta | 19 | Observed 2026-11-21 · source benchlm:models |
| 141 | [Llama 4 Maverick](https://modelscale.dev/models/llama-4-maverick) | Meta | 17.8 | Observed 2026-11-21 · source benchlm:models |
| 142 | [GPT-4.1 nano](https://modelscale.dev/models/gpt-4-1-nano) | OpenAI | 17.3 | Observed 2026-11-21 · source benchlm:models |
| 143 | [Qwen3-Omni-30B-A3B-Instruct](https://modelscale.dev/models/qwen3-omni-30b-a3b-instruct) | Alibaba | 16.4 | Observed 2026-11-21 · source benchlm:models |
| 144 | [Llama 4 Scout](https://modelscale.dev/models/llama-4-scout) | Meta | 15.5 | Observed 2026-11-21 · source benchlm:models |
| 145 | [Gemini 2.5 Flash](https://modelscale.dev/models/gemini-2-5-flash) | Google | 14.9 | Observed 2026-11-21 · source benchlm:models |
| 146 | [Granite-4.0-H-350M](https://modelscale.dev/models/granite-4-0-h-350m) | IBM | 14.6 | Observed 2026-11-21 · source benchlm:models |
| 147 | [Nova Pro](https://modelscale.dev/models/nova-pro) | Amazon | 14 | Observed 2026-11-21 · source benchlm:models |
| 148 | [Granite-4.0-350M](https://modelscale.dev/models/granite-4-0-350m) | IBM | 13.2 | Observed 2026-11-21 · source benchlm:models |
| 149 | [Nemotron Ultra 253B](https://modelscale.dev/models/nemotron-ultra-253b) | NVIDIA | 11.4 | Observed 2026-11-21 · source benchlm:models |
| 150 | [Gemma 3 27B](https://modelscale.dev/models/gemma-3-27b) | Google | 10.5 | Observed 2026-11-21 · source benchlm:models |
| 151 | [LFM2.5-VL-1.6B-Extract](https://modelscale.dev/models/lfm2-5-vl-1-6b-extract) | LiquidAI | 8.5 | Observed 2026-11-21 · source benchlm:models |
| 152 | [Exaone 4.0 32B](https://modelscale.dev/models/exaone-4-0-32b) | LG AI Research | 4.1 | Observed 2026-11-21 · source benchlm:models |
| 153 | [Phi-4](https://modelscale.dev/models/phi-4) | Microsoft | 0 | Observed 2026-11-21 · source benchlm:models |
| 153 | [Phi-4-reasoning](https://modelscale.dev/models/phi-4-reasoning) | Microsoft | 0 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
