τ²-bench results leaderboard

This route is a sourced ledger for published τ²-bench results. Most current rows come from Artificial Analysis's telecom implementation, while named provider rows can use telecom, airline, retail, or aggregate setups.

Models ranked by their published τ²-Bench Tool-Agent-User Evaluation value

Rank	Model	Provider	Published domain success or pass^k results
1	GLM-5.2	Z.AI	99.1
2	GPT-5.4	OpenAI	98.9
3	Claude Fable 5	Anthropic	98.5
3	GLM-5-Turbo	Z.AI	98.5
3	GLM-5V-Turbo	Z.AI	98.5
3	Step 3.7 Flash	StepFun	98.5
7	GLM-5	Z.AI	98.2
8	GPT-5.5	OpenAI	98
9	GLM-5.1	Z.AI	97.7
9	Grok 4.3	xAI	97.7
9	Qwen3.6 Plus	Alibaba	97.7
12	DeepSeek V4 Pro 0813	DeepSeek	96.2
13	GLM-4.7	Z.AI	95.9
13	Kimi K2.5	Moonshot AI	95.9
13	Kimi K2.5 (Reasoning)	Moonshot AI	95.9
13	Kimi K2.6	Moonshot AI	95.9
13	Qwen 3.6 Max (preview)	Alibaba	95.9
18	Gemini 3.1 Pro	Google	95.6
19	Gemini 3.5 Flash	Google	95.3
19	Qwen3.6-35B-A3B	Alibaba	95.3
21	MiMo-V2-Pro	Xiaomi	95
22	Qwen3.7 Max	Alibaba	94.7
23	Claude Opus 4.8	Anthropic	94.4
24	MiMo-V2.5-Pro	Xiaomi	94.2
24	Mistral Medium 3.5 128B	Mistral	94.2
24	Qwen3.6-27B	Alibaba	94.2
27	Qwen3.5-27B	Alibaba	93.9
28	Qwen3.5-122B-A10B	Alibaba	93.6
29	GPT-5.4 mini	OpenAI	93.4
30	Grok 4.1 Fast (Reasoning)	xAI	93.3
31	Qwen3.7 Plus	Alibaba	93
32	GPT-5.4 nano	OpenAI	92.5
33	Claude Opus 4.6 (Adaptive)	Anthropic	92.1
33	GPT-5.2-Codex	OpenAI	92.1
35	Muse Spark	Meta	91.5
36	MiMo-V2-Omni	Xiaomi	91.2
37	Kimi K2.7 Code	Moonshot AI	90.1
37	Trinity-Large-Preview	Arcee AI	90.1
37	Trinity-Large-Thinking	Arcee AI	90.1
40	Claude Opus 4.5 Thinking	Anthropic	89.5
41	Qwen3.5-35B-A3B	Alibaba	89.2
42	MiniMax M3	MiniMax	88.9
43	Claude Opus 4.7 (Adaptive)	Anthropic	88.6
44	LFM2.5-8B-A1B	LiquidAI	88.07
45	Gemini 3 Pro	Google	87.1
46	GPT-5 (medium)	OpenAI	86.5
47	Claude Opus 4.5	Anthropic	86.3
47	GPT-5.6 Terra	OpenAI	86.3
47	Solar Pro 3	Upstage	86.3
50	GPT-5.3 Codex	OpenAI	86
50	Ling 2.6 Flash	InclusionAI	86
52	GPT-5.6 Sol	OpenAI	85.1
53	Command A+	Cohere	85
54	Claude Opus 4.6	Anthropic	84.8
54	GPT-5 (high)	OpenAI	84.8
54	GPT-5.2	OpenAI	84.8
54	MiniMax M2.7	MiniMax	84.8
58	MiMo-V2-Flash	Xiaomi	83.9
58	Qwen3.5 397B	Alibaba	83.9
58	Qwen3.5 397B (Reasoning)	Alibaba	83.9
61	Nemotron 3 Ultra	NVIDIA	83.3
62	GPT-5.1-Codex	OpenAI	83
62	GPT-5.1-Codex-Max	OpenAI	83
64	GPT-5.1	OpenAI	81.9
65	o3	OpenAI	80.7
66	LLaDA2.2-flash	InclusionAI	80.33
67	PMTernary Bonsai 2 27B	Prism ML	80.22
68	Claude Sonnet 4.6	Anthropic	79.5
69	DeepSeek V3.2	DeepSeek	78.9
70	Agents-A1-4B	InternScience	78.2
71	GLM-4.6	Z.AI	76.9
72	Grok Code Fast 1	xAI	75.7
73	Grok 4	xAI	74.9
74	K-Exaone	LG AI Research	74.3
74	Qwen3 Max	Alibaba	74.3
76	Claude Opus 4.7	Anthropic	74
77	Claude 4.1 Opus Thinking	Anthropic	71.4
78	Nemotron 3 Super 100B	NVIDIA	67.8
79	GPT-OSS 120B	OpenAI	65.8
79	Grok 4 Fast (Reasoning)	xAI	65.8
81	Grok 4.1 Fast	xAI	63.7
82	o1	OpenAI	62.6
83	Kimi K2	Moonshot AI	61.1
84	GPT-OSS 20B	OpenAI	60.2
85	Gemma 4 31B	Google	59.9
86	LLaDA2.2-mini	InclusionAI	57.5
87	Gemini 2.5 Pro	Google	54.1
88	GPT-4.1 mini	OpenAI	52.9
89	Claude 4 Sonnet	Anthropic	52.3
90	GPT-4.1	OpenAI	47.1
91	SASarvam 105B	Sarvam	46.8
92	GLM-4.5-Air	Z.AI	46.5
93	Nemotron 3 Nano Omni 30B A3B	NVIDIA	45.3
94	Gemma 4 26B A4B	Google	43.6
95	Gemini 3 Flash	Google	43.3
96	Mistral Small 4	Mistral	41.2
96	Mistral Small 4 (Reasoning)	Mistral	41.2
98	Nemotron 3 Nano 30B	NVIDIA	40.9
99	DeepSeek V3.1 (Reasoning)	DeepSeek	37.4
99	North Mini Code	Cohere	37.4
101	DeepSeek-R1	DeepSeek	36.5
102	Gemma 4 12B	Google	36.3
103	DeepSeek V3.1	DeepSeek	34.8
104	SASarvam 30B	Sarvam	34.5
105	Solar Pro 2	Upstage	31.9
106	Mistral Large 2	Mistral	30.7
107	o3-mini	OpenAI	28.7
108	FAUltravox v0.6 Llama 3.3 70B	Fixie AI	26.6
109	GPT-4o	OpenAI	25.1
110	Mistral Large 3	Mistral	24.6
111	Mistral Medium 3	Mistral	24.3
112	DeepSeek V3	DeepSeek	22.8
112	Granite-4.0-1B	IBM	22.8
114	Qwen3-Omni-30B-A3B-Thinking	Alibaba	21.3
115	Claude 3 Haiku	Anthropic	21.1
116	Gemma 4 E2B	Google	20.8
116	Gemma 4 E4B	Google	20.8
118	Exaone 4.0 1.2B	LG AI Research	20.5
119	Granite-4.0-H-1B	IBM	19.6
120	Llama 3.1 405B	Meta	19
121	Llama 4 Maverick	Meta	17.8
122	GPT-4.1 nano	OpenAI	17.3
123	Qwen3-Omni-30B-A3B-Instruct	Alibaba	16.4
124	Llama 4 Scout	Meta	15.5
125	Gemini 2.5 Flash	Google	14.9
126	Granite-4.0-H-350M	IBM	14.6
127	Nova Pro	Amazon	14
128	Granite-4.0-350M	IBM	13.2
129	Nemotron Ultra 253B	NVIDIA	11.4
130	Gemma 3 27B	Google	10.5
131	LFM2.5-VL-1.6B-Extract	LiquidAI	8.5
132	Exaone 4.0 32B	LG AI Research	4.1
133	Phi-4	Microsoft	0

Evidence key: ObservedLast good

Rows are ordered by the value τ²-Bench: Evaluating Conversational Agents in a Dual-Control Environment published, highest first. The source does not state whether a higher value is the better result, so this page does not either: for a benchmark that measures a rate of failure, read the table from the bottom.