SWE-bench Verified leaderboard

A curated, human-verified subset of SWE-bench that tests models on resolving real GitHub issues from popular open-source Python repositories like Django, Flask, and scikit-learn.

Models ranked by their published Software Engineering Benchmark Verified value

Rank	Model	Provider	Code patch generation
1	Claude Opus 5	Anthropic	96
2	Claude Mythos 5	Anthropic	95.5
3	Claude Fable 5	Anthropic	95
4	Claude Opus 4.8	Anthropic	88.6
5	Claude Opus 4.7 (Adaptive)	Anthropic	87.6
6	OAOrnith-1.5-397B	Ornith AI	86
7	Claude Sonnet 5	Anthropic	85.2
8	GPT-5.3 Codex	OpenAI	85
9	DAOrnith-1.0-397B	DeepReinforce AI	82.4
10	Claude Opus 4.5	Anthropic	80.9
11	Claude Opus 4.6	Anthropic	80.84
12	DeepSeek V4 Pro 0813	DeepSeek	80.6
13	MiniMax M3	MiniMax	80.5
14	Qwen3.7 Max	Alibaba	80.4
15	TMInkling-Small	Thinking Machines Lab	80.2
15	Kimi K2.6	Moonshot AI	80.2
17	GPT-5.2	OpenAI	80
18	Claude Sonnet 4.6	Anthropic	79.6
19	DeepSeek V4 Flash 0731	DeepSeek	79
19	OAOrnith-1.5-35B-A3B	Ornith AI	79
21	Qwen3.6 Plus	Alibaba	78.8
22	BTBTL-4	Bad Theory Labs	78.4
22	DSdots3-note Preview	Dots Studio	78.4
24	MiMo-V2-Pro	Xiaomi	78
25	GLM-5	Z.AI	77.8
26	APApodex 1.1	Apodex	77.7
26	Qwen3.7 Plus	Alibaba	77.7
28	TMInkling	Thinking Machines Lab	77.6
28	Mistral Medium 3.5 128B	Mistral	77.6
30	Muse Spark	Meta	77.4
31	Claude Sonnet 4.5	Anthropic	77.2
31	Qwen3.6-27B	Alibaba	77.2
33	Kimi K2.5	Moonshot AI	76.8
33	Kimi K2.5 (Reasoning)	Moonshot AI	76.8
35	Grok 4.20	xAI	76.7
36	Qwen3.5 397B	Alibaba	76.2
37	Muse Glimmer 30B	Meta	76
38	DAOrnith-1.0-35B	DeepReinforce AI	75.6
39	MiMo-V2-Omni	Xiaomi	74.8
40	Laguna M.1	Poolside	74.6
41	Claude 4.1 Opus	Anthropic	74.5
42	Hy3 Preview	Tencent	74.4
43	GLM-4.7	Z.AI	73.8
44	MAI-Thinking-1	Microsoft	73.5
45	MiMo-V2-Flash	Xiaomi	73.4
45	Qwen3.6-35B-A3B	Alibaba	73.4
47	Claude Haiku 4.5	Anthropic	73.3
48	Claude 4 Sonnet	Anthropic	72.7
49	MAI-Code-1.1-Flash	Microsoft	72.6
50	Qwen3.5-27B	Alibaba	72.4
51	Qwen3.5-122B-A10B	Alibaba	72
52	Nemotron 3 Ultra	NVIDIA	71.9
53	Laguna XS 2.1	Poolside	70.9
54	Grok Code Fast 1	xAI	70.8
55	OAOrnith-1.5-9B	Ornith AI	70.6
55	Solar Pro 4	Upstage	70.6
57	Solar Open 2	Upstage	70.4
58	Laguna XS.2	Poolside	69.9
59	DAOrnith-1.0-9B	DeepReinforce AI	69.4
60	Qwen3.5-35B-A3B	Alibaba	69.2
61	K-EXAONE 2.0	LG AI Research	68.2
61	LongCat-Flash-Lite-Sparse	Meituan	68.2
63	Gemini 2.5 Pro	Google	63.8
64	PMTernary Bonsai 2 27B	Prism ML	60.8
65	Granite 4.2 30B	IBM	57
66	GPT-4.1	OpenAI	54.6
67	ZYZAYA1-74B-Preview	Zyphra	53.2
68	Nemotron 3.5 Lightning 30B A3B NVFP4	NVIDIA	52.8
69	o3-mini	OpenAI	49.3
70	LLaDA2.2-flash	InclusionAI	49.28
71	Claude 3.5 Sonnet	Anthropic	49
72	Granite 4.2 8B	IBM	47.67
73	OPMiniCPM5-2B	OpenBMB	46.4
74	DeepSeek V3	DeepSeek	42
75	GPT-4.1 mini	OpenAI	23.6

Evidence key: Observed

Rows are ordered by the value SWE-bench: Can Language Models Resolve Real-World GitHub Issues? published, highest first. The source does not state whether a higher value is the better result, so this page does not either: for a benchmark that measures a rate of failure, read the table from the bottom.