Coding benchmark
SWE-Rebench leaderboard
Every model the catalog carries a published SWE-Rebench value for, ranked by that value.
A continuously updated software engineering benchmark by Nebius using fresh GitHub issues to avoid contamination. Models are evaluated 5 times per problem under a fixed ReAct scaffolding; the Resolved Rate (best pass@1) is reported.
SWE-Rebench ranking
13 models with a published SWE-Rebench value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.
| Rank | Model | Provider | Code patch generation |
|---|---|---|---|
| 1 |  Claude Opus 4.6 | Anthropic | 65.3 |
| 2 |  GLM-5 | Z.AI | 62.8 |
| 3 | GLM-5.1 | Z.AI | 62.7 |
| 4 |  DeepSeek V3.2 | DeepSeek | 60.9 |
| 5 | Claude Sonnet 4.6 | Anthropic | 60.7 |
| 6 |  Qwen3.5-27B | Alibaba | 58.9 |
| 7 | GLM-4.7 | Z.AI | 58.7 |
| 8 |  Kimi K2.5 | Moonshot AI | 58.5 |
| 9 |  GPT-5.3 Codex | OpenAI | 58.2 |
| 10 |  Composer 2 | Cursor | 58 |
| 11 | Qwen3.5-35B-A3B | Alibaba | 53.7 |
| 12 |  MiniMax M2.7 | MiniMax | 51.9 |
| 13 |  Gemma 4 31B | 41.6 |
Evidence key: Observed
Rows are ordered by the value SWE-Rebench: Contamination-Free Evaluation of Software Engineering Agents published, highest first. The source does not state whether a higher value is the better result, so this page does not either: for a benchmark that measures a rate of failure, read the table from the bottom.