# VulcanBench v3 leaderboard

Coding benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/coding-vulcan-bench

Every model the catalog carries a published VulcanBench v3 value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Coding |
| Measure | Pass@1 with low, medium, and high effort |
| Tasks | 23 post-cutoff repository tasks in the v3 report |
| Difficulty | Professional multi-file software engineering |
| Published by | [VulcanBench](https://github.com/morganlinton/VulcanBench/tree/main) |

An open software-engineering benchmark built from real merged post-cutoff pull requests across Python, Rust, TypeScript, JavaScript, and Go repositories. Replaced by VulcanBench Frontier v4 in September 2026.

## VulcanBench v3 ranking

14 models with a published VulcanBench v3 value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Pass@1 with low, medium, and high effort | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Grok 4.5](https://modelscale.dev/models/grok-4-5) | xAI | 89.9 | Observed 2026-11-21 · source benchlm:models |
| 2 | [Claude Fable 5](https://modelscale.dev/models/claude-fable-5) | Anthropic | 89.5 | Observed 2026-11-21 · source benchlm:models |
| 3 | [DeepSeek V4 Flash 0731](https://modelscale.dev/models/deepseek-v4-flash-max) | DeepSeek | 88.4 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Claude Opus 5](https://modelscale.dev/models/claude-opus-5) | Anthropic | 87.0 | Observed 2026-11-21 · source benchlm:models |
| 4 | [GPT-5.6 Sol](https://modelscale.dev/models/gpt-5-6-sol) | OpenAI | 87.0 | Observed 2026-11-21 · source benchlm:models |
| 4 | [GPT-5.6 Terra](https://modelscale.dev/models/gpt-5-6-terra) | OpenAI | 87.0 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Grok 4.6](https://modelscale.dev/models/grok-4-6) | xAI | 87.0 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Muse Spark 1.2](https://modelscale.dev/models/muse-spark-1-2) | Meta | 87.0 | Observed 2026-11-21 · source benchlm:models |
| 9 | [GPT-5.6 Luna](https://modelscale.dev/models/gpt-5-6-luna) | OpenAI | 85.5 | Observed 2026-11-21 · source benchlm:models |
| 10 | [Qwen3.8-27B](https://modelscale.dev/models/qwen3-8-27b) | Alibaba | 82.6 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Qwen3.8 Max](https://modelscale.dev/models/qwen3-8-max) | Alibaba | 81.2 | Observed 2026-11-21 · source benchlm:models |
| 12 | [GLM-5.3](https://modelscale.dev/models/glm-5-3) | Z.AI | 78.3 | Observed 2026-11-21 · source benchlm:models |
| 13 | [Claude Haiku 4.5](https://modelscale.dev/models/claude-haiku-4-5) | Anthropic | 76.2 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Kimi K3](https://modelscale.dev/models/kimi-3) | Moonshot AI | 73.7 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
