# CompileBench leaderboard

external benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/external-compile-bench

CompileBench (building real projects from source, pass@1). Every model the catalog carries a published CompileBench value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | external |
| Measure | Agent in a Docker Linux terminal builds a project from source; the binary is checked |
| Tasks | 15 build tasks on cowsay, jq, GNU coreutils and curl, three attempts each (45 attempts) |
| Difficulty | Dependency resolution, legacy toolchains, static linking and cross-compilation |
| Published by | [CompileBench: can AI compile 22-year-old code? (Quesma)](https://quesma.com/blog/introducing-compilebench/) |

Quesma's CompileBench hands an agent the unmodified source of an open-source project such as curl, jq or GNU coreutils and asks for a working binary. Tasks include static and musl builds, cross-compiling to Windows and ARM64, and reviving code from 2003.

## CompileBench ranking

17 models with a published CompileBench value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Agent in a Docker Linux terminal builds a project from source; the binary is checked | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Claude Sonnet 4.5](https://modelscale.dev/models/claude-sonnet-4-5) | Anthropic | 86.7 | Observed 2026-11-21 · source benchlm:models |
| 1 | [GPT-5](https://modelscale.dev/models/gpt-5) | OpenAI | 86.7 | Observed 2026-11-21 · source benchlm:models |
| 1 | [GPT-5 (high)](https://modelscale.dev/models/gpt-5-high) | OpenAI | 86.7 | Last good 2026-11-18 · source benchlm:models |
| 4 | [Claude Sonnet 4.5 Thinking](https://modelscale.dev/models/claude-sonnet-4-5-thinking) | Anthropic | 84.4 | Observed 2026-11-21 · source benchlm:models |
| 5 | [Claude Haiku 4.5 Thinking](https://modelscale.dev/models/claude-haiku-4-5-thinking) | Anthropic | 73.3 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Grok 4](https://modelscale.dev/models/grok-4) | xAI | 71.1 | Observed 2026-11-21 · source benchlm:models |
| 7 | [Claude Haiku 4.5](https://modelscale.dev/models/claude-haiku-4-5) | Anthropic | 68.9 | Observed 2026-11-21 · source benchlm:models |
| 8 | [DeepSeek V3.1](https://modelscale.dev/models/deepseek-v3-1) | DeepSeek | 64.4 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Grok Code Fast 1](https://modelscale.dev/models/grok-code-fast-1) | xAI | 64.4 | Observed 2026-11-21 · source benchlm:models |
| 10 | [Grok 4 Fast (Reasoning)](https://modelscale.dev/models/grok-4-fast-reasoning) | xAI | 55.6 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Gemini 2.5 Pro](https://modelscale.dev/models/gemini-2-5-pro) | Google | 53.3 | Observed 2026-11-21 · source benchlm:models |
| 11 | [GPT-4.1](https://modelscale.dev/models/gpt-4-1) | OpenAI | 53.3 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Qwen3 Max](https://modelscale.dev/models/qwen3-max) | Alibaba | 53.3 | Observed 2026-11-21 · source benchlm:models |
| 14 | [GLM-4.5](https://modelscale.dev/models/glm-4-5) | Z.AI | 48.9 | Observed 2026-11-21 · source benchlm:models |
| 15 | [GPT-4.1 mini](https://modelscale.dev/models/gpt-4-1-mini) | OpenAI | 46.7 | Observed 2026-11-21 · source benchlm:models |
| 16 | [GPT-OSS 120B](https://modelscale.dev/models/gpt-oss-120b) | OpenAI | 42.2 | Observed 2026-11-21 · source benchlm:models |
| 17 | [Gemini 2.5 Flash](https://modelscale.dev/models/gemini-2-5-flash) | Google | 40.0 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
