# HumanEval leaderboard

Coding benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/coding-humaneval

Evaluating Large Language Models Trained on Code. Every model the catalog carries a published HumanEval value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Coding |
| Measure | Python function generation |
| Tasks | 164 problems |
| Difficulty | Introductory to intermediate programming |
| Published by | [Evaluating Large Language Models Trained on Code](https://arxiv.org/abs/2107.03374) |

HumanEval tests whether generated Python functions pass unit tests on 164 handwritten problems. These scores describe function-level correctness; they do not measure repository work or a coding agent.

## HumanEval ranking

23 models with a published HumanEval value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Python function generation | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [o4-mini](https://modelscale.dev/models/o4-mini) | OpenAI | 97.3 | Observed 2026-11-21 · source benchlm:models |
| 2 | [BTL-3](https://modelscale.dev/models/btl-3) | Bad Theory Labs | 95.12 | Observed 2026-11-21 · source benchlm:models |
| 3 | [o1-mini](https://modelscale.dev/models/o1-mini) | OpenAI | 92.4 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Sarvam 30B](https://modelscale.dev/models/sarvam-30b) | Sarvam | 92.1 | Observed 2026-11-21 · source benchlm:models |
| 5 | [Qwen2.5-VL-32B](https://modelscale.dev/models/qwen2-5-vl-32b) | Alibaba | 91.5 | Observed 2026-11-21 · source benchlm:models |
| 6 | [DeepSeek-V2.5](https://modelscale.dev/models/deepseek-v2-5) | DeepSeek | 89 | Observed 2026-11-21 · source benchlm:models |
| 7 | [GPT-4.5](https://modelscale.dev/models/gpt-4-5) | OpenAI | 88.6 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Mistral Small 3.1](https://modelscale.dev/models/mistral-small-3-1) | Mistral | 88.41 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Grok 2](https://modelscale.dev/models/grok-2) | xAI | 88.4 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Llama 3.3 70B](https://modelscale.dev/models/llama-3-3-70b) | Meta | 88.4 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Claude 3.5 Haiku](https://modelscale.dev/models/claude-3-5-haiku) | Anthropic | 88.1 | Observed 2026-11-21 · source benchlm:models |
| 12 | [GPT-4o mini](https://modelscale.dev/models/gpt-4o-mini) | OpenAI | 87.2 | Observed 2026-11-21 · source benchlm:models |
| 13 | [MiniMax-Text-01](https://modelscale.dev/models/minimax-text-01) | MiniMax | 86.9 | Observed 2026-11-21 · source benchlm:models |
| 14 | [Grok 2 mini](https://modelscale.dev/models/grok-2-mini) | xAI | 85.7 | Observed 2026-11-21 · source benchlm:models |
| 15 | [Phi-4](https://modelscale.dev/models/phi-4) | Microsoft | 82.6 | Observed 2026-11-21 · source benchlm:models |
| 16 | [Llama 3.1 70B](https://modelscale.dev/models/llama-3-1-70b) | Meta | 80.5 | Observed 2026-11-21 · source benchlm:models |
| 17 | [Granite-4.0-1B](https://modelscale.dev/models/granite-4-0-1b) | IBM | 74 | Observed 2026-11-21 · source benchlm:models |
| 18 | [Soofi S 30B-A3B](https://modelscale.dev/models/soofi-s-30b-a3b) | Soofi Project | 73.8 | Observed 2026-11-21 · source benchlm:models |
| 19 | [Granite-4.0-H-1B](https://modelscale.dev/models/granite-4-0-h-1b) | IBM | 73 | Observed 2026-11-21 · source benchlm:models |
| 20 | [Llama 3.1 8B](https://modelscale.dev/models/llama-3-1-8b) | Meta | 72.6 | Observed 2026-11-21 · source benchlm:models |
| 21 | [Phi-3.5-mini](https://modelscale.dev/models/phi-3-5-mini) | Microsoft | 62.8 | Observed 2026-11-21 · source benchlm:models |
| 22 | [Granite-4.0-350M](https://modelscale.dev/models/granite-4-0-350m) | IBM | 39 | Observed 2026-11-21 · source benchlm:models |
| 23 | [Granite-4.0-H-350M](https://modelscale.dev/models/granite-4-0-h-350m) | IBM | 38 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
