# BinaryAudit leaderboard

external benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/external-binary-audit

BinaryAudit (backdoor detection in compiled binaries). Every model the catalog carries a published BinaryAudit value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | external |
| Measure | Agent with Ghidra and radare2 in a terminal, Harbor framework |
| Tasks | 19 backdoor and timebomb detection tasks on stripped binaries, three runs each (57 runs); 6 tooling and 8 clean-binary tasks reported alongside |
| Difficulty | Reverse engineering without source code or symbols |
| Published by | [We hid backdoors in ~40 MB binaries and asked AI + Ghidra to find them (Quesma)](https://quesma.com/blog/introducing-binaryaudit/) |

Quesma's BinaryAudit asks AI agents to find backdoors and timebombs planted in compiled, stripped binaries of real server software, using the reverse engineering tools Ghidra and radare2. The score here is the share of detection runs that found the planted code.

## BinaryAudit ranking

18 models with a published BinaryAudit value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Agent with Ghidra and radare2 in a terminal, Harbor framework | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Claude Opus 4.6](https://modelscale.dev/models/claude-opus-4-6) | Anthropic | 49.1 | Observed 2026-11-21 · source benchlm:models |
| 1 | [Gemini 3.1 Pro](https://modelscale.dev/models/gemini-3-1-pro) | Google | 49.1 | Observed 2026-11-21 · source benchlm:models |
| 3 | [GPT-5.2-Codex](https://modelscale.dev/models/gpt-5-2-codex) | OpenAI | 45.6 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Gemini 3 Pro](https://modelscale.dev/models/gemini-3-pro) | Google | 43.9 | Observed 2026-11-21 · source benchlm:models |
| 5 | [GPT-5.3 Codex](https://modelscale.dev/models/gpt-5-3-codex) | OpenAI | 42.1 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Claude Opus 4.5](https://modelscale.dev/models/claude-opus-4-5) | Anthropic | 36.8 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Gemini 3 Flash](https://modelscale.dev/models/gemini-3-flash) | Google | 36.8 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Claude Sonnet 4.5](https://modelscale.dev/models/claude-sonnet-4-5) | Anthropic | 26.3 | Observed 2026-11-21 · source benchlm:models |
| 9 | [GLM-4.7](https://modelscale.dev/models/glm-4-7) | Z.AI | 24.6 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Kimi K2.5](https://modelscale.dev/models/kimi-k2-5) | Moonshot AI | 24.6 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Gemini 2.5 Pro](https://modelscale.dev/models/gemini-2-5-pro) | Google | 21.1 | Observed 2026-11-21 · source benchlm:models |
| 12 | [GLM-5](https://modelscale.dev/models/glm-5) | Z.AI | 19.3 | Observed 2026-11-21 · source benchlm:models |
| 13 | [GPT-5.2](https://modelscale.dev/models/gpt-5-2) | OpenAI | 17.5 | Observed 2026-11-21 · source benchlm:models |
| 13 | [GPT-5.4](https://modelscale.dev/models/gpt-5-4) | OpenAI | 17.5 | Observed 2026-11-21 · source benchlm:models |
| 15 | [DeepSeek V3.2](https://modelscale.dev/models/deepseek-v3-2) | DeepSeek | 12.3 | Observed 2026-11-21 · source benchlm:models |
| 16 | [Claude Haiku 4.5](https://modelscale.dev/models/claude-haiku-4-5) | Anthropic | 10.5 | Observed 2026-11-21 · source benchlm:models |
| 17 | [Grok 4](https://modelscale.dev/models/grok-4) | xAI | 8.8 | Observed 2026-11-21 · source benchlm:models |
| 18 | [Grok 4.1 Fast](https://modelscale.dev/models/grok-4-1-fast) | xAI | 7.0 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
