# SWE-Atlas Codebase QnA leaderboard

Coding benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/coding-swe-atlas-codebase-qna

Every model the catalog carries a published SWE-Atlas Codebase QnA value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Coding |
| Measure | Mean pass@1 |
| Tasks | 124 codebase questions across 11 repositories |
| Difficulty | Production codebase comprehension |
| Published by | [Muse Spark 1.3 Evaluation Methodology](https://research.meta.ai/static/muse-spark-1-3-multimodal-evaluation-methodology) |

A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.

## SWE-Atlas Codebase QnA ranking

8 models with a published SWE-Atlas Codebase QnA value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Mean pass@1 | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Ling 3.1 Flash](https://modelscale.dev/models/ling-3-1-flash) | InclusionAI | 64.5 | Observed 2026-11-21 · source benchlm:models |
| 2 | [Muse Spark 1.3](https://modelscale.dev/models/muse-spark-1-3) | Meta | 59.4 | Observed 2026-11-21 · source benchlm:models |
| 3 | [LongCat-2.5](https://modelscale.dev/models/longcat-2-5) | Meituan | 58.3 | Observed 2026-11-21 · source benchlm:models |
| 4 | [Muse Spark Open](https://modelscale.dev/models/muse-spark-open) | Meta | 58.0 | Observed 2026-11-21 · source benchlm:models |
| 5 | [SWE-2 Flash](https://modelscale.dev/models/swe-2-flash) | Cognition | 50.0 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Beam 1.0 501B-A23B FP8](https://modelscale.dev/models/reflection-beam-fp8) | Reflection AI | 32.2 | Observed 2026-11-21 · source benchlm:models |
| 7 | [Beam 1.0 501B-A23B](https://modelscale.dev/models/reflection-beam) | Reflection AI | 31.8 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Beam 1.0 501B-A23B NVFP4](https://modelscale.dev/models/reflection-beam-nvfp4) | Reflection AI | 31.4 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
