Coding benchmark
SWE-Atlas Codebase QnA leaderboard
Every model the catalog carries a published SWE-Atlas Codebase QnA value for, ranked by that value.
CategoryCoding
MeasureMean pass@1
Tasks124 codebase questions across 11 repositories
DifficultyProduction codebase comprehension
Published byMuse Spark 1.3 Evaluation Methodology
A code-comprehension benchmark covering production repositories across Go, Python, C, and TypeScript.
SWE-Atlas Codebase QnA ranking
1 model with a published SWE-Atlas Codebase QnA value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.
| Rank | Model | Provider | Mean pass@1 |
|---|---|---|---|
| 1 |  Muse Spark 1.3 | Meta | 59.4 |
Evidence key: Observed
Rows are ordered by the value Muse Spark 1.3 Evaluation Methodology published, highest first. The source does not state whether a higher value is the better result, so this page does not either: for a benchmark that measures a rate of failure, read the table from the bottom.