# NoLiMa leaderboard

external benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/external-no-li-ma-original-reports

NoLiMa original-task reports. Every model the catalog carries a published NoLiMa value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | external |
| Measure | Character-name answer accuracy by context length |
| Tasks | 58 question-needle pairs; five haystacks; 26 placements through 32K |
| Difficulty | — |
| Published by | [NoLiMa: Long-Context Evaluation Beyond Literal Matching](https://arxiv.org/abs/2502.05167v2) |

Selected author accuracy curves for long-context retrieval with minimal wording overlap. Original NoLiMa stays separate from Hard, and 64K/128K use fewer placements. The displayed score is 32K accuracy, outside overall and category rankings.

## NoLiMa ranking

No model in the catalog has a published NoLiMa score. The benchmark is defined by NoLiMa: Long-Context Evaluation Beyond Literal Matching, but the catalog carries no value for it yet. An absent value is shown as absent here rather than as a zero.

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.