# ScarfBench leaderboard

Coding benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/coding-scarf-bench

ScarfBench: Cross-Framework Application Migration in Enterprise Java. Every model the catalog carries a published ScarfBench value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Coding |
| Measure | Percent of smoke tests passed after migration (Kaggle) |
| Tasks | 204 framework migration tasks across 34 enterprise Java applications |
| Difficulty | Whole-application framework migration |
| Published by | [ScarfBench: A Benchmark for Cross-Framework Application Migration in Enterprise Java](https://arxiv.org/abs/2605.06754) |

IBM Research's benchmark for coding agents that migrate complete enterprise Java applications between Jakarta EE, Quarkus and Spring while keeping them working.

## ScarfBench ranking

11 models with a published ScarfBench value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Percent of smoke tests passed after migration (Kaggle) | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [Gemini 4 Argon](https://modelscale.dev/models/gemini-4-argon) | Google | 75.2 | Observed 2026-11-21 · source benchlm:models |
| 2 | [Claude Opus 5.5](https://modelscale.dev/models/claude-opus-5-5) | Anthropic | 70.5 | Observed 2026-11-21 · source benchlm:models |
| 3 | [Claude Opus 5](https://modelscale.dev/models/claude-opus-5) | Anthropic | 69.8 | Observed 2026-11-21 · source benchlm:models |
| 4 | [DeepSeek V4.1 Flash 1017](https://modelscale.dev/models/deepseek-v4-1-flash) | DeepSeek | 69.0 | Observed 2026-11-21 · source benchlm:models |
| 5 | [GPT-6.1 Sol](https://modelscale.dev/models/gpt-6-1-sol) | OpenAI | 68.4 | Observed 2026-11-21 · source benchlm:models |
| 6 | [Gemini 3.8 Flash](https://modelscale.dev/models/gemini-3-8-flash) | Google | 66.0 | Observed 2026-11-21 · source benchlm:models |
| 7 | [GPT-6 Astra](https://modelscale.dev/models/gpt-6-astra) | OpenAI | 65.4 | Observed 2026-11-21 · source benchlm:models |
| 8 | [Qwen3.8 Max](https://modelscale.dev/models/qwen3-8-max) | Alibaba | 63.7 | Observed 2026-11-21 · source benchlm:models |
| 9 | [Kimi K3](https://modelscale.dev/models/kimi-3) | Moonshot AI | 61.3 | Observed 2026-11-21 · source benchlm:models |
| 10 | [Gemini 3.1 Pro](https://modelscale.dev/models/gemini-3-1-pro) | Google | 37.6 | Observed 2026-11-21 · source benchlm:models |
| 11 | [Gemma 4 26B A4B](https://modelscale.dev/models/gemma-4-26b-a4b) | Google | 24.4 | Observed 2026-11-21 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.
