# JevBench 1.5 leaderboard

Reasoning benchmark

> Every figure below is reproduced as its upstream source published it: nothing is modelled, estimated, interpolated or converted. `Unavailable` means no source published the value — it is never a zero. Each value carries its evidence state and the date it was observed.

Page: https://modelscale.dev/leaderboards/benchmarks/reasoning-jev-bench15

Every model the catalog carries a published JevBench 1.5 value for, ranked by that value.

## Benchmark record

| Field | Value |
| --- | --- |
| Category | Reasoning |
| Measure | Official option A harmonic composite with published uncertainty |
| Tasks | 904 open decisions plus 720 sealed decisions per complete run |
| Difficulty | Choice, Noul, and Score decisions with sealed generalization tests |
| Published by | [JevBench v1.5 frozen method and disclosures](https://github.com/fstandhartinger/jevbench/blob/bb05a335bc809e61b20c0f745d25499a82b326fc/docs/METHOD-v1.5-README.md) |

Typed-decision evaluation across 1,624 decisions, combining chance-corrected intelligence, probability calibration, latency, and cost.

## JevBench 1.5 ranking

214 models with a published JevBench 1.5 value, ordered by that value, highest first. Models the source has not scored on this benchmark are not listed — they are unmeasured, not last.

| Rank | Model | Provider | Official option A harmonic composite with published uncertainty | Evidence |
| --- | --- | --- | --- | --- |
| 1 | [classifier.dev (fast tier)](https://modelscale.dev/models/jevbench-classifier-dev-fast) | mrmps (@michael_chomsky) | 74.66 | Last good 2026-10-01 · source benchlm:models |
| 1 | [classifier.dev (fast)](https://modelscale.dev/models/classifier-dev-fast) | classifier.dev | 74.66 | Observed 2026-10-23 · source benchlm:models |
| 3 | [Cygnet](https://modelscale.dev/models/cygnet) | blockbrain | 73.70 | Observed 2026-10-23 · source benchlm:models |
| 3 | [Cygnet (blockbrain, frozen Gemma-4-12B-it)](https://modelscale.dev/models/jevbench-cygnet) | blockbrain | 73.70 | Last good 2026-10-01 · source benchlm:models |
| 5 | [Winnow-12B Q8](https://modelscale.dev/models/jevbench-winnow-12b) | Eldan Ring | 73.23 | Last good 2026-10-01 · source benchlm:models |
| 5 | [Winnow-12B Q8](https://modelscale.dev/models/winnow-12b) | Eldan Ring | 73.23 | Observed 2026-10-23 · source benchlm:models |
| 7 | [Jev 1.13.0](https://modelscale.dev/models/jev-1-13-0) | TypeSafe AI | 72.13 | Observed 2026-10-23 · source benchlm:models |
| 7 | [Jev 1.13.0 (TypeSafe AI)](https://modelscale.dev/models/jevbench-jev-1-13-0) | TypeSafe AI | 72.13 | Last good 2026-10-01 · source benchlm:models |
| 9 | [JevK5 v0.3 (4B)](https://modelscale.dev/models/jevbench-jevk5-v0-3-4b) | unknown | 71.90 | Last good 2026-10-01 · source benchlm:models |
| 9 | [JevK5 v0.3 4B](https://modelscale.dev/models/jevk5-v0-3-4b) | allebee | 71.90 | Observed 2026-10-23 · source benchlm:models |
| 11 | [Plumb-4B](https://modelscale.dev/models/plumb-4b) | crh225 | 71.56 | Observed 2026-10-23 · source benchlm:models |
| 11 | [Plumb-4B (crh225, JevK5 v0.2 + LoRA)](https://modelscale.dev/models/jevbench-plumb-4b) | crh225 | 71.56 | Last good 2026-10-01 · source benchlm:models |
| 13 | [Jev-Omni](https://modelscale.dev/models/jev-omni) | akhilaaa3 | 71.50 | Observed 2026-10-23 · source benchlm:models |
| 13 | [Jev-Omni (akhilaaa3, Gemma-4-12B merged)](https://modelscale.dev/models/jevbench-jev-omni) | akhilaaa3 | 71.50 | Last good 2026-10-01 · source benchlm:models |
| 15 | [decider-4b v2](https://modelscale.dev/models/decider-4b-v2) | Mapika | 71.28 | Observed 2026-10-23 · source benchlm:models |
| 15 | [decider-4b v2 (Mapika)](https://modelscale.dev/models/jevbench-decider-4b-v2) | Mapika | 71.28 | Last good 2026-10-01 · source benchlm:models |
| 17 | [Decision 4B v1.2](https://modelscale.dev/models/decision-4b-v12) | FlyMy.AI | 70.83 | Observed 2026-10-23 · source benchlm:models |
| 17 | [Decision 4B v1.2 (FlyMyJev, Qwen3.5-4B + LoRA)](https://modelscale.dev/models/jevbench-decision-4b-v12) | unknown | 70.83 | Last good 2026-10-01 · source benchlm:models |
| 19 | [Imajev-4B (RTX 5090)](https://modelscale.dev/models/imajev-4b-rtx5090-a2) | mohit67890 | 70.39 | Observed 2026-10-23 · source benchlm:models |
| 19 | [Imajev-4B (RTX 5090)](https://modelscale.dev/models/jevbench-imajev-4b-rtx5090-a2) | unknown | 70.39 | Last good 2026-10-01 · source benchlm:models |
| 21 | [Decision 4B v1.1](https://modelscale.dev/models/decision-4b-v11) | FlyMy.AI | 70.39 | Observed 2026-10-23 · source benchlm:models |
| 21 | [Decision 4B v1.1 (FlyMyJev, Qwen3.5-4B + LoRA)](https://modelscale.dev/models/jevbench-decision-4b-v11) | unknown | 70.39 | Last good 2026-10-01 · source benchlm:models |
| 23 | [Manchego v2.1](https://modelscale.dev/models/jevbench-manchego-v21-v15) | oraculumai | 68.81 | Last good 2026-10-01 · source benchlm:models |
| 23 | [Manchego v2.1](https://modelscale.dev/models/manchego-v2-1) | oraculumai | 68.81 | Observed 2026-10-23 · source benchlm:models |
| 25 | [SemIf Qwen3.5-4B](https://modelscale.dev/models/semif-qwen3-5-4b) | Theodore Lee (TheoLeeCJ) | 68.66 | Observed 2026-10-23 · source benchlm:models |
| 25 | [SemIf, formerly OpenJev (Qwen3.5-4B, TheoLeeCJ)](https://modelscale.dev/models/jevbench-semif-qwen3-5-4b) | Theodore Lee (TheoLeeCJ) | 68.66 | Last good 2026-10-01 · source benchlm:models |
| 27 | [spark-s1-4b-v6](https://modelscale.dev/models/spark-s1-4b-v6) | Abhishek Rai (abhishek085) | 68.16 | Observed 2026-10-23 · source benchlm:models |
| 27 | [spark-s1-4b-v6 (Open Spark Jev, abhishek085)](https://modelscale.dev/models/jevbench-spark-s1-4b-v6) | Abhishek Rai (abhishek085) | 68.16 | Last good 2026-10-01 · source benchlm:models |
| 29 | [metask-jev-4b](https://modelscale.dev/models/jevbench-metask-jev-4b) | Wayfind (metask-ai) | 67.48 | Last good 2026-10-01 · source benchlm:models |
| 29 | [metask-jev-4b](https://modelscale.dev/models/metask-jev-4b) | Wayfind (metask-ai) | 67.48 | Observed 2026-10-23 · source benchlm:models |
| 31 | [Hopper](https://modelscale.dev/models/hopper) | HopitAI | 67.47 | Observed 2026-10-23 · source benchlm:models |
| 31 | [Hopper](https://modelscale.dev/models/jevbench-hopper) | HopitAI | 67.47 | Last good 2026-10-01 · source benchlm:models |
| 33 | [Malkuth-4B](https://modelscale.dev/models/malkuth-4b) | newfull5 (dhtocks) | 66.77 | Observed 2026-10-23 · source benchlm:models |
| 33 | [Malkuth-4B (newfull5, Kev post-train)](https://modelscale.dev/models/jevbench-malkuth-4b) | newfull5 (dhtocks) | 66.77 | Last good 2026-10-01 · source benchlm:models |
| 35 | [Surogate Rune 26B-A4B v3 (RTX PRO 6000)](https://modelscale.dev/models/jevbench-surogate-rune-26b-a4b-v3-rtxpro6000-a2) | unknown | 66.47 | Last good 2026-10-01 · source benchlm:models |
| 35 | [Surogate Rune 26B-A4B v3 (RTX PRO 6000)](https://modelscale.dev/models/surogate-rune-26b-a4b-v3-rtxpro6000-a2) | Surogate | 66.47 | Observed 2026-10-23 · source benchlm:models |
| 37 | [reflex 4B](https://modelscale.dev/models/reflex-4b) | kshetrajna12 | 65.24 | Observed 2026-10-23 · source benchlm:models |
| 37 | [reflex 4B (kshetrajna12)](https://modelscale.dev/models/jevbench-reflex-4b) | kshetrajna12 | 65.24 | Last good 2026-10-01 · source benchlm:models |
| 39 | [jev-local (Qwen3.5-9B)](https://modelscale.dev/models/jevbench-jev-local) | us (GitHub) | 65.24 | Last good 2026-10-01 · source benchlm:models |
| 39 | [jev-local Qwen3.5-9B](https://modelscale.dev/models/jev-local) | us (GitHub) | 65.24 | Observed 2026-10-23 · source benchlm:models |
| 41 | [djev](https://modelscale.dev/models/djev) | Maisa | 64.16 | Observed 2026-10-23 · source benchlm:models |
| 41 | [djev (Maisa, diffusion-gemma)](https://modelscale.dev/models/jevbench-djev) | Maisa (David Villalón) | 64.16 | Last good 2026-10-01 · source benchlm:models |
| 43 | [Raw Qwen3 4B Instruct 2507 direct logits](https://modelscale.dev/models/jevbench-raw-qwen3-4b-instruct-2507) | Alibaba Qwen / neutral reproduction | 62.14 | Last good 2026-10-01 · source benchlm:models |
| 43 | [Raw Qwen3 4B Instruct 2507 direct logits](https://modelscale.dev/models/raw-qwen3-4b-instruct-2507) | Alibaba | 62.14 | Observed 2026-10-23 · source benchlm:models |
| 45 | [jqv (Qwen3-32B zero-shot)](https://modelscale.dev/models/jevbench-jqv) | hjmurmur (Octalab) | 60.77 | Last good 2026-10-01 · source benchlm:models |
| 45 | [jqv Qwen3-32B](https://modelscale.dev/models/jqv) | Octalab | 60.77 | Observed 2026-10-23 · source benchlm:models |
| 47 | [JevK5 v0.2.0](https://modelscale.dev/models/jevbench-jevk5-v02) | allebee | 58.11 | Last good 2026-10-01 · source benchlm:models |
| 47 | [JevK5 v0.2.0](https://modelscale.dev/models/jevk5-v0-2-0) | allebee | 58.11 | Observed 2026-10-23 · source benchlm:models |
| 49 | [Qwen3.5-9B Jev-like data-mix v2](https://modelscale.dev/models/jevbench-qwen35-9b-jev-data-mix-v2) | jsaurabh | 53.02 | Last good 2026-10-01 · source benchlm:models |
| 49 | [Qwen3.5-9B Jev-like data-mix v2](https://modelscale.dev/models/qwen35-9b-jev-data-mix-v2) | jsaurabh | 53.02 | Observed 2026-10-23 · source benchlm:models |
| 51 | [Standard One 8B](https://modelscale.dev/models/standardone-8b) | Standard Thinking | 47.79 | Observed 2026-10-23 · source benchlm:models |
| 51 | [Standard One 8B (Standard Thinking)](https://modelscale.dev/models/jevbench-standardone-8b) | Standard Thinking (myeongho12) | 47.79 | Last good 2026-10-01 · source benchlm:models |
| 53 | [NInfer Qwen3.8-Flash-Next mixed](https://modelscale.dev/models/jevbench-ninfer-qwen3-8-flash-next) | Igor L. / NInfer contributors | 47.47 | Last good 2026-10-01 · source benchlm:models |
| 53 | [NInfer Qwen3.8-Flash-Next mixed](https://modelscale.dev/models/ninfer-qwen3-8-flash-next) | Igor L. / NInfer contributors | 47.47 | Observed 2026-10-23 · source benchlm:models |
| 55 | [Instinct Dual 4B](https://modelscale.dev/models/instinct-dual-4b) | ZooWork | 46.97 | Observed 2026-10-23 · source benchlm:models |
| 55 | [Instinct Dual 4B](https://modelscale.dev/models/jevbench-instinct-dual-4b) | ZooWork / pierre-srp | 46.97 | Last good 2026-10-01 · source benchlm:models |
| 57 | [swanOne](https://modelscale.dev/models/swanone) | blockbrain | 46.56 | Observed 2026-10-23 · source benchlm:models |
| 57 | [swanOne (blockbrain, Qwen3.8-Flash-Next NVFP4)](https://modelscale.dev/models/jevbench-swanone) | blockbrain | 46.56 | Last good 2026-10-01 · source benchlm:models |
| 59 | [Raw Qwen3 8B direct logits](https://modelscale.dev/models/jevbench-raw-qwen3-8b) | Alibaba Qwen / neutral reproduction | 45.22 | Last good 2026-10-01 · source benchlm:models |
| 59 | [Raw Qwen3 8B direct logits](https://modelscale.dev/models/raw-qwen3-8b) | Alibaba | 45.22 | Observed 2026-10-23 · source benchlm:models |
| 61 | [decider-2b](https://modelscale.dev/models/decider-2b) | Mapika | 45.10 | Observed 2026-10-23 · source benchlm:models |
| 61 | [decider-2b (Mapika)](https://modelscale.dev/models/jevbench-decider-2b) | Mapika | 45.10 | Last good 2026-10-01 · source benchlm:models |
| 63 | [system-one (Qwen3-8B, Sean Goedecke)](https://modelscale.dev/models/jevbench-system-one-sg) | Sean Goedecke | 44.14 | Last good 2026-10-01 · source benchlm:models |
| 63 | [system-one Qwen3-8B](https://modelscale.dev/models/system-one-sg) | Sean Goedecke | 44.14 | Observed 2026-10-23 · source benchlm:models |
| 65 | [system-one-open](https://modelscale.dev/models/system-one-open) | mithalouni | 42.44 | Observed 2026-10-23 · source benchlm:models |
| 65 | [system-one-open (Gemma 4 E2B LoRA on an L4)](https://modelscale.dev/models/jevbench-system-one-open) | mithalouni | 42.44 | Last good 2026-10-01 · source benchlm:models |
| 67 | [Autoloops – Gemma 4 31B IT](https://modelscale.dev/models/jevbench-kushal-gemma4-31b-it-autoloops) | Autoloops | 40.53 | Last good 2026-10-01 · source benchlm:models |
| 67 | [Autoloops Gemma 4 31B IT](https://modelscale.dev/models/autoloops-gemma-4-31b-it) | Autoloops | 40.53 | Observed 2026-10-23 · source benchlm:models |
| 69 | [GPT-6 Luna (low reasoning effort)](https://modelscale.dev/models/jevbench-gpt-6-luna-low) | OpenAI | 40.48 | Last good 2026-10-01 · source benchlm:models |
| 69 | [GPT-6 Luna (low)](https://modelscale.dev/models/gpt-6-luna-low) | OpenAI | 40.48 | Observed 2026-10-23 · source benchlm:models |
| 71 | [GPT-6 Luna (default medium reasoning effort)](https://modelscale.dev/models/jevbench-gpt-6-luna) | OpenAI | 38.75 | Last good 2026-10-01 · source benchlm:models |
| 71 | [GPT-6 Luna (medium)](https://modelscale.dev/models/gpt-6-luna-medium) | OpenAI | 38.75 | Observed 2026-10-23 · source benchlm:models |
| 73 | [JevOne](https://modelscale.dev/models/jevbench-jevone) | Juspay | 38.25 | Last good 2026-10-01 · source benchlm:models |
| 73 | [JevOne](https://modelscale.dev/models/jevone) | Juspay | 38.25 | Observed 2026-10-23 · source benchlm:models |
| 75 | [kev 4B](https://modelscale.dev/models/kev-4b) | Jared Palmer | 38.07 | Observed 2026-10-23 · source benchlm:models |
| 75 | [kev 4B (research preview)](https://modelscale.dev/models/jevbench-kev-4b) | Jared Palmer | 38.07 | Last good 2026-10-01 · source benchlm:models |
| 77 | [kev 8B](https://modelscale.dev/models/kev-8b) | Jared Palmer | 34.15 | Observed 2026-10-23 · source benchlm:models |
| 77 | [kev 8B (research preview)](https://modelscale.dev/models/jevbench-kev-8b) | Jared Palmer | 34.15 | Last good 2026-10-01 · source benchlm:models |
| 79 | [open-alternative-jev (Qwen3.5-4B, IkerMoel)](https://modelscale.dev/models/jevbench-open-alternative-jev) | IkerMoel | 33.56 | Last good 2026-10-01 · source benchlm:models |
| 79 | [open-alternative-jev Qwen3.5-4B](https://modelscale.dev/models/open-alternative-jev) | IkerMoel | 33.56 | Observed 2026-10-23 · source benchlm:models |
| 81 | [Bespoke Nimble 9B](https://modelscale.dev/models/nimble-9b) | Bespoke Labs | 31.82 | Observed 2026-10-23 · source benchlm:models |
| 81 | [Bespoke Nimble 9B (Bespoke Labs)](https://modelscale.dev/models/jevbench-nimble-9b) | Bespoke Labs | 31.82 | Last good 2026-10-01 · source benchlm:models |
| 83 | [Malkuth-2B](https://modelscale.dev/models/malkuth-2b) | newfull5 (dhtocks) | 29.90 | Observed 2026-10-23 · source benchlm:models |
| 83 | [Malkuth-2B (newfull5, Kev post-train)](https://modelscale.dev/models/jevbench-malkuth-2b) | newfull5 (dhtocks) | 29.90 | Last good 2026-10-01 · source benchlm:models |
| 85 | [openjev-sglang (Qwen3.6-35B-A3B on SGLang)](https://modelscale.dev/models/jevbench-openjev-sglang) | ekzhang | 29.03 | Last good 2026-10-01 · source benchlm:models |
| 85 | [openjev-sglang Qwen3.6-35B-A3B](https://modelscale.dev/models/openjev-sglang) | ekzhang | 29.03 | Observed 2026-10-23 · source benchlm:models |
| 87 | [decider-35b-a3b](https://modelscale.dev/models/decider-35b-a3b) | Mapika | 27.50 | Observed 2026-10-23 · source benchlm:models |
| 87 | [decider-35b-a3b (Mapika)](https://modelscale.dev/models/jevbench-decider-35b-a3b) | Mapika | 27.50 | Last good 2026-10-01 · source benchlm:models |
| 89 | [local-jev Qwen3.5-4B](https://modelscale.dev/models/jevbench-localjev-qwen3-5-4b) | Amith Chandrappa (amithgc) | 25.82 | Last good 2026-10-01 · source benchlm:models |
| 89 | [local-jev Qwen3.5-4B](https://modelscale.dev/models/localjev-qwen3-5-4b) | Amith Chandrappa (amithgc) | 25.82 | Observed 2026-10-23 · source benchlm:models |
| 91 | [Nemotron Diffusion 8B (optimized vLLM)](https://modelscale.dev/models/nemotron-diffusion-8b) | pst2154 | 25.68 | Observed 2026-10-23 · source benchlm:models |
| 91 | [Nemotron Diffusion 8B (pst2154, optimized vLLM)](https://modelscale.dev/models/jevbench-nemotron-diffusion-8b) | pst2154 | 25.68 | Last good 2026-10-01 · source benchlm:models |
| 93 | [Open-Jev 9B](https://modelscale.dev/models/open-jev-zefan-9b) | Zefan Cai (@Zefan_Cai) | 24.36 | Observed 2026-10-23 · source benchlm:models |
| 93 | [Open-Jev 9B (Zefan Cai)](https://modelscale.dev/models/jevbench-open-jev-zefan-9b) | Zefan Cai (@Zefan_Cai) | 24.36 | Last good 2026-10-01 · source benchlm:models |
| 95 | [Decision 2B (FlyMy.AI, v59)](https://modelscale.dev/models/jevbench-decision-2b) | FlyMy.AI (@denti) | 22.52 | Last good 2026-10-01 · source benchlm:models |
| 95 | [Decision 2B v59](https://modelscale.dev/models/decision-2b) | FlyMy.AI | 22.52 | Observed 2026-10-23 · source benchlm:models |
| 97 | [GPT-5.6 Luna (low reasoning effort)](https://modelscale.dev/models/jevbench-gpt-5-6-luna) | OpenAI | 22.37 | Last good 2026-10-01 · source benchlm:models |
| 97 | [GPT-5.6 Luna (low)](https://modelscale.dev/models/gpt-5-6-luna-low) | OpenAI | 22.37 | Observed 2026-10-23 · source benchlm:models |
| 99 | [typecastlm](https://modelscale.dev/models/typecastlm) | Mikhail Gribov | 21.83 | Observed 2026-10-23 · source benchlm:models |
| 99 | [typecastlm (Mikhail Gribov, Qwen3.5-4B computed head)](https://modelscale.dev/models/jevbench-typecastlm) | Mikhail Gribov | 21.83 | Last good 2026-10-01 · source benchlm:models |
| 101 | [JEV Qwen3.5-9B Base NVFP4](https://modelscale.dev/models/jev-qwen3-5-9b-base-nvfp4) | WilfLin | 20.07 | Observed 2026-10-23 · source benchlm:models |
| 101 | [JEV Qwen3.5-9B Base NVFP4](https://modelscale.dev/models/jevbench-jev-qwen3-5-9b-base-nvfp4) | WilfLin | 20.07 | Last good 2026-10-01 · source benchlm:models |
| 103 | [Gemini 3.1 Flash-Lite](https://modelscale.dev/models/gemini-3-1-flash-lite) | Google | 19.58 | Observed 2026-10-23 · source benchlm:models |
| 104 | [AutoJev-27B](https://modelscale.dev/models/autojev-27b) | denis-pplx | 19.54 | Observed 2026-10-23 · source benchlm:models |
| 104 | [AutoJev-27B (denis-pplx, Qwen3.8-27B)](https://modelscale.dev/models/jevbench-autojev-27b) | unknown | 19.54 | Last good 2026-10-01 · source benchlm:models |
| 106 | [AutoJev-27B (RTX PRO 6000)](https://modelscale.dev/models/autojev-27b-rtxpro6000-a2) | denis-pplx | 19.48 | Observed 2026-10-23 · source benchlm:models |
| 106 | [AutoJev-27B (RTX PRO 6000)](https://modelscale.dev/models/jevbench-autojev-27b-rtxpro6000-a2) | unknown | 19.48 | Last good 2026-10-01 · source benchlm:models |
| 108 | [NInfer Qwen3.8-27B NVFP4](https://modelscale.dev/models/jevbench-ninfer-qwen3-8-27b) | Igor L. / NInfer contributors | 18.74 | Last good 2026-10-01 · source benchlm:models |
| 108 | [NInfer Qwen3.8-27B NVFP4](https://modelscale.dev/models/ninfer-qwen3-8-27b) | Igor L. / NInfer contributors | 18.74 | Observed 2026-10-23 · source benchlm:models |
| 110 | [Eikos-27B](https://modelscale.dev/models/eikos-27b) | caiovicentino1 | 18.50 | Observed 2026-10-23 · source benchlm:models |
| 110 | [Eikos-27B (caiovicentino1, Qwen3.8-27B)](https://modelscale.dev/models/jevbench-eikos-27b) | unknown | 18.50 | Last good 2026-10-01 · source benchlm:models |
| 112 | [NInfer Qwen3.8-27B NVFP4 (T=1.5)](https://modelscale.dev/models/jevbench-ninfer-qwen3-8-27b-t1-5) | Igor L. / NInfer contributors | 18.48 | Last good 2026-10-01 · source benchlm:models |
| 112 | [NInfer Qwen3.8-27B NVFP4 (T=1.5)](https://modelscale.dev/models/ninfer-qwen3-8-27b-t1-5) | Igor L. / NInfer contributors | 18.48 | Observed 2026-10-23 · source benchlm:models |
| 114 | [Instinct (ZooWork, Qwen3.8-27B)](https://modelscale.dev/models/jevbench-instinct) | rayrain-srp (ZooWork) | 18.32 | Last good 2026-10-01 · source benchlm:models |
| 114 | [Instinct Qwen3.8-27B](https://modelscale.dev/models/instinct) | ZooWork | 18.32 | Observed 2026-10-23 · source benchlm:models |
| 116 | [OpenJev (thinking, BF16)](https://modelscale.dev/models/jevbench-openjev-thinking) | razorback16 | 17.94 | Last good 2026-10-01 · source benchlm:models |
| 116 | [OpenJev (thinking, BF16)](https://modelscale.dev/models/openjev-thinking) | razorback16 | 17.94 | Observed 2026-10-23 · source benchlm:models |
| 118 | [djev (thinking)](https://modelscale.dev/models/djev-thinking) | Maisa | 17.40 | Observed 2026-10-23 · source benchlm:models |
| 118 | [djev (thinking)](https://modelscale.dev/models/jevbench-djev-thinking) | David Villalon / Maisa | 17.40 | Last good 2026-10-01 · source benchlm:models |
| 120 | [LitJev (Qwen3.8-27B)](https://modelscale.dev/models/jevbench-litjev) | Zhengxu Yu | 16.31 | Last good 2026-10-01 · source benchlm:models |
| 120 | [LitJev Qwen3.8-27B](https://modelscale.dev/models/litjev) | Zhengxu Yu | 16.31 | Observed 2026-10-23 · source benchlm:models |
| 122 | [Bev / Bonsai 27B](https://modelscale.dev/models/bev-bonsai-27b) | Reza Sayar | 15.77 | Observed 2026-10-23 · source benchlm:models |
| 122 | [Bev / Bonsai 27B](https://modelscale.dev/models/jevbench-bev-bonsai-27b) | Reza Sayar | 15.77 | Last good 2026-10-01 · source benchlm:models |
| 124 | [Raw Phi-4 mini direct logits](https://modelscale.dev/models/jevbench-raw-phi-4-mini) | Microsoft / neutral reproduction | 15.22 | Last good 2026-10-01 · source benchlm:models |
| 124 | [Raw Phi-4 mini direct logits](https://modelscale.dev/models/raw-phi-4-mini) | Microsoft | 15.22 | Observed 2026-10-23 · source benchlm:models |
| 126 | [OpenSourceJev (Qwen3.5-4B Q4_K_M, native llama.cpp)](https://modelscale.dev/models/jevbench-opensourcejev-qwen35-4b-q4km) | sabeel111 | 13.25 | Last good 2026-10-01 · source benchlm:models |
| 126 | [OpenSourceJev Qwen3.5-4B Q4_K_M](https://modelscale.dev/models/opensourcejev-qwen35-4b-q4km) | sabeel111 | 13.25 | Observed 2026-10-23 · source benchlm:models |
| 128 | [reflex-27b](https://modelscale.dev/models/reflex-27b) | kshetrajna12 | 13.19 | Observed 2026-10-23 · source benchlm:models |
| 128 | [reflex-27b (Qwen3.8-27B)](https://modelscale.dev/models/jevbench-reflex-27b) | kshetrajna12 | 13.19 | Last good 2026-10-01 · source benchlm:models |
| 130 | [Open-Jev 2B](https://modelscale.dev/models/open-jev-zefan-2b) | Zefan Cai (@Zefan_Cai) | 9.07 | Observed 2026-10-23 · source benchlm:models |
| 130 | [Open-Jev 2B (Zefan Cai)](https://modelscale.dev/models/jevbench-open-jev-zefan-2b) | Zefan Cai (@Zefan_Cai) | 9.07 | Last good 2026-10-01 · source benchlm:models |
| 132 | [GLiNER2 large](https://modelscale.dev/models/gliner2-large) | Fastino | 8.40 | Observed 2026-10-23 · source benchlm:models |
| 132 | [GLiNER2 large (Fastino)](https://modelscale.dev/models/jevbench-gliner2-large) | Fastino | 8.40 | Last good 2026-10-01 · source benchlm:models |
| 134 | [Qwen3-Reranker-4B](https://modelscale.dev/models/jevbench-qwen3-reranker-4b) | Qwen | 7.06 | Last good 2026-10-01 · source benchlm:models |
| 134 | [Qwen3-Reranker-4B](https://modelscale.dev/models/qwen3-reranker-4b) | Alibaba | 7.06 | Observed 2026-10-23 · source benchlm:models |
| 136 | [DeepSeek V4.1 Flash 1017](https://modelscale.dev/models/deepseek-v4-1-flash) | DeepSeek | 6.65 | Observed 2026-10-23 · source benchlm:models |
| 137 | [SimpleJev (Qwen3.5-0.8B, CPU)](https://modelscale.dev/models/jevbench-simplejev-qwen3-5-0-8b) | sabeel111 / Featherless AI | 4.00 | Last good 2026-10-01 · source benchlm:models |
| 137 | [SimpleJev Qwen3.5-0.8B](https://modelscale.dev/models/simplejev-qwen3-5-0-8b) | Featherless AI | 4.00 | Observed 2026-10-23 · source benchlm:models |
| 139 | [SimpleJev Qwen3.8-27B](https://modelscale.dev/models/jevbench-simplejev-qwen3-8-27b) | Featherless AI | 3.36 | Last good 2026-10-01 · source benchlm:models |
| 139 | [SimpleJev Qwen3.8-27B](https://modelscale.dev/models/simplejev-qwen3-8-27b) | Featherless AI | 3.36 | Observed 2026-10-23 · source benchlm:models |
| 141 | [decision-machine-1](https://modelscale.dev/models/decision-machine-1) | milliseconds.ai | 3.24 | Observed 2026-10-23 · source benchlm:models |
| 141 | [decision-machine-1 (milliseconds.ai)](https://modelscale.dev/models/jevbench-decision-machine-1) | milliseconds.ai (Baptiste Laget) | 3.24 | Last good 2026-10-01 · source benchlm:models |
| 143 | [GLiNER2.5 multi](https://modelscale.dev/models/gliner2-5-multi) | Fastino | 2.72 | Observed 2026-10-23 · source benchlm:models |
| 143 | [GLiNER2.5 multi (Fastino, 287M)](https://modelscale.dev/models/jevbench-gliner2-5-multi) | Fastino | 2.72 | Last good 2026-10-01 · source benchlm:models |
| 145 | [Bosun v3.1 0.6B](https://modelscale.dev/models/bosun-v31-0-6b) | Clause Logic | 2.50 | Observed 2026-10-23 · source benchlm:models |
| 145 | [Bosun v3.1 0.6B](https://modelscale.dev/models/jevbench-bosun-v31-0-6b) | Clause Logic | 2.50 | Last good 2026-10-01 · source benchlm:models |
| 147 | [GLiNER2 (Fastino, gliner2.5-base)](https://modelscale.dev/models/jevbench-gliner2) | Fastino | 2.27 | Last good 2026-10-01 · source benchlm:models |
| 147 | [GLiNER2.5 base](https://modelscale.dev/models/gliner2-5-base) | Fastino | 2.27 | Observed 2026-10-23 · source benchlm:models |
| 149 | [Deem 0.8B v1](https://modelscale.dev/models/deem-0-8-v1) | LibertAI | 2.14 | Observed 2026-10-23 · source benchlm:models |
| 149 | [Deem 0.8B v1](https://modelscale.dev/models/jevbench-deem-0-8-v1) | LibertAI | 2.14 | Last good 2026-10-01 · source benchlm:models |
| 151 | [JevAct](https://modelscale.dev/models/jevact) | einptein | 1.52 | Observed 2026-10-23 · source benchlm:models |
| 151 | [JevAct (einptein, jev1-2b-v2)](https://modelscale.dev/models/jevbench-jevact) | einptein | 1.52 | Last good 2026-10-01 · source benchlm:models |
| 153 | [CLM-8B](https://modelscale.dev/models/clm-8b) | Contrastive-LM | 1.51 | Observed 2026-10-23 · source benchlm:models |
| 153 | [CLM-8B (Contrastive-LM, clm-latest)](https://modelscale.dev/models/jevbench-clm-8b) | Contrastive-LM (Kwok, Kang, Suresh, Saad-Falcon, Pavone, Ré, Mirhoseini) | 1.51 | Last good 2026-10-01 · source benchlm:models |
| 155 | [kev 0.6B](https://modelscale.dev/models/kev-0-6b) | Jared Palmer | 1.26 | Observed 2026-10-23 · source benchlm:models |
| 155 | [kev 0.6B (research preview)](https://modelscale.dev/models/jevbench-kev-0-6b) | Jared Palmer | 1.26 | Last good 2026-10-01 · source benchlm:models |
| 157 | [Raw Qwen3 0.6B direct logits](https://modelscale.dev/models/jevbench-raw-qwen3-0-6b) | Alibaba Qwen / neutral reproduction | 1.08 | Last good 2026-10-01 · source benchlm:models |
| 157 | [Raw Qwen3 0.6B direct logits](https://modelscale.dev/models/raw-qwen3-0-6b) | Alibaba | 1.08 | Observed 2026-10-23 · source benchlm:models |
| 159 | [GLiNER2.5 small](https://modelscale.dev/models/gliner2-5-small) | Fastino | 0.89 | Observed 2026-10-23 · source benchlm:models |
| 159 | [GLiNER2.5 small (Fastino, 74M)](https://modelscale.dev/models/jevbench-gliner2-5-small) | Fastino | 0.89 | Last good 2026-10-01 · source benchlm:models |
| 161 | [Raw Qwen3 1.7B direct logits](https://modelscale.dev/models/jevbench-raw-qwen3-1-7b) | Alibaba Qwen / neutral reproduction | 0.85 | Last good 2026-10-01 · source benchlm:models |
| 161 | [Raw Qwen3 1.7B direct logits](https://modelscale.dev/models/raw-qwen3-1-7b) | Alibaba | 0.85 | Observed 2026-10-23 · source benchlm:models |
| 163 | [Mirror](https://modelscale.dev/models/jevbench-mirror) | Bluusun | 0.22 | Last good 2026-10-01 · source benchlm:models |
| 163 | [Mirror](https://modelscale.dev/models/mirror) | Bluusun | 0.22 | Observed 2026-10-23 · source benchlm:models |
| 165 | [ZeroEntropy zerank-2](https://modelscale.dev/models/jevbench-zerank-2) | ZeroEntropy | 0.13 | Last good 2026-10-01 · source benchlm:models |
| 165 | [ZeroEntropy zerank-2](https://modelscale.dev/models/zerank-2) | ZeroEntropy | 0.13 | Observed 2026-10-23 · source benchlm:models |
| 167 | [jeff](https://modelscale.dev/models/jeff) | Logan Markewich | 0.12 | Observed 2026-10-23 · source benchlm:models |
| 167 | [jeff (Logan Markewich, GLiFormer 400M)](https://modelscale.dev/models/jevbench-jeff) | Logan Markewich | 0.12 | Last good 2026-10-01 · source benchlm:models |
| 169 | [smalljev semantic-v9](https://modelscale.dev/models/jevbench-smalljev) | Aditya (isHeSatoshi) | 0.07 | Last good 2026-10-01 · source benchlm:models |
| 169 | [smalljev semantic-v9](https://modelscale.dev/models/smalljev) | Aditya (isHeSatoshi) | 0.07 | Observed 2026-10-23 · source benchlm:models |
| 171 | [Laya multilingual](https://modelscale.dev/models/jevbench-laya-multilingual) | Convai Innovations | 0.02 | Last good 2026-10-01 · source benchlm:models |
| 171 | [Laya multilingual](https://modelscale.dev/models/laya-multilingual) | Convai Innovations | 0.02 | Observed 2026-10-23 · source benchlm:models |
| 173 | [OpenDecision (ModernBERT-large zero-shot)](https://modelscale.dev/models/jevbench-opendecision) | Deepan Wadhwa | 0.01 | Last good 2026-10-01 · source benchlm:models |
| 173 | [OpenDecision ModernBERT-large](https://modelscale.dev/models/opendecision) | Deepan Wadhwa | 0.01 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Alibaba GTE Reranker ModernBERT-base](https://modelscale.dev/models/gte-reranker-modernbert-base) | Alibaba | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Alibaba GTE Reranker ModernBERT-base](https://modelscale.dev/models/jevbench-gte-reranker-modernbert-base) | Alibaba-NLP | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [BAAI bge-reranker-v2-m3](https://modelscale.dev/models/bge-reranker-v2-m3) | BAAI | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [BAAI bge-reranker-v2-m3](https://modelscale.dev/models/jevbench-bge-reranker-v2-m3) | BAAI | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Certo v1](https://modelscale.dev/models/certo) | AltSlate Labs | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Certo v1 (AltSlate Labs)](https://modelscale.dev/models/jevbench-certo) | AltSlate Labs | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Decision Fast (FlyMy.AI, v53a)](https://modelscale.dev/models/jevbench-decision-fast) | FlyMy.AI (@denti) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Decision Fast v53a](https://modelscale.dev/models/decision-fast) | FlyMy.AI | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [kev 0.5B](https://modelscale.dev/models/jevbench-kev-0-5b) | Jared Palmer | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [kev 0.5B](https://modelscale.dev/models/kev-0-5b) | Jared Palmer | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Laya](https://modelscale.dev/models/laya) | Convai Innovations | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Laya (Convai Innovations, ModernBERT-large 421M)](https://modelscale.dev/models/jevbench-laya) | Convai Innovations | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Laya typed-decisions](https://modelscale.dev/models/jevbench-laya-typed-decisions) | Convai Innovations | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Laya typed-decisions](https://modelscale.dev/models/laya-typed-decisions) | Convai Innovations | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [lev-350m](https://modelscale.dev/models/lev-350m) | Franck Verrot (franckverrot) | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [lev-350m (Franck Verrot, LFM2.5-350M)](https://modelscale.dev/models/jevbench-lev-350m) | Franck Verrot (franckverrot) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Mixedbread mxbai-rerank-base-v2](https://modelscale.dev/models/jevbench-mxbai-rerank-base-v2) | Mixedbread | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Mixedbread mxbai-rerank-base-v2](https://modelscale.dev/models/mxbai-rerank-base-v2) | Mixedbread | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Needle 3 (2-bit)](https://modelscale.dev/models/needle-3) | Cactus Compute | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Needle 3 (Cactus, 2-bit, local CPU)](https://modelscale.dev/models/jevbench-needle-3) | Cactus Compute | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Needle 3 (options as tools)](https://modelscale.dev/models/needle-3-tools) | Cactus Compute | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Needle 3, options as tools (post-hoc adapter mode)](https://modelscale.dev/models/jevbench-needle-3-tools) | Cactus Compute | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Open Jev JSON Canvas](https://modelscale.dev/models/open-jev-json-canvas-joshuasp) | JoshuaSP | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Open Jev JSON Canvas (JoshuaSP)](https://modelscale.dev/models/jevbench-open-jev-json-canvas-joshuasp) | JoshuaSP | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [open-jev-deberta-v3-large](https://modelscale.dev/models/open-jev-deberta-v3-large) | Kotoba Labs | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [open-jev-deberta-v3-large (local CPU)](https://modelscale.dev/models/jevbench-open-jev-deberta-v3-large) | Kotoba Labs | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [openJev Verdict](https://modelscale.dev/models/openjev-verdict) | Hemant (heman10x) | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [openJev Verdict (heman10x, ModernBERT-base 151M)](https://modelscale.dev/models/jevbench-openjev-verdict) | Hemant (heman10x) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [openJev Verdict 1.4](https://modelscale.dev/models/jevbench-openjev-verdict-1-4) | Hemant (heman10x) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [openJev Verdict 1.4](https://modelscale.dev/models/openjev-verdict-1-4) | Hemant (heman10x) | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Qwen3.5-0.8B Decision Model](https://modelscale.dev/models/mghafiri-qwen3-5-0-8b-decision-model) | Mourad Ghafiri | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [Qwen3.5-0.8B Decision Model (Mourad Ghafiri)](https://modelscale.dev/models/jevbench-mghafiri-qwen3-5-0-8b-decision-model) | Mourad Ghafiri | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Qwen3.8 27B (Chutes TEE)](https://modelscale.dev/models/jevbench-qwen3-8-27b) | Qwen / Chutes | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Qwen3.8-27B (Chutes TEE)](https://modelscale.dev/models/qwen3-8-27b-chutes) | Alibaba | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [SimpleJev Qwen3.6-35B-A3B](https://modelscale.dev/models/jevbench-simplejev-qwen3-6-35b-a3b) | Featherless AI | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [SimpleJev Qwen3.6-35B-A3B](https://modelscale.dev/models/simplejev-qwen3-6-35b-a3b) | Featherless AI | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [verdict-small](https://modelscale.dev/models/verdict-small) | Manavarya09 (Manav) | 0.00 | Observed 2026-10-23 · source benchlm:models |
| 175 | [verdict-small (Manavarya09, multilingual-e5-small 118M)](https://modelscale.dev/models/jevbench-verdict-small) | Manavarya09 (Manav) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Von (wfzyx, Option-Marker 395M)](https://modelscale.dev/models/jevbench-von-395m) | wfzyx (Victor Hugo) | 0.00 | Last good 2026-10-01 · source benchlm:models |
| 175 | [Von 395M](https://modelscale.dev/models/von-395m) | wfzyx (Victor Hugo) | 0.00 | Observed 2026-10-23 · source benchlm:models |

Rows are ordered by the value the source published, highest first. The source does not state whether a higher value is the better result, so this document does not either: for a benchmark that measures a rate of failure, read the table from the bottom.