Benchmark Heaven JevBench (Capability) Leaderboard
decision163 models · Updated October 2026
On the Benchmark Heaven JevBench (Capability) benchmark, vLLM-SR d3 (27B) ranks #1 with a score of 83.0, while Sage 1.3.0 offers the best score-per-dollar at $10.00/1M output tokens. The full ranking, with cost per million tokens, is below.
💰 Best value
Sage 1.3.0 — score 78.6 at $10.00/1M output tokens
| # | Model | Score | ECE | p50 latency | Input / 1M |
|---|---|---|---|---|---|
| 1 | vLLM-SR d3 (27B)vLLM Semantic Router | 83.0 | — | — | — |
| 2 | Perplexity Decider v1.1 (27B)Perplexity | 82.8 | — | — | $0.02 |
| 3 | Quyet-1.0-Largechinhnc | 81.7 | — | — | — |
| 4 | Torchcast Decision 27BTorchcast AI | 81.6 | — | — | — |
| 5 | Jeff-1.0-Largejgeuter | 81.5 | — | — | — |
| 6 | metask-jev-rain-31B-trainedMeTask-Rain | 79.6 | — | — | — |
| 7 | decisio v0.8.0 on gemma-4-31B-itaminry | 79.5 | — | — | — |
| 8 | amos-1-largeScira (mukaddamzaid) | 79.3 | — | — | — |
| 9 | wity-1Wity (AlphaNimble) | 79.1 | — | — | — |
| 10 | Kev 27BJared Palmer | 79.0 | — | — | — |
| 11 | SPX-CD-ProSurdAI | 78.7 | — | — | — |
| 12 | Sage 1.3.0Levanto Labs | 78.6 | — | — | $0.05 |
| 13 | metask-jev-rain-12B-trainedMeTask-Rain | 77.8 | — | — | — |
| 14 | deck31b (Gemma 4 31B)krishna-gogineni-765 | 77.6 | — | — | — |
| 15 | Eikos-27Bcaiovicentino1 | 77.2 | — | — | — |
| 16 | JevTypeSafe AI | 77.1 | — | — | $0.04 |
| 17 | Fastino GLiDEFastino | 77.0 | — | — | $0.15 |
| 18 | rene-1-31b-fp8salfatigroup | 76.0 | — | — | — |
| 19 | OpenJevrazorback16 | 75.8 | — | — | — |
| 20 | Cloudflare clefCloudflare | 75.3 | — | — | — |
| 21 | h2o-lightning-4bH2O.ai | 75.0 | — | — | — |
| 22 | simple-jev · Qwen3.8-27B (featherless)featherless-ai | 74.9 | — | — | — |
| 23 | Liquid d1 (hosted)Liquid AI | 74.5 | — | — | — |
| 24 | SPX-CD-FlashSurdAI | 74.3 | — | — | — |
| 25 | JEV-27BAutoTrust AI Lab | 73.8 | — | — | — |
| 26 | Mercury DecideInception | 73.6 | — | — | $0.02 |
| 27 | bobcat-flash-1.2sanghwa-na | 73.5 | — | — | — |
| 28 | GPT-6 Luna Decisions (OpenAI Decisions API)OpenAI | 73.5 | — | — | $0.10 |
| 29 | Surogate Rune 26B-A4B v3Surogate | 73.5 | — | — | — |
| 30 | RSI-Jev v6.1 VL 27B (effort auto)shgao | 73.5 | — | — | — |
| 31 | Autoloops – Gemma 4 31B ITAutoloops | 72.7 | — | — | — |
| 32 | decider-12bMapika | 72.5 | — | — | — |
| 33 | swanOneblockbrain | 72.4 | — | — | — |
| 34 | Xor 26B-A4B v1.1Juspay Technologies | 72.4 | — | — | — |
| 35 | Microsoft-Decision-1Microsoft | 72.2 | — | — | $0.04 |
| 36 | torchcast-decision-12bTorchcast AI | 71.7 | — | — | — |
| 37 | autojev-27bdenis-pplx | 71.6 | — | — | — |
| 38 | vLLM-SR Decision 2.0 Vega 27BvLLM Semantic Router | 71.3 | — | — | — |
| 39 | Jev-Omniakhilaaa3 | 71.3 | — | — | — |
| 40 | Winnow-12BEldanRing | 71.3 | — | — | — |
| 41 | Cygnetblockbrain | 70.9 | — | — | — |
| 42 | decisio v0.8.0 on gemma-4-12B-itaminry | 70.8 | — | — | — |
| 43 | JPT-35B-A3Bkirp | 70.7 | — | — | — |
| 44 | Decider chat · Gemma-4-31BMapika | 70.5 | — | — | — |
| 45 | Instinct (ZooWork)ZooWork | 70.5 | — | — | — |
| 46 | jebadiah-27bFrontier Infra | 70.3 | — | — | — |
| 47 | GEV-26B-DecideAutoTrust AI Lab | 70.0 | — | — | — |
| 48 | Blink v0.3 26B-A4B (NVFP4)Pixilab.ai | 70.0 | — | — | — |
| 49 | decisio v0.9.0 on gemma-4-12B-itaminry | 70.0 | — | — | — |
| 50 | NInfer Qwen3.8-Flash-Next mixedigorls | 69.7 | — | — | — |
| 51 | attest-31B-v1.0attest.engineering | 69.5 | — | — | — |
| 52 | Wald-4BORG2 AI | 69.3 | — | — | — |
| 53 | Xor 1.2Juspay Technologies | 68.9 | — | — | — |
| 54 | torchcast-decision-4bTorchcast AI | 68.7 | — | — | — |
| 55 | SimpleJev Qwen3.6-35B-A3B (Featherless)Featherless AI | 68.3 | — | — | $0.00 |
| 56 | Hopper 12B (trained)HopitAI | 68.2 | — | — | — |
| 57 | SPX-CD-OmniSurdAI | 68.2 | — | — | — |
| 58 | NInfer Qwen3.8-27B NVFP4igorls | 68.0 | — | — | — |
| 59 | Open-Jev-27B-v1.1ZefanCai | 68.0 | — | — | — |
| 60 | Aplomb 1EmpirioLabs AI | 67.5 | — | — | — |
| 61 | Bespoke Nimble 9B v3Bespoke Labs | 66.8 | — | — | — |
| 62 | LitJevzhengxuyu | 66.6 | — | — | — |
| 63 | BevReza2kn | 66.2 | — | — | — |
| 64 | Decider 35B-A3BMapika | 66.2 | — | — | — |
| 65 | Metask-Jev-4BMetask-AI-Lab | 66.0 | — | — | — |
| 66 | plumb-4bcrh225 | 65.5 | — | — | — |
| 67 | Decider 4BMapika | 64.8 | — | — | — |
| 68 | JevK5 v0.3alibiserikbay | 64.2 | — | — | — |
| 69 | Cloudflare clef-flashCloudflare | 63.9 | — | — | — |
| 70 | Vansa-3.4Vansa | 63.7 | — | — | $0.03 |
| 71 | deck-4B v1.0krishna-gogineni-765 | 63.6 | — | — | — |
| 72 | amos-1-mediumScira (mukaddamzaid) | 63.2 | — | — | — |
| 73 | janus-4bcmxu | 62.9 | — | — | — |
| 74 | jqvOctalab-Inc | 61.9 | — | — | — |
| 75 | imajev-4bmohit67890 | 61.9 | — | — | — |
| 76 | Open-Jev-9BZefanCai | 61.9 | — | — | — |
| 77 | Kahn1-Qwen3.5-4BOkura66 | 61.8 | — | — | — |
| 78 | levInterfaze | 61.7 | — | — | — |
| 79 | malkuth-4bdhtocks | 61.1 | — | — | — |
| 80 | qwen3.5-9b-jev-data-mix-v2jsaurabh | 60.9 | — | — | — |
| 81 | decision-4b-v1.2FlyMy.AI | 60.4 | — | — | — |
| 82 | jiwo 4B v2eljiwo | 60.3 | — | — | — |
| 83 | Standard One 8B SH (v3)Standard Thinking | 60.0 | — | — | — |
| 84 | Quyet-1.0-Mediumchinhnc | 59.3 | — | — | — |
| 85 | decision-4bevalengine | 59.1 | — | — | — |
| 86 | reflex 4Bkshetrajna12 | 59.0 | — | — | — |
| 87 | openjev-sglang (Qwen3.6-35B-A3B)Eric Zhang | 58.7 | — | — | — |
| 88 | StandardOne-8BStandard Thinking | 58.4 | — | — | — |
| 89 | Cloudflare clef-omni (30B-A3B)Cloudflare | 58.4 | — | — | $0.15 |
| 90 | Manchego v2.1nschlaepfer | 58.3 | — | — | — |
| 91 | messier-oneagentmessier | 58.1 | — | — | — |
| 92 | Diffusion Jev | 58.1 | — | — | — |
| 93 | pjev-2bKailune-AI | 57.8 | — | — | — |
| 94 | jev-localus | 57.5 | — | — | — |
| 95 | Instinct Dual 4BZooWork | 57.2 | — | — | — |
| 96 | decision-4b-v1.1FlyMy.AI | 57.1 | — | — | — |
| 97 | JEV-Qwen3.5-9B-Base-NVFP4WIlfLin | 57.1 | — | — | — |
| 98 | uniphore-4b-deciderUniphore | 57.1 | — | — | — |
| 99 | CoCo-Decision-4B-KoCorners | 57.0 | — | — | — |
| 100 | APUS-Reflex-9Bapus-ailab | 56.9 | — | — | — |
| 101 | typecastlm-qwen3.5-3.8bmihailgribov | 56.5 | — | — | — |
| 102 | Bespoke-Nimble-9BBespoke Labs | 56.5 | — | — | — |
| 103 | Standard One 3B SH (v2)Standard Thinking | 55.6 | — | — | — |
| 104 | SemIfTheoLeeCJ | 55.4 | — | — | — |
| 105 | local-jev Qwen3.5-4Bamithgc | 55.0 | — | — | — |
| 106 | decision-2b-previewFlyMy.AI | 54.9 | — | — | — |
| 107 | spark-s1-4b-v6abhishek085 | 54.0 | — | — | — |
| 108 | RYOTIDE-Qwen9csabag | 53.5 | — | — | — |
| 109 | system-one-open (Gemma 4 E2B LoRA)mithalouni | 51.9 | — | — | — |
| 110 | classone-qwen3.5-9bdevops-thiago | 50.5 | — | — | — |
| 111 | kev-8bJared Palmer | 50.0 | — | — | — |
| 112 | LiquidAI d1-3BLiquid AI | 49.5 | — | — | — |
| 113 | JADEtheunnecessarythings | 49.3 | — | — | — |
| 114 | open-alternative-jevikermoel | 49.1 | — | — | — |
| 115 | OpenSourceJevsabeel111 | 48.5 | — | — | — |
| 116 | Open-Jev-2BZefanCai | 47.7 | — | — | — |
| 117 | EXAONE-4.0-1.2B-JEV v0.3carrtesy | 47.6 | — | — | — |
| 118 | malkuth-2bdhtocks | 47.6 | — | — | — |
| 119 | Nemotron Diffusion 8Bpst2154 | 47.5 | — | — | — |
| 120 | jiwo 0.8B v2eljiwo | 47.2 | — | — | — |
| 121 | decision-machine-1milliseconds.ai | 46.1 | — | — | — |
| 122 | decisor-4bunderlabs | 45.8 | — | — | — |
| 123 | Decider 2BMapika | 44.6 | — | — | — |
| 124 | LAYAConvai Innovations | 44.3 | — | — | $0.00 |
| 125 | Kev 4B r10Jared Palmer | 44.0 | — | — | $0.04 |
| 126 | lev-350mFranck Verrot | 43.4 | — | — | — |
| 127 | certo-decision-modelaltslate | 43.1 | — | — | — |
| 128 | gutsykouhxp | 43.0 | — | — | — |
| 129 | PurpleMIST-Mini-1.0Olaverse Lab | 43.0 | — | — | — |
| 130 | smalljev semantic-v9isHeSatoshi | 43.0 | — | — | — |
| 131 | jul-decision-minicpm5-2busejul | 42.8 | — | — | — |
| 132 | qwen3.5-0.8B-decision-modelmghafiri | 42.6 | — | — | — |
| 133 | krino-2bGuru0381 | 42.4 | — | — | — |
| 134 | watt-flash-0.1zaitlabs | 41.6 | — | — | — |
| 135 | decision-fast-previewFlyMy.AI | 40.6 | — | — | — |
| 136 | RLCD ModernBERT 151Mheman10x | 40.4 | — | — | — |
| 137 | Quyet-1.0-Small-ENchinhnc | 39.5 | — | — | — |
| 138 | classone-gemma4-e4bdevops-thiago | 38.4 | — | — | — |
| 139 | classone-qwen3.5-4bdevops-thiago | 38.0 | — | — | — |
| 140 | OpenDecisiondeepanwadhwa | 38.0 | — | — | — |
| 141 | vega-08b-public-intentsnandakishorm | 37.7 | — | — | — |
| 142 | verdict-smallManav2op | 37.6 | — | — | — |
| 143 | Kev 0.6BJared Palmer | 37.0 | — | — | — |
| 144 | WaterSheepsamratduttaofficial | 35.5 | — | — | — |
| 145 | open-jev DeBERTa-v3-largecom-kotobalabs | 35.4 | — | — | — |
| 146 | modernbert-jev-v2ksonny4 | 34.6 | — | — | — |
| 147 | GLiNER2.5-DecideFastino | 34.5 | — | — | — |
| 148 | kev-0.5bJared Palmer | 32.9 | — | — | — |
| 149 | LiquidAI d1-omni-600MLiquid AI | 32.8 | — | — | — |
| 150 | Quyet-1.0-Smallchinhnc | 32.5 | — | — | — |
| 151 | SimpleJevfeatherless-ai | 31.6 | — | — | — |
| 152 | Quyet-1.0-Tinychinhnc | 31.1 | — | — | — |
| 153 | Open Jev JSON CanvasJoshuaSP | 30.6 | — | — | — |
| 154 | Phocinae-Largha-150M-v1Phocinae | 30.3 | — | — | — |
| 155 | system-onesgoedecke | 29.8 | — | — | — |
| 156 | JevAct (jev1-2b-v2)einptein | 28.8 | — | — | — |
| 157 | Mirror | 25.8 | — | — | — |
| 158 | tacet-sonatacodepawl | 24.3 | — | — | — |
| 159 | CLM-v0.1-8BContrastive-LM | 20.1 | — | — | — |
| 160 | classone-qwen3.5-2bdevops-thiago | 19.2 | — | — | — |
| 161 | deem-0.8-v1LibertAIDAI | 18.1 | — | — | — |
| 162 | classone-gemma4-e2bdevops-thiago | 15.2 | — | — | — |
| 163 | Julia 1Supersonic Labs | 9.8 | — | — | — |
What does Benchmark Heaven JevBench (Capability) test?
Dataset is used to resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting).
Frequently asked questions
What is the Benchmark Heaven JevBench (Capability) benchmark?
Dataset is used to resolve references in a question, perhaps to multiple input positions, and perform discrete operations over them (such as addition, counting, or sorting).
Which model is best on Benchmark Heaven JevBench (Capability)?
vLLM-SR d3 (27B) (vLLM Semantic Router) currently leads the Benchmark Heaven JevBench (Capability) leaderboard with a score of 83.0.
What is the best value model on Benchmark Heaven JevBench (Capability)?
Sage 1.3.0 offers the best score-per-dollar on Benchmark Heaven JevBench (Capability) — a score of 78.6 at $10.00 per 1M output tokens.
Pricing is indicative — confirm with the provider before production use. Updated October 2026.