Advisor / Leaderboard
Local LLM benchmark leaderboard
Median measured generation speed (tokens/s) per hardware, model and quantization class. 252 entries from public llama.cpp results plus 64 that include our own runs. Download CSV.
Measured tokens/sec
Short-context generation (tg128) in tokens/s; the median of the runs listed. Click a header to sort. Entries aggregate public community benchmarks under their upstream terms (see sources); entries marked “own” include measurements by Local LLM Advisor, licensed CC BY 4.0: attribute as Data: Local LLM Advisor (localllmadvisor.com).
| Hardware | Model | Quant | Params | Gen tok/s | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|---|---|
| NVIDIA DGX Spark | deepseek2 30B.A3B | Q8 | 29.94B MoE | 48.7 | 2364 | 1 | public |
| NVIDIA DGX Spark | gemma3 4B | Q4 | 3.88B | 81.1 | 5949 | 1 | public |
| NVIDIA DGX Spark | gpt-oss 120B | Q4 | 116.83B MoE | 56.8 | 1993 | 6 | public |
| NVIDIA DGX Spark | gpt-oss 20B | Q4 | 20.91B MoE | 86.2 | 3798 | 7 | public |
| NVIDIA DGX Spark | Llama 2 7B | Q4 | 6.74B | 54.7 | 3210 | 24 | public |
| NVIDIA DGX Spark | Llama 3.2 1B Instruct | Q4 | 1.5B | 132.3 | 7127 | 2 | public |
| NVIDIA DGX Spark | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 34.2 | 1998 | 2 | public |
| NVIDIA DGX Spark | nemotron 120B.A12B | Q4 | 120.67B MoE | 19.9 | 769 | 1 | public |
| NVIDIA DGX Spark | nemotron 4B | Q8 | 3.97B | 52.9 | 2762 | 1 | public |
| NVIDIA DGX Spark | nemotron_h_moe 120B.A12B | Q4 | 120.67B MoE | 16.6 | 730 | 1 | public |
| NVIDIA DGX Spark | nemotron_h_moe 31B.A3.5B | Q4 | 31.58B MoE | 63.1 | 1899 | 2 | public |
| NVIDIA DGX Spark | qwen2 7B | Q8 | 7.62B | 29.4 | 2250 | 1 | public |
| NVIDIA DGX Spark | Qwen2.5 14B Instruct | Q4 | 14.77B | 18.3 | 1092 | 2 | public |
| NVIDIA DGX Spark | Qwen3-Coder 30B-A3B | Q8 | 30.53B MoE | 61.1 | 2987 | 1 | public |
| NVIDIA DGX Spark | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 60 | 2789 | 1 | public |
| NVIDIA DGX Spark | Qwen3.6 35B-A3B | Q8 | 35.95B MoE | 49.2 | 2572 | 1 | public |
| NVIDIA DGX Spark | qwen35moe 122B.A10B | Q4 | 122.11B MoE | 21.3 | 1075 | 1 | public |
| NVIDIA DGX Spark | qwen3next 80B.A3B | Q4 | 79.67B MoE | 49.3 | 1743 | 1 | public |
| NVIDIA DGX Spark | qwen3vlmoe 235B.A22B | Q4 | 235.09B MoE | 13 | 529 | 1 | public |
| Apple M3 Ultra | gpt-oss 20B | Q4 | 20.91B MoE | 115.5 | 2817 | 1 | public |
| Apple M3 Ultra | Llama 2 7B | Q4 | 6.74B | 114.6 | 1117 | 3 | public |
| Apple M3 Ultra | Llama 3.2 1B Instruct | Q4 | 1.5B | 177.9 | 5000 | 3 | public |
| Apple M3 Ultra | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 63 | 1086 | 2 | public |
| Apple M3 Ultra | Qwen2.5 14B Instruct | Q4 | 14.77B | 35.8 | 568 | 3 | public |
| Apple M4 | llama 1B | F16 | 1.1B | 46.2 | 1376 | 1 | public |
| Apple M4 | llama 1B | Q4 | 1.1B | 121.2 | 1335 | 1 | public |
| Apple M4 | llama 1B | Q8 | 1.1B | 75.3 | 1334 | 1 | public |
| Apple M4 | Llama 2 7B | Q4 | 6.74B | 23.3 | 212 | 1 | public |
| Apple M4 | Llama 3.2 1B Instruct | Q4 | 1.5B | 74.2 | 1028 | 5 | public |
| Apple M4 | llama 3B | Q4 | 3.21B | 46 | 530 | 1 | public |
| Apple M4 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 16.1 | 157 | 4 | public |
| Apple M4 | qwen2 7B | Q4 | 7.62B | 22.6 | 240 | 1 | public |
| Apple M4 | Qwen2.5 14B Instruct | Q4 | 14.77B | 8.8 | 83 | 4 | public |
| Apple M4 Max (32-core GPU) | gpt-oss 20B | Q4 | 20.91B MoE | 92.4 | 1277 | 1 | public |
| Apple M4 Max (32-core GPU) | Llama 2 7B | F16 | 6.74B | 25.8 | 769 | 2 | public |
| Apple M4 Max (32-core GPU) | Llama 2 7B | Q4 | 6.74B | 70.8 | 739 | 2 | public |
| Apple M4 Max (32-core GPU) | Llama 2 7B | Q8 | 6.74B | 44.3 | 711 | 2 | public |
| Apple M4 Max (32-core GPU) | Llama 3.2 1B Instruct | Q4 | 1.5B | 166.4 | 3255 | 1 | public |
| Apple M4 Max (32-core GPU) | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 45.5 | 540 | 1 | public |
| Apple M4 Max (32-core GPU) | Qwen2.5 14B Instruct | Q4 | 14.77B | 25.6 | 285 | 1 | public |
| Apple M4 Max (40-core GPU) | Llama 3.2 1B Instruct | Q4 | 1.5B | 180 | 3817 | 3 | public |
| Apple M4 Max (40-core GPU) | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 52.9 | 632 | 3 | public |
| Apple M4 Max (40-core GPU) | minicpm-v47 35B-A3B | Q4 | 35B MoE | 177.8 | 693 | 1 | public |
| Apple M4 Max (40-core GPU) | Qwen2.5 14B Instruct | Q4 | 14.77B | 29 | 328 | 3 | public |
| Apple M4 Pro | Llama 3.2 1B Instruct | Q4 | 1.5B | 112 | 1948 | 6 | public |
| Apple M4 Pro | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 32.5 | 349 | 5 | public |
| Apple M4 Pro | Qwen2.5 14B Instruct | Q4 | 14.77B | 17.1 | 170 | 6 | public |
| Apple M5 | Llama 3.2 1B Instruct | Q4 | 1.5B | 99.1 | 1275 | 3 | public |
| Apple M5 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 20.2 | 189 | 3 | public |
| Apple M5 | Qwen2.5 14B Instruct | Q4 | 14.77B | 10.2 | 96 | 2 | public |
| Apple M5 Max (32-core GPU) | Llama 3.2 1B Instruct | Q4 | 1.5B | 229.3 | 3635 | 1 | public |
| Apple M5 Max (32-core GPU) | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 62.2 | 637 | 1 | public |
| Apple M5 Max (32-core GPU) | Qwen2.5 14B Instruct | Q4 | 14.77B | 34.2 | 343 | 1 | public |
| Apple M5 Max (40-core GPU) | Llama 2 7B | F16 | 6.74B | 35.4 | 3190 | 1 | public |
| Apple M5 Max (40-core GPU) | Llama 2 7B | Q4 | 6.74B | 119.1 | 3347 | 1 | public |
| Apple M5 Max (40-core GPU) | Llama 2 7B | Q8 | 6.74B | 72.5 | 3220 | 1 | public |
| Apple M5 Max (40-core GPU) | Qwen2.5 14B Instruct | Q4 | 14.77B | 39.6 | 393 | 2 | public |
| Apple M5 Max (40-core GPU) | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 82.6 | 1 | public | |
| Apple M5 Max (40-core GPU) | Qwen3.6 35B-A3B | Q8 | 35.95B MoE | 71.4 | 1 | public | |
| Apple M5 Max (40-core GPU) | Tess-4-27B (Qwen3.6-27B fine-tune) | Q4 | 27B | 28.3 | 1 | public | |
| Apple M5 Max (40-core GPU) | Tess-4-27B (Qwen3.6-27B fine-tune) | Q8 | 27B | 17.3 | 1 | public | |
| Apple M5 Pro | Llama 3.2 1B Instruct | Q4 | 1.5B | 175 | 2409 | 2 | public |
| Apple M5 Pro | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 44.1 | 376 | 4 | public |
| Apple M5 Pro | Qwen2.5 14B Instruct | Q4 | 14.77B | 23.7 | 216 | 5 | public |
| Apple M6 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 25.2 | 263 | 1 | public |
| GeForce RTX 3060 12GB | Gemma 4 12B | Q4 | 11.96B | 39.7 | 1277 | 1 | own + public |
| GeForce RTX 3060 12GB | Llama 2 7B | Q4 | 6.74B | 75.6 | 2273 | 4 | public |
| GeForce RTX 3060 12GB | Llama 3.2 1B Instruct | Q4 | 1.5B | 185 | 6042 | 1 | public |
| GeForce RTX 3060 12GB | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 51.3 | 1483 | 1 | public |
| GeForce RTX 3060 12GB | Qwen2.5 14B Instruct | Q4 | 14.77B | 26.6 | 760 | 1 | public |
| GeForce RTX 3060 12GB | Qwen3-VL 8B | Q4 | 8.77B | 61.5 | 1983 | 1 | own + public |
| GeForce RTX 3060 12GB | Qwen3-VL 8B | Q8 | 8.77B | 39.3 | 2087 | 1 | own + public |
| GeForce RTX 3060 12GB | Qwen3.5 4B | Q4 | 4.66B | 85.3 | 2997 | 1 | own + public |
| GeForce RTX 3060 12GB | Qwen3.5 4B | Q8 | 4.66B | 62.6 | 3133 | 1 | own + public |
| GeForce RTX 3060 12GB | Qwen3.5 9B | Q4 | 9.65B | 55.3 | 1849 | 1 | own + public |
| GeForce RTX 3060 12GB | Qwen3.5 9B | Q8 | 9.65B | 36.6 | 1945 | 1 | own + public |
| GeForce RTX 3090 | Gemma 4 12B | Q4 | 11.96B | 86.5 | 3272 | 1 | own + public |
| GeForce RTX 3090 | Gemma 4 12B | Q8 | 11.96B | 57.2 | 3408 | 1 | own + public |
| GeForce RTX 3090 | Gemma 4 26B-A4B | Q4 | 25.81B MoE | 140.3 | 4664 | 1 | own + public |
| GeForce RTX 3090 | Gemma 4 31B | Q4 | 31.27B | 38.8 | 1292 | 1 | own + public |
| GeForce RTX 3090 | gpt-oss 20B | Q4 | 20.91B MoE | 189.5 | 6092 | 16 | own + public |
| GeForce RTX 3090 | gpt-oss 20B | Q8 | 20.91B MoE | 204.2 | 5952 | 1 | own + public |
| GeForce RTX 3090 | Llama 2 7B | Q4 | 6.74B | 151 | 4517 | 35 | public |
| GeForce RTX 3090 | Llama 3.2 1B Instruct | Q4 | 1.5B | 269.9 | 13482 | 1 | public |
| GeForce RTX 3090 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 96.3 | 3550 | 1 | public |
| GeForce RTX 3090 | Mistral Small 3.2 24B | Q4 | 24.01B | 53.2 | 1897 | 1 | own + public |
| GeForce RTX 3090 | Qwen2.5 14B Instruct | Q4 | 14.77B | 56.2 | 1950 | 1 | public |
| GeForce RTX 3090 | Qwen3-Coder 30B-A3B | Q4 | 30.53B MoE | 206.3 | 4305 | 1 | own + public |
| GeForce RTX 3090 | Qwen3-VL 8B | Q4 | 8.77B | 134.6 | 4898 | 1 | own + public |
| GeForce RTX 3090 | Qwen3-VL 8B | Q8 | 8.77B | 91.4 | 5109 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.5 4B | Q4 | 4.66B | 179.1 | 7366 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.5 4B | Q8 | 4.66B | 138 | 7631 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.5 9B | Q4 | 9.65B | 122.1 | 4632 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.5 9B | Q8 | 9.65B | 85.7 | 4788 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.6 27B | Q4 | 27.78B | 42.2 | 1516 | 1 | own + public |
| GeForce RTX 3090 | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 159 | 4053 | 1 | own + public |
| GeForce RTX 4060 | Llama 3.2 1B Instruct | Q4 | 1.5B | 133.7 | 5353 | 1 | public |
| GeForce RTX 4060 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 38.3 | 1544 | 1 | public |
| GeForce RTX 4060 Ti 16GB | Gemma 4 12B | Q4 | 11.96B | 35.4 | 2186 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Gemma 4 12B | Q8 | 11.96B | 20.6 | 2216 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | gpt-oss 20B | Q4 | 20.91B MoE | 103.7 | 4049 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | gpt-oss 20B | Q8 | 20.91B MoE | 94.4 | 4110 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Llama 3.2 1B Instruct | Q4 | 1.5B | 210.6 | 9225 | 1 | public |
| GeForce RTX 4060 Ti 16GB | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 48.3 | 2233 | 1 | public |
| GeForce RTX 4060 Ti 16GB | Mistral Small 3.2 24B | Q4 | 24.01B | 19.1 | 1206 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen2.5 14B Instruct | Q4 | 14.77B | 25.6 | 1239 | 1 | public |
| GeForce RTX 4060 Ti 16GB | Qwen3-VL 8B | Q4 | 8.77B | 54.2 | 3406 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen3-VL 8B | Q8 | 8.77B | 32.5 | 3559 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen3.5 4B | Q4 | 4.66B | 82.4 | 5288 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen3.5 4B | Q8 | 4.66B | 54.3 | 5330 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen3.5 9B | Q4 | 9.65B | 48.3 | 3191 | 1 | own + public |
| GeForce RTX 4060 Ti 16GB | Qwen3.5 9B | Q8 | 9.65B | 30.4 | 3360 | 1 | own + public |
| GeForce RTX 4070 12GB | Gemma 4 12B | Q4 | 11.96B | 58.5 | 2928 | 1 | own + public |
| GeForce RTX 4070 12GB | Llama 2 7B | Q4 | 6.74B | 91.9 | 3737 | 2 | public |
| GeForce RTX 4070 12GB | Llama 3.2 1B Instruct | Q4 | 1.5B | 283 | 12718 | 1 | public |
| GeForce RTX 4070 12GB | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 76.4 | 3192 | 1 | public |
| GeForce RTX 4070 12GB | Qwen2.5 14B Instruct | Q4 | 14.77B | 37.9 | 1692 | 1 | public |
| GeForce RTX 4070 12GB | Qwen3-VL 8B | Q4 | 8.77B | 89.7 | 4489 | 1 | own + public |
| GeForce RTX 4070 12GB | Qwen3-VL 8B | Q8 | 8.77B | 55 | 4714 | 1 | own + public |
| GeForce RTX 4070 12GB | Qwen3.5 4B | Q4 | 4.66B | 129.5 | 6623 | 1 | own + public |
| GeForce RTX 4070 12GB | Qwen3.5 4B | Q8 | 4.66B | 89 | 7082 | 1 | own + public |
| GeForce RTX 4070 12GB | Qwen3.5 9B | Q4 | 9.65B | 79.8 | 4237 | 1 | own + public |
| GeForce RTX 4070 12GB | Qwen3.5 9B | Q8 | 9.65B | 51.3 | 4418 | 1 | own + public |
| GeForce RTX 4090 | Gemma 4 12B | Q4 | 11.96B | 102 | 6465 | 1 | own + public |
| GeForce RTX 4090 | Gemma 4 12B | Q8 | 11.96B | 64.2 | 6420 | 1 | own + public |
| GeForce RTX 4090 | Gemma 4 26B-A4B | Q4 | 25.81B MoE | 169.3 | 9165 | 1 | own + public |
| GeForce RTX 4090 | Gemma 4 31B | Q4 | 31.27B | 44.4 | 2403 | 1 | own + public |
| GeForce RTX 4090 | gpt-oss 20B | Q4 | 20.91B MoE | 254.1 | 10346 | 11 | own + public |
| GeForce RTX 4090 | gpt-oss 20B | Q8 | 20.91B MoE | 258.4 | 12257 | 1 | own + public |
| GeForce RTX 4090 | Llama 2 7B | Q4 | 6.74B | 166.9 | 10830 | 41 | public |
| GeForce RTX 4090 | Llama 3.1 8B | Q4 | 8.03B | 150.9 | 1 | public | |
| GeForce RTX 4090 | Llama 3.2 1B Instruct | Q4 | 1.5B | 292 | 22430 | 2 | public |
| GeForce RTX 4090 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 105.3 | 7146 | 2 | public |
| GeForce RTX 4090 | Mistral Small 3.2 24B | Q4 | 24.01B | 61.6 | 3303 | 1 | own + public |
| GeForce RTX 4090 | Qwen2.5 14B Instruct | Q4 | 14.77B | 49.1 | 3506 | 3 | public |
| GeForce RTX 4090 | Qwen3-14B | Q4 | 14.77B | 83.9 | 1 | public | |
| GeForce RTX 4090 | Qwen3-8B | Q4 | 8.19B | 133.3 | 2 | public | |
| GeForce RTX 4090 | Qwen3-8B | Q8 | 8.19B | 90.1 | 1 | public | |
| GeForce RTX 4090 | Qwen3-Coder 30B-A3B | Q4 | 30.53B MoE | 266 | 8590 | 1 | own + public |
| GeForce RTX 4090 | Qwen3-VL 8B | Q4 | 8.77B | 158.5 | 10766 | 1 | own + public |
| GeForce RTX 4090 | Qwen3-VL 8B | Q8 | 8.77B | 102.3 | 10242 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.5 4B | Q4 | 4.66B | 220.2 | 14882 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.5 4B | Q8 | 4.66B | 158.3 | 15455 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.5 9B | Q4 | 9.65B | 141.6 | 9762 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.5 9B | Q8 | 9.65B | 95.1 | 9850 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.6 27B | Q4 | 27.78B | 48.4 | 2795 | 1 | own + public |
| GeForce RTX 4090 | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 190.5 | 7962 | 1 | own + public |
| GeForce RTX 5070 | Gemma 4 12B | Q4 | 11.96B | 70.1 | 3168 | 1 | own + public |
| GeForce RTX 5070 | Llama 2 7B | Q4 | 6.74B | 127.9 | 5484 | 2 | public |
| GeForce RTX 5070 | Llama 3.2 1B Instruct | Q4 | 1.5B | 100.7 | 8328 | 1 | public |
| GeForce RTX 5070 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 55.9 | 2927 | 1 | public |
| GeForce RTX 5070 | Qwen2.5 14B Instruct | Q4 | 14.77B | 33.3 | 1511 | 2 | public |
| GeForce RTX 5070 | Qwen3-VL 8B | Q4 | 8.77B | 111.9 | 4912 | 1 | own + public |
| GeForce RTX 5070 | Qwen3-VL 8B | Q8 | 8.77B | 72.4 | 5041 | 1 | own + public |
| GeForce RTX 5070 | Qwen3.5 4B | Q4 | 4.66B | 154.5 | 7449 | 1 | own + public |
| GeForce RTX 5070 | Qwen3.5 4B | Q8 | 4.66B | 115.7 | 7724 | 1 | own + public |
| GeForce RTX 5070 | Qwen3.5 9B | Q4 | 9.65B | 101.3 | 4652 | 1 | own + public |
| GeForce RTX 5070 | Qwen3.5 9B | Q8 | 9.65B | 67.6 | 4757 | 1 | own + public |
| GeForce RTX 5070 Ti | gpt-oss 20B | Q4 | 20.91B MoE | 189.5 | 6333 | 2 | public |
| GeForce RTX 5070 Ti | Llama 2 7B | Q4 | 6.74B | 166 | 6911 | 18 | public |
| GeForce RTX 5070 Ti | Llama 3.2 1B Instruct | Q4 | 1.5B | 145.3 | 12339 | 2 | public |
| GeForce RTX 5070 Ti | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 50.1 | 3116 | 2 | public |
| GeForce RTX 5070 Ti | Qwen2.5 14B Instruct | Q4 | 14.77B | 31.3 | 1994 | 2 | public |
| GeForce RTX 5080 | devstral-24b | Q4 | 23.6B | 35.6 | 7356 | 2 | public |
| GeForce RTX 5080 | gpt-oss 20B | Q4 | 20.91B MoE | 187.5 | 9447 | 10 | public |
| GeForce RTX 5080 | Llama 2 7B | Q4 | 6.74B | 182.5 | 7425 | 18 | public |
| GeForce RTX 5080 | Llama 3.2 1B Instruct | Q4 | 1.5B | 196.8 | 15539 | 3 | public |
| GeForce RTX 5080 | llama3.2-3b | Q4 | 3.2B | 305.3 | 18041 | 4 | public |
| GeForce RTX 5080 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 35.5 | 4045 | 2 | public |
| GeForce RTX 5080 | Qwen2.5 14B Instruct | Q4 | 14.77B | 33.1 | 2416 | 3 | public |
| GeForce RTX 5080 | qwen2.5-14b | Q4 | 14.8B | 95.6 | 5039 | 4 | public |
| GeForce RTX 5080 | qwen2.5-7b | Q4 | 7.6B | 178.2 | 10059 | 4 | public |
| GeForce RTX 5080 | qwen2.5-coder-14b | Q4 | 14.8B | 94.1 | 4926 | 4 | public |
| GeForce RTX 5080 | qwen2.5-coder-7b | Q4 | 7.6B | 172.7 | 9579 | 4 | public |
| GeForce RTX 5090 | Gemma 4 12B | Q6 | 11.96B | 122.3 | 7160 | 1 | public |
| GeForce RTX 5090 | Gemma 4 31B | Q4 | 31.27B | 76.4 | 4255 | 1 | public |
| GeForce RTX 5090 | Gemma 4 31B | Q6 | 31.27B | 55.4 | 3165 | 1 | public |
| GeForce RTX 5090 | gpt-oss 120B | Q4 | 116.83B MoE | 46.5 | 484 | 1 | public |
| GeForce RTX 5090 | gpt-oss 20B | Q4 | 20.91B MoE | 298.3 | 12930 | 5 | public |
| GeForce RTX 5090 | LFM2.5-8B-A1B | Q8 | 8B MoE | 563.1 | 30681 | 1 | public |
| GeForce RTX 5090 | Llama 2 7B | Q4 | 6.74B | 289 | 14474 | 30 | public |
| GeForce RTX 5090 | Llama 3.2 1B Instruct | Q4 | 1.5B | 119.8 | 16760 | 2 | public |
| GeForce RTX 5090 | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 66.3 | 6297 | 1 | public |
| GeForce RTX 5090 | Mistral Small 4 119B | Q4 | 119B | 35.9 | 243 | 1 | public |
| GeForce RTX 5090 | Muse-Glimmer-30B | Q4 | 30B | 85.7 | 4740 | 1 | public |
| GeForce RTX 5090 | Nemotron-3-Nano-30B-A3B | Q4 | 31.58B MoE | 363.7 | 10674 | 2 | public |
| GeForce RTX 5090 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q2 | 32.91B MoE | 401.1 | 12431 | 2 | public |
| GeForce RTX 5090 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q3 | 32.91B MoE | 390.4 | 11877 | 2 | public |
| GeForce RTX 5090 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q4 | 31.58B MoE | 386 | 11864 | 2 | public |
| GeForce RTX 5090 | NVIDIA-Nemotron-3.5-Lightning-30B-A3B | Q5 | 32.91B MoE | 348.7 | 11067 | 1 | public |
| GeForce RTX 5090 | Ornith 1.5 35B-A3B | Q4 | 35.51B MoE | 303.2 | 8953 | 1 | public |
| GeForce RTX 5090 | Ornith 1.5 9B | Q4 | 8.95B | 225.2 | 11938 | 1 | public |
| GeForce RTX 5090 | Ornith 1.5 9B | Q6 | 8.95B | 187.7 | 10164 | 1 | public |
| GeForce RTX 5090 | Qwable-3.6-27B | Q4 | 27B | 77.6 | 3809 | 1 | public |
| GeForce RTX 5090 | Qwable-5-27B-Coder | Q6 | 27B | 62.9 | 3211 | 1 | public |
| GeForce RTX 5090 | Qwen2.5 14B Instruct | Q4 | 14.77B | 41.8 | 3734 | 2 | public |
| GeForce RTX 5090 | Qwen3-Coder-Next | Q2 | 79.67B | 224.9 | 4447 | 1 | public |
| GeForce RTX 5090 | Qwen3.6 27B | Q4 | 27.78B | 80.7 | 4620 | 2 | public |
| GeForce RTX 5090 | Qwen3.6 27B | Q6 | 27.78B | 61.9 | 3222 | 1 | public |
| GeForce RTX 5090 | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 271 | 9240 | 1 | public |
| GeForce RTX 5090 | Qwen3.6 35B-A3B | Q6 | 35.95B MoE | 259.7 | 8322 | 1 | public |
| GeForce RTX 5090 | Qwen3.6-27B-MTP | Q4 | 27.32B | 77 | 3836 | 1 | public |
| GeForce RTX 5090 | Qwen3.6-27B-MTP-pi-tune | Q6 | 27B | 62.9 | 3198 | 1 | public |
| GeForce RTX 5090 | Qwen3.6-28B-REAP-A3B | Q6 | 28B | 246.6 | 8227 | 1 | public |
| GeForce RTX 5090 | Qwen3.8-27B | Q2 | 27.32B | 109 | 3621 | 2 | public |
| GeForce RTX 5090 | Qwen3.8-27B | Q3 | 27.32B | 96.2 | 3944 | 1 | public |
| GeForce RTX 5090 | Qwen3.8-27B | Q4 | 27.32B | 77.6 | 3860 | 2 | public |
| GeForce RTX 5090 | Qwen3.8-27B | Q6 | 27.32B | 63.1 | 3223 | 1 | public |
| GeForce RTX 5090 | Qwen3.8-27B | Q8 | 27.32B | 53.1 | 3834 | 1 | public |
| GeForce RTX 5090 | Qwopus3.6-27B-Coder | Q5 | 27B | 70.5 | 3761 | 1 | public |
| GeForce RTX 5090 | Qwopus3.6-27B-Coder-Compat | Q6 | 27B | 62.9 | 3211 | 1 | public |
| GeForce RTX 5090 | Qwopus3.8-27B-Flash | Q6 | 27.32B | 63 | 3148 | 1 | public |
| RTX PRO 6000 Blackwell Workstation | gpt-oss 120B | Q4 | 116.83B MoE | 196.3 | 5518 | 1 | public |
| RTX PRO 6000 Blackwell Workstation | gpt-oss 20B | Q4 | 20.91B MoE | 286.9 | 11522 | 1 | public |
| RTX PRO 6000 Blackwell Workstation | Llama 2 7B | Q4 | 6.74B | 270.9 | 15737 | 12 | public |
| RTX PRO 6000 Blackwell Workstation | Llama 3.2 1B Instruct | Q4 | 1.5B | 191.4 | 20440 | 1 | public |
| RTX PRO 6000 Blackwell Workstation | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 85.8 | 6346 | 1 | public |
| RTX PRO 6000 Blackwell Workstation | Qwen2.5 14B Instruct | Q4 | 14.77B | 64.1 | 4416 | 1 | public |
| Radeon RX 7900 XTX | Gemma 4 31B | Q8 | 31.27B | 22.7 | 852 | 2 | public |
| Radeon RX 7900 XTX | gemma3 27B | Q4 | 27.01B | 34 | 755 | 2 | public |
| Radeon RX 7900 XTX | gemma3 27B | Q6 | 27.01B | 6.9 | 421 | 2 | public |
| Radeon RX 7900 XTX | GLM-4.5-Air | Q3 | 110.47B MoE | 12 | 258 | 2 | public |
| Radeon RX 7900 XTX | Llama 2 7B | Q4 | 6.74B | 144.2 | 3098 | 166 | public |
| Radeon RX 7900 XTX | Llama 3.2 1B Instruct | Q4 | 1.5B | 78.4 | 4089 | 2 | public |
| Radeon RX 7900 XTX | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 42.3 | 875 | 2 | public |
| Radeon RX 7900 XTX | Qwen2.5 14B Instruct | Q4 | 14.77B | 26.8 | 491 | 2 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | cognitivecomputations_Dolphin-Mistral-24B-Venice-Edition-Q6_K | Q6 | 23.57B | 11.5 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Gemma 4 26B-A4B | Q4 | 25.81B MoE | 61.6 | 1287 | 2 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Gemma 4 31B | Q4 | 31.27B | 10.5 | 4 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | gpt-oss 120B | Q4 | 116.83B MoE | 56.4 | 727 | 2 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | LFM2.5 8B-A1B | Q4 | 8B MoE | 171.2 | 3398 | 3 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Llama 2 7B | Q4 | 6.74B | 50.6 | 1112 | 54 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Llama 3.1 70B | Q4 | 70B | 4.7 | 22 | 1 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Llama 3.2 1B Instruct | Q4 | 1.5B | 107.4 | 1150 | 8 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Llama 4 Scout 109B | Q4 | 109B | 18.3 | 331 | 1 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Meta Llama 3.1 8B Instruct | Q4 | 8.03B | 21.4 | 240 | 3 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Nemotron 3 Nano 30B-A3B | Q4 | 30B MoE | 76 | 1312 | 1 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Nemotron 3 Super 120B-A12B | Q4 | 120B MoE | 18.4 | 296 | 3 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | ornith-1.0-35b-Q4_K_M | Q4 | 34.66B | 74.1 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen2.5 14B Instruct | Q4 | 14.77B | 11.7 | 111 | 9 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3 0.6B | Q8 | 0.6B | 266 | 13112 | 1 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3-Coder 30B-A3B | Q4 | 30.53B MoE | 96.9 | 1399 | 18 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3-Next 80B-A3B | Q4 | 80B MoE | 59.1 | 676 | 3 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.5 35B-A3B | Q4 | 35B MoE | 64.9 | 1080 | 1 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 27B | Q3 | 27.78B | 14.4 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 27B | Q4 | 27.78B | 13.4 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 27B | Q5 | 27.78B | 11 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 27B | Q6 | 27.78B | 9.5 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 35B-A3B | Q4 | 35.95B MoE | 63.4 | 1106 | 6 | public |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.6 35B-A3B | Q8 | 35.95B MoE | 53.5 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | Qwen3.8-27B-UD-Q4_K_XL | Q4 | 27.32B | 11.9 | 1 | public | |
| AMD Ryzen AI Max+ 395 (Strix Halo) | qwen35moe 122B.A10B | Q6 | 124.64B MoE | 19 | 146 | 9 | public |
FAQ
Where do these numbers come from?
Public llama.cpp benchmark results (llama.cpp discussions, LocalScore, open datasets) plus our own llama-bench sweeps on rented GPUs. Each row is a median across runs; the CSV lists source ids per row.
Can I reuse the data?
Our own measurements and curated data are CC BY 4.0 with attribution to Local LLM Advisor (localllmadvisor.com). Third-party rows keep their upstream licenses.
Submit your benchmark to add your hardware.