Best local LLMs for 64GB VRAM
What you can run with 64 GB of GPU memory (63 GB usable after ~1 GB overhead), at 16k context with an f16 KV cache. Speeds are estimated from the memory bandwidth of a RTX PRO 6000 Blackwell Workstation (1792 GB/s) and should be read as a ceiling for a card or pool of that size.
Models that fit 64 GB, largest first
| Model | Params | Quant | Fit | Memory | Tok/s | Speed source |
|---|---|---|---|---|---|---|
| gpt-oss 120B | 116.83B MoE | Q4_K_M | MoE offload to CPU | 63.4 GB | 251 | estimated |
| GLM-4.5-Air | 110.47B MoE | Q4_K_M | MoE offload to CPU | 76.1 GB | 28 | estimated |
| Qwen3.6 35B-A3B | 35.95B MoE | Q8_0 | fits fully | 37.3 GB | 320 | estimated |
| Gemma 4 31B | 31.27B | Q8_0 | fits fully | 35.3 GB | 30 | estimated |
| Qwen3-Coder 30B-A3B | 30.53B MoE | Q8_0 | fits fully | 34.1 GB | 281 | estimated |
| Qwen3.6 27B | 27.78B | Q8_0 | fits fully | 29.8 GB | 34 | estimated |
| Gemma 4 26B-A4B | 25.81B MoE | Q8_0 | fits fully | 27.9 GB | 245 | estimated |
| Mistral Small 3.2 24B | 24.01B | Q8_0 | fits fully | 27.7 GB | 39 | estimated |
| gpt-oss 20B | 20.91B MoE | Q8_0 | fits fully | 12.5 GB | 473 | estimated |
| Gemma 4 12B | 11.96B | Q8_0 | fits fully | 13.7 GB | 75 | estimated |
| Qwen3.5 9B | 9.65B | Q8_0 | fits fully | 10.1 GB | 103 | estimated |
| Qwen3-VL 8B | 8.77B | Q8_0 | fits fully | 11.1 GB | 113 | estimated |
| Qwen3.5 4B | 4.66B | Q8_0 | fits fully | 5.1 GB | 220 | estimated |
Copy-paste commands for the top pick
Qwen3.6 35B-A3B Q8_0, 16k context.
llama-server -hf unsloth/Qwen3.6-35B-A3B-GGUF:Q8_0 -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.6-35B-A3B-GGUF:Q8_0
How much context fits in 64 GB?
Largest context where weights plus KV cache stay inside 63 GB, by KV cache type (q8_0 halves and q4_0 quarters the cache). Native limits and YaRN extension apply per model.
| Model | Weights | f16 KV | q8_0 KV | q4_0 KV |
|---|---|---|---|---|
| Qwen3.6 35B-A3B Q8_0 | 36.9 GB | 1M | 1M | 1M |
| Gemma 4 31B Q8_0 | 33.1 GB | 256k | 256k | 256k |
| Qwen3-Coder 30B-A3B Q8_0 | 32.5 GB | 256k | 512k | 1M |
| Qwen3.6 27B Q8_0 | 28.6 GB | 256k | 512k | 1M |
| Gemma 4 26B-A4B Q8_0 | 27.3 GB | 256k | 256k | 256k |
| Mistral Small 3.2 24B Q8_0 | 25.1 GB | 128k | 128k | 128k |
| gpt-oss 20B Q8_0 | 12.1 GB | 128k | 128k | 128k |
| Gemma 4 12B Q8_0 | 13.1 GB | 256k | 256k | 256k |
Try the exact numbers for your setup in the interactive advisor.
What does not fit fully in 64 GB
gpt-oss 120B (needs 63 GB at 16k, ~251 tok/s with offload); GLM-4.5-Air (needs 76 GB at 16k, ~28 tok/s with offload).
Hardware pages near 64 GB
Mac mini (2024) M4 Pro 64GB · 2x RTX 3090 (48GB total) · MacBook Pro 14/16 M5 Pro 48GB
FAQ
What is the best local LLM for 64GB VRAM?
By size, Qwen3.6 35B-A3B Q8_0 is the largest model in our dataset that fits fully at 16k context (37.3 GB, ~320 tok/s (estimated)).
How many models fit in 64GB VRAM?
11 of 13 models fit fully at 16k context with an f16 KV cache; 2 more run only with CPU offload.
How much context can I use with 64GB?
It depends on the model: for Qwen3.6 35B-A3B the KV cache allows up to 1M tokens at f16 and 1M with a q4_0 cache.
Neighbouring tiers: 48 GB · 96 GB · all tiers. Numbers come from the same memory-fit math as the advisor; sources and methods.