Best local LLMs for 8GB VRAM
What you can run with 8 GB of GPU memory (7 GB usable after ~1 GB overhead), at 16k context with an f16 KV cache. Speeds use data measured on GeForce RTX 4060 where available.
Models that fit 8 GB, largest first
| Model | Params | Quant | Fit | Memory | Tok/s | Speed source |
|---|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | 35.95B MoE | UD-Q4_K_M | MoE offload to CPU | 22.5 GB | 36 | estimated |
| Gemma 4 31B | 31.27B | Q4_K_M | partial offload (slow) | 20.5 GB | 4 | estimated |
| Qwen3-Coder 30B-A3B | 30.53B MoE | Q4_K_M | MoE offload to CPU | 20.2 GB | 34 | estimated |
| Qwen3.6 27B | 27.78B | Q4_K_M | partial offload (slow) | 18.1 GB | 4 | estimated |
| Gemma 4 26B-A4B | 25.81B MoE | UD-Q4_K_M | MoE offload to CPU | 17.5 GB | 29 | estimated |
| Mistral Small 3.2 24B | 24.01B | Q4_K_M | partial offload (slow) | 17.0 GB | 5 | estimated |
| gpt-oss 20B | 20.91B MoE | Q4_K_M | MoE offload to CPU | 12.0 GB | 44 | estimated |
| Gemma 4 12B | 11.96B | Q4_K_M | partial offload (slow) | 7.7 GB | 21 | estimated |
| Qwen3.5 9B | 9.65B | Q4_K_M | fits fully | 6.3 GB | 33 | scaled from measured |
| Qwen3-VL 8B | 8.77B | Q4_K_M | partial offload (slow) | 7.4 GB | 31 | estimated |
| Qwen3.5 4B | 4.66B | Q8_0 | fits fully | 5.1 GB | 42 | scaled from measured |
Copy-paste commands for the top pick
Qwen3.5 9B Q4_K_M, 16k context.
llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M
How much context fits in 8 GB?
Largest context where weights plus KV cache stay inside 7 GB, by KV cache type (q8_0 halves and q4_0 quarters the cache). Native limits and YaRN extension apply per model.
| Model | Weights | f16 KV | q8_0 KV | q4_0 KV |
|---|---|---|---|---|
| Qwen3.5 9B Q4_K_M | 5.7 GB | 32k | 64k | 128k |
| Qwen3.5 4B Q8_0 | 4.5 GB | 64k | 128k | 256k |
Try the exact numbers for your setup in the interactive advisor.
What does not fit fully in 8 GB
Qwen3.6 35B-A3B (needs 23 GB at 16k, ~36 tok/s with offload); Gemma 4 31B (needs 21 GB at 16k, ~4 tok/s with offload); Qwen3-Coder 30B-A3B (needs 20 GB at 16k, ~34 tok/s with offload); Qwen3.6 27B (needs 18 GB at 16k, ~4 tok/s with offload); Gemma 4 26B-A4B (needs 17 GB at 16k, ~29 tok/s with offload); Mistral Small 3.2 24B (needs 17 GB at 16k, ~5 tok/s with offload); gpt-oss 20B (needs 12 GB at 16k, ~44 tok/s with offload); Gemma 4 12B (needs 8 GB at 16k, ~21 tok/s with offload); Qwen3-VL 8B (needs 7 GB at 16k, ~31 tok/s with offload).
GPUs with 8 GB
GeForce RTX 4060. See their pages: 8GB VRAM class (RTX 4060 8GB, 32GB RAM).
Hardware pages near 8 GB
FAQ
What is the best local LLM for 8GB VRAM?
By size, Qwen3.5 9B Q4_K_M is the largest model in our dataset that fits fully at 16k context (6.3 GB, ~33 tok/s (scaled from measured)).
How many models fit in 8GB VRAM?
2 of 13 models fit fully at 16k context with an f16 KV cache; 9 more run only with CPU offload.
How much context can I use with 8GB?
It depends on the model: for Qwen3.5 9B the KV cache allows up to 32k tokens at f16 and 128k with a q4_0 cache.
Neighbouring tiers: 12 GB · all tiers. Numbers come from the same memory-fit math as the advisor; sources and methods.