Best local LLMs for 16GB VRAM
What you can run with 16 GB of GPU memory (15 GB usable after ~1 GB overhead), at 16k context with an f16 KV cache. Speeds use data measured on GeForce RTX 5070 Ti where available.
Models that fit 16 GB, largest first
| Model | Params | Quant | Fit | Memory | Tok/s | Speed source |
|---|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | 35.95B MoE | UD-Q4_K_M | MoE offload to CPU | 22.5 GB | 71 | estimated |
| Gemma 4 31B | 31.27B | Q4_K_M | partial offload (slow) | 20.5 GB | 9 | estimated |
| Qwen3-Coder 30B-A3B | 30.53B MoE | Q4_K_M | MoE offload to CPU | 20.2 GB | 79 | estimated |
| Qwen3.6 27B | 27.78B | Q4_K_M | partial offload (slow) | 18.1 GB | 13 | estimated |
| Gemma 4 26B-A4B | 25.81B MoE | UD-Q4_K_M | MoE offload to CPU | 17.5 GB | 98 | estimated |
| Mistral Small 3.2 24B | 24.01B | Q4_K_M | partial offload (slow) | 17.0 GB | 19 | estimated |
| gpt-oss 20B | 20.91B MoE | Q8_0 | fits fully | 12.5 GB | 189 | scaled from measured |
| Gemma 4 12B | 11.96B | Q8_0 | fits fully | 13.7 GB | 21 | scaled from measured |
| Qwen3.5 9B | 9.65B | Q8_0 | fits fully | 10.1 GB | 29 | scaled from measured |
| Qwen3-VL 8B | 8.77B | Q8_0 | fits fully | 11.1 GB | 32 | scaled from measured |
| Qwen3.5 4B | 4.66B | Q8_0 | fits fully | 5.1 GB | 54 | scaled from measured |
Copy-paste commands for the top pick
gpt-oss 20B Q8_0, 16k context.
llama-server -hf unsloth/gpt-oss-20b-GGUF:Q8_0 -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/gpt-oss-20b-GGUF:Q8_0
How much context fits in 16 GB?
Largest context where weights plus KV cache stay inside 15 GB, by KV cache type (q8_0 halves and q4_0 quarters the cache). Native limits and YaRN extension apply per model.
| Model | Weights | f16 KV | q8_0 KV | q4_0 KV |
|---|---|---|---|---|
| gpt-oss 20B Q8_0 | 12.1 GB | 64k | 128k | 128k |
| Gemma 4 12B Q8_0 | 13.1 GB | 64k | 128k | 256k |
| Qwen3.5 9B Q8_0 | 9.5 GB | 128k | 256k | 512k |
| Qwen3-VL 8B Q8_0 | 8.7 GB | 32k | 64k | 128k |
| Qwen3.5 4B Q8_0 | 4.5 GB | 256k | 512k | 1M |
Try the exact numbers for your setup in the interactive advisor.
What does not fit fully in 16 GB
Qwen3.6 35B-A3B (needs 23 GB at 16k, ~71 tok/s with offload); Gemma 4 31B (needs 21 GB at 16k, ~9 tok/s with offload); Qwen3-Coder 30B-A3B (needs 20 GB at 16k, ~79 tok/s with offload); Qwen3.6 27B (needs 18 GB at 16k, ~13 tok/s with offload); Gemma 4 26B-A4B (needs 17 GB at 16k, ~98 tok/s with offload); Mistral Small 3.2 24B (needs 17 GB at 16k, ~19 tok/s with offload).
GPUs with 16 GB
GeForce RTX 4060 Ti 16GB, GeForce RTX 5070 Ti, GeForce RTX 5080. See their pages: 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) · RTX 5070 Ti 16GB · RTX 5080 16GB.
Hardware pages near 16 GB
RTX 3060 12GB (32GB RAM) · 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) · RTX 5070 12GB · RTX 4070 12GB · RTX 5070 Ti 16GB · RTX 5080 16GB · Mac mini / iMac M4 16GB
FAQ
What is the best local LLM for 16GB VRAM?
By size, gpt-oss 20B Q8_0 is the largest model in our dataset that fits fully at 16k context (12.5 GB, ~189 tok/s (scaled from measured)).
How many models fit in 16GB VRAM?
5 of 13 models fit fully at 16k context with an f16 KV cache; 6 more run only with CPU offload.
How much context can I use with 16GB?
It depends on the model: for gpt-oss 20B the KV cache allows up to 64k tokens at f16 and 128k with a q4_0 cache.
Neighbouring tiers: 12 GB · 24 GB · all tiers. Numbers come from the same memory-fit math as the advisor; sources and methods.