Best local LLMs for 12GB VRAM
What you can run with 12 GB of GPU memory (11 GB usable after ~1 GB overhead), at 16k context with an f16 KV cache. Speeds use data measured on GeForce RTX 4070 12GB where available.
Models that fit 12 GB, largest first
| Model | Params | Quant | Fit | Memory | Tok/s | Speed source |
|---|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | 35.95B MoE | UD-Q4_K_M | MoE offload to CPU | 22.5 GB | 49 | estimated |
| Gemma 4 31B | 31.27B | Q4_K_M | partial offload (slow) | 20.5 GB | 6 | estimated |
| Qwen3-Coder 30B-A3B | 30.53B MoE | Q4_K_M | MoE offload to CPU | 20.2 GB | 47 | estimated |
| Qwen3.6 27B | 27.78B | Q4_K_M | partial offload (slow) | 18.1 GB | 7 | estimated |
| Gemma 4 26B-A4B | 25.81B MoE | UD-Q4_K_M | MoE offload to CPU | 17.5 GB | 46 | estimated |
| Mistral Small 3.2 24B | 24.01B | Q4_K_M | partial offload (slow) | 17.0 GB | 8 | estimated |
| gpt-oss 20B | 20.91B MoE | Q4_K_M | MoE offload to CPU | 12.0 GB | 121 | estimated |
| Gemma 4 12B | 11.96B | Q6_K | fits fully | 10.4 GB | 43 | scaled from measured |
| Qwen3.5 9B | 9.65B | Q8_0 | fits fully | 10.1 GB | 51 | measured |
| Qwen3-VL 8B | 8.77B | Q6_K | fits fully | 9.1 GB | 62 | scaled from measured |
| Qwen3.5 4B | 4.66B | Q8_0 | fits fully | 5.1 GB | 89 | measured |
Copy-paste commands for the top pick
Gemma 4 12B Q6_K, 16k context.
llama-server -hf unsloth/gemma-4-12b-it-GGUF:Q6_K -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/gemma-4-12b-it-GGUF:Q6_K
How much context fits in 12 GB?
Largest context where weights plus KV cache stay inside 11 GB, by KV cache type (q8_0 halves and q4_0 quarters the cache). Native limits and YaRN extension apply per model.
| Model | Weights | f16 KV | q8_0 KV | q4_0 KV |
|---|---|---|---|---|
| Gemma 4 12B Q6_K | 9.8 GB | 32k | 64k | 128k |
| Qwen3.5 9B Q8_0 | 9.5 GB | 32k | 64k | 128k |
| Qwen3-VL 8B Q6_K | 6.7 GB | 16k | 32k | 64k |
| Qwen3.5 4B Q8_0 | 4.5 GB | 128k | 256k | 512k |
Try the exact numbers for your setup in the interactive advisor.
What does not fit fully in 12 GB
Qwen3.6 35B-A3B (needs 23 GB at 16k, ~49 tok/s with offload); Gemma 4 31B (needs 21 GB at 16k, ~6 tok/s with offload); Qwen3-Coder 30B-A3B (needs 20 GB at 16k, ~47 tok/s with offload); Qwen3.6 27B (needs 18 GB at 16k, ~7 tok/s with offload); Gemma 4 26B-A4B (needs 17 GB at 16k, ~46 tok/s with offload); Mistral Small 3.2 24B (needs 17 GB at 16k, ~8 tok/s with offload); gpt-oss 20B (needs 12 GB at 16k, ~121 tok/s with offload).
GPUs with 12 GB
GeForce RTX 3060 12GB, GeForce RTX 4070 12GB, GeForce RTX 5070. See their pages: RTX 3060 12GB (32GB RAM) · RTX 5070 12GB · RTX 4070 12GB.
Hardware pages near 12 GB
RTX 3060 12GB (32GB RAM) · 8GB VRAM class (RTX 4060 8GB, 32GB RAM) · 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) · RTX 5070 12GB · RTX 4070 12GB · RTX 5070 Ti 16GB · RTX 5080 16GB · Mac mini / iMac M4 16GB
FAQ
What is the best local LLM for 12GB VRAM?
By size, Gemma 4 12B Q6_K is the largest model in our dataset that fits fully at 16k context (10.4 GB, ~43 tok/s (scaled from measured)).
How many models fit in 12GB VRAM?
4 of 13 models fit fully at 16k context with an f16 KV cache; 7 more run only with CPU offload.
How much context can I use with 12GB?
It depends on the model: for Gemma 4 12B the KV cache allows up to 32k tokens at f16 and 128k with a q4_0 cache.
Neighbouring tiers: 8 GB · 16 GB · all tiers. Numbers come from the same memory-fit math as the advisor; sources and methods.