Advisor / Hardware / 8GB VRAM class (RTX 4060 8GB, 32GB RAM)

Best local LLMs for 8GB VRAM class (RTX 4060 8GB, 32GB RAM)

GeForce RTX 4060. Memory-fit math with KV cache, speeds labelled by how they were obtained, and commands you can paste.

7.0 GBusable for model + KV cache

272 GB/smemory bandwidth

24 GBRAM available for offload

2measured data points on file

Sponsor slot. No sponsor yet and no link.

Best for Coding · 16k context

Model	Quant	Fit	Memory	Speed (generation)
Qwen3.5 9B	Q4_K_M	fits fully	6.3 GB	~33 tok/s scaled from measured src
Qwen3-Coder 30B-A3B	Q4_K_M	MoE offload	20.2 GB	~34 tok/s estimated
Qwen3.6 35B-A3B	UD-Q4_K_M	MoE offload	22.5 GB	~36 tok/s estimated
Gemma 4 26B-A4B	UD-Q4_K_M	MoE offload	17.5 GB	~29 tok/s estimated
gpt-oss 20B	Q4_K_M	MoE offload	12.0 GB	~44 tok/s estimated

llama.cpp · top pick

llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 -fa on --jinja -ngl 99

ollama · top pick

OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M

Best for Chat / general · 16k context

Model	Quant	Fit	Memory	Speed (generation)
Qwen3.5 9B	Q4_K_M	fits fully	6.3 GB	~33 tok/s scaled from measured src
Qwen3.5 4B	Q8_0	fits fully	5.1 GB	~42 tok/s scaled from measured src
Gemma 4 12B	Q4_K_M	partial offload	7.7 GB	~21 tok/s estimated
Qwen3.6 35B-A3B	UD-Q4_K_M	MoE offload	22.5 GB	~36 tok/s estimated
Qwen3-Coder 30B-A3B	Q4_K_M	MoE offload	20.2 GB	~34 tok/s estimated

llama.cpp · top pick

llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 -fa on --jinja -ngl 99

ollama · top pick

OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M

Best for Vision · 16k context

Model	Quant	Fit	Memory	Speed (generation)
Qwen3.5 9B	Q4_K_M	fits fully	6.3 GB	~33 tok/s scaled from measured src
Qwen3.5 4B	Q8_0	fits fully	5.1 GB	~42 tok/s scaled from measured src
Gemma 4 12B	Q4_K_M	partial offload	7.7 GB	~21 tok/s estimated
Qwen3.6 35B-A3B	UD-Q4_K_M	MoE offload	22.5 GB	~36 tok/s estimated
Gemma 4 26B-A4B	UD-Q4_K_M	MoE offload	17.5 GB	~29 tok/s estimated

llama.cpp · top pick

llama-server -hf unsloth/Qwen3.5-9B-GGUF:Q4_K_M -c 16384 -fa on --jinja -ngl 99

ollama · top pick

OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/Qwen3.5-9B-GGUF:Q4_K_M

All 11 models that fit, fastest first · 16k context

Click a column header to sort. 2 of 11 speeds here are measured or scaled from measured; the rest are bandwidth estimates. Filter by use case, speed source, fit, license and more in the interactive advisor →

Model	Params	Quant	Fit	Memory	Tok/s	Source
gpt-oss 20B	20.91B MoE	Q4_K_M	MoE offload	12.0 GB	~44	estimated
Qwen3.5 4B	4.66B	Q8_0	fits fully	5.1 GB	~42	scaled from measured
Qwen3.6 35B-A3B	35.95B MoE	UD-Q4_K_M	MoE offload	22.5 GB	~36	estimated
Qwen3-Coder 30B-A3B	30.53B MoE	Q4_K_M	MoE offload	20.2 GB	~34	estimated
Qwen3.5 9B	9.65B	Q4_K_M	fits fully	6.3 GB	~33	scaled from measured
Qwen3-VL 8B	8.77B	Q4_K_M	partial offload	7.4 GB	~31	estimated
Gemma 4 26B-A4B	25.81B MoE	UD-Q4_K_M	MoE offload	17.5 GB	~29	estimated
Gemma 4 12B	11.96B	Q4_K_M	partial offload	7.7 GB	~21	estimated
Mistral Small 3.2 24B	24.01B	Q4_K_M	partial offload	17.0 GB	~5	estimated
Qwen3.6 27B	27.78B	Q4_K_M	partial offload	18.1 GB	~4	estimated
Gemma 4 31B	31.27B	Q4_K_M	partial offload	20.5 GB	~4	estimated

Long context on this hardware · up to 1M tokens

Largest context whose KV cache still fits next to the weights (same placement as the 16k result, KV types f16 / q8_0 / q4_0). Models are capped at their verified maximum; extended means beyond the native maximum using YaRN, only listed where the model card documents it. Long context needs flash attention (-fa on, already in the commands) and a lot of memory; a quantized KV cache saves memory but can cost accuracy. Speed at depth is an estimate (penalty fitted to measured depth runs up to 128k).

Try it in the advisor: 128k · 256k · 512k · 1M

Model	Native max	Max ctx f16	Max ctx q8_0	Max ctx q4_0	Est. tok/s at max f16 ctx
Qwen3.6 35B-A3B UD-Q4_K_M	256k	256k	512k extended	1M extended	~12 estimated
Gemma 4 26B-A4B UD-Q4_K_M	256k	256k	256k	256k	~11 estimated
Gemma 4 12B Q4_K_M	256k	256k	256k	256k	~14 estimated
gpt-oss 20B Q4_K_M	128k	128k	128k	128k	~20 estimated
Qwen3-VL 8B Q4_K_M	256k	128k	256k	512k extended	~8 estimated
Qwen3.6 27B Q4_K_M	256k	128k	256k	512k extended	~3 estimated
Gemma 4 31B Q4_K_M	256k	128k	256k	256k	~3 estimated
Qwen3.5 4B Q8_0	256k	64k	128k	256k	~31 estimated
Qwen3-Coder 30B-A3B Q4_K_M	256k	64k	128k	256k	~10 estimated
Mistral Small 3.2 24B Q4_K_M	128k	64k	128k	128k	~3 estimated
Qwen3.5 9B Q4_K_M	256k	32k	64k	128k	~29 estimated

Affiliate box: “cheapest upgrade” for 8GB VRAM class (RTX 4060 8GB, 32GB RAM). No affiliate links yet.

Open the interactive advisor with this hardware →

Speeds marked measured are medians of real runs (short-context generation) from these sources; scaled from measured means a measurement of a same-size or same-model-different-quant run, scaled by bytes read; estimated comes from memory bandwidth (spec source: https://www.techpowerup.com/gpu-specs/geforce-rtx-4060.c4107) × an efficiency factor of 0.68 calibrated to measured runs on this hardware class. 2 measured data points on file for this hardware · submit your benchmark. System RAM here is an example (32 GB, ddr5-5600). Model data last verified 2026-10-14. Ranking = fits and fast enough first, then larger; not a quality benchmark.

Page generated 2026-10-16 by tools/generate_pages.js