Can 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) run gpt-oss 20B?
Measured results for gpt-oss 20B (20.91B MoE, 3.6B active) on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM), plus what fits and the commands to run it.
Short answer
Yes. gpt-oss 20B Q8_0 fits fully on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) at 16k context, using 12.5 of 15.0 GB usable, and generates about 94.4 tokens/s in our own run. For Q8_0, our bandwidth-only estimate was ~76 tok/s against a measured median of ~94 (20% too pessimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 103.7 | 103.7–103.7 | 4049 | 1 | includes our runs |
| Q8 class (median) | 94.4 | 94.4–94.4 | 4110 | 1 | includes our runs |
| Q4_K_M (our run, depth 0) | 103.7 | 3 repeats | 4049 | 3 | own measurement |
| Q8_0 (our run, depth 0) | 94.4 | 3 repeats | 4110 | 3 | own measurement |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench b11658 on a rented GPU.
Speed vs context depth measured
Our own llama-bench runs on GeForce RTX 4060 Ti 16GB (flash attention on, 3 repeats): generation and prompt speed with the given number of tokens already in the context.
| Quant | Context depth | Generation tok/s | vs depth 0 | Prompt tok/s |
|---|---|---|---|---|
| Q4_K_M | 0 | 103.7 | 100% | 4049 |
| Q4_K_M | 4k | 98.3 | 95% | 3849 |
| Q4_K_M | 16k | 89.4 | 86% | 3189 |
| Q8_0 | 0 | 94.4 | 100% | 4110 |
| Q8_0 | 4k | 89.8 | 95% | 3861 |
| Q8_0 | 16k | 82.3 | 87% | 3174 |
Memory and context limits
Q8_0 weights are 12.1 GB; the 16k f16 KV cache adds 0.41 GB. Largest context that fits in 15.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 64k |
| q8_0 | 128k |
| q4_0 | 128k |
Native limit 128k. Full KV cache table for gpt-oss 20B.
Copy-paste commands
The advisor's pick for this hardware: Q8_0, 16k context.
llama-server -hf unsloth/gpt-oss-20b-GGUF:Q8_0 -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/gpt-oss-20b-GGUF:Q8_0
FAQ
How fast is gpt-oss 20B on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM)?
About 94.4 tokens/s for generation (Q8_0 on llama.cpp CUDA, 128-token generation), 4110 tokens/s for prompt processing. Speed drops as the context fills; see the depth table.
How much memory does gpt-oss 20B need on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM)?
The advisor's pick, Q8_0, is 12.1 GB of weights plus 0.41 GB of KV cache at 16k context (12.5 GB), against 15.0 GB usable.
What is the maximum context for gpt-oss 20B on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM)?
With an f16 KV cache about 64k tokens, 128k with q8_0 and 128k with q4_0, bounded by the model's own limit.
Related
All models on 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM) · gpt-oss 20B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: RTX 5070 Ti 16GB vs 16GB VRAM class (RTX 4060 Ti 16GB, 32GB RAM)