Recommended models for RTX 4090 24GB
11 of 13 models fit fully in GPU memory at 16k context. Fastest three:
Qwen3-Coder 30B-A3B
Q4_K_M
~266 tok/s
measured
gpt-oss 20B
Q8_0
~258 tok/s
measured
Qwen3.6 35B-A3B
UD-Q4_K_M
~190 tok/s
measured
All models and commands
· Data:
Local LLM Advisor (localllmadvisor.com)
, CC BY 4.0