Can RTX PRO 6000 96GB run GLM-4.5-Air?
Measured results for GLM-4.5-Air (110.47B MoE, 12B active) on RTX PRO 6000 96GB, plus what fits and the commands to run it.
Short answer
Yes. GLM-4.5-Air Q4_K_M fits fully on RTX PRO 6000 96GB at 16k context, using 76.1 of 95.0 GB usable, and generates about 138.3 tokens/s in our own run. As the context fills it slows down: 104.5 tok/s with 16k tokens already in context (76% of the empty-context speed), measured. For Q5_K_M, our bandwidth-only estimate was ~109 tok/s against a measured median of ~121 (10% too pessimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 138.3 | 138.3–138.3 | 3635 | 1 | includes our runs |
| Q4_K_M (our run, depth 0) | 138.3 | 3 repeats | 3635 | 3 | own measurement |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench b11658 on a rented GPU.
Speed vs context depth measured
Our own llama-bench runs on RTX PRO 6000 Blackwell Workstation (flash attention on, 3 repeats): generation and prompt speed with the given number of tokens already in the context.
| Quant | Context depth | Generation tok/s | vs depth 0 | Prompt tok/s |
|---|---|---|---|---|
| Q4_K_M | 0 | 138.3 | 100% | 3635 |
| Q4_K_M | 4k | 117.9 | 85% | 3305 |
| Q4_K_M | 16k | 104.5 | 76% | 2265 |
Alternatives
Less memory that still fits GLM-4.5-Air
Smaller memory usually means a cheaper card or Mac (we do not list prices).
- Mac Studio M5 Max 128GB: Q5_K_M, 89 GB usable, ~33 tok/s estimated
- MacBook Pro 14/16 M5 Max 128GB: Q5_K_M, 89 GB usable, ~33 tok/s estimated
- MacBook Pro 14/16 (2024) M4 Max 128GB: Q5_K_M, 89 GB usable, ~22 tok/s estimated
Other models on RTX PRO 6000 96GB, similar size
- gpt-oss 120B: Q8_0, fits fully, ~261 tok/s (measured)
- Nemotron 3.5 Super VL 120B-A12B (hardware page): Q4_K_M, fits fully, ~162 tok/s (scaled from measured)
- Llama 3.3 70B (hardware page): Q8_0, fits fully, ~15 tok/s (estimated)
- Qwen3.6 35B-A3B (hardware page): Q8_0, fits fully, ~247 tok/s (measured)
Memory and context limits
Q5_K_M weights are 83.5 GB; the 16k f16 KV cache adds 3.09 GB. Largest context that fits in 95.0 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 32k |
| q8_0 | 64k |
| q4_0 | 128k |
Native limit 128k. Full KV cache table for GLM-4.5-Air.
Copy-paste commands
The advisor's pick for this hardware: Q5_K_M, 16k context.
llama-server -hf unsloth/GLM-4.5-Air-GGUF:Q5_K_M -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama serve ollama run hf.co/unsloth/GLM-4.5-Air-GGUF:Q5_K_M
FAQ
How fast is GLM-4.5-Air on RTX PRO 6000 96GB?
About 138.3 tokens/s for generation (Q4_K_M on llama.cpp CUDA, 128-token generation), 3635 tokens/s for prompt processing. Speed drops as the context fills; see the depth table.
How much memory does GLM-4.5-Air need on RTX PRO 6000 96GB?
The advisor's pick, Q5_K_M, is 83.5 GB of weights plus 3.09 GB of KV cache at 16k context (86.6 GB), against 95.0 GB usable. The measured Q4_K_M run needs 76.1 GB.
Is the GLM-4.5-Air speed on RTX PRO 6000 96GB measured or estimated?
Measured: our own llama-bench run on a rented RTX PRO 6000 Blackwell Workstation, including runs with 4k to 16k tokens already in context.
What can I use instead of RTX PRO 6000 96GB for GLM-4.5-Air?
With less memory, Mac Studio M5 Max 128GB (~33 tok/s), MacBook Pro 14/16 M5 Max 128GB (~33 tok/s), MacBook Pro 14/16 (2024) M4 Max 128GB (~22 tok/s) also fit it fully.
What is the maximum context for GLM-4.5-Air on RTX PRO 6000 96GB?
With an f16 KV cache about 32k tokens, 64k with q8_0 and 128k with q4_0, bounded by the model's own limit.
Related
All models on RTX PRO 6000 96GB · GLM-4.5-Air on all hardware · VRAM tier · all measured pages · leaderboard