Can NVIDIA DGX Spark 128GB run gpt-oss 120B?
Measured results for gpt-oss 120B (116.83B MoE, 5.1B active) on NVIDIA DGX Spark 128GB, plus what fits and the commands to run it.
Short answer
Yes. gpt-oss 120B Q8_0 fits fully on NVIDIA DGX Spark 128GB at 16k context, using 64.0 of 108.3 GB usable, and generates about 56.8 tokens/s. For Q8_0, our bandwidth-only estimate was ~54 tok/s against a measured median of ~57 (4% too pessimistic).
Measured generation speed by quantization
| Quant | Generation tok/s | Range | Prompt tok/s | Runs | Origin |
|---|---|---|---|---|---|
| Q4 class (median) | 56.8 | 45.3–62.4 | 1993 | 6 | public runs |
Generation = tg128 (128 generated tokens, short context). Public rows are medians of community llama.cpp results (sources); our own runs use llama-bench on a rented GPU.
Memory and context limits
Q8_0 weights are 63.4 GB; the 16k f16 KV cache adds 0.61 GB. Largest context that fits in 108.3 GB usable:
| KV cache type | Max context |
|---|---|
| f16 | 128k |
| q8_0 | 128k |
| q4_0 | 128k |
Native limit 128k. Full KV cache table for gpt-oss 120B.
Copy-paste commands
The advisor's pick for this hardware: Q8_0, 16k context.
llama-server -hf unsloth/gpt-oss-120b-GGUF:Q8_0 -c 16384 -fa on --jinja -ngl 99
OLLAMA_CONTEXT_LENGTH=16384 ollama run hf.co/unsloth/gpt-oss-120b-GGUF:Q8_0
FAQ
How fast is gpt-oss 120B on NVIDIA DGX Spark 128GB?
About 56.8 tokens/s for generation (Q4 class, median of 6 public runs). Speed drops as the context fills; see the depth table on this hardware's other pages.
How much memory does gpt-oss 120B need on NVIDIA DGX Spark 128GB?
The advisor's pick, Q8_0, is 63.4 GB of weights plus 0.61 GB of KV cache at 16k context (64.0 GB), against 108.3 GB usable.
What is the maximum context for gpt-oss 120B on NVIDIA DGX Spark 128GB?
With an f16 KV cache about 128k tokens, 128k with q8_0 and 128k with q4_0, bounded by the model's own limit.
Related
All models on NVIDIA DGX Spark 128GB · gpt-oss 120B on all hardware · VRAM tier · all measured pages · leaderboard
Compare: AMD Ryzen AI Max+ 395 (Strix Halo) 128GB vs NVIDIA DGX Spark 128GB