State of local LLM hardware, October 2026

Key findings

• Memory decides what you can run; bandwidth decides how fast. At 16k context, 8 GB fits 2 of the 13 models; every model fits from 96 GB up (chart).
• Mixture-of-experts models are much faster than dense models of similar size. On the same GPU at Q4_K_M, the MoE model generated 3.8x as many tokens per second as its dense counterpart (range 3.6x to 3.9x over 4 comparisons) (chart).
• Long context costs speed. At 16k tokens of context, generation kept a median of 91% of its depth-0 speed for dense models and 89% for MoE models across 63 full-GPU runs (chart).
• Q8_0 costs speed relative to Q4_K_M. Median Q8_0 speed was 66% of Q4_K_M for dense models (21 comparisons) and 93% for MoE models (3).
• Same model, wide spread. gpt-oss 20B at Q4 ranges from 86.2 tok/s on NVIDIA DGX Spark to 298.3 tok/s on GeForce RTX 5090; per 100 GB/s of memory bandwidth the spread runs from 14.1 (Apple M3 Ultra) to 36 (GeForce RTX 4060 Ti 16GB) (chart).

Speed by hardware, on one model

The same model on every machine that has it measured removes the model as a variable. We have no price data in the dataset and do not state prices, so instead of price tiers this compares speed per memory bandwidth; memory sizes are on each hardware page.

Bars mix our own runs with public community medians (different drivers, builds and backends), so small gaps are within noise. gpt-oss 20B is the one model with the widest hardware coverage in the dataset.

What fits at each memory tier

Same fit math as the VRAM tier pages; the largest model that fits fully at 512 GB is gpt-oss 120B. Offload to system RAM (not counted here) extends this at a large speed cost for dense models.

The cost of long context

We ran each model at depth 0, 4096 and 16384 tokens with llama-bench (flash attention on). Median retention across 64 full-GPU runs at 4k was 96% (min 84%, max 99%) and at 16k 91% (min 65%, max 95%).

Beyond our own runs, the site's depth model (fitted on 156 measured points, up to 128k tokens of depth, median absolute error 3.4%) estimates the speed at long context, up to 1M tokens where a model supports it; those figures are labelled estimated on each page.

MoE versus dense

MoE models read only their active experts per token, so generation is limited by active parameters rather than total size, while the whole model still has to fit in memory.

Which hardware we have measured

Public rows come from community llama.cpp results (sources); entries without our own runs are medians of other people's benchmarks. We have not yet measured Macs or AMD GPUs ourselves, so those numbers are public data. Send yours via the submit form.

Methods

• Our runs: llama-bench from llama.cpp (build b11658), CUDA 12.8, flash attention on, all layers on GPU, ubatch 512, 3 repeats (median reported), pp512 and tg128, run on rented GPUs (vast.ai) on 2026-10-16. Quants are GGUF Q4_K_M and Q8_0 from public repositories. Per-run details are in the dataset. 62 of the 253 rows used MoE expert offload to system RAM (different --n-cpu-moe settings per depth) and are left out of the depth, MoE and quantization comparisons, because offload settings differ between rows.
• Public rows: medians of community llama.cpp benchmarks, aggregated per hardware, model and quantization class; see sources and attribution.
• Fit math: weights plus KV cache at 16k context (f16) plus about 1 GB overhead must fit in GPU memory; model sizes come from the published GGUF files.
• Limits: single-user generation speed, short prompts; not a quality benchmark; GPU clocks, drivers and build versions differ between sources; our runs cover 6 GPUs only.

Reuse and attribution

Our own measurements and curated data are licensed CC BY 4.0. Please attribute as Data: Local LLM Advisor (localllmadvisor.com) and link to this post or the leaderboard. Download the CSV or the Hugging Face dataset. Third-party rows keep their upstream licenses. Charts on this page are inline SVG you may copy with the same attribution.