Which local LLM should you run on your hardware?
Pick your GPU, Mac or RAM. Get models that fit (weights + KV cache), a speed figure labelled measured, scaled or estimated, and a copy-paste command for llama.cpp or Ollama.
How these numbers are made
Fit
GGUF file size (read from Hugging Face) + KV cache (computed from each model's config; sliding-window and linear-attention layers handled) + ~1 GB overhead per GPU.
Speed
Memory bandwidth × efficiency ÷ bytes read per token. Efficiency is calibrated against real runs. Real results vary with backend, drivers and context.
Long context
Each model is capped at its verified maximum context. Beyond it, only models whose card documents YaRN are offered, flagged extended. Speed at depth and time-to-first-token are always estimates (penalty fitted to measured runs up to 128k; beyond that it is extrapolated). Long context needs flash attention and lots of memory.
Badges
measured median of real runs.
scaled from measured a same-size or same-model-other-quant run, scaled by bytes read.
estimated bandwidth formula, not a benchmark.
Ranking
Fits and is fast enough first, then bigger. A heuristic, not a quality benchmark. Assumptions are listed in hardware.json.
Data changelog
Hardware pages: all configs · Submit your benchmark · Data sources & attribution