Can My PC Run This LLM?
Check whether a local language model fits in your memory. Enter your GPU VRAM, unified memory or system RAM, pick a model and a quantization, and see whether it fits, how much memory is left or missing, and a lower-precision format or smaller model to try.
Result
Enter the memory you have and the model size to check.
- Estimated need
- Recommended (with headroom)
- Available
- Margin
This is a memory check only. It does not predict speed (tokens per second) or guarantee that a particular runtime supports the model.
Examples
Worked examples (KV cache in FP16, runtime overhead 1 GiB, 10% headroom):
| Memory | Model · quantization · context | Result |
|---|---|---|
| 8 GiB GPU | Llama 3.1 8B · Q4_K_M · 8,192 | Fits comfortably. Spare memory beyond the recommended amount: 0.29 GiB. |
| 24 GiB GPU | Qwen2.5 32B Instruct · Q4_K_M · 8,192 | Fits, but tight. It fits with 1.98 GiB to spare, which is less than the 10% headroom: expect trouble with longer prompts or other running apps. |
| 24 GiB GPU | Llama 3.1 70B · Q4_K_M · 8,192 | Does not fit. Short by 19.88 GiB. |
| 64 GiB unified | Llama 3.1 70B · Q4_K_M · 8,192 | Fits comfortably. Spare memory beyond the recommended amount: 12.28 GiB. |
Formula & Methodology
The check compares the memory you have with the estimated need:
available = memory − reservedneed = weights + KV cache + runtime overheadfits comfortably if available ≥ need × (1 + headroom ÷ 100)fits, but tight if need ≤ available < need × (1 + headroom ÷ 100)What this check does not cover
- Speed: fitting in memory says nothing about tokens per second, which depends on memory bandwidth, compute and the runtime.
- Offloading: layers split between GPU and system RAM are not modelled; the tool asks whether everything fits in the memory you chose.
- Models with multi-head latent attention, sliding-window or hybrid attention can need much less KV cache than the standard formula.
- Runtimes differ in their buffers and in which models and formats they support.
- For typical sizes by model and format, see the LLM VRAM reference tables.
Method and references
These primary sources support the methods this tool relies on. Having read them does not verify every result the calculator produces, and figures listed as assumptions are ToolQuarter’s own.
The check uses the same weight and KV-cache arithmetic as the VRAM calculator and compares it with the memory you enter. The reserved memory and the fit thresholds are ToolQuarter heuristics, and fitting in memory says nothing about speed.
- llama.cpp: quantize README: Source of the bits per weight of the GGUF quantization formats (for example Q4_K_M 4.8944), measured on Llama-3.1-8B. Other models and files vary. ()
- PagedAttention paper (arXiv:2309.06180v1): Gives the KV cache per token as 2 (keys and values) × hidden size × layers × bytes per value, with a worked example of 800 KB per token for a 13B model in FP16. This tool writes the same product with KV heads and head dimension. ()
- llama.cpp: server README: Documents the --gpu-layers option, the maximum number of layers stored in VRAM, which is how one runtime can place only part of a model on the GPU. This tool does not model offloading. ()
- NIST: Prefixes for binary multiples: Defines the gibibyte as 2^30 bytes (1,073,741,824) and the gigabyte as 10^9 bytes, which is why GB and GiB differ by about 7%. ()
ToolQuarter assumptions (no primary source):
- The reserved memory (0.5 GiB for a GPU, 4 GiB for unified memory or CPU), the 1 GiB runtime overhead and the 10% headroom that separate “fits comfortably”, “fits, but tight” and “does not fit” are ToolQuarter heuristics, not measured limits.
- How much of a unified pool the GPU can address, and the idea that memory bandwidth usually limits CPU speed, are general statements by ToolQuarter with no source read here. The result is a memory check on paper: it does not estimate speed or confirm that a runtime supports the model.
How to Use
- Choose where the model will run and enter the memory you have in GiB.
- Pick a model preset or enter its parameters, then choose a quantization and a context length.
- Open “Architecture” (a preset fills it in) so the KV cache is included.
- Read whether it fits, then follow the suggestions if it does not.
Frequently Asked Questions
Can I run a model that is larger than my VRAM?
Some runtimes can offload part of a model to system RAM or the CPU, which works but is usually much slower. This tool answers a stricter question: does everything fit in the memory you selected?
Does fitting in memory mean it will be fast?
No. Speed depends on memory bandwidth, GPU compute and the runtime. This tool checks memory only and never promises a speed.
What can I do if it does not fit?
Use a lower-precision quantization, shorten the context, reduce the number of concurrent sequences, store the KV cache in a lower precision, choose a smaller model, or add memory (for example with several GPUs).
How is unified memory handled?
The GPU and CPU share one pool, so the whole pool is available to the model minus what the system and other apps use. Some systems also limit how much of the pool the GPU can address; raise the reserved amount to account for that.
Why is a result marked “tight” instead of “fits”?
The estimate has uncertainty: runtimes add buffers and real usage can peak above the average. “Tight” means it fits on paper but leaves less than your headroom, so small differences could make it fail.
Last updated: