LLM VRAM Calculator

Estimate how much GPU or system memory a large language model needs to run locally. Pick a model preset or enter its size, choose a quantization and a context length, and see the weights, KV cache and overhead, with every assumption visible and editable.

Processed locally in your browser. Your data is not uploaded to our servers.

Preset figures come from official model documentation, last verified October 3, 2026. You can edit every number.

Source: Llama 3 paper, Table 3

For example 8 for an 8B model. Mixture-of-experts models must load all of their parameters.

Prompt plus generated tokens kept in memory.

1 for a single chat. Each extra sequence needs its own KV cache.

Architecture (needed for the KV cache)

Find these numbers in the model’s config.json or model card. Nothing is guessed: leave all three empty to leave the KV cache out.

Usually the hidden size divided by the number of attention heads.

Only used to show what grouped-query attention saves.

Assumption for the runtime context and compute buffers. About 0.5–1 GiB is typical.

Assumption: extra margin on top of the estimate. 10–20% is a sensible range.

Result

Estimate

Enter the number of parameters to see the estimate.

Weights and KV cache follow exact formulas applied to your numbers. The effective bits of a quantization format, the runtime overhead and the headroom are assumptions: real memory use depends on the runtime and its version.

Examples

Worked examples (runtime overhead 1 GiB, KV cache in FP16):

ModelQuantization · contextWeightsKV cacheTotal
Llama 3.1 8BQ4_K_M · 8,1924.56 GiB1 GiB6.56 GiB
Llama 3.1 70BQ4_K_M · 8,19239.88 GiB2.5 GiB43.38 GiB
Qwen2.5 7B InstructFP16 · 32,76814.17 GiB1.75 GiB16.92 GiB

Reference tables: LLM memory by model and format

Weight storage is estimated from parameters × bits per weight ÷ 8; KV cache and runtime overhead are additional.

Both tables assume: context 8,192 tokens; sequences 1; KV cache precision FP16; runtime overhead 1 GiB. They leave out headroom: the calculator’s default headroom is 10%, so add it if you want a recommended figure.

All values are in GiB (one GiB is 1,073,741,824 bytes), shown to two decimal places. The models listed are the calculator’s presets.

Weight storage only
ModelFP16Q8_0Q4_K_M
Llama 3.1 8B14.907.924.56
Llama 3.1 70B130.3969.2739.88
Llama 3.1 405B754.37400.80230.76
Qwen2.5 7B Instruct14.177.534.34
Qwen2.5 14B Instruct27.3814.558.38
Qwen2.5 32B Instruct60.5432.1618.52
Qwen3 8B15.278.114.67

Excludes KV cache, runtime overhead, activations and headroom.

Estimated total memory before runtime-specific placement
ModelKV cacheTotal FP16Total Q8_0Total Q4_K_M
Llama 3.1 8B1.0016.909.926.56
Llama 3.1 70B2.50133.8972.7743.38
Llama 3.1 405B3.94759.31405.73235.70
Qwen2.5 7B Instruct0.4415.618.975.77
Qwen2.5 14B Instruct1.5029.8817.0510.88
Qwen2.5 32B Instruct2.0063.5435.1621.52
Qwen3 8B1.1317.4010.246.80

Includes the KV cache and runtime overhead stated above. Excludes headroom, activation peaks and runtime allocations that are not modelled.

What these tables do not tell you

  • These are formula-based estimates, not measured benchmarks. A configuration that fits on paper may still not run in a particular runtime.
  • The Q8_0 and Q4_K_M sizes use the effective bits per weight published for llama.cpp’s GGUF formats, measured on one model (Llama-3.1-8B); actual files can vary.
  • Totals are estimates before runtime-specific placement. They do not mean that all memory behaves as one pool, or that a listed model fits on one GPU; offloading and multi-GPU placement are not modelled.
  • The tables list dense models. Mixture-of-experts models need the total parameter count and separate treatment of where the experts are stored.
  • Models with sliding-window or latent attention can use much less KV cache than the standard formula used here.

Formula & Methodology

The calculator adds three parts:

weights = parameters × bits per weight ÷ 8KV cache = 2 × layers × KV heads × head dimension × tokens × bytes per value × sequencestotal = weights + KV cache + runtime overheadrecommended = total × (1 + headroom ÷ 100)

What this estimate does not cover

  • Mixture-of-experts models must load every expert, so use the total parameter count, not the “active” count.
  • Models with multi-head latent attention, sliding-window or hybrid attention can use much less KV cache than the standard formula.
  • Prompt processing can briefly need more memory than steady-state generation, and each runtime adds its own buffers.
  • GB (10^9 bytes) and GiB (2^30 bytes) differ by about 7%: GPUs and RAM sold as “24 GB” are normally 24 GiB.

Method and references

These primary sources support the methods this tool relies on. Having read them does not verify every result the calculator produces, and figures listed as assumptions are ToolQuarter’s own.

Weights and KV cache follow arithmetic on the numbers you enter. The runtime overhead and headroom are ToolQuarter assumptions, and real memory use depends on the runtime.

  • llama.cpp: quantize README: Source of the bits per weight of the GGUF quantization formats (for example Q4_K_M 4.8944), measured on Llama-3.1-8B. Other models and files vary. ()
  • PagedAttention paper (arXiv:2309.06180v1): Gives the KV cache per token as 2 (keys and values) × hidden size × layers × bytes per value, with a worked example of 800 KB per token for a 13B model in FP16. This tool writes the same product with KV heads and head dimension, so it also covers models that share KV heads. ()
  • Mixtral of Experts paper (arXiv:2401.04088v1): Defines total and active parameters for a mixture-of-experts model: each token has access to 47B parameters but uses 13B. The paper does not discuss memory. ()
  • llama.cpp: server README: Documents --cpu-moe and --n-cpu-moe, which keep mixture-of-experts weights in CPU memory instead of VRAM, and --gpu-layers, the maximum number of layers stored in VRAM. ()
  • NIST: Prefixes for binary multiples: Defines the gibibyte as 2^30 bytes (1,073,741,824) and the gigabyte as 10^9 bytes, which is why GB and GiB differ by about 7%. ()

ToolQuarter assumptions (no primary source):

  • The runtime overhead (1 GiB by default) and the headroom (10%) are ToolQuarter heuristics, not figures from a source. Real overhead depends on the runtime and its version, and prompt processing can briefly need more memory than steady-state generation; activation memory is not modelled.
  • Using the total parameter count for mixture-of-experts models is ToolQuarter’s inference: all experts have to be stored somewhere, but some runtimes can keep expert weights in system RAM instead of VRAM (llama.cpp’s --cpu-moe option, for example). That a card sold as “24 GB” has 24 GiB is a working assumption: check the manufacturer’s specification.

How to Use

  1. Choose a model preset, or enter the parameters in billions.
  2. Pick a quantization and the context length you plan to use.
  3. Open “Architecture” and enter layers, KV heads and head dimension (a preset fills them in) to include the KV cache.
  4. Adjust the overhead and headroom assumptions if you know your runtime, then read the total.

Frequently Asked Questions

How much VRAM do I need to run an 8B model?

It depends on quantization and context. For example, at Q4_K_M an 8B model needs about 4.6 GiB for weights; with an 8K context (about 1 GiB of KV cache) and roughly 1 GiB of runtime overhead, plan for about 6.6 GiB, or about 7.2 GiB with 10% headroom. FP16 weights alone are about 14.9 GiB.

Why does a longer context need more memory?

The KV cache stores keys and values for every token in the context in every layer, so it grows linearly with the context length and with the number of concurrent sequences. Long contexts can need more memory than the weights themselves.

Is this result exact?

The weight and KV-cache parts are exact arithmetic on the numbers you enter. The effective bits per weight of a quantization format, the runtime overhead and the headroom are estimates and assumptions, so treat the total as a planning figure and keep some margin.

What is the difference between GB and GiB?

1 GB is 1,000,000,000 bytes and 1 GiB is 1,073,741,824 bytes. Memory in GPUs and RAM is normally counted in GiB even when it is labelled GB, so this tool shows both and compares in GiB.

Does a mixture-of-experts model need less memory?

Not for the weights: every expert must be loaded, so memory follows the total parameter count. Only the computation per token is lower.

Last updated: