LLM Model Size & Quantization Calculator

Enter a model’s parameter count to see how large its weights are in every common format, from FP32 down to Q2, in GB and GiB, and how much each quantization saves compared with FP16. Add your own bits per weight for any other format.

Processed locally in your browser. Your data is not uploaded to our servers.

For example 8 for an 8B model. Mixture-of-experts models must load all of their parameters.

Adds a row for a format that is not listed, for example 4.5.

Result

Estimate

Enter the number of parameters to see the sizes.

Model size by format
FormatBits per weightSize (GB)Size (GiB)Compared with FP16
FP32323229.8+100%
FP16161614.90%
BF16161614.90%
Q8_0≈ 8.50088.57.92−46.9%
INT8887.45−50%
Q6_K≈ 6.56336.566.11−59%
Q5_K_M≈ 5.70365.75.31−64.4%
Q4_K_M≈ 4.89444.894.56−69.4%
Q3_K_M≈ 3.99643.72−75%
Q2_K≈ 3.15933.162.94−80.3%

Q8_0 to Q2_K are GGUF formats: their bits per weight are the whole-model averages published in the llama.cpp quantize README (measured on Llama-3.1-8B, including the layers that keep higher precision). Effective bits per weight vary by model and quantization configuration.

Examples

Worked examples (sizes in GiB):

ParametersFP16Q8_0Q4_K_M
7B13.046.933.99
8B14.97.924.56
70B130.3969.2739.88

Formula & Methodology

The size of the weights is:

size in bytes = parameters × bits per weight ÷ 8GB = bytes ÷ 1,000,000,000GiB = bytes ÷ 1,073,741,824change vs FP16 = (bits per weight ÷ 16 − 1) × 100

About the formats

  • FP32 stores each weight in 32 bits; FP16 and BF16 use 16 bits (BF16 keeps FP32’s numeric range with less precision).
  • INT8 stores 8-bit integers; Q8_0 is a GGUF block format that also stores a scale per block, so it is slightly above 8 bits.
  • Q6_K, Q5_K_M, Q4_K_M, Q3_K_M and Q2_K are GGUF “K” quantizations: fewer bits per weight, lower quality as the number falls.

What this size does not include

  • Running a model also needs memory for the KV cache (which grows with context length) and for the runtime: use the LLM VRAM Calculator for the total.
  • Real files add metadata and other storage overhead, and effective bits per weight vary by model and quantization configuration, so treat the result as an estimate. A “within a few percent” margin is an unvalidated ToolQuarter estimate; actual variation can be larger.
  • Mixture-of-experts models store every expert: use the total parameter count.

Method and references

These primary sources support the methods this tool relies on. Having read them does not verify every result the calculator produces, and figures listed as assumptions are ToolQuarter’s own.

The size is plain arithmetic on the parameter count. The bits per weight of GGUF formats come from llama.cpp’s published measurements.

  • llama.cpp: quantize README: Source of the bits per weight for Q8_0, Q6_K, Q5_K_M, Q4_K_M, Q3_K_M and Q2_K (for example Q4_K_M 4.8944). They are whole-file averages measured on Llama-3.1-8B, which is why they differ from the number in the name. ()
  • PyTorch: Tensor Attributes: Defines the 32-bit float, the 16-bit float (1 sign, 5 exponent and 10 mantissa bits), bfloat16 (1, 8 and 7) and the 8-bit integer behind the FP32, FP16, BF16 and INT8 rows. ()

ToolQuarter assumptions (no primary source):

  • Size = parameters × bits per weight ÷ 8 estimates weight storage from the entered parameter count. Embedding weights are included when they are part of that count. FP32, FP16, BF16 and INT8 use nominal bit widths; the estimate does not separately account for additional storage overhead such as quantization scales or file metadata.
  • Effective bits per weight vary by model and quantization configuration, and llama.cpp measured one model. Any “within a few percent” margin is an unvalidated ToolQuarter estimate, and actual variation can be larger. Using the total parameter count for mixture-of-experts models is ToolQuarter’s guidance, with no source read for it.

How to Use

  1. Enter the number of parameters in billions (for example 8 for an 8B model).
  2. Optionally add a custom bits-per-weight value for another format.
  3. Read the size of the weights in each format, in GB and GiB, and the saving against FP16.
  4. Use the LLM VRAM Calculator to add the KV cache and runtime overhead.

Frequently Asked Questions

Why isn’t Q4_K_M exactly 4 bits per weight?

GGUF “K” quantizations store small blocks with scale values and keep some tensors at higher precision, so the whole-model average is higher than the name suggests: about 4.89 bits for Q4_K_M in the published llama.cpp figures.

Is the file size the same as the memory I need?

No. Running a model also needs memory for the KV cache, which grows with context length, and for runtime overhead. Use the LLM VRAM Calculator for the total.

What is the difference between FP16 and BF16?

Both use 16 bits, so their size is the same. BF16 keeps the wide numeric range of FP32 with less precision; FP16 has more precision but a narrower range.

Should I compare GB or GiB with my disk or GPU?

Disk and download sizes are usually quoted in GB, while GPU and RAM capacity is normally counted in GiB. This tool shows both so you can compare like with like.

How accurate is the size of a real model file?

It is an approximation: real files add metadata and other storage overhead, and effective bits per weight vary by model and quantization configuration. A “within a few percent” margin is an unvalidated ToolQuarter estimate; actual variation can be larger.

Last updated: