RAG Context Budget Calculator

In retrieval-augmented generation (RAG), the chunks you retrieve have to share the context window with everything else. Enter the window, what the system prompt, history and tools use, the reply you want to reserve and your chunk size, and see the maximum number of chunks that fit and how much room remains. Overlap does not enter the calculation: it affects how you index, not what a retrieved chunk costs in the prompt.

Processed locally in your browser. Your data is not uploaded to our servers.

Check your modelโ€™s documentation for its context window. Input and output usually share it.

Instructions and role description sent with every request.

Earlier messages kept in the request.

Function schemas and results returned to the model.

Room to keep free for the answer, often your maximum output setting.

The size of each retrieved chunk.

Separators, source labels or citation ids added around each chunk. An assumption: set it to match your prompt.

Covers message formatting and tokenizer differences. An assumption.

Often called top-k. Checks whether that many fit.

Result

Enter a context window and your token amounts to see how many chunks fit.

Examples

Worked examples (5% margin, 20 extra tokens per chunk):

ScenarioWindowReservedChunk sizeMaximum chunks
Chatbot with history32,7688,00051243
Support answers128,0009,000256407
Long-context research1,000,00024,0001,024886

Formula & Methodology

The budget is exact arithmetic on your numbers:

available = window โˆ’ (system + history + tools + reply) โˆ’ margintokens per chunk = chunk size + extra tokens per chunkmaximum chunks = floor(available รท tokens per chunk)headroom = available โˆ’ maximum chunks ร— tokens per chunk

Good to know

  • Chunk overlap is not an input here. It changes how the corpus is indexed, not what a retrieved chunk costs in the prompt. But if neighbouring chunks that overlap are both retrieved, the shared text is sent twice: merge or deduplicate them.
  • Retrieving the maximum is rarely best. More chunks add noise and cost; many systems retrieve a handful and re-rank.
  • Input and output usually share the context window, so the reply is reserved first. Some models also set a separate output limit.
  • The extra tokens per chunk and the safety margin are assumptions. Measure a real prompt with the token counter and adjust.
  • Context windows differ by model and change over time: enter your modelโ€™s window yourself.

How to Use

  1. Enter the context window of your model, or pick a common size.
  2. Fill in the tokens used by the system prompt, history and tools, and the reply you want to reserve.
  3. Enter your chunk size, the extra tokens per chunk and a safety margin.
  4. Optionally enter how many chunks you want, then read the maximum, the headroom and any advice.

Frequently Asked Questions

How many chunks can I retrieve for my context window?

Subtract everything else from the window (system prompt, history, tools, reply and a margin), then divide what remains by the tokens per chunk and round down. This calculator does that and shows the headroom left.

Does chunk overlap reduce how many chunks fit?

Not directly. Overlap is decided when you index the documents. At retrieval time each chunk costs its own size. If two overlapping neighbours are both retrieved, the shared text is sent twice, so merge or deduplicate them.

Should I fill the whole window with chunks?

Usually not. Extra chunks add noise, latency and cost, and long contexts can dilute the relevant passages. A modest number of high-quality chunks, often re-ranked, tends to work better than the maximum.

What are the extra tokens per chunk?

The separators, source names or citation ids you put around each chunk in the prompt. They add up across many chunks, so set the value to match your prompt format.

Why keep a safety margin?

Your token counts are estimates and the real request adds message formatting. A margin of a few percent prevents an overflow caused by a count that is slightly off.

Last updated: