RAG Context Budget Calculator
In retrieval-augmented generation (RAG), the chunks you retrieve have to share the context window with everything else. Enter the window, what the system prompt, history and tools use, the reply you want to reserve and your chunk size, and see the maximum number of chunks that fit and how much room remains. Overlap does not enter the calculation: it affects how you index, not what a retrieved chunk costs in the prompt.
Result
Enter a context window and your token amounts to see how many chunks fit.
- Reserved (prompt, history, tools, reply)
- Safety margin
- Available for chunks
- Tokens per chunk (with extras)
- Maximum chunks
- Tokens used by those chunks
- Headroom left
- Window used
Examples
Worked examples (5% margin, 20 extra tokens per chunk):
| Scenario | Window | Reserved | Chunk size | Maximum chunks |
|---|---|---|---|---|
| Chatbot with history | 32,768 | 8,000 | 512 | 43 |
| Support answers | 128,000 | 9,000 | 256 | 407 |
| Long-context research | 1,000,000 | 24,000 | 1,024 | 886 |
Formula & Methodology
The budget is exact arithmetic on your numbers:
available = window โ (system + history + tools + reply) โ margintokens per chunk = chunk size + extra tokens per chunkmaximum chunks = floor(available รท tokens per chunk)headroom = available โ maximum chunks ร tokens per chunkGood to know
- Chunk overlap is not an input here. It changes how the corpus is indexed, not what a retrieved chunk costs in the prompt. But if neighbouring chunks that overlap are both retrieved, the shared text is sent twice: merge or deduplicate them.
- Retrieving the maximum is rarely best. More chunks add noise and cost; many systems retrieve a handful and re-rank.
- Input and output usually share the context window, so the reply is reserved first. Some models also set a separate output limit.
- The extra tokens per chunk and the safety margin are assumptions. Measure a real prompt with the token counter and adjust.
- Context windows differ by model and change over time: enter your modelโs window yourself.
How to Use
- Enter the context window of your model, or pick a common size.
- Fill in the tokens used by the system prompt, history and tools, and the reply you want to reserve.
- Enter your chunk size, the extra tokens per chunk and a safety margin.
- Optionally enter how many chunks you want, then read the maximum, the headroom and any advice.
Frequently Asked Questions
How many chunks can I retrieve for my context window?
Subtract everything else from the window (system prompt, history, tools, reply and a margin), then divide what remains by the tokens per chunk and round down. This calculator does that and shows the headroom left.
Does chunk overlap reduce how many chunks fit?
Not directly. Overlap is decided when you index the documents. At retrieval time each chunk costs its own size. If two overlapping neighbours are both retrieved, the shared text is sent twice, so merge or deduplicate them.
Should I fill the whole window with chunks?
Usually not. Extra chunks add noise, latency and cost, and long contexts can dilute the relevant passages. A modest number of high-quality chunks, often re-ranked, tends to work better than the maximum.
What are the extra tokens per chunk?
The separators, source names or citation ids you put around each chunk in the prompt. They add up across many chunks, so set the value to match your prompt format.
Why keep a safety margin?
Your token counts are estimates and the real request adds message formatting. A margin of a few percent prevents an overflow caused by a count that is slightly off.
Last updated: