Home / AI & DevOps / LLM Context Window and RAG Chunk Calculator

AI & DevOps

LLM Context Window and RAG Chunk Calculator

Calculate RAG chunk count, context-window fit, max chunks that fit, and prompt token budget.

RAGcontext windowchunk sizeLLMtokens

LLM context window and RAG chunk calculator

Estimate chunk count, retrieved context size, and whether your RAG prompt fits inside an LLM context window.

Model limits change; enter your actual model limit if needed.
English planning rule: roughly 1.3 tokens per word.
Chunks created
48
From document size, chunk size, and overlap
Context fit
Fits
6.3% of context window
Retrieved context
4800
top-k × chunk size
Max chunks that fit
155
After prompt and output reserve
Estimated document tokens
33250
Use a tokenizer for production accuracy
Remaining tokens
119900
After input and output reserve
Use this as a planning estimate. Exact token counts depend on the model tokenizer and prompt format.

What this estimate does and does not do

This calculator is a planning tool for Retrieval-Augmented Generation (RAG) prompts. It estimates how many chunks a document creates, how much retrieved context will be inserted into a prompt, and whether that prompt fits inside a model’s context window after reserving room for the answer.

It does not run your actual text splitter or model tokenizer. Real systems may split on paragraphs, markdown headings, sentence boundaries, code functions, or semantic sections rather than fixed tokens. Exact token counts also depend on the model tokenizer and prompt format. Use this calculator for design trade-offs, then verify with your actual tokenizer and RAG pipeline before production.

Formula used

The calculator uses a sliding-window chunk model:

estimated document tokens = words × tokens per word, or entered tokens directly
effective stride = chunk size - overlap
chunks = 1 + ceil((document tokens - chunk size) / effective stride), if document > chunk size
embedded token estimate = document tokens + (chunks - 1) × overlap
retrieved context = top-k retrieved chunks × chunk size
input prompt = system prompt + query + history/tools + retrieved context
total budget needed = input prompt + output reserve
fits = total budget needed ≤ context window

The formula intentionally separates indexing-time chunk count from query-time context budget. You may create thousands of chunks in a vector store, but only retrieve a small top-k set into each LLM call.

Worked example

For a 25,000-word English document, using 1.33 tokens/word, 800-token chunks, 100-token overlap, top-k of 6, a 128k context window, a 1,000-token system prompt, a 300-token query, and a 2,000-token output reserve:

document tokens ≈ 25,000 × 1.33 = 33,250
stride = 800 - 100 = 700
chunks = 1 + ceil((33,250 - 800) / 700) = 48 chunks
retrieved context = 6 × 800 = 4,800 tokens
total budget = 1,000 + 300 + 4,800 + 2,000 = 8,100 tokens
8,100 ≤ 128,000, so it fits

How to use the result

  • If the prompt does not fit, reduce retrieved chunks, reduce chunk size, reduce history/tool context, lower output reserve, or choose a larger-context model.
  • If context utilisation is very high, leave more headroom for longer responses, tool schemas, citations, or prompt growth.
  • If chunk count is much higher than expected, reduce overlap or increase chunk size.
  • If retrieval quality is weak, test smaller chunks for precision or larger chunks for more surrounding context.

Assumptions and limitations

  • The chunk count assumes fixed-size token chunks with fixed overlap.
  • Logical splitters may create smaller or larger chunks depending on document structure.
  • Word-to-token estimates are rough. A common English planning rule is about 1 token per 0.75 words, or around 1.3 tokens per word.
  • Overlap increases the number of stored vectors and the number of embedded tokens.
  • Very large context windows do not always improve answer quality; focused retrieved context is often cheaper and faster.
  • For cost planning, pair this with the LLM Token Cost Calculator.

Frequently asked questions

How do you calculate the number of RAG chunks?

If the document is larger than one chunk, the calculator uses one first chunk plus enough additional chunks to cover the remaining tokens with a stride of chunk size minus overlap.

What chunk size should I use for RAG?

There is no universal best size. Many RAG systems start around 256–1024 tokens, with smaller chunks for precise facts and larger chunks for long-form reasoning or legal/technical material.

How much chunk overlap should I use?

A common starting point is roughly 10–20% overlap. More overlap can preserve boundary context, but it increases vector count, embedding cost, and storage.

Why reserve output tokens?

The context window is shared by the input prompt and the model response. Reserving output space helps prevent the answer from being cut off or the request exceeding the limit.

Are the token counts exact?

No. Word-to-token conversion is an estimate. Production systems should count tokens with the target model tokenizer because code, non-English text, JSON, and symbols can tokenize differently.