RAG Chunking Strategies: Fixed vs Recursive vs Semantic (and What to Pick)

Your RAG bot keeps giving vague or half-finished answers, and the instinct is to blame the model. Usually the real culprit is upstream: the chunk it retrieved was half a sentence, or a whole page of noise that buried the one line that mattered. RAG chunking strategies are how you cut documents so the right passage is whole, findable, and small enough to rank, before the model ever sees it.

There is a whole genre of "8 chunking strategies" posts that crown semantic chunking the winner and move on. The honest picture is messier: the public benchmarks flatly disagree, because they ran on different documents with different embedders. So this is a how-to for picking a method, a size, and an overlap that work on your data, not a leaderboard to copy. Chunking only matters once you have committed to retrieval; if your corpus fits a model's window, weigh long context vs RAG before you tune any of this.

Why does chunking decide whether your RAG answers are good?#

Retrieval ranks chunks by embedding similarity, then the model only sees the chunks you hand it. Chunk too small and you lose the context that makes a passage answerable; chunk too big and the relevant sentence drowns in filler and ranks lower. Chunking strategies exist to sit you at the right point on that dial.

That dial is the whole game. A chunk is the unit your retriever scores and your model reads, so its size sets two things at once: how precisely a query can match it, and how much surrounding context comes along for the ride.

The tradeoff fits in one line. Smaller chunks raise precision and lower context per hit; bigger chunks raise context and lower precision. Picking a chunk size is just choosing where on that line you want to sit for your content and your queries.

There is a second, quieter cost to oversized chunks. An embedding model compresses a whole chunk into one vector, so a 1,500-token chunk that spans five topics produces a blurry average vector that matches no single query well. Right-sizing chunks keeps each vector about one idea, which is what lets similarity search actually discriminate. If your bot already answers badly, why an n8n RAG chatbot gives wrong answers walks the rest of the failure chain.

Fixed, recursive, and semantic chunking: how each one splits#

The three methods differ only in where they cut. Fixed slices every N tokens regardless of meaning. Recursive walks a hierarchy of natural boundaries, paragraphs to sentences, and cuts at the largest one that still fits. Semantic embeds each sentence and breaks where the topic shifts, paying one embedding call per sentence.

Recursive splits on structure for near-zero cost; semantic splits on meaning but pays an embedding call per sentence. Source: Chroma, Firecrawl.

In more detail:

• Fixed-size. Cut every N tokens with a sliding overlap. Fastest, zero model calls, predictable sizes, but it happily cuts mid-sentence and mid-table. Good for prototyping and uniform text, not production.
• Recursive. Walk a separator hierarchy and cut at the largest natural boundary that still fits under the size cap, then add overlap. This is the sensible default: readable chunks at no index-time cost.
• Semantic. Embed each sentence and start a new chunk where similarity to the running chunk drops below a threshold. It splits on meaning rather than layout, but costs one embedding call per sentence and needs threshold tuning.

In practice, recursive is the default because it gets you most of the benefit of structure-aware splitting for none of the cost of semantic. And the splitter is only half the retrieval story: comparing n8n embedding models matters just as much for whether the right chunk ranks first.

Worth naming briefly: sentence-based splitters (good for chat and Q&A), page or section chunking (wins on table-heavy PDFs, where NVIDIA's 2024 test reported 0.648 accuracy for page-level), and late chunking, where you embed the whole document then split, so each chunk embedding carries document context (Jina reported +6.5 nDCG@10 on a long-document corpus and roughly zero on a short-document one).

Here is the order the recursive splitter actually tries, which is why its chunks stay readable:

Recursive splitting cuts at the biggest natural boundary that still fits, so chunks stay readable. This is the default for most pipelines.

What chunk size and overlap should you start with?#

Start at about 512 tokens with 10 to 20 percent overlap using the recursive splitter, then tune from there. Smaller chunks sharpen retrieval but carry less context; bigger chunks keep clauses whole but blur precision. Count tokens, not characters, so your chunks actually match your embedder's limit.

Size by content type, as starting points to A/B test, not laws:

Overlap, honestly, is a small insurance policy: carrying 10 to 20 percent of tokens into the next chunk helps a concept that straddles a boundary survive in at least one chunk. But it is not free. Overlap inflates your index size and cost, and a January 2026 study found it added no measurable benefit with sparse (SPLADE) retrieval while still raising indexing cost. So keep a little overlap for dense retrieval, and reconsider it if you lean sparse.

Now the part the listicles skip: do not pick a method off someone else's leaderboard, because the leaderboards conflict.

Different corpora, embedders, and metrics, so these bars are NOT comparable head to head. That is the point: measure on your own data. Sources: Chroma

Look at the spread. Chroma's research puts recursive splitting near 88 to 89 percent recall and semantic around 0.92; a separate Vecta run from February 2026 puts recursive at 69 percent and semantic at 54 percent; and a NAACL 2025 Findings paper concluded that fixed 200-word chunks match or beat semantic chunking. Those numbers use different corpora, embedders, and metrics, so they are not comparable head to head. That is exactly the point: the only score that transfers to your project is the one you measure on your own documents.

Which chunking strategy should you actually pick?#

For most pipelines, pick the recursive splitter at about 512 tokens with 10 to 20 percent overlap. It is fast, free at index time, structure-aware, and competitive in every published benchmark. Reach for semantic or contextual chunking only when your own retrieval metrics show a real gap on dense, unstructured prose.

Default to recursive; escalate to page-level or semantic only when the content or your retrieval metrics demand it.

The short version of that decision:

• Prototyping or MVP: fixed-size and move on. Do not spend chunking-tuning time before you have a working retrieval loop.
• Structured content (headings, paragraphs, markdown): recursive splitter at ~512 tokens with overlap. This is where most pipelines should stop.
• Tables or paginated PDFs: chunk by page or section and keep tables intact. A split table is worse than a slightly oversized chunk.
• Still missing after recursive plus overlap: add context to chunks first, then test semantic chunking, and budget for the per-sentence embedding cost.

That "add context first" step is the lever most people skip.

Often the win is contextualizing chunks, not switching splitters. Source: Anthropic Contextual Retrieval.

Anthropic's Contextual Retrieval prepends chunk-specific context before embedding, and on their eval it cut retrieval failures by 35 percent with contextual embeddings, 49 percent when paired with contextual BM25, and 67 percent once reranking was added. With prompt caching the contextualizing step cost about $1.02 per million document tokens. Pairing dense and sparse retrieval the same way is covered in hybrid search with BM25 and vectors. The takeaway: contextualizing chunks often beats switching splitters, for less effort.

How do you chunk in n8n (and where it bites)?#

In n8n, chunking lives in the Default Data Loader sub-node, which takes a Text Splitter. The built-in match for recursive splitting is the Recursive Character Text Splitter, with Chunk Size and Chunk Overlap fields. There is no native semantic-chunking node as of June 2026, so semantic work means a Code or LLM step.

Set Chunk Size to around 512 and Chunk Overlap to around 50 to 100 to match the recommendation above, and confirm the splitter is counting what you expect, tokens or characters, for your embedder. The core set also includes a Character Text Splitter and a Token Text Splitter.

The gotcha is that there is no semantic splitter in the core nodes today, so semantic or contextual chunking means a Code node that embeds sentences and cuts on similarity, or an LLM/HTTP step that contextualizes chunks before the Default Data Loader runs. Plan for that build only if your metrics justify it.

Once your chunks look clean, measure retrieval on a small labeled set of your own questions, change one variable at a time (size, then overlap, then method), and keep only the change your numbers reward. To choose the store those chunks land in, pgvector vs Chroma vs Qdrant compares the options. That measure-then-tune loop, not a leaderboard, is the actual chunking strategy.