Text Splitter for Embeddings

Why chunking decides retrieval quality

In a retrieval pipeline, each chunk is embedded into a single vector, and a query retrieves the chunks whose vectors sit closest to it. That single vector has to represent the whole chunk, which is why chunk boundaries determine how well retrieval works — more than the choice of embedding model, in most systems.

Chunks that are too small lose the context that makes them meaningful. A sentence reading "this reduced costs by 40%" is useless in isolation, because the vector carries no information about what "this" was. It will match a query about cost reduction and then fail to answer it.

Chunks that are too large produce an averaged vector spanning several topics. It sits in the middle of everything and is strongly similar to nothing, so it is retrieved for vague queries and missed for precise ones — and when it is retrieved, most of the context window it consumes is irrelevant.

Splitting on meaning, not on arithmetic

The naive approach cuts every N characters, which severs sentences mid-clause. Half a sentence embeds badly: the vector falls between two meanings and represents neither.

The recursive approach used here splits on the largest natural boundary that fits. Paragraphs first, because a paragraph is usually a complete thought and is the right default for prose. Sentences when you need finer granularity, at the cost of losing some surrounding context. Markdown headings when the document is structured, so each section stays whole — which is usually the best choice for documentation.

A hard character cut is still applied when a single unit genuinely exceeds the window, because something has to give. But it is the fallback rather than the strategy.

Choosing size and overlap

Chunk size should be matched to what you are retrieving for. Somewhere between 300 and 800 characters suits question-answering over prose, where you want a focused passage. Larger chunks, 1,000 to 2,000 characters, suit summarisation and topic-level retrieval where breadth matters more than precision.

Check your embedding model's own input limit as well. Most have one, and text beyond it is silently truncated — so an oversized chunk can be embedded from only its first portion, which is a quiet and confusing failure.

Overlap exists for facts that straddle a boundary. Without it, a definition at the end of one chunk and its use at the start of the next are never retrieved together. Ten to twenty percent of the chunk size is the usual range.

More overlap is not better. Past about half the window, each chunk advances only slightly, which multiplies your chunk count, your embedding cost and your storage, while returning near-duplicate results that crowd out genuinely different passages. This tool clamps overlap at half the window for that reason.

Practical notes

The character offset recorded with each chunk is worth keeping. It lets you cite a retrieved passage back to its position in the source document, which is what makes a system's answers auditable rather than merely plausible.

Very short chunks are flagged because they are usually an accident — a stray heading, a caption, a list item that ended up alone. Their embeddings carry too little signal to be useful, and they add noise to retrieval.

The token estimate is a rough guide to embedding cost. Embedding is far cheaper per token than generation, but a large corpus re-embedded on every pipeline change adds up, and knowing the total before you start is worth the moment it takes.

Everything runs in your browser, so a corporate handbook or an internal knowledge base can be chunked without being uploaded anywhere.