Skip to content

Which chunking strategy for which document? ​

The full chunking guide describes every strategy in depth. This page answers a narrower question: given the documents you actually have, which family should you start from, and what will still go wrong.

Start from the document, not the tool ​

Chunking strategies differ in one thing: how they choose boundaries. Whether a boundary rule works depends far more on the shape of your documents than on the rule itself. Three questions sort most corpora:

  1. Does the text carry usable structure? Headings, numbered sections, tables, code blocks. Contracts, manuals, standards and technical documentation usually do. Chat logs, transcripts and scraped prose usually do not.
  2. How long is a typical document? Under a page, whole-document embedding is often enough. Over ten pages, boundary choice starts to dominate retrieval quality.
  3. What do questions look like? Factoid lookups reward small, precise chunks. "Explain the termination conditions" rewards chunks that keep their surrounding context.

Decision table ​

Your documentsStart withWhyWhat still breaks
Short notes, tickets, FAQs (under a page)No chunkingOne embedding per document already fits the model window and preserves everything.Long tail of oversized items; retrieval returns the whole item even for one-line answers.
Plain prose without headings (articles, transcripts)Recursive delimiter or sentence chunkingParagraph and sentence boundaries are the only structure available, and these rules respect them.Topic drift inside a chunk; answers that depend on a sentence two paragraphs back.
Prose where topics shift without visible markers (research notes, meeting minutes)Semantic similarity chunkingBoundaries follow meaning rather than punctuation.Extra embedding cost at index time; boundaries are still cuts, so context outside the chunk is lost.
Markdown, HTML, codeFormat-structure chunkingThe format already marks sections and blocks; splitting on them keeps units coherent.Sections that exceed the chunk budget fall back to size-based cuts; heading context is not attached to the chunk.
PDFs with tables, figures and nested sections (contracts, manuals, filings)Partition-then-pack or hierarchical chunkingElements are detected first, so tables and captions stay whole; hierarchy lets you retrieve coarse or fine.Parser quality caps everything downstream; parent-child schemes add index complexity and still return isolated units.
Very long documents with cross-references (standards, legal codes)Hierarchical chunking plus retrieval that returns ancestorsQuestions span levels; a clause is meaningless without its section.Most implementations retrieve a leaf and hope the parent is enough; the full path is rarely reconstructed.
Anything where recall must not miss boundary factsLate chunkingToken embeddings see the whole document before pooling, so a chunk's vector reflects its neighbours.Requires long-context embedding models; the returned text is still a slice.

The failure that survives every row ​

Every strategy in the table still returns a slice of a flat sequence. The better ones choose better cut points, but the retrieved unit arrives without the trail that says what it belongs to. That is why chunk size and overlap tuning plateaus: the parameters move the boundary, they do not remove it.

POMA chunksets change the unit instead of the boundary. A chunkset is a complete root-to-leaf path through the document, so every retrieved sentence carries its chapter, section and paragraph context. That is the approach behind PrimeCut, which parses the document itself or takes the output of a parser you already run.

TL;DR

Match the strategy to the document: no chunking for short items, recursive or sentence rules for plain prose, semantic when topics shift invisibly, structure-aware and hierarchical for anything with headings and tables. Then plan for the failure they share, which is context lost at the boundary.

Continue reading ​